MDSkin-Net: Multi-Task Skin Lesion Analysis Driven by Pattern Analysis Priors and Spatial Alignment Regularization

Yijian Li; Saad Bedros; Paul Bigliardi; Mei Bigliardi Qi; Vassilios Morellas; Nikolaos Papanikolopoulos

cs.CVSep 25, 2026

MDSkin-Net: Multi-Task Skin Lesion Analysis Driven by Pattern Analysis Priors and Spatial Alignment Regularization

Authors: Yijian Li, Saad Bedros, Paul Bigliardi, Mei Bigliardi Qi, Vassilios Morellas, Nikolaos Papanikolopoulos

Organizations: University of Minnesota, Minneapolis, MN, USA

Abstract

Reliable skin lesion segmentation and classification are central to dermoscopic computer-aided diagnosis. Existing multi-task frameworks couple the two tasks architecturally without clinical knowledge, while knowledge-injecting approaches rely on the macroscopic ABCD rule, which was not designed for dermoscopy. Dermoscopic diagnosis is grounded in Pattern Analysis, a microscopic framework structured around dermoscopic features. We propose MDSkin-Net, which incorporates cue-level Pattern Analysis priors into a hybrid CNN-Transformer architecture. At its core is a Pattern Analysis-Guided Attention Module (PAGAM) comprising three priors motivated by distinct dermoscopic cues: an improved Efficient Channel Attention (iECA), a Multi-Scale Spatial Attention (MSSA), and a Biased Asymmetry Attention (BAA). We further introduce a multi-scale spatial alignment regularization (MSAR) that uses the segmentation ground-truth mask as hierarchical soft supervision, confining the classification head to lesion-localized evidence and coupling both task pathways through a shared spatial prior. Trained exclusively on the ISIC 2017 training split without external dermoscopy data, the MDSkin-Net ensemble transfers robustly under zero-shot evaluation, reaching a Dice Similarity Coefficient (DSC) of 92.38% and a melanoma AUC of 97.84%on PH2, and a DSC of 88.92% on the ISIC 2018 Task 1 test set. On the in-domain ISIC 2017 benchmark, the ensemble attains a mean Area Under the Curve (AUC) of 91.60% across the two classification tasks (melanoma and seborrheic keratosis vs. rest), and a DSC of 84.72% for segmentation. Classification remains competitive with baselines; in-domain segmentation trails single-task specialists, yet the proposed priors and alignment regularization yield representations that generalize consistently across cohorts of different scales.

Figures & tables

Fig. 1: Architecture of the Proposed MDSkin-Net

Cohort	Images	Annotation	Role here
ISIC 2017 [ 22 ]	2,750	label + mask	train / val / test
PH2 [ 23 ]	200	label + mask	zero-shot, small
ISIC 2018 T1 [ 24 , 25 ]	1,000	mask only	zero-shot, large

TABLE I: Dermoscopy cohorts used in this work. Only ISIC 2017 contributes training data; the other two are held out entirely and seen only at inference.

Method	Year	Segmentation				Classification
Method	Year	DSC	SE	SP	ACC	AUC	SE	SP	ACC
U-Net [ 18 ]	2015	89.36	91.25	95.88	92.33	—	—	—	—
PSPNet [ 29 ]	2017	84.55	79.82	93.65	86.33	—	—	—	—
Attn-UNet [ 30 ]	2018	90.03	92.05	96.40	92.76	—	—	—	—
UNet++ [ 31 ]	2018	88.58	88.20	92.44	90.31	—	—	—	—
MB-DCNN † [ 11 ]	2020	92.60	97.90	95.30	96.50	95.60	82.50	90.00	88.50

TABLE II: Comparison with state-of-the-art methods on skin lesion benchmarks. (a) Cross-dataset generalization on the PH2 dataset (small cohort) under zero-shot evaluation; (b) Segmentation on the ISIC2017 official test set; (c) Classification on the ISIC2017 official test set.

Method	DSC	IoU	SE	SP	ACC
Ours	$88.81{\scriptstyle\pm 0.45}$	$81.30{\scriptstyle\pm 0.69}$	$94.31{\scriptstyle\pm 0.10}$	$96.48{\scriptstyle\pm 0.31}$	$95.38{\scriptstyle\pm 0.21}$
Ours (ens.)	88.92	81.45	94.39	96.51	95.42

TABLE III: Cross-dataset generalization on the ISIC 2018 dataset (larger cohort) under zero-shot segmentation evaluation.

Setting	Value
Optimizer	AdamW
Peak learning rate	$5\times 10^{-4}$
Weight decay	$1\times 10^{-4}$
LR multiplier, BAA	$3.0\times$ base
LR multiplier, encoder	$0.05\times$ base
Schedule	cosine annealing

TABLE IV: Training configuration. PyTorch, single NVIDIA A100.

Model	BAA	iECA	MSSA	MSAR	Params	Segmentation Metrics				Classification Metrics
Model	BAA	iECA	MSSA	MSAR	(M)	DSC	IoU	SE	SP	AUC	F1	SE	SP
I: Baseline	—	—	—	—	33.97	83.76 $\pm$ 0.94	75.09 $\pm$ 1.21	81.02 $\pm$ 2.69	99.02 $\pm$ 0.29	85.34 $\pm$ 1.58	57.14 $\pm$ 5.81	64.96 $\pm$ 9.25	85.51 $\pm$ 8.07
II: +BAA	$\checkmark$	—	—	—	33.97	84.84 $\pm$ 0.58	76.51 $\pm$ 0.84	81.47 $\pm$ 1.54	99.18 $\pm$ 0.14	84.48 $\pm$ 1.78	59.41 $\pm$ 0.97	56.13 $\pm$ 3.56	92.06 $\pm$ 1.69
III: +iECA	$\checkmark$	$\checkmark$	—	—	33.97	84.87 $\pm$ 0.50	76.50 $\pm$ 0.51	81.00 $\pm$ 0.65	99.21 $\pm$ 0.06	85.00 $\pm$ 1.65	60.05 $\pm$ 1.87	52.42 $\pm$ 1.31	94.62 $\pm$ 0.90
IV: +MSSA	$\checkmark$	$\checkmark$	$\checkmark$	—	34.06	84.94 $\pm$ 0.54	76.42 $\pm$ 0.78	82.38 $\pm$ 1.54	98.95 $\pm$ 0.10	84.90 $\pm$ 1.44	59.18 $\pm$ 2.11	61.82 $\pm$ 8.21	88.61 $\pm$ 4.31
V: Full model	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	34.08	85.43 $\pm$ 0.64	77.16 $\pm$ 0.79	82.57 $\pm$ 1.92	98.99 $\pm$ 0.37	85.37 $\pm$ 0.92	58.42 $\pm$ 3.83	50.71 $\pm$ 5.22	94.48 $\pm$ 1.81

TABLE V: Ablation study of different components on the ISIC 2017 dataset.

Model	Configuration	Segmentation		Classification
Model	Configuration	DSC	IoU	AUC	F1
I	Standard MHA (all layers)	83.76 $\pm$ 0.94	75.09 $\pm$ 1.21	85.34 $\pm$ 1.58	57.14 $\pm$ 1.58
VI	BAA (last two layers)	85.22 $\pm$ 0.53	77.00 $\pm$ 0.66	83.86 $\pm$ 0.82	55.10 $\pm$ 2.20
VII	BAA (all layers)	84.58 $\pm$ 0.79	76.24 $\pm$ 1.03	84.40 $\pm$ 1.64	55.61 $\pm$ 8.01
II	BAA (last layer only)	84.84 $\pm$ 0.58	76.51 $\pm$ 0.84	84.48 $\pm$ 1.78	59.41 $\pm$ 0.97

TABLE VI: Ablation study of different BAA layer configurations on ISIC2017 dataset.

ISIC2017 (in-domain)
Model	Align Mode	DSC	IoU	AUC	F1
VIII	None	84.94 $\pm$ 0.54	76.42 $\pm$ 0.78	84.90 $\pm$ 1.44	59.18 $\pm$ 2.11
IX	Deepest	85.15 $\pm$ 0.21	76.87 $\pm$ 0.31	86.43 $\pm$ 0.93	61.39 $\pm$ 5.08
V	Multi-scale	85.43 $\pm$ 0.64	77.16 $\pm$ 0.79	85.37 $\pm$ 1.92	58.42 $\pm$ 3.83
PH2 (zero-shot)
VIII	None	90.58 $\pm$ 1.06	83.44 $\pm$ 1.80	91.31 $\pm$ 3.39	65.87 $\pm$ 9.32

TABLE VII: Ablation study of spatial alignment strategies on ISIC2017 and PH2 dataset.

Model	Loss Strategy	Weights			Segmentation		Classification
Model	Loss Strategy	$w_{\text{seg}}$	$w_{\text{cls}}$	$w_{\text{align}}$	DSC	IoU	AUC	F1
ISIC2017 (in-domain)
X	Fixed (1:1)	1.0	1.0	1.0	85.40 $\pm$ 0.48	77.08 $\pm$ 0.60	83.76 $\pm$ 1.28	56.07 $\pm$ 2.72
XI	Fixed (1:5)	1.0	5.0	1.0	85.48 $\pm$ 1.05	77.28 $\pm$ 1.32	85.38 $\pm$ 0.87	56.43 $\pm$ 0.12
XII	Fixed (1:10)	1.0	10.0	1.0	85.17 $\pm$ 0.98	76.71 $\pm$ 1.25	84.35 $\pm$ 1.79	54.81 $\pm$ 2.44
V	Fixed (1:3)	1.0	3.0	1.0	85.43 $\pm$ 0.64	77.16 $\pm$ 0.79	85.37 $\pm$ 1.92	58.42 $\pm$ 3.83

TABLE VIII: Ablation study of different loss weighting strategies on ISIC2017 and PH2 datasets.

Fig. 2: Visualization of segmentation performance between our proposed model and other SOTA methods. From left to right, the columns sequentially display: (1) the original input image; (2) the ground truth mask; (3) the prediction by U-Net; (4) the prediction by Attention U-Net; (5) the prediction by Unet++; (6) the prediction by TransU-Net; (7) the prediction by SwinU-Net; and (8) the segmentation output of our full model.

Fig. 3: Visual ablation analysis of MDSkin-Net using seed 42. From left to right, the columns sequentially display: (1) the original input image; (2) the ground truth mask; (3) the prediction of the baseline hybrid model; (4) the baseline integrated with the BAA module; (5) the further addition of the iECA module; (6) the inclusion of the MSSA module; (7) the segmentation output of the full architecture; and (8) the learned MSAR spatial prior applied along the classification path

Fig. 4: Precision-recall curves comparing the baseline and proposed ensemble models on the ISIC 2017 test set. The proposed architecture achieves a higher AP (0.7047) than the baseline ensemble (0.6788).

Explore similar work

Privileged Lesion-Context Relational Distillation for Mask-Free Skin Lesion Classification
Jul 21, 2026Abu Mukaddim Rahi, Md Mithun Hossain, Md Zulficar Hasan Joy +2Skin Lesion Classification Medical Image Classification
Department of IT in Artificial Intelligence, Victoria University, Footscray, VIC, Australia · BUBT Resarch Graduate School, Bangladesh University of Business and Technology, Dhaka, Bangladesh · Department of Artificial Intelligence, Daffodil Institute of Information Technology, Dhaka, Bangladesh +2
Geometry-Aware Superpixel Graph Transformer with Metadata for Skin Lesion Classification
Jun 18, 2026Muhammad Azeem, Tanveer Hussain, Amr Ahmed +1Skin Lesion Classification Graph Representation Learning
Edge Hill University, Ormskirk, Lancashire, L39 4QP, United Kingdom
RABC-Net: Reliability-Aware Annotation-Free Skin Lesion Segmentation for Low-Resource Dermoscopy
Apr 7, 2026Yujie Yao, Yuhaohang He, Junjie Huang +6Skin Lesion Segmentation Boundary-Aware Segmentation
College of Information Engineering, Sichuan Agricultural University, Ya’an, China

Jul 21, 2026J/K move · Enter open · S save

Privileged Lesion-Context Relational Distillation for Mask-Free Skin Lesion Classification

Abu Mukaddim Rahi, Md Mithun Hossain, Md Zulficar Hasan Joy, M. F. Mridha+1

Department of IT in Artificial Intelligence, Victoria University, Footscray, VIC, Australia · BUBT Resarch Graduate School, Bangladesh University of Business and Technology, Dhaka, Bangladesh · Department of Artificial Intelligence, Daffodil Institute of Information Technology, Dhaka, Bangladesh+2

Skin Lesion Classification Medical Image Classification Relational Knowledge Distillation Learning Using Privileged Information

Open paper AskPDF arXiv

Accurate skin lesion classification can benefit from lesion segmentation masks, but requiring masks or an auxiliary segmentation model during inference reduces clinical practicality and increases computational complexity. This work introduces Privileged Lesion-Context Relational Distillation (PLCRD), a teacher-student framework that exploits lesion masks exclusively during training while preserving image-only inference. The privileged teacher jointly analyzes the original dermoscopic image and its mask-guided lesion region to learn lesion-specific and contextual diagnostic representations. An image-only student is then trained through complementary knowledge-transfer mechanisms that convey the teacher's diagnostic distribution, lesion-focused attention, inter-lesion relational geometry, and lesion-context structure. PLCRD decomposes deep representations into lesion and contextual embeddings and transfers their relational organization through inter-lesion similarity alignment, lesion-context affinity matching, separation regularization, and class-aware relational learning. This formulation avoids direct feature matching between heterogeneous teacher and student architectures and enables the student to internalize mask-informed diagnostic structure without accessing masks at deployment. The framework was evaluated on HAM10000 using lesion-disjoint data partitioning and externally validated on ISIC 2018 without retraining. PLCRD achieved a lesion-level macro-F1 of 0.773 +/- 0.018, balanced accuracy of 0.764 +/- 0.023, and macro-AUROC of 0.976 +/- 0.002 on HAM10000, together with a macro-F1 of 0.732 +/- 0.008 on ISIC 2018. The results indicate that privileged lesion annotations can be transformed into transferable relational knowledge, yielding a practical and interpretable approach to mask-free skin lesion classification.

Abstract

Explore similar work

Privileged Lesion-Context Relational Distillation for Mask-Free Skin Lesion Classification

Geometry-Aware Superpixel Graph Transformer with Metadata for Skin Lesion Classification

RABC-Net: Reliability-Aware Annotation-Free Skin Lesion Segmentation for Low-Resource Dermoscopy