MDSkin-Net: Multi-Task Skin Lesion Analysis Driven by Pattern Analysis Priors and Spatial Alignment Regularization

Yijian Li; Saad Bedros; Paul Bigliardi; Mei Bigliardi Qi; Vassilios Morellas; Nikolaos Papanikolopoulos

cs.CVSep 25, 2026

MDSkin-Net: Multi-Task Skin Lesion Analysis Driven by Pattern Analysis Priors and Spatial Alignment Regularization

Authors: Yijian Li, Saad Bedros, Paul Bigliardi, Mei Bigliardi Qi, Vassilios Morellas, Nikolaos Papanikolopoulos

Organizations: University of Minnesota, Minneapolis, MN, USA

Abstract

Reliable skin lesion segmentation and classification are central to dermoscopic computer-aided diagnosis. Existing multi-task frameworks couple the two tasks architecturally without clinical knowledge, while knowledge-injecting approaches rely on the macroscopic ABCD rule, which was not designed for dermoscopy. Dermoscopic diagnosis is grounded in Pattern Analysis, a microscopic framework structured around dermoscopic features. We propose MDSkin-Net, which incorporates cue-level Pattern Analysis priors into a hybrid CNN-Transformer architecture. At its core is a Pattern Analysis-Guided Attention Module (PAGAM) comprising three priors motivated by distinct dermoscopic cues: an improved Efficient Channel Attention (iECA), a Multi-Scale Spatial Attention (MSSA), and a Biased Asymmetry Attention (BAA). We further introduce a multi-scale spatial alignment regularization (MSAR) that uses the segmentation ground-truth mask as hierarchical soft supervision, confining the classification head to lesion-localized evidence and coupling both task pathways through a shared spatial prior. Trained exclusively on the ISIC 2017 training split without external dermoscopy data, the MDSkin-Net ensemble transfers robustly under zero-shot evaluation, reaching a Dice Similarity Coefficient (DSC) of 92.38% and a melanoma AUC of 97.84%on PH2, and a DSC of 88.92% on the ISIC 2018 Task 1 test set. On the in-domain ISIC 2017 benchmark, the ensemble attains a mean Area Under the Curve (AUC) of 91.60% across the two classification tasks (melanoma and seborrheic keratosis vs. rest), and a DSC of 84.72% for segmentation. Classification remains competitive with baselines; in-domain segmentation trails single-task specialists, yet the proposed priors and alignment regularization yield representations that generalize consistently across cohorts of different scales.

Figures & tables

Fig. 1: Architecture of the Proposed MDSkin-Net

Cohort	Images	Annotation	Role here
ISIC 2017 [ 22 ]	2,750	label + mask	train / val / test
PH2 [ 23 ]	200	label + mask	zero-shot, small
ISIC 2018 T1 [ 24 , 25 ]	1,000	mask only	zero-shot, large

TABLE I: Dermoscopy cohorts used in this work. Only ISIC 2017 contributes training data; the other two are held out entirely and seen only at inference.

Method	Year	Segmentation				Classification
Method	Year	DSC	SE	SP	ACC	AUC	SE	SP	ACC
U-Net [ 18 ]	2015	89.36	91.25	95.88	92.33	—	—	—	—
PSPNet [ 29 ]	2017	84.55	79.82	93.65	86.33	—	—	—	—
Attn-UNet [ 30 ]	2018	90.03	92.05	96.40	92.76	—	—	—	—
UNet++ [ 31 ]	2018	88.58	88.20	92.44	90.31	—	—	—	—
MB-DCNN † [ 11 ]	2020	92.60	97.90	95.30	96.50	95.60	82.50	90.00	88.50

TABLE II: Comparison with state-of-the-art methods on skin lesion benchmarks. (a) Cross-dataset generalization on the PH2 dataset (small cohort) under zero-shot evaluation; (b) Segmentation on the ISIC2017 official test set; (c) Classification on the ISIC2017 official test set.

Method	DSC	IoU	SE	SP	ACC
Ours	$88.81{\scriptstyle\pm 0.45}$	$81.30{\scriptstyle\pm 0.69}$	$94.31{\scriptstyle\pm 0.10}$	$96.48{\scriptstyle\pm 0.31}$	$95.38{\scriptstyle\pm 0.21}$
Ours (ens.)	88.92	81.45	94.39	96.51	95.42

TABLE III: Cross-dataset generalization on the ISIC 2018 dataset (larger cohort) under zero-shot segmentation evaluation.

Setting	Value
Optimizer	AdamW
Peak learning rate	$5\times 10^{-4}$
Weight decay	$1\times 10^{-4}$
LR multiplier, BAA	$3.0\times$ base
LR multiplier, encoder	$0.05\times$ base
Schedule	cosine annealing

TABLE IV: Training configuration. PyTorch, single NVIDIA A100.

Model	BAA	iECA	MSSA	MSAR	Params	Segmentation Metrics				Classification Metrics
Model	BAA	iECA	MSSA	MSAR	(M)	DSC	IoU	SE	SP	AUC	F1	SE	SP
I: Baseline	—	—	—	—	33.97	83.76 $\pm$ 0.94	75.09 $\pm$ 1.21	81.02 $\pm$ 2.69	99.02 $\pm$ 0.29	85.34 $\pm$ 1.58	57.14 $\pm$ 5.81	64.96 $\pm$ 9.25	85.51 $\pm$ 8.07
II: +BAA	$\checkmark$	—	—	—	33.97	84.84 $\pm$ 0.58	76.51 $\pm$ 0.84	81.47 $\pm$ 1.54	99.18 $\pm$ 0.14	84.48 $\pm$ 1.78	59.41 $\pm$ 0.97	56.13 $\pm$ 3.56	92.06 $\pm$ 1.69
III: +iECA	$\checkmark$	$\checkmark$	—	—	33.97	84.87 $\pm$ 0.50	76.50 $\pm$ 0.51	81.00 $\pm$ 0.65	99.21 $\pm$ 0.06	85.00 $\pm$ 1.65	60.05 $\pm$ 1.87	52.42 $\pm$ 1.31	94.62 $\pm$ 0.90
IV: +MSSA	$\checkmark$	$\checkmark$	$\checkmark$	—	34.06	84.94 $\pm$ 0.54	76.42 $\pm$ 0.78	82.38 $\pm$ 1.54	98.95 $\pm$ 0.10	84.90 $\pm$ 1.44	59.18 $\pm$ 2.11	61.82 $\pm$ 8.21	88.61 $\pm$ 4.31
V: Full model	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	34.08	85.43 $\pm$ 0.64	77.16 $\pm$ 0.79	82.57 $\pm$ 1.92	98.99 $\pm$ 0.37	85.37 $\pm$ 0.92	58.42 $\pm$ 3.83	50.71 $\pm$ 5.22	94.48 $\pm$ 1.81

TABLE V: Ablation study of different components on the ISIC 2017 dataset.

Model	Configuration	Segmentation		Classification
Model	Configuration	DSC	IoU	AUC	F1
I	Standard MHA (all layers)	83.76 $\pm$ 0.94	75.09 $\pm$ 1.21	85.34 $\pm$ 1.58	57.14 $\pm$ 1.58
VI	BAA (last two layers)	85.22 $\pm$ 0.53	77.00 $\pm$ 0.66	83.86 $\pm$ 0.82	55.10 $\pm$ 2.20
VII	BAA (all layers)	84.58 $\pm$ 0.79	76.24 $\pm$ 1.03	84.40 $\pm$ 1.64	55.61 $\pm$ 8.01
II	BAA (last layer only)	84.84 $\pm$ 0.58	76.51 $\pm$ 0.84	84.48 $\pm$ 1.78	59.41 $\pm$ 0.97

TABLE VI: Ablation study of different BAA layer configurations on ISIC2017 dataset.

ISIC2017 (in-domain)
Model	Align Mode	DSC	IoU	AUC	F1
VIII	None	84.94 $\pm$ 0.54	76.42 $\pm$ 0.78	84.90 $\pm$ 1.44	59.18 $\pm$ 2.11
IX	Deepest	85.15 $\pm$ 0.21	76.87 $\pm$ 0.31	86.43 $\pm$ 0.93	61.39 $\pm$ 5.08
V	Multi-scale	85.43 $\pm$ 0.64	77.16 $\pm$ 0.79	85.37 $\pm$ 1.92	58.42 $\pm$ 3.83
PH2 (zero-shot)
VIII	None	90.58 $\pm$ 1.06	83.44 $\pm$ 1.80	91.31 $\pm$ 3.39	65.87 $\pm$ 9.32

TABLE VII: Ablation study of spatial alignment strategies on ISIC2017 and PH2 dataset.

Model	Loss Strategy	Weights			Segmentation		Classification
Model	Loss Strategy	$w_{\text{seg}}$	$w_{\text{cls}}$	$w_{\text{align}}$	DSC	IoU	AUC	F1
ISIC2017 (in-domain)
X	Fixed (1:1)	1.0	1.0	1.0	85.40 $\pm$ 0.48	77.08 $\pm$ 0.60	83.76 $\pm$ 1.28	56.07 $\pm$ 2.72
XI	Fixed (1:5)	1.0	5.0	1.0	85.48 $\pm$ 1.05	77.28 $\pm$ 1.32	85.38 $\pm$ 0.87	56.43 $\pm$ 0.12
XII	Fixed (1:10)	1.0	10.0	1.0	85.17 $\pm$ 0.98	76.71 $\pm$ 1.25	84.35 $\pm$ 1.79	54.81 $\pm$ 2.44
V	Fixed (1:3)	1.0	3.0	1.0	85.43 $\pm$ 0.64	77.16 $\pm$ 0.79	85.37 $\pm$ 1.92	58.42 $\pm$ 3.83

TABLE VIII: Ablation study of different loss weighting strategies on ISIC2017 and PH2 datasets.

Fig. 2: Visualization of segmentation performance between our proposed model and other SOTA methods. From left to right, the columns sequentially display: (1) the original input image; (2) the ground truth mask; (3) the prediction by U-Net; (4) the prediction by Attention U-Net; (5) the prediction by Unet++; (6) the prediction by TransU-Net; (7) the prediction by SwinU-Net; and (8) the segmentation output of our full model.

Fig. 3: Visual ablation analysis of MDSkin-Net using seed 42. From left to right, the columns sequentially display: (1) the original input image; (2) the ground truth mask; (3) the prediction of the baseline hybrid model; (4) the baseline integrated with the BAA module; (5) the further addition of the iECA module; (6) the inclusion of the MSSA module; (7) the segmentation output of the full architecture; and (8) the learned MSAR spatial prior applied along the classification path

Fig. 4: Precision-recall curves comparing the baseline and proposed ensemble models on the ISIC 2017 test set. The proposed architecture achieves a higher AP (0.7047) than the baseline ensemble (0.6788).

Explore similar work

Jul 21, 2026

Privileged Lesion-Context Relational Distillation for Mask-Free Skin Lesion Classification

Accurate skin lesion classification can benefit from lesion segmentation masks, but requiring masks or an auxiliary segmentation model during inference reduces clinical practicality and increases computational complexity. This work introduces Privileged Lesion-Context Relational Distillation (PLCRD), a teacher-student framework that exploits lesion masks exclusively during training while preserving image-only inference. The privileged teacher jointly analyzes the original dermoscopic image and its mask-guided lesion region to learn lesion-specific and contextual diagnostic representations. An image-only student is then trained through complementary knowledge-transfer mechanisms that convey the teacher's diagnostic distribution, lesion-focused attention, inter-lesion relational geometry, and lesion-context structure. PLCRD decomposes deep representations into lesion and contextual embeddings and transfers their relational organization through inter-lesion similarity alignment, lesion-context affinity matching, separation regularization, and class-aware relational learning. This formulation avoids direct feature matching between heterogeneous teacher and student architectures and enables the student to internalize mask-informed diagnostic structure without accessing masks at deployment. The framework was evaluated on HAM10000 using lesion-disjoint data partitioning and externally validated on ISIC 2018 without retraining. PLCRD achieved a lesion-level macro-F1 of 0.773 +/- 0.018, balanced accuracy of 0.764 +/- 0.023, and macro-AUROC of 0.976 +/- 0.002 on HAM10000, together with a macro-F1 of 0.732 +/- 0.008 on ISIC 2018. The results indicate that privileged lesion annotations can be transformed into transferable relational knowledge, yielding a practical and interpretable approach to mask-free skin lesion classification.

Abu Mukaddim Rahi, Md Mithun Hossain, Md Zulficar Hasan Joy +2

Department of IT in Artificial Intelligence, Victoria University, Footscray, VIC, Australia · BUBT Resarch Graduate School, Bangladesh University of Business and Technology, Dhaka, Bangladesh · Department of Artificial Intelligence, Daffodil Institute of Information Technology, Dhaka, Bangladesh +2

Skin Lesion Classification Medical Image Classification Relational Knowledge Distillation Learning Using Privileged Information

Ask PDF

Jun 18, 2026

Geometry-Aware Superpixel Graph Transformer with Metadata for Skin Lesion Classification

Automated skin cancer classification from dermoscopic images remains challenging due to heterogeneous lesion structure, strong intra-class variability, and subtle visual differences between benign and malignant cases. Existing CNN/ViT pipelines typically rely on global or patch-level features and often combine patient metadata via late fusion, which limits spatially grounded multimodal reasoning. We present a novel region-based graph learning framework that explicitly models lesions as graphs of spatially coherent superpixel regions represented as frozen CNN features. To capture fine-grained lesion arrangements, we encode inter-regional geometry as edge attributes and introduce a dedicated metadata context node connected to all regions, providing structured integration of demographic/clinical variables within the same relational space. Node representations are updated using our edge-aware graph transformer followed by attention-driven propagation, and a final graph-level embedding for benign-malignant classification. Experiments on four public benchmarks demonstrate that explicit region-level relational modeling and graph-native multimodal fusion yield consistent gains over the state-of-the-art. Consequently, we establish a new graph-centric perspective in which CNN features are modeled as relational nodes and improved through contextual integration, yielding more expressive and robust classifications.

Muhammad Azeem, Tanveer Hussain, Amr Ahmed +1

Edge Hill University, Ormskirk, Lancashire, L39 4QP, United Kingdom

Skin Lesion Classification Graph Representation Learning Multimodal Classification Medical Image Classification Graph Transformer

Ask PDF

Apr 7, 2026

RABC-Net: Reliability-Aware Annotation-Free Skin Lesion Segmentation for Low-Resource Dermoscopy

Pixel-level annotation is costly in low-resource dermoscopy. We present RABC-Net, a reliability-aware annotation-free segmentation system that combines pseudo-label reliability learning, restricted target-domain adaptation, and Reliability-Adaptive Boundary Calibration (RABC). The system decouples reliability learning from deployment: uncertainty-aware pseudo-label interaction shapes robust representations during training, while the image-only inference path is preserved and RABC performs local logit-space calibration from boundary confidence, uncertainty, and foreground probability. No manual masks are used for training or target-domain adaptation; validation labels, when available, are used only for final operating-point selection. Across ISIC-2017, ISIC-2018, and PH2, RABC-Net achieves macro-average DICE/JAC of 86.58%/79.47% and consistent matched-protocol results. Controlled within-study analyses show that RABC provides localized gains over nonlearned boundary correction, while the overall result comes from the full reliability-aware system. Adaptation updates only 3.50% of model parameters, image-only inference runs at 87.4 FPS, and the selected operating points use

σ=0

on all three datasets, indicating that learned calibration avoids extra smoothing at deployment.

Yujie Yao, Yuhaohang He, Junjie Huang +6

College of Information Engineering, Sichuan Agricultural University, Ya’an, China

Skin Lesion Segmentation Boundary-Aware Segmentation Pseudo-Labeling Unsupervised Domain Adaptation

Ask PDF