Recent advances in dermatological image analysis have been driven by large-scale annotated datasets; however, most existing benchmarks focus on dermatoscopic images and lack patient-authored queries and clinical context, limiting their applicability to patient-centered care. To address this gap, we introduce DermaVQA-DAS, an extension of the DermaVQA dataset that supports two complementary tasks: closed-ended question answering (QA) and dermatological lesion segmentation. Central to this work is the Dermatology Assessment Schema (DAS), a novel expert-developed framework that systematically captures clinically meaningful dermatological features in a structured and standardized form. DAS comprises 36 high-level and 27 fine-grained assessment questions, with multiple-choice options in English and Chinese. Leveraging DAS, we provide expert-annotated datasets for both closed QA and segmentation and benchmark state-of-the-art multimodal models. For segmentation, we evaluate multiple prompting strategies and show that prompt design impacts performance: the default prompt achieves the best results under Mean-of-Max and Mean-of-Mean evaluation aggregation schemes, while an augmented prompt incorporating both patient query title and content yields the highest performance under majority-vote-based microscore evaluation, achieving a Jaccard index of 0.395 and a Dice score of 0.566 with BiomedParse. For closed-ended QA, overall performance is strong across models, with average accuracies ranging from 0.729 to 0.798; o3 achieves the best overall accuracy (0.798), closely followed by GPT-4.1 (0.796), while Gemini-1.5-Pro shows competitive performance within the Gemini family (0.783). We publicly release DermaVQA-DAS, the DAS schema, and evaluation protocols to support and accelerate future research in patient-centered dermatological vision-language modeling (https://osf.io/72rp3).
Figures & tables
Figure 1: An example from the DermaVQA-DAS dataset with the original consumer health query and the accompanying images, as well as the created closed-ended question-answer pairs and segmentation annotations.
Category
Type
Question
Percent
General
Site
How much of the body is effected?
0.99
General
Site
1 Where is the affected area?
0.98
Skin Specific
Affected area description
1 What label best describes the affected area?
0.98
Skin Specific
Lesion Color
Compared to the normal surrounding skin, what is the color of the skin lesion?
0.97
Skin Specific
Lesion Count
How many skin lesions are there?
0.94
General
Size
1 How large are the affected areas? Please specify which affected area for each selection.
0.94
Table 1: Question Relevance By Encounter Percentage (Top 20)
CQID
Question Type
Slots
# Options
Example Options
CQID010
Site Extent
1
4
single spot; widespread
CQID011
Site Location
6
8
head; upper extremities
CQID012
Size
6
4
thumb nail; palm
CQID015
Onset
1
7
within days; within months
CQID020
Skin Description
6
10
raised; flat; crust
CQID025
Itch
1
3
yes; no
Table 2: Extract of the Dermatology Assessment Schema (DAS). Repeated slots enable representation of multiple affected sites or lesions. Example options are illustrative rather than exhaustive.
Split
Encounters
Images
Masks
Train
842
2,474
7,448
Valid
56
157
472
Test
100
314
944
Total
998
2,945
8,864
Table 3: Segmentation dataset statistics across splits.
Mean of Max
Mean of Mean
Majority Vote
Model
Jaccard
Dice
Jaccard
Dice
Jaccard
Dice
MedSAM (no prompt)
0.4097
0.5250
0.3299
0.4386
0.2597
0.4123
BiomedParse (prompt I)
0.5088
0.6133
0.4253
0.5315
0.3863
0.5573
BiomedParse (prompt II)
0.5034
0.6068
0.4215
0.5268
0.3893
0.5604
BiomedParse (prompt III)
0.5081
0.6120
0.4243
0.5298
0.3948
0.5661
Table 4: Segmentation performance under different prompting strategies. Prompt I (default: "Highlight the abnormal skin lesion that appears different from surrounding healthy skin." ), Prompt II (prompt augmented with patient query title), and Prompt III (prompt augmented with query title and content). Best results within each aggregation scheme are shown in bold.
System
CQID010
CQID011
CQID012
CQID015
CQID020
CQID025
CQID034
CQID035
CQID036
ALL
Gemini-1.5-Pro
0.650
0.876
0.793
0.820
0.746
0.900
0.660
0.890
0.710
0.783
Gemini 2.0 Flash
0.640
0.873
0.769
0.790
0.757
0.930
0.630
0.840
0.680
0.768
Claude 3.7 Sonnet (2025-02-19)
0.620
0.853
0.780
0.840
0.720
0.960
0.560
0.880
0.640
0.761
o4-mini (2025-04-16)
0.660
0.847
0.590
0.810
0.735
0.970
0.620
0.900
0.430
0.729
GPT-4.1
0.670
0.881
0.830
0.850
0.757
0.930
0.640
0.880
0.730
0.796
o3 (2025-04-16)
0.650
0.893
0.835
0.830
0.756
0.980
0.600
0.900
0.740
0.798
Table 5: Results of the closed-ended question answering task. Best performance per category is shown in bold; second-best is underlined.
QID
Spearman
Desc
CQID010
-0.147
Site Extent
CQID015
0.116
Onset
CQID025
0.116
Itch
CQID034
0.134
Lesion Color
CQID035
-0.001
Lesion Count
CQID036
0.046
Texture
Table 6: Correlation Between Site Number and Accuracy
This study presents a methodology for constructing a clinically verified dataset of dermatoscopic images for medical informatics research. The relevance of the work is driven by the fact that the performance of automated diagnostic support systems depends not only on the volume of images, but also on the reproducibility of the image acquisition procedure, the completeness of structured metadata, and the reliability of diagnostic labels. International collections were primarily created under conditions that differ substantially from routine Russian outpatient practice and mobile dermatoscopy. The proposed methodology integrates three interconnected components: (1) a standard operating procedure (SOP) for acquiring images via mobile dermatoscopy, (2) an information model comprising 16 structured metadata fields organized into six clinically oriented blocks in ISIC-compatible notation, and (3) a multi-stage expert verification of diagnostic labels (initial clinical annotation, consensus review by three specialists, and histological confirmation of all malignant neoplasms). Using this methodology, a dataset of 1,026 unique dermatoscopic images from 443 patients was collected between June 2025 and May 2026. From 1,044 initial records, 18 duplicates were excluded. The dataset includes nine nosological categories; all 39 malignant lesions (18 melanomas, 15 basal cell carcinomas, and 6 squamous cell carcinomas) were histologically verified. Patient age ranged from 2 to 90 years (median 38), with 279 females (63%) and 164 males (37%). Each image is accompanied by expert-annotated dermatoscopic structures and an explicit verification_stage field indicating the level of diagnostic confirmation. The resulting dataset serves as a pilot clinically verified resource suitable for independent model evaluation, domain shift analysis, interpretability studies, and further expansion.
Kozachok Elena Sergeevna
Ivannikov Institute for System Programming of RAS, Moscow, Russia
Skin lesion segmentation without annotated masks requires the target to be specified through other cues. Existing approaches rely on dermoscopy-specific rules, annotation-derived model priors, or user-selected regions. A general text-to-image diffusion transformer offers a text interface to the target, but its concept-word attention is unreliable on dermoscopy images: in the model's attention-output space, the word-derived skin-to-lesion direction is poorly aligned with the direction separating real skin from real lesions. We show that the direction derived from generated healthy-skin and lesion images is substantially better aligned. Based on this finding, we propose GENIQ, a training-free method that constructs initial skin and lesion prototypes from generated images and refines them using the most skin-like and lesion-like patches from unlabelled dermoscopy images. It scores test patches along the refined direction and applies a gated threshold to suppress contextual spill-over around small lesions. Without inference-time annotations or additional models, GENIQ achieves Dice scores of 0.85-0.88 on three dermoscopy test sets. It outperforms zero-shot and word-based attention methods on all three sets, and a hand-designed dermoscopy pipeline in Dice and IoU on two.
Hongyu Cao, Guangfu Guo, Jinghan Zhang +3
Clemson University, Clemson, SC, USA · Analog Devices, Inc., USA · Wake Forest University, USA
Large vision-language models (LVLMs) demonstrate strong performance in dermatology; however, evaluating diagnostic reasoning for rare conditions remains largely unexplored. Existing benchmarks focus on common diseases and assess only final accuracy, overlooking the clinical reasoning process, which is critical for complex cases. We address this gap by constructing DermCase, a long-context benchmark derived from peer-reviewed case reports. Our dataset contains 26,030 multi-modal image-text pairs and 6,354 clinically challenging cases, each annotated with comprehensive clinical information and step-by-step reasoning chains. To enable reliable evaluation, we establish DermLIP-based similarity metrics that achieve stronger alignment with dermatologists for assessing differential diagnosis quality. Benchmarking 22 leading LVLMs exposes significant deficiencies across diagnosis accuracy, differential diagnosis, and clinical reasoning. Fine-tuning experiments demonstrate that instruction tuning substantially improves performance while Direct Preference Optimization (DPO) yields minimal gains. Systematic error analysis further reveals critical limitations in current models' reasoning capabilities.
Yang Liu, Jiyao Yang, Hongjin Zhao +10
Carnegie Mellon University · University of Alabama at Birmingham · Australian National University +3