cs.CVAug 17, 2026

Picking the Right Image to Classify: Reliable-Input Selection in Teledermatology

Authors: Fabian Gröger, Marco Weishaupt, Philippe Gottfrois, Simone Lionetti, Linda Wermelinger, Nipun Ranasekara, Ludovic Amruthalingam, Alexander A. Navarini, +1 more

Organizations: University of Basel · Lucerne University of Applied Sciences and Arts · University Hospital Basel

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Dermatology models face distribution shifts in teledermatology settings, where submitted images differ from the training data in lighting, angle, distance, focus, and framing. These test-time images are ordinary clinical photographs, but some fall outside the model's training conditions, leading the model to often misclassify them due to shifts in acquisition between training and deployment. When multiple images of the same case exist (several photos of one patient or lesion), a natural way to improve accuracy is therefore to select the image the model is most likely to classify correctly. We call this task reliable-input selection. An oracle that, for each case, selects a correctly classified image when one exists raises weighted F1 by about 20 percentage points on average across six dermatology datasets and nine frozen backbones. This oracle is an upper bound that sees the labels, whereas a selector must choose blindly. Capturing this gain in practice is hard. A selector that needs no pretraining data applies to any frozen model, including those whose data is not public. It must judge reliability from quantities the model exposes at inference: its embeddings, their norms, and its confidence. We benchmark four such training-data-free selectors: the embedding norm, the neighborhood consensus among a case's images, the stability of the prediction under small perturbations, and the model's own confidence. No training-data-free selector substantially narrows this oracle gap. The best of them is the model's own confidence, but it recovers only a small part of the gap on the clinical datasets. A small labeled reference set does not help either: the best selector overall, a fusion of confidence and Mahalanobis distance, still leaves most of the gap. To our knowledge, this is the first study to introduce and benchmark reliable input selection, a clinically important, unsolved task.

Figures & tables

Figure 1: Reliable-input selection. In teledermatology a patient submits several images of their skin condition, varying in body site and acquisition. Here one matches the model’s training conditions (standardized) and two differ in angle, distance, or lighting (off-condition). A frozen model embeds each image as zvz_{v} and classifies the standardized image correctly ( ✓ ) but the off-condition images incorrectly ( ×\times ), silent failures that selection avoids. The task is to score the images and classify the selected one.
Figure 2: A large oracle gap that current selectors barely narrow. Weighted F1 on the six datasets, grouped by multi-image regime (mean over nine frozen backbones and 1010 seeds, error bars show ±1\pm 1 standard deviation across backbones). An oracle that picks a correct image per case (green) reaches F1 well above random selection (gray), whereas the best training-data-free selector (blue) recovers only a small part of that gap. We baseline against random throughout. A fixed best view is omitted, as it is defined only for the aligned datasets and is not deployable (see Sec. 4.1 ).
 per-patientviewpoint / modalityrepeated capture 
MethodPASSIONDermaCon-INSCINderm7ptPAD-UFESHAM10000mean
majority vote+3.9+3.9+2.6+2.6+1.3+1.3+0.2+0.2+0.7+0.7+1.0+1.0+1.6+1.6
soft vote+6.3+6.3+6.0+6.0+2.4+2.4+4.8+4.8+3.6+3.6+3.6+3.6+4.4+4.4
training-data-free
embedding norm+0.1+0.1+0.4+0.4−0.2-0.2+0.6+0.6+0.4+0.4+0.2+0.2+0.3+0.3
neighborhood consensus+1.1+1.1+0.7+0.7+1.0+1.0+1.0+1.0+0.5+0.5+0.5+0.5+0.8+0.8
Table 1: No selector recovers much of the oracle gap (weighted-F1 gain over random selection, percentage points, mean over nine backbones, 1010 seeds). The strongest training-data-free selector is the model’s own confidence. A fusion with a class-conditional Mahalanobis score (a small labeled reference set) is best overall, yet both leave the bulk of the oracle gap (last row) unexploited. Majority vote and soft vote aggregate a case’s images instead of selecting one. Datasets are grouped by multi-image regime. The standard deviation across backbones is at most 33 points.
Figure 3: Why selection is hard: ample room, weak signal. Left: every case is all-correct (selection irrelevant), no-image-correct (unrescuable), or mixed (rescuable). The mixed group is large, so there is room to rescue. Right: AUROC of each training-data-free score for per-image correctness on the mixed cases. The embedding norm and neighborhood consensus sit at chance. Only the classifier’s confidence is clearly above it, and even then only moderately, too weak to exploit that room. All panels use all six datasets, in the same order as Fig. 2 . The dermoscopy datasets (derm7pt, HAM10000) show the same pattern as the clinical ones. Means over nine backbones and 1010 seeds. Error bars on the right panel show ±1\pm 1 standard deviation across backbones.

Explore similar work

CardsList
Sep 2, 2026

Beyond Blur: A Semantic Tri-view Pipeline for Teledermatology Gradability via Skin Micro-relief

Smartphone skin photographs are indispensable to teledermatology, yet assessing the diagnostic suitability of submitted cases (gradability) remains a critical bottleneck in mobile care workflows. Dermatologists routinely review multiple photographic views (regional, angled, and close-up) to identify consistent textural detail rather than relying on a single image. We present the Semantic Tri-view Pipeline, an interpretable architecture for automated teledermatology gradability screening that formalizes epidermal micro-relief as a computable biomarker of image quality. Using an expert-annotated subset of the public SCIN dataset, we train a lightweight DeepLabV3+ model to segment micro-relief fidelity. These spatial masks are then aggregated across up to three case views with a logistic regression classifier, leveraging viewpoint redundancy to support robustness under uncontrolled smartphone acquisition. This approach learns context-aware, clinically intelligible heuristics, such as penalizing high-fidelity texture in regional distance views. Evaluated at a predefined 90% sensitivity operating point, the system's apparent errors largely reflect subjective clinical variance on borderline cases where clinicians rely on non-visual metadata. On SCIN, performance improves from an AUC of 0.81 (80.6% PPV) on variance-heavy majority-consensus cases to 0.96 (97.7% PPV) on optically unambiguous unanimous cases. Overall, this work delivers an interpretable, privacy-by-design, edge-ready system that can provide real-time feedback during case submission to filter ungradable photo sets before review.

Robert Engel

Avalytiq San Francisco, USA

Explainable Medical Image AnalysisImage Quality Assessment
AskPDF
Aug 26, 2026

Label-Free Foundational Model Selection for Medical Image Classification under Distribution Shift via Pseudo Label Discrepancy

Foundation models are increasingly deployed for medical image analysis. However, under the inter-institutional distribution shift typical of deployment, their performance varies widely and cannot be known without target-domain labels, which are rarely available. This leaves a practical question unresolved: given several candidate foundational models and labeled-data from a source domain, which one to deploy in an unlabeled target domain? We propose a label-free selection criterion built on SUDO, a framework for evaluating clinical AI systems without ground-truth annotations. SUDO partitions the unlabeled target data by predicted probability and, for each region, measures a pseudo-label discrepancy reflecting class contamination; aggregated across regions, this yields a score (AURCC) requiring neither target annotation nor fine-tuning. We show that AURCC can be used to rank a variety of vision-language models (BioMedCLIP, CXR-CLIP, CheXzero, MedCLIP, MedImageInsight, CLIP) on chest X-ray classification across three inter-hospital shift scenarios, under zero-shot and MLP-probe regimes. The AURCC ranking recovers the ground-truth ranking with Spearman rho up to 0.943 (p<0.05). Against the natural baseline of ranking by held-out source accuracy, AURCC is competitive when the labeled source is large and yields a more accurate ranking once it is small; the regime of interest in resource-constrained settings.

Juan Iñaki Larrea, Lucas Mansilla, Enzo Ferrante

Applied Artificial Intelligence Lab (LIAA), Computer Sciences Institute (ICC), Universidad de Buenos Aires - CONICET, Argentina · Research Institute for Signals, Systems and Computational Intelligence, sinc(i), Universidad Nacional del Litoral - CONICET, Argentina

Model SelectionMedical Imaging Foundation ModelsMedical Image ClassificationDistribution ShiftMedical VLMs
AskPDF
Oct 2, 2026

Bridging Research and Practice: A Systematic Evaluation of Generalist and Dermatology-Specific Models in Clinical Skin Lesion Classification

The application of machine learning to dermatology has grown substantially in recent years, moving beyond proof-of-concept studies toward potential applications. However, clinical dermatology remains a challenging and still open problem. Diagnostic assessment is often ambiguous, and skin lesions exhibit high variability, compounded by differences in acquisition modality, device quality, and patient demographics. These factors hinder the development of robust models suitable for safe and equitable clinical use. To support translation into practice, it is essential to systematically evaluate how contemporary models generalize across heterogeneous data sources. In this work, we benchmark a diverse set of architectures on recent dermatology datasets, spanning dermoscopic images and smartphone-based clinical photographs. We assess the robustness of recent general-purpose and medical vision-language models, as well as foundation models, and compare them against task-specific dermatology classifiers, including embedding-based approaches and convolutional neural networks. Our study provides an evaluation of model performance under distribution shifts, modality changes, and demographic variability. By quantifying the gap between current state-of-the-art models and the requirements of clinical deployment, we aim to contribute to the development of reliable, accessible, and clinically applicable AI systems for dermatology.

Emanoel dos Santos, Kelvin Cunha, Rodrigo Mota +8

Centro de Informática, Universidade Federal de Pernambuco (UFPE), Brazil · Hospital das Clinicas, Ebserh, UFPE, Pernambuco, Brazil

Skin Lesion ClassificationCross-Dataset GeneralizationMedical Imaging Foundation ModelsMedical VLMsVLM Robustness
AskPDF