Organizations: Department of Computer Science, National Yang Ming Chiao Tung University, Hsinchu, 300093, Taiwan · Department of Computer Science, National Tsing Hua University, Hsinchu, 300093, Taiwan
Multi-label few-shot learning (MLFSL) remains a significant challenge in medical image analysis (MIA). Current metric-based meta-learning methods face two critical limitations in MIA. First, conventional prototype generation often entangles irrelevant disease information, leading to contaminated prototypes and degraded performance. Second, prior studies typically enforce inter-class separability in embedding space, largely neglecting the inherent correlations among diseases. To overcome these challenges, we propose Prototype Purification and Regulation (PPR), a novel MLFSL framework for MIA. PPR first performs prototype purification by leveraging sample-level comorbidity scores to emphasize disease-specific features, producing purified prototypes that better characterize each disease. Building upon these purified prototypes, PPR further addresses the underexplored problem of inter-class prototype distance in MIA by incorporating disease-level comorbidity statistics to adaptively regulate inter-class similarity, forming a comorbidity-aware embedding space. Overall, PPR sequentially enables the model to capture pure disease features and inter-class relationships for reliable MLFSL in MIA. Extensive experiments across four chest X-ray benchmark datasets, including cross-domain evaluation, show that PPR consistently outperforms state-of-the-art methods, significantly improving disease detection while demonstrating robust generalization and clinical applicability.
Figures & tables
Figure 1: Comparison of prototype generation strategies in a 3-way 3-shot example. Gradient-color samples indicate mixed feature representations caused by multi-label co-occurrence, while single-colored samples correspond to pure single-disease features. (a) Support samples are encoded into the embedding space. (b) Existing methods derive a reference feature by simply averaging support features, then perform similarity-weighted averaging based on this biased reference, resulting in a contaminated prototype. (c) Our method enables the model to capture pure features by aligning the reference feature toward disease-specific representations, producing a purified prototype that better preserves disease characteristics in the embedding space.
Figure 2: Overview of the proposed PPR framework in a 3-way 3-shot example. (a) Purification: The reference feature is aligned toward purer representations by leveraging the label distribution derived from the sample comorbidity score as a purity prior, thereby emphasizing discriminative features and generating purified prototypes. (b) Regulation: prototype similarities are adaptively constrained based on disease comorbidity statistics to capture inter-disease relationships. (c) Classification: query features are concatenated with the purified prototypes and fed into the relation network for final prediction.
5-shot
Dataset
Method
maF1
miF1
maR
miR
maP
miP
mAP
ProtoNet+NLC [ 40 ]
34.75
37.13
36.73
36.73
38.75
38.73
39.54
RelationNet [ 40 ]
41.24
44.11
60.22
60.23
36.03
36.02
46.46
LPN+NLC [ 40 ]
32.17
37.06
35.60
35.60
37.52
37.54
39.45
BCR [ 3 ]
46.12
46.78
70.38
70.38
35.51
35.51
46.62
CheXpert [ 17 ]
PPR (Ours)
49.82
50.41
86.84
86.85
35.72
35.72
46.37
Table 1: Results of overall performance comparison (%) in the 3-way 5-shot setting. The best scores are highlighted in bold and the second-best in blue.
10-shot
Dataset
Method
maF1
miF1
maR
miR
maP
miP
mAP
ProtoNet+NLC [ 40 ]
36.69
38.81
38.48
38.47
40.87
40.82
40.30
RelationNet [ 40 ]
48.34
49.09
79.87
79.87
35.69
35.69
46.40
LPN+NLC [ 40 ]
44.73
47.07
47.38
47.45
47.08
47.13
44.60
BCR [ 3 ]
47.68
47.85
74.80
74.80
35.25
35.25
46.21
CheXpert [ 17 ]
PPR (Ours)
50.14
50.35
87.62
87.63
35.59
35.59
46.53
Table 2: Results of overall performance comparison (%) in the 3-way 10-shot setting. The best scores are highlighted in bold and the second-best in blue.
CheXpert [ 17 ]
MIMIC [ 20 ]
CXR14 [ 45 ]
Method
shot
Pneumonia
Lung Lesion
Fracture
Enl. Card.
Lung Lesion
Fracture
Hernia
Edema
Pneumonia
Fibrosis
ProtoNet+NLC [ 40 ]
37.16
37.03
37.08
36.64
36.57
36.64
42.67
42.71
42.69
42.57
RelationNet [ 40 ]
45.01
45.11
45.10
33.70
33.60
33.76
35.95
35.99
35.87
35.95
LPN+NLC [ 40 ]
36.44
36.48
36.58
35.46
35.54
35.44
44.22
44.31
44.12
44.31
BCR [ 3 ]
47.16
47.21
47.21
44.04
44.09
44.02
36.20
36.57
35.92
36.57
PPR (Ours)
5
50.58
50.63
50.64
47.34
47.36
47.31
45.92
46.06
45.87
45.88
Table 3: Results of individual disease performance comparison using F1 score (%) in the 3-way 5-shot and 3-way 10-shot settings. The best scores are highlighted in bold and the second-best in blue.
Dataset
Method
maF1
miF1
maR
miR
maP
miP
mAP
ProtoNet+NLC [ 40 ]
45.67
47.85
47.89
47.92
47.64
47.64
45.72
RelationNet [ 40 ]
39.07
42.24
45.61
45.67
41.16
41.22
54.32
LPN+NLC [ 40 ]
44.73
47.07
47.38
47.45
47.08
47.13
44.60
BCR [ 3 ]
46.18
47.11
49.42
49.44
46.57
46.56
56.41
CheXpert [ 17 ] →CXR-LT [ 13 ]
PPR (Ours)
51.43
51.70
62.61
62.61
44.28
44.28
57.45
ProtoNet+NLC [ 40 ]
40.19
37.23
34.87
36.92
36.91
36.81
39.78
Table 4: Performance comparison results of cross-domain rare disease classification (%) in the 3-way 5-shot setting. The best scores are highlighted in bold and the second-best in blue.
Dataset
Method
maF1
miF1
maR
miR
maP
miP
mAP
Chexpert [ 17 ]
Baseline (mean)
41.62
43.31
56.45
56.46
35.71
35.71
47.08
Baseline (ws)
46.74
47.66
72.50
72.51
35.94
35.94
46.40
Baseline (ws)+ Lpur
48.27
48.98
78.26
78.28
35.77
35.78
46.93
PPR
49.82
50.41
86.84
86.85
35.72
35.72
46.37
MIMIC [ 20 ]
Baseline (mean)
43.08
44.04
58.83
58.83
35.79
35.79
46.94
Baseline (ws)
38.18
38.94
45.15
45.15
35.38
35.37
46.42
Table 5: Ablation study of loss functions and different prototype generation methods in the 3-way 5-shot setting (%).
Dataset
Method
maF1
miF1
maR
miR
maP
miP
mAP
Chexpert [ 17 ]
max
47.09
48.47
76.13
76.14
35.83
35.83
46.55
min
47.25
48.24
75.26
75.27
35.78
35.78
46.69
PPR
49.82
50.41
86.84
86.85
35.72
35.72
46.37
MIMIC [ 20 ]
max
40.51
42.6
55.53
55.53
35.52
35.51
46.82
min
42.54
43.66
58.54
58.54
35.74
35.73
46.49
PPR
46.27
47.08
70.87
70.88
35.54
35.55
46.67
Table 6: Ablation study of different prototype regulation methods in the 3-way 5-shot setting (%).
Figure 3: Visualization of representations trained on CheXpert in the 3-way 5-shot setting. The radius indicates intra-class compactness, with bars closer to the circumference denoting higher intra-class feature similarity. The angle reflects inter-class distance, where a wider angle corresponds to a greater distance between disease classes. The dotted line denotes the comorbidity statistics j between two diseases used in training. Base and novel classes are displayed in the upper and lower plots, respectively.
Figure 4: Hyperparameter analysis of δ on CheXpert in the 3-way 5-shot setting.
Figure 5: Hyperparameter analysis of λ on CheXpert in the 3-way 5-shot setting.
Federated learning (FL) enables multiple clinical institutions to collaboratively train a shared disease classifier without centralizing patient data. In practice, however, each institution annotates only the pathologies within its area of expertise, so the federation operates under task heterogeneity: each client holds labels for a strict subset of the target disease categories while the remaining classes are entirely unobserved at that site. Existing gradient-based FL methods fail under this setting because they require hundreds of communication rounds to converge and because missing class labels introduce systematic false-negative bias that the model cannot correct without a principled mechanism. We propose an analytic federated learning framework for multi-label medical image classification under task heterogeneity. The proposed method replaces iterative gradient optimization with three closed-form operations: a balanced label projection that neutralizes class-imbalance bias by normalizing positive and negative contributions to equal total mass; a per-class absolute aggregation law that independently assembles the optimal ridge-regression classifier for each disease category from the sufficient statistics uploaded by its annotating clients; and an optional analytic pseudo-label refinement round that propagates missing-class knowledge from a confidence-filtered teacher classifier to non-annotating clients. The entire procedure requires at most two communication rounds, irrespective of the degree of task heterogeneity or the number of participating clients. Experiments on ChestXray14 under four progressively severe missing-class configurations demonstrate that the proposed method consistently outperforms the state-of-the-art federated multi-label method FedMLP by up to 18.44 BACC points and 13.24 AUC points, while reducing the communication.
Afsaneh Mahanipour, Hana Khamfroush
Department of Computer Science University of Kentucky Lexington, KY, USA
Current methods for multimodal medical image classification (M3IC) face two major challenges. First, the prevailing "fusion after unimodal image embedding" paradigm cannot fully exploit the complementary and correlated information in the multimodal data. Second, the scarcity of labeled multimodal medical images, coupled with their substantial domain shift from natural images, impedes the use of cutting-edge Vision Foundation Models (VFMs) for medical image embedding. To jointly address the challenges, we propose a novel Early Intervention (EI) framework. Treating one modality as target and the rest as reference, EI harnesses high-level semantic tokens from the reference as intervention tokens to steer the target modality's embedding process at an early stage. For parameter-efficient VFM adaptation, we introduce Mixture of varied-rank LoRAs (MoR), which employs a small set of LoRA experts with distinct ranks and a bypass-allowed router. Extensive experiments on three public datasets, covering retinal disease recognition, skin lesion recognition, and knee anomaly classification, verify the effectiveness of the proposed method against a number of competitive baselines.
Medical diagnosis requires the effective synthesis of visual manifestations and clinical metadata. However, existing methods often treat metadata as isolated tags, failing to exploit the rich semantic knowledge embedded in clinical descriptions. We propose PRIMA (Pre-training with Risk-integrated Image-Metadata Alignment), a framework that integrates domain-specific knowledge into multi-modal representation learning. We first curate a corpus of risk--disease correlations via Retrieval-Augmented Generation (RAG) to refine Clinical ModernBERT, embedding diagnostic priors into the text encoder. To bridge the modality gap, we introduce a dual-encoder pre-training strategy utilizing DINOv3 and our refined Clinical ModernBERT, optimized by a suite of four complementary loss functions. These losses are designed to capture multi-granular semantic alignment and handle the ambiguity of clinical correlations through soft labels. Finally, we leverage Qwen3 to fuse these aligned features for precise disease classification. Extensive experiments demonstrate that PRIMA effectively harmonizes pixel-level features with abstract clinical expertise. Across PAD-UFES-20 and AQUA, PRIMA achieves average F1-scores of 72.04% and 85.22%, outperforming strong image-only, metadata-fusion, and medical vision-language baselines. Notably, our framework achieves strong performance without requiring massive data collection or exhaustive computational resources. Our code is available at https://github.com/yqwang01/PRIMA.
Yiqing Wang, Chunming He, Ziyun Yang +5
Department of Biomedical Engineering, Duke University 101 Science Drive, Durham, NC 27705 USA · Department of Ophthalmology and Visual Sciences, University of Michigan 1000 Wall Street, Ann Arbor, MI, USA