Organizations: Faculty of Computer Science and Engineering, Ho Chi Minh City University of Technology, 268 Ly Thuong Kiet Street, Dien Hong Ward, Ho Chi Minh City, Vietnam · Vietnam National University Ho Chi Minh City, Dong Hoa Ward, Ho Chi Minh City, Vietnam · Applied AI & Data Science Division, AK Technologies Company Limited, Ho Chi Minh City, Vietnam
Multi-label chest X-ray classification has attracted considerable attention in recent years, with the effective use of visual representations and clinical semantic knowledge playing an important role. This study proposes a framework that combines unimodal representations from RAD-DINO with vision--language representations from BioViL-T for the classification of 14 labels in the MIMIC-CXR-JPG dataset. The RAD-DINO and BioViL-T embeddings and their combined representation are refined separately in latent space before being normalized and fused across the three branches. In addition to improving classification performance, the study aims to clarify the role of each embedding source and the degree to which they complement one another. Experiments show that RAD-DINO outperforms BioViL-T when used independently, whereas early fusion further improves the results, indicating that the two embedding sources contain complementary information. The best-performing model achieves a mean AUROC of 0.840 and an mAP of 0.467. Ablation analysis shows that hybrid fusion provides consistent and statistically significant improvements over early fusion when each embedding source is refined in latent space, suggesting that fusion effectiveness depends on the quality of the representation supplied by each branch. However, the study has only been evaluated internally on MIMIC-CXR-JPG; its generalizability to data from other healthcare institutions therefore remains to be validated. The source code is available at: https://anonymous.4open.science/r/mimic-report-C210/.
Figures & tables
Figure 1: Overview of the latent hybrid fusion architecture.
Configuration
AUROC (95% CI)
Δ AUROC
mAP (95% CI)
DenseNet121 [ 20 ]
0.773
–
–
RAD-DINO only
0.827 [0.825, 0.828]
0.053
0.443 [0.439, 0.448]
BioViL-T only
0.805 [0.804, 0.807]
0.032
0.408 [0.404, 0.411]
Early fusion
0.837 [0.835, 0.838]
0.064
0.464 [0.459, 0.469]
Hybrid fusion
0.840 [0.839, 0.842]
0.067
0.467 [0.462, 0.471]
Table 1: Comparison with the DenseNet121 results reported by Singh [ 20 ] .
Model
Reference model
Δ AUROC
p
Sig.
Δ mAP
p
Sig.
Early fusion
BioViL-T only
0.031
3.61×10−9
✓
0.056
7.51×10−8
✓
Early fusion
RAD-DINO only
0.010
1.01×10−6
✓
0.020
2.54×10−6
✓
Hybrid fusion
Early fusion
0.003
6.30×10−5
✓
0.003
3.28×10−4
✓
Table 2: Statistical tests across experiments.
Model
AUROC ↑
mAP ↑
Macro F1 ↑
Comparison on 13 labels (excluding No Finding)
MCX-Net 13 [ 21 ]
0.816
–
0.451
Ours 13
0.837
–
0.464
Comparison on 14 labels
DenseNet121 [ 20 ]
0.773
–
–
MBRANet [ 12 ]
0.805
–
–
Table 3: Comparison of the proposed method with other models on MIMIC-CXR.
Figure 2: ROC curves of latent hybrid fusion for the 14 labels on the test set. Mean AUROC ranges from 0.742 for Enlarged Cardiomediastinum to 0.928 for Support Devices .
ID
Configuration
AUROC (95% CI)
Comparison
Δ AUROC
p
Sig.
Latent-refinement block
A1
Embedding only
0.8356 [0.8340, 0.8372]
A2
A1 + Autoencoder
0.8366 [0.8346, 0.8386]
A2 − A1
+0.0010
2.74×10−2
×
A3
A2 + noise
0.8369 [0.8354, 0.8385]
A3 − A1
+0.0013
1.64×10−4
✓
Prediction head (Latent-refinement gain under different heads)
A4
MLP head
0.8368 [0.8352, 0.8384]
A4 − A1
+0.0012
1.68×10−2
×
Table 4: Ablation over the latent-refinement block, the prediction head, and the fusion stage. Each value is the mean macro AUROC and 95% confidence interval across five seeds; p is the two-sided paired t -test p-value across seeds.
Tier
Labels
Δ AUROC
Excl. 0
Δ AP
Excl. 0
Common ( >15% )
8
+0.0022
8/8
+0.0038
8/8
Mid ( 5 – 15% )
3
+0.0036
3/3
+0.0024
2/3
Rare ( <5% )
3
+0.0067
3/3
+0.0012
0/3
Table 5: Improvement of hybrid fusion over early fusion by label-prevalence tier (exploratory).
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Label
Positive studies
Prevalence
Atelectasis
52,818
25.017%
Cardiomegaly
47,579
22.535%
Consolidation
14,190
6.721%
Edema
37,934
17.967%
Enlarged Cardiomediastinum
15,497
7.340%
Fracture
4,590
2.174%
Appendix
Table 1: Label distribution in the aligned cohort.
Item
Value
Aligned studies
211,130
Disease labels
14
RAD-DINO embedding dimension
768
BioViL-T embedding dimension
128
Random seeds
42,43,44,45,46
Number of independent runs
5
Appendix
Table 2: Cohort and reproducibility settings.
Split
Ratio
Studies
Train
70%
147,791
Validation
10%
21,113
Test
20%
42,226
Total
100%
211,130
Appendix
Table 3: Train, validation, and test set sizes shared across all five random seeds.
Parameter
Setting
Model dimensions
Predictor hidden dimension
512
Latent dimension
256
Feed-forward dimension
1024
Training schedule
Batch size
1024
Appendix
Table 4: Training and model-selection hyperparameters.
Configuration
Params
FLOPs
Latency
(M)
(M/study)
(ms/study)
RAD-DINO only
7.1
13.1
0.057
BioViL-T only
6.1
11.8
0.057
Early fusion
7.3
13.4
0.057
Hybrid fusion
20.5
38.3
0.170
Appendix
Table 5: Inference and training cost of the fusion stage, measured on a single NVIDIA A100 GPU.
Figure 1: Per-label performance of latent hybrid fusion. Each cell shows the mean ± standard deviation across five seeds.
Label
MBRANet [ 12 ]
Ours
Δ
Atelectasis
0.779
0.828
+0.049
Cardiomegaly
0.825
0.820
−0.005
Consolidation
0.788
0.797
+0.009
Edema
0.833
0.893
+0.060
Cardio.
0.823
0.742
−0.081
Fracture
0.703
0.773
+0.070
Appendix
Table 6: Per-label AUROC comparison between MBRANet and the proposed method on MIMIC-CXR.
Comparison
Metric
Δ
CI
Rel. gain (%)
Headroom (%)
Early − BioViL-T
AUROC
0.0314
[0.0301, 0.0327]
3.90
16.15
mAP
0.0560
[0.0544, 0.0579]
13.75
–
Macro F1 e
0.0451
[0.0437, 0.0464]
10.33
–
Early − RAD-DINO
AUROC
0.0104
[0.0097, 0.0111]
1.25
5.98
mAP
0.0203
[0.0193, 0.0215]
4.59
–
Macro F1 e
0.0122
[0.0111, 0.0133]
2.59
–
Appendix
Table 7: Patient-level clustered bootstrap of paired metric differences ( B=2,000 subject-level resamples; within each replicate the paired difference is computed per seed on an independently drawn subject resample and averaged across the five seeds). Confirmatory comparisons report simultaneous 99.17% confidence intervals; Macro F1 is exploratory and reports 95% intervals.
Comparison
Uncorrected
Bonferroni
Median p
n
%
n
%
Early − BioViL-T
70
100.0
69
98.6
1.7\times10−24
Early − RAD-DINO
61
87.1
52
74.3
2.7\times10−11
Hybrid − Early
59
84.3
39
55.7
1.2\times10−3
Appendix
Table 8: Per-label DeLong tests for the three confirmatory comparisons, computed on the one-study-per-patient subset of the test set.
Label
RAD-DINO latent
BioViL-T latent
Early fusion
Atelectasis
0.182±0.038
0.168±0.038
0.650±0.049
Cardiomegaly
0.196±0.059
0.252±0.038
0.552±0.063
Consolidation
0.168±0.080
0.280±0.049
0.552±0.063
Edema
0.224±0.059
0.196±0.031
0.580±0.049
Cardio.
0.252±0.080
0.196±0.059
0.552±0.038
Fracture
0.210±0.111
0.112±0.063
0.678±0.063
Appendix
Table 9: Label-wise late-fusion weights for each label. Each value is reported as the mean ± standard deviation across five seeds.
Label
Pearson (score)
Spearman (score)
Pearson (error)
Atelectasis
0.867
0.874
0.972
Cardiomegaly
0.873
0.875
0.971
Consolidation
0.835
0.842
0.948
Edema
0.893
0.891
0.948
Enlarged Cardio.
0.846
0.837
0.965
Fracture
0.466
0.456
0.817
Appendix
Table 10: Between-branch correlation of predicted scores and errors for the RAD-DINO and BioViL-T branches.
Label
Both ✓
Only R
Only B
Both ×
Disagr.
Acc. R
Acc. B
Oracle
Atelectasis
0.658
0.073
0.065
0.204
0.138
0.731
0.723
0.796
Cardiomegaly
0.655
0.076
0.077
0.192
0.153
0.731
0.732
0.808
Consolidation
0.785
0.069
0.052
0.095
0.120
0.853
0.836
0.905
Edema
0.788
0.051
0.043
0.118
0.094
0.839
0.831
0.882
Enlarged Cardio.
0.664
0.101
0.087
0.148
0.188
0.765
0.751
0.852
Fracture
0.824
0.109
0.034
0.033
0.143
0.933
0.858
0.967
Appendix
Table 11: Decision-level agreement between the RAD-DINO and BioViL-T branches.
Configuration
Metric
Mean [95% CI]
Δ vs. full [95% CI]
Full (three branches)
mAUROC
0.8404 [0.8388, 0.8419]
–
mAP
0.4666 [0.4619, 0.4713]
–
w/o RAD-DINO latent
mAUROC
0.8367 [0.8352, 0.8381]
−0.0037 [ −0.0039 , −0.0035 ]
mAP
0.4604 [0.4558, 0.4650]
−0.0062 [ −0.0065 , −0.0058 ]
w/o BioViL-T latent
mAUROC
0.8382 [0.8365, 0.8399]
−0.0022 [ −0.0025 , −0.0018 ]
mAP
0.4643 [0.4595, 0.4692]
−0.0023 [ −0.0029 , −0.0016 ]
Appendix
Table 12: Leave-one-branch-out analysis of the main configuration.
Configuration
ECE
Brier score
RAD-DINO only
0.0125±0.0006
0.0858±0.0002
BioViL-T only
0.0125±0.0005
0.0887±0.0002
Early fusion
0.0139±0.0000
0.0836±0.0002
Hybrid fusion
0.0140±0.0002
0.0832±0.0002
Appendix
Table 13: Calibration after per-label Platt scaling fitted on the validation set (mean ± standard deviation across five seeds).
Aggregation scheme
Weights (F/L)
mAUROC [95% CI]
mAP [95% CI]
Frontal only
–
0.8394 [0.8381, 0.8406]
0.4661 [0.4615, 0.4707]
Frontal + lateral
0.5/0.5
0.8398 [0.8381, 0.8414]
0.4652 [0.4605, 0.4700]
0.6/0.4
0.8404 [0.8388, 0.8419]
0.4666 [0.4619, 0.4713]
0.7/0.3
0.8401 [0.8383, 0.8410]
0.4663 [0.4609, 0.4707]
Appendix
Table 14: Sensitivity of the main configuration to the frontal–lateral aggregation scheme.
σmax
mAUROC
mAP
0.00
0.8368±0.0011
0.4636±0.0039
0.20
0.8368±0.0011
0.4638±0.0039
0.65
0.8369±0.0013
0.4638±0.0041
1.00
0.8369±0.0013
0.4638±0.0040
1.25
0.8369±0.0013
0.4638±0.0039
1.50
0.8370±0.0012
0.4638±0.0039
Appendix
Table 15: Sensitivity of the early-fusion branch to the noise ceiling σmax .
Quantity
Mean ± SD
Latent transformation, z=g(z,0)
cos(z,z)
0.8510 ± 0.0093
∥z−z∥2
5.4684 ± 0.1642
Relative L2 deviation
0.5465 ± 0.0151
∥z∥2
10.0288 ± 0.1116
Autoencoder reconstruction
Appendix
Table 16: Geometry of the latent transformation at s=0 and reconstruction quality of the autoencoder, on the early-fusion branch. Values are mean ± standard deviation across five seeds.
Configuration
Mean
Δ
p
Macro AUROC
Applied, z=g(z,0)
0.8369
–
–
Bypassed, z=z
0.8356
−0.0013
6.10×10−4
Objective removed
0.8369
−0.0000
8.34×10−1
mAP
Applied, z=g(z,0)
0.4638
–
–
Appendix
Table 17: Inference-time role of the latent denoiser on the early-fusion branch
Representation
Fusion rule
macro AUROC
mAP
Refined
Uniform ( 1/3 )
0.8404±0.0013
0.4666±0.0038
Refined
Learned label-wise
0.8398±0.0011
0.4674±0.0039
Raw
Uniform ( 1/3 )
0.8362±0.0012
0.4570±0.0036
Raw
Learned label-wise
0.8364±0.0010
0.4621±0.0033
Effect
Δ AUROC ( p )
Δ mAP ( p )
Representation ∣ uniform
+0.0042 ( 1.73×10−5 ) ✓
+0.0096 ( 1.46×10−6 ) ✓
Appendix
Table 18: Two-factor decomposition of the A8 − A9 contrast: representation quality crossed with the late-fusion rule. Cell values are mean ± standard deviation across five seeds.
Chest X-ray classification uses machine learning to detect and categorize diseases and findings in radiographs for screening, triage, and clinical decision support.
A central challenge in multimodal learning is how to fuse modalities that differ substantially in predictive strength. When one modality provides a much stronger representation than another, indiscriminate cross-modal interaction can introduce weak, redundant, or unstable signals, limiting or even degrading performance relative to the strongest unimodal representation. To address this problem, we propose CheXFuse, a multimodal fusion framework based on Guided Differential Joint Attention (GuDJA). CheXFuse combines image and text token representations within a joint multimodal sequence, where learned token guidance emphasizes informative representations and differential attention reduces less useful attention responses. We evaluate CheXFuse on the Indiana University (IU) Chest X-ray dataset, using binary (normal vs. abnormal) classification as the downstream task. Experiments are conducted under both original-report and masked-report settings, with the latter serving as a controlled stress test under reduced textual informativeness. CheXFuse with GuDJA achieves the strongest mean overall performance across the evaluated baselines and ablations, reaching 95.73% accuracy, 95.79% F1-score, 96.54% recall, and 95.28% sensitivity at 95% specificity (Sens@95Spec) with original reports, and 95.58%, 95.60%, 95.60%, and 94.34%, respectively, under masked reports. These results suggest that GuDJA provides a promising fusion strategy under asymmetric or partially reduced modality informativeness.
Muntaqim Ahmed Raju, Priyanka Siddappa, Md Shifat Haider Al Amin +2
University of Massachusetts Lowell, USA · NYC Health & Hospitals/Jacobi-NCB, USA
Long-tailed chest X-ray classification requires visual representations that capture both common abnormalities and subtle, infrequent findings. We propose Med-AR-8B and Med-AR-2B, two radiology-native autoregressive vision-language models pretrained with structured reports, abnormality-focused text, and region annotations. We evaluate the transfer of their visual encoders to multi-label classification against contrastive, self-supervised, and supervised pretrained encoders, including Med-CLIP, CheXFound, EVA-Base, ARK, and BioViL-T, using a common ML-Decoder classification head. To assess fine-grained recognition, we also construct LLM-expanded, report-derived label sets for MIMIC-CXR and CheXpert. Across PadChest, MIMIC-CXR, and CheXpert, Med-AR-8B outperforms Med-CLIP in mean AUROC and AUPRC for head, medium, and tail findings. On MIMIC-CXR, it increases tail-label mean AUPRC from 0.1033 to 0.1441. Med-AR-2B achieves the strongest discrimination results on PadChest. Across the broader encoder comparison, a Med-AR variant achieves the highest mean AUROC and AUPRC in every reported prevalence group on each public dataset. Both Med-AR variants also achieve lower excess area under the risk-coverage curve than Med-CLIP on all three public datasets, indicating improved selective-prediction performance under the evaluated protocol. Internal results are metric-dependent, with Med-CLIP retaining advantages in overall and tail AUPRC and in selective prediction. These findings establish Med-AR as a strong pretraining recipe for long-tailed chest X-ray classification on the evaluated public benchmarks and demonstrate the value of assessing discrimination and selective prediction together.
Chest radiography is a widely used imaging modality for thoracic disease diagnosis, yet its conventional interpretation remains time-consuming and heavily dependent on expert knowledge. While deep learning has improved diagnostic efficiency through automated feature extraction, challenges such as class imbalance and the localization of multiple co-existing pathologies remain unsolved. In this paper, inspired by the strength of Convolutional Block Attention Module (CBAM) in feature refinement and the capability of CNN blocks in feature extraction, we propose a strategy to integrate CBAM into traditional CNN blocks to enhance performance in multi-label classification tasks. Our method achieves a mean AUC of 0.8695 on ChestXray14 dataset, outperforming several state-of-the-art baselines.Our source code is available at: https://github.com/NNNguyenDuyyy/FETC_CBAM_Enhanced_CNN.git
Department of Artificial Intelligence FPT University Ho Chi Minh City, Vietnam · Department of Computing Fundamental FPT University Ho Chi Minh City, Vietnam