cs.LGNov 5, 2026

Leakage- and Label-Aware Benchmarking of Class-Imbalance Strategies for Multimodal Chest X-Ray and EHR Classification

Authors: Dang-Khoa Bui, Quang-Huy Tran, Minh-Khoi Nguyen-Bui, Duc-Tuan Ngo, Thanh-Trong Tran, Tuan-Khoi Nguyen, Hoang-Anh Ngo

Organizations: Faculty of Computer Science and Engineering, Ho Chi Minh City University of Technology (HCMUT), VNU-HCM, Vietnam · Applied AI & Data Science Division, AK Technologies Company Limited, Ho Chi Minh City, Vietnam · Vietnam National University Ho Chi Minh City, Vietnam

Abstract

Benchmarks of class-imbalance strategies for multimodal chest X-ray (CXR) and electronic health record (EHR) prediction are often evaluated on record-level splits of cohorts in which one patient contributes several radiographs. We audit such a benchmark on 29,083 MIMIC-CXR radiographs of 10,479 patients with eight diagnosis targets. First, the record-split benchmark was misleading: a lookup table that copies the label of a development-set radiograph with the same EHR vector nearly matched the best model, and on unseen patients every model fell to the level of predicting every case positive. Second, tracing the labels back to MIMIC-IV explained why: each patient's labels are keyword matches on the diagnosis codes of a single admission, copied, with one EHR vector per patient, to all of the patient's radiographs, so about a third of the radiographs carry labels that differ from the codes of their own admission. Third, under patient-separated evaluation with admission-aligned and rule-refined labels, the initial conclusions do not hold: in the evaluated configurations no imbalance strategy raised the observed mean macro F2 over no augmentation by more than 0.01, tuned tree ensembles showed no consistent advantage over tuned logistic regression, and the released EHR variables stayed at the level of the all-positive baseline, whereas the diagnostic history available at imaging time and image embeddings exceeded it, including, with rule-refined labels, a CheXpert-trained encoder that never saw these radiographs; RAD-DINO had seen 96.6% of them during self-supervised pretraining. Adding either encoder to the diagnostic history gave no statistically significant F2 gain; exploratory AUROC gains were significant with RAD-DINO but not with DenseNet. We provide reusable diagnostics, a leakage-aware protocol and a checklist.

Figures & tables

Explore similar work

CardsList
  1. When Does Multimodal Learning Help in Healthcare? A Benchmark on EHR and Chest X-Ray Fusion

    Feb 27, 2026Kejing Yin, Haizhou Xu, Wenfang Yao +5Multimodal FusionMultimodal Model Evaluation

  2. Loss Design and Architecture Selection for Long-Tailed Multi-Label Chest X-Ray Classification

    Mar 2, 2026Nikhileswara Rao SulakeLong-Tailed ClassificationChest X-Ray Classification

  3. Improving Imbalanced Multi-Label Chest X-Ray Diagnosis via CBAM-Enhanced CNN Backbones

    May 4, 2026Duy Nguyen Huu, Duy Hoang Khuong, Ngu Huynh Cong VietClass-Imbalanced LearningChest X-Ray Classification