eess.IVNov 9, 2026

Is a CNN-Transformer Ensemble Worth It When a Chest X-Ray Foundation Model Is Available?

Authors: Aditya Seelaboyina, Irfan Khan

Organizations: Department of Computer Science and Engineering Texas A&M University College Station, TX, USA · Department of Marine Engineering Technology, in a courtesy joint appointment with the Department of Electrical & Computer Engineering Texas A&M University, TX, USA

Abstract

Ensembles of ImageNet-pretrained convolutional networks and Vision Transformers are a common way to improve multi-label chest X-ray classification. We ask whether such an ensemble is still worth its cost when a model pretrained on chest radiographs is available. On the official patient-disjoint split of NIH ChestX-ray14 (25,596 test images, 2,797 patients), we train five ImageNet-pretrained backbones and the EVA-X-S foundation model with validation-only model selection, compare five fusion rules, and estimate every difference with a patient-level paired bootstrap. EVA-X-S alone reached a test macro-AUROC of 0.828; the five-model ensemble reached 0.814 (difference -0.0136, 95% CI -0.0178 to -0.0098) at about 7.8 times the summed backbone latency and ten times the parameters. Averaging the five ImageNet models with EVA-X-S changed AUROC by -0.0056 (-0.0095 to -0.0021). With all five ImageNet backbones trained with the EVA-X-S recipe, their ensemble scored 0.817, still -0.0109 (-0.0142 to -0.0076) below EVA-X-S, and across three seeds EVA-X-S varied by at most 0.001 AUROC. Under the ImageNet models' recipe, however, EVA-X-S fell to 0.798. Our full-stacking configuration never outperformed calibrated averaging. In this setting, the tested ImageNet ensembles did not improve on a single chest X-ray foundation model fine-tuned with its own recipe.

Figures & tables

Explore similar work

CardsList
  1. Frozen Foundation-Model Embeddings Discard Small-Lesion Signal in Chest Radiography: Implications for Pre-Deployment Evaluation

    Jun 10, 2026Raajitha Muthyala, Zhenan Yin, Alekhya Jilla +6Chest X-Ray ClassificationMedical Imaging Foundation Models

  2. A data- and compute-efficient chest X-ray foundation model beyond aggressive scaling

    Feb 26, 2026Chong Wang, Yabin Zhang, Yunhe Gao +9Medical Imaging Foundation ModelsMedical VLMs

  3. Vision-language models for chest radiography do not always need the image

    Jun 16, 2026Mahshad Lotfinia, Sebastian Ziegelmayer, Lisa Adams +4VLM EvaluationMedical Visual Question Answering