A data- and compute-efficient chest X-ray foundation model beyond aggressive scaling
Authors: Chong Wang, Yabin Zhang, Yunhe Gao, Maya Varma, Clemence Mottez, Faidra Patsatzi, Jiaming Liu, Jin Long, Jean-Benoit Delbrouck, Sergios Gatidis, Akshay S. Chaudhari, Curtis P. Langlotz
Organizations: Stanford Center for Artificial Intelligence in Medicine and Imaging, Stanford University, Palo Alto, CA, USA. · Department of Radiology, Stanford University, Stanford, CA, USA. · Department of Computer Science, Stanford University, Stanford, CA, USA. · Department of Pediatrics, Stanford University, Stanford, CA, USA. · Department of Biomedical Data Science, Stanford University, Stanford, CA, USA. · Department of Medicine, Stanford University, Stanford, CA, USA.
Foundation models for medical imaging are typically pretrained on increasingly large datasets, following a "scale-at-all-costs" paradigm. However, this strategy faces two critical challenges: large-scale medical datasets often contain substantial redundancy and severe class imbalance that bias representation learning toward over-represented patterns, and indiscriminate training regardless of heterogeneity in data quality incurs considerable computational inefficiency. Here we demonstrate that active, principled data curation during pretraining can serve as a viable, cost-effective alternative to brute-force dataset enlargement. We introduce CheXficient, a chest X-ray (CXR) foundation model that selectively prioritizes informative training samples. CheXficient is pretrained on only 22.7% of 1,235,004 paired CXR images and reports while consuming under 27.3% of the total compute budget, yet achieving comparable or superior performance to its full-data counterpart and other large-scale pretrained models. We assess CheXficient across 20 individual benchmarks spanning 5 task types, including non-adapted off-the-shelf evaluations (zero-shot findings classification and crossmodal retrieval) and adapted downstream tasks (disease prediction, semantic segmentation, and radiology report generation). Further analyses show that CheXficient systematically prioritizes under-represented training samples, improving generalizability on long-tailed or rare conditions. Overall, our work offers practical insights into the data and computation demands for efficient pretraining and downstream adaptation of medical vision-language foundation models.
Figures & tables
Figure 1: Overview of this study. (a) CheXficient pretraining strategy. Paired chest X-ray (CXR) images and radiology reports are employed. A data curator selectively determines which image–report pairs are included in training, followed by optimization with an InfoNCE Oord et al. (2018) contrastive loss. (b) Data curation mechanism. The data curator prioritizes image–report pairs based on their proximity to a set of learnable prototypes. Samples farther from the prototypes are assigned higher priority, while more redundant data closer to the prototypes is down-sampled. (c) Evaluation protocol. After pretraining, CheXficient is evaluated on non-adapted tasks without any architectural or weight modifications, including (i) zero-shot finding classification and (ii) zero-shot cross-modal retrieval. CheXficient can also be adapted to downstream tasks via fine-tuning, including (iii) multi-class disease prediction, (iv) semantic segmentation, and (v) radiology report generation. (d) Data efficiency analysis. Zero-shot findings classification performance (average AUROC on 8 datasets) versus pretraining data size compared with alternative pretraining strategies and models. (e) Overall performance. CheXficient achieves better or comparable performances with full-data pretraining, while greatly outperforming other existing large-scale pretrained models on diverse evaluation benchmarks spanning 5 task types: retrieval (in Recall@1), segmentation (in Dice score), and report generation (in RadGraph), adapted and non-adapted classification (in AUROC).
Figure 2: (a) Feature distribution of training samples from the full set (CheXfull, n = 1,235K), random subset (CheXrandom, n = 280K), and curated subset (CheXficient, n = 280K). Qualitative histograms of feature embeddings projected onto a two-dimensional PCA space (Colorbars are independently normalized to reveal the structural organization of the PCA space, absolute density values are therefore not directly comparable across methods). The curated subset occupies distinct and long-tailed regions in the feature space, compared with both the full and random subsets. (b) Quantitative analysis of local feature density (left) for samples from the full set, random subset, and curated subset, measured by the average k -nearest neighbor (kNN) distance ( k=20 ) computed in the raw feature space. p -values are reported from Welch’s t-test by comparing each subset against the full set. The low-density proportion (right) denotes the fraction of samples falling within the top 25% lowest-density regions of the full dataset. (c) CDF plot of the average kNN distance. Curated samples (red) are shifted toward higher kNN distances, indicating enrichment in low-density regions, whereas the random subset (purple) closely overlaps with the full set (yellow).
Figure 3: Performance on zero-shot findings classification. We evaluate the pretrained models on 8 public datasets. Among them, SIIM-PTX, Pneumonia2017, and TBX11K are from external domains unseen in pretraining, while the remaining 5 datasets are used for internal evaluation. Compared to CheXfull, CheXficient achieves higher AUROC on three datasets ( p<0.05 ), comparable performance on four datasets ( p>0.05 ), and lower performance on one dataset ( p<0.05 ). Across all datasets, CheXficient achieves higher AUROC than CheXrandom. We present the mean ± 95% confidence interval (CI) of AUROC for each model. The listed p -values are computed using two-sided t -tests.
Figure 4: Performance on zero-shot cross-modal retrieval. We evaluate the pretrained models on the public CheXpert and MIMIC-CXR benchmarks. CheXficient achieves performance comparable to CheXfull on both datasets ( p>0.05 ), while outperforming CheXrandom. We report the mean ± 95% CI of Recall@1 (The recall of retrieving the exact paired CXR report (or image) within the top-1 result for a given CXR image (or report)).
Figure 5: Zero-shot findings classification performance under different pretraining data budgets. The performance of both CheXficient and CheXrandom progressively increases on all 8 benchmarks as the quantity of pretraining samples increases. CheXficient consistently outperforms CheXrandom at all data budgets and across all benchmarks, demonstrating higher data efficiency. When pretrained on only 280K curated image–report pairs (22.7% of the full data, except for ChestX-ray14), CheXficient performs better or similarly to the full-data pretrained model CheXfull.
Figure 6: Zero-shot findings classification performance under different pretraining computation budgets. The three models are pretrained for the same number of epochs. CheXficient (280K) consistently outperforms CheXrandom (280K) and CheXfull (1235K) on all evaluation benchmarks when using the same pretraining time, highlighting the compute efficiency of curated pretraining of CheXficient.
Figure 7: Comparison of CheXficient with other pretrained models in label-efficient disease prediction across three representative datasets: CheXpert ( n = 500, 14 categories), MIMIC-CXR ( n = 3,082, 14 categories), and ChestXray14 ( n = 25,596, 15 categories). Performance is reported at varying proportions of labeled training data. Vertical dashed lines indicate the amount of labeled data required for CheXficient to match the performance of its counterpart CheXrandom.
Figure 8: Comparison of CheXficient with other pretrained models in annotation-efficient anatomical and pathological segmentation across three datasets: SIIM-PTX ( n = 1,372), JSRT ( n = 50), and VinDr-RibCXR ( n = 49). Performance is reported at varying proportions of annotated training samples. Vertical dashed lines indicate the amount of annotated data required for CheXficient to match the performance of its counterpart CheXrandom.
Figure 9: Downstream radiology report ( Findings section) generation performance (%) on official test split of MIMIC-CXR ( n = 2,461). Text in parentheses denotes the pretraining data size of the model.
Figure 10: Downstream radiology report ( Findings section) generation performance (%) on official test split of ReXGradient-160K ( n = 10,000). Text in parentheses denotes the pretraining data size of the model.
Figure 11: Impact of curated pretraining on per-disease generalization in VinDr-CXR. Top: disease label distributions from the VinDr-CXR training set (the y-axis uses a symmetric logarithmic scale to visualize disease label frequencies spanning multiple orders of magnitude). Bottom: corresponding zero-shot per-disease AUROC results on the VinDr-CXR test set. Curated pretraining (CheXficient) re-balances label distributions and yields improved zero-shot generalization for most disease categories compared to non-curated pretraining (CheXfull and CheXrandom). Note that the disease label ‘edema’ is not shown in the figure as there are no samples with this label in the official test data.
Figure 12: Adaptability of CheXficient in detecting long-tailed conditions (Calcification, Enlarged PA (pulmonary artery), and Rib fracture) under a two-way (no finding versus each long-tailed condition) k -shot learning setting. The results demonstrate the ability of CheXficient to adapt to data-scarce conditions using only a few labeled training samples. Box plots show AUROC distributions over 20 experimental replicates, with the median (central line), interquartile range (box indicating the 25th and 75th percentiles), whiskers extending to 1.5 times the interquartile range, and outliers shown as individual points. p -values from two-sided t -tests are reported.
Source dataset
Country
Patients
Labels
Text tokens
Image-text pairs
5%-ile
Average
95%-ile
Real-world Findings (87.4%)
MIMIC-CXR
U.S.
65,379
–
42
94
176
354,619
ReXGradient-160K
U.S.
95,716
–
35
76
152
238,965
CheXpert-Plus
U.S.
64,725
–
38
98
209
222,116
PadChest
Spain
67,625
–
6
30
76
160,672
Table 1: An extensive corpus of 1.235 million CXR image-report pairs employed for model pretraining in our experiments. We exclude images that do not belong to standard views (i.e., frontal or lateral). Real-world Findings are extracted from the free-text radiology reports. Synthetic Findings are generated using template sentences derived from clinical diagnostic labels.
Dataset
Learning setup
Task
Clinical conditions
# Train
# Test
Internal evaluation
Non-adapted
Classification
14 thoracic findings
0
3,082
Non-adapted
Retrieval
–
0
3,082
Adapted
Classification
14 thoracic findings
227,459 (1–100%)
3,082
MIMIC-CXR
Adapted
Report generation
–
146,909
2,461
CheXpert
Non-adapted
Classification
14 thoracic findings
0
500
Table 2: Summary of the evaluation datasets used in this study. We evaluate on a total of 11 datasets (6 seen datasets for internal evaluation and 5 unseen datasets for external evaluation), comprising diverse evaluation benchmarks across 5 task types. We follow the official train/test splits provided by each dataset, except for CheXpert, where 1,000 paired CXR samples were excluded from the training set for retrieval evaluation.
Model
Image encoder
Parameter size
Image resolution
Pretraining data size
Pretraining type
Data accessibility
CheXficient
ViT-Base
86 M
378×378
280 K
image-text
public
CheXZero Tiu et al. (2022)
ViT-Base
86 M
224×224
377 K
image-text
public
BioViL-T Bannur et al. (2023)
ResNet-50
25.6 M
512×512
377 K
image-text
public
LLaVA-Rad Zambrano Chaves et al. (2025)
ViT-Base
86 M
518×518
697 K
image-text
public
RAD-DINO Perez-Garcia et al. (2025)
ViT-Base
86 M
518×518
883 K
self-supervised
public + private
CheXagent Chen et al. (2024)
ViT-Large
307 M
512×512
1,077 K
image-text
public
Extended Data Table 1: Comparative overview of CheXficient with other large-scale pretrained medical foundation models.
Extended Data Fig. 1: Performance (AUPRC) on zero-shot findings classification. We evaluate the pretrained models on 8 public datasets. Among them, SIIM-PTX, Pneumonia2017, and TBX11K are from external domains unseen in pretraining, while the remaining 5 datasets are used for internal evaluation. Compared to CheXfull, CheXficient achieves higher AUROC on 3 datasets ( p<0.05 ), and comparable performance on 5 datasets ( p> 0.05). For each dataset, CheXficient outperforms CheXrandom by large margins. We present the mean ± 95% confidence interval (CI) of AUPRC for each model. The listed p -values are calculated using two-sided t -tests.
Extended Data Fig. 2: Performance on zero-shot cross-modal retrieval (Image → Findings and Findings → Image). We evaluate the pretrained models on the public CheXpert and MIMIC-CXR benchmarks. CheXficient achieves performance comparable to CheXfull on both datasets ( p>0.05 ), while outperforming CheXrandom. We report the mean ± 95% CI of Recall@1 (The recall of retrieving the exact paired CXR Findings section (or image) within the top-1 result for a given CXR image (or Findings section)).
Extended Data Fig. 3: Performance on zero-shot cross-modal retrieval (Image → Impression and Impression → Image). We evaluate the pretrained models on the public CheXpert and MIMIC-CXR benchmarks. CheXficient achieves performance comparable to CheXfull on both datasets ( p>0.05 ), while outperforming CheXrandom. We report the mean ± 95% CI of Recall@1 (The recall of retrieving the exact paired CXR Impression section (or image) within the top-1 result for a given CXR image (or Impression section)).
Extended Data Fig. 4: Performance (AUPRC) comparison of CheXficient with other pretrained models in label-efficient disease prediction across three representative datasets: CheXpert ( n = 500, 14 categories), MIMIC-CXR ( n = 3,082, 14 categories), and ChestXray14 ( n = 25,596, 15 categories). Performance is reported at varying proportions of labeled training data. Vertical dashed lines indicate the amount of labeled data required for CheXficient to match the performance of its counterpart CheXrandom.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Distribution of patient age in the full training set (CheXfull) and the curated subset (CheXficient). The curated subset includes relatively more samples from two age groups: pediatric patients (0–5 years old) and older adults (over 65 years old).
Figure 6: Distribution of patient race in the full training set (CheXfull) and the curated subset (CheXficient). The two sets show similar race distributions.
Figure 7: Distribution of frontal and lateral scans in the full training set (CheXfull) and the curated subset (CheXficient). The curated subset contains a higher proportion of frontal scans in comparison with the full training set.
Figure 8: Distribution of CXR report token lengths in the full training set (CheXfull) and the curated subset (CheXficient). The curated subset demonstrates a higher proportion of longer CXR reports with high informativeness.
Dataset
ziimg
zitxt
concat(ziimg,zitxt)
SIIM-PTX
0.8920 [0.8858, 0.8981]
0.8937 [0.8900, 0.8975]
0.9074 [0.9001, 0.9148]
Pneumonia2017
0.9481 [0.9416, 0.9546]
0.9490 [0.9447, 0.9532]
0.9659 [0.9597, 0.9722]
TBX11K
0.8209 [0.8145, 0.8274]
0.8174 [0.8110, 0.8238]
0.8248 [0.8183, 0.8313]
CheXpert
0.8407 [0.8361, 0.8453]
0.8335 [0.8290, 0.8380]
0.8535 [0.8501, 0.8569]
MIMIC-CXR
0.6613 [0.6558, 0.6669]
0.6758 [0.6708, 0.6807]
0.6866 [0.6824, 0.6908]
ChestX-ray14
0.7191 [0.7166, 0.7216]
0.7312 [0.7255, 0.7369]
0.7308 [0.7281, 0.7336]
Appendix
Table 2: Effect of different feature embeddings used in CheXficient pretraining, e.g., image-only ziimg , text-only zitxt , and image-text concatenation concat(ziimg,zitxt) , on 280K curated CXR data pairs. Zero-shot findings classification performance (AUROC) is reported across eight datasets.
Figure 9: Effect of the number of prototypes used in CheXficient pretraining on zero-shot findings classification performance across eight datasets.
The rapid growth of medical imaging has fueled the development of Foundation Models (FMs) to reduce the growing, unsustainable workload on radiologists. While recent FMs have shown the power of large-scale pre-training to CT and MRI analysis, there remains significant room to optimize how these models learn from complex radiological volumes. Building upon the Curia framework, this work introduces Curia-2, which significantly improves the original pre-training strategy and representation quality to better capture the specificities of radiological data. The proposed methodology enables scaling the architecture up to billion-parameter Vision Transformers, marking a first for multi-modal CT and MRI FMs. Furthermore, we formalize the evaluation of these models by extending and restructuring CuriaBench into two distinct tracks: a 2D track tailored for slice-based vision models and a 3D track for volumetric benchmarking. Our results demonstrate that Curia-2 outperforms all FMs on vision-focused tasks and fairs competitively to vision-language models on clinically complex tasks such as finding detection. Weights will be made publicly available to foster further research.
Antoine Saporta, Baptiste Callard, Corentin Dancette +5
Raidium, 27 rue du Faubourg Saint-Jacques, 75014 Paris, France. · Department of Vascular and Oncological Interventional Radiology, Hôpital Européen Georges Pompidou, AP-HP, Paris, France · Faculté de Santé, Université Paris-Cité, Paris, France.
Recent medical multimodal foundation models are built as multimodal LLMs (MLLMs) by connecting a CLIP-pretrained vision encoder to an LLM using LLaVA-style finetuning. This two-stage, decoupled approach introduces a projection layer that can distort visual features. This is especially concerning in medical imaging where subtle cues are essential for accurate diagnoses. In contrast, early-fusion generative approaches such as Chameleon eliminate the projection bottleneck by processing image and text tokens within a single unified sequence, enabling joint representation learning that leverages the inductive priors of language models. We present CheXmix, a unified early-fusion generative model trained on a large corpus of chest X-rays paired with radiology reports. We expand on Chameleon's autoregressive framework by introducing a two-stage multimodal generative pretraining strategy that combines the representational strengths of masked autoencoders with MLLMs. The resulting models are highly flexible, supporting both discriminative and generative tasks at both coarse and fine-grained scales. Our approach outperforms well-established generative models across all masking ratios by 6.0% and surpasses CheXagent by 8.6% on AUROC at high image masking ratios on the CheXpert classification task. We further inpaint images over 51.0% better than text-only generative models and outperform CheXagent by 45% on the GREEN metric for radiology report generation. These results demonstrate that CheXmix captures fine-grained information across a broad spectrum of chest X-ray tasks. Our code is at: https://github.com/StanfordMIMI/CheXmix.
Ashwin Kumar, Robbie Holland, Corey Barrett +8
Stanford AIMI, Stanford University · Oracle Health AI · Department of Radiology, Stanford University
Foundation models have shown significant potential across diverse fields and have been applied to medical imaging, but their robustness across patient populations remains unclear. In this work, we investigate foundation models' ability to generalize from adult to pediatric chest X-ray classification. To the best of our knowledge, this is the first benchmark to compare multiple foundation models under a common adult-to-pediatric evaluation setting while also analyzing the effect of adaptation depth. We harmonize VinDr-CXR and PediCXR into a five-label classification task and evaluate six foundation models and two CNN baselines. All models are adapted using adult data only and tested in both the in-domain and pediatric domains using linear probing, partial fine-tuning, and full fine-tuning. The benchmark reveals a clear and consistent trend: all models suffer a substantial performance degradation when transferred from adult to pediatric data, with drops exceeding 10 percentage points. While Macro-AUROC values reach approximately 92% on the adult test set, performance decreases to around 78% on the pediatric domain. Another important finding is that increasing the depth of adaptation does not significantly improve cross-population generalization. Moreover, strong in-domain performance does not translate into better performance on pediatric data.
Sara Inácio, Hugo Proença, João C. Neves
University of Beira Interior, Portugal · NOVA LINCS · IT: Instituto de Telecomunicações