Efficient Chest X-ray Representation Learning via Semantic-Partitioned Contrastive Learning

Wangyu Feng; Shawn Young; Lijian Xu

cs.CVMar 7, 2026

Efficient Chest X-ray Representation Learning via Semantic-Partitioned Contrastive Learning

Authors: Wangyu Feng, Shawn Young, Lijian Xu

Organizations: Shenzhen University of Advanced Technology, Shenzhen, China

Abstract

Self-supervised learning (SSL) has emerged as a powerful paradigm for Chest X-ray (CXR) analysis under limited annotations. Yet, existing SSL strategies remain suboptimal for medical imaging. Masked image modeling allocates substantial computation to reconstructing high-frequency background details with limited diagnostic value. Contrastive learning, on the other hand, often depends on aggressive augmentations that risk altering clinically meaningful structures. We introduce Semantic-Partitioned Contrastive Learning (S-PCL), an efficient pre-training framework tailored for CXR representation learning. Instead of reconstructing pixels or relying on heavy augmentations, S-PCL randomly partitions patch tokens from a single CXR into two non-overlapping semantic subsets. Each subset provides a complementary but incomplete view. The encoder must maximize agreement between these partitions, implicitly inferring global anatomical layout and local pathological cues from partial evidence. This semantic partitioning forms an internal bottleneck that enforces long-range dependency modeling and structural coherence. S-PCL eliminates the need for hand-crafted augmentations, auxiliary decoders, and momentum encoders. The resulting architecture is streamlined, computationally efficient, and easy to scale. Extensive experiments on large-scale CXR benchmarks, including ChestX-ray14, CheXpert, RSNA Pneumonia and SIIM-ACR Pneumothorax, show that S-PCL achieves competitive performance while attaining the lowest GFLOPs and superior accuracy among existing SSL approaches.

Figures & tables

Figure 1: Our occluded image contrastive learning: Through non-overlapping occluding, distinct tokens within an image are categorized as intraclass, while across-image tokens within a batch are viewed as interclass.

Training with clinical reports
Method	GFLOPs	GPU Hrs	ChestX-ray14			CheXpert			RSNA Pneu.
Method	GFLOPs	GPU Hrs	1%	10%	100%	1%	10%	100%	1%	10%	100%
M3AE Chen et al. (2022)	34.4	710	-	-	-	86.2	87.3	87.9	89.0	90.8	92.3
REFERS Zhou et al. (2022)	54.8	-	76.7	80.9	84.7	87.2	88.1	88.2	89.4	91.6	92.7
MRM Zhou et al. (2023)	13.1	800	79.4	84.0	85.9	88.5	88.5	88.7	91.3	92.7	93.3
Training without clinical reports

Table 1: Comparison of our method with specialist approaches for disease classification. Performance is measured using macro - averaged AUC. The labeling ratio X% indicates the proportion of the fully annotated training set used for supervised fine - tuning. Best results are highlighted in bold.

Figure 2: Efficiency and scaling comparison of MRM Zhou et al. (2023) , Medical MAE Xiao et al. (2023) , M3AE Chen et al. (2022) , and our method on CheXpert fine-tuning, measured by mAUC (%).

Method	Backbone	Atelectasis	Cardiomegaly	Consolidation	Edema	Effusion	mAUC (%)
Label-Assemble Kang et al. (2023)	DN121	82.1	85.9	94.4	89.2	93.6	89.0
Medical MAE Xiao et al. (2023)	ViT-B/16	82.7	83.5	92.5	93.8	94.1	89.3
S-PCL (Ours)	ViT-B/16	80.3	95.4	80.4	94.1	95.6	89.1

Table 2: Performance comparison on the CheXpert benchmark . We report the Area Under the Curve (AUC) for five clinical observations and their mean (mAUC). Best results are highlighted in bold.

Model	Mean	Atelectasis	Cardiomegaly	Effusion	Infiltration	Mass	Nodule	Pneumonia	Pneumothorax	Consolidation	Edema	Emphysema	Fibrosis	Pleural Thicken	Hernia
ConVIRT Zhang et al. (2022)	56.0	45.1	44.3	63.2	65.1	61.6	57.2	63.6	54.1	63.7	70.2	41.9	47.4	56.0	51.2
GLoRIA Huang et al. (2021)	81.8	82.6	83.3	86.0	66.4	81.8	73.5	71.0	84.5	81.3	89.8	93.1	78.9	76.1	97.5
MRM Zhou et al. (2023)	85.9	84.2	93.0	89.6	71.8	88.2	78.5	77.3	90.2	82.2	91.0	94.3	86.7	81.4	94.4
S-PCL (Ours)	84.1	82.3	87.7	91.4	75.7	80.8	75.0	77.3	92.5	79.0	89.4	93.5	82.6	80.1	90.3

Table 3: Performance comparison on the ChestX-ray14 benchmark. We report the AUC scores across all 14 categories. The best results are in bold and the second-best results are underlined.

Methods	Semantic Segmentation			Object Detection
	SIIM-ACR Pneumothorax			RSNA Pneumonia
	1%	10%	100%	1%	10%	100%
GLoRIA Huang et al. (2021)	37.4	57.1	64.0	11.6	16.1	24.8
MGCA Wang et al. (2022)	49.7	59.3	64.2	12.9	16.8	24.9
MedKLIP Wu et al. (2023)	50.2	60.8	63.9	8.9	16.3	24.5
S-PCL (Ours)	38.8	61.7	65.1	13.2	16.9	25.6

Table 4: Results of semantic segmentation and object detection. Best results are highlighted in bold. Each model is fine-tuned with 1%, 10%, 100% training data.

Figure 3: t-SNE visualization of the learned global representations on CheXpert benchmark.

Medical Image Representation Learning

Medical image representation learning builds reusable visual features for diagnosis, segmentation, retrieval, and other clinical imaging tasks.

See all 260 papers

Apr 27, 2026

SparseContrast: Dynamic Sparse Attention for Efficient and Accurate Contrastive Learning in Medical Imaging

We propose SparseContrast, a new framework that merges dynamic sparse attention with contrastive learning for medical imaging, with a focus on chest X-ray disease detection in low-data settings. Traditional contrastive learning methods rely on dense attention mechanisms, which are computationally expensive and often process redundant regions in medical images. To resolve this, SparseContrast introduces a sparse attention mechanism that selectively concentrates on diagnostically pertinent areas, markedly decreasing computational burden without compromising accuracy. The framework adaptively trims attention maps in the training phase, directed by a compact saliency predictor which concurrently optimizes sparsity and feature quality. This method not only speeds up training and inference by as much as 40% relative to dense attention benchmarks but also boosts diagnostic accuracy by focusing on areas of clinical importance. Moreover, the approach remains indifferent to the selection of backbone architecture, which permits its application to both convolutional and transformer-based models. Experiments show SparseContrast attains comparable or better performance in disease identification tasks with greater efficiency relative to current approaches. The proposed framework delivers a practical approach for implementing contrastive learning in medical imaging settings with limited resources, where computational efficiency and diagnostic accuracy are paramount.

Paarth Prasad, Ruchika Malhotra

Department of Software Engineering, Delhi Technological University

Contrastive Learning Chest X-Ray Classification Sparse Attention Medical Image Representation Learning

Ask PDF

Dec 19, 2025

AnyCXR: Human Anatomy Segmentation of Chest X-ray at Any Acquisition Position using Multi-stage Domain Randomized Synthetic Data with Imperfect Annotations and Conditional Joint Annotation Regularization Learning

Robust anatomical segmentation of chest X-rays (CXRs) remains challenging due to the scarcity of comprehensive annotations and the substantial variability of real-world acquisition conditions. We propose AnyCXR, a unified framework that enables generalizable multi-organ segmentation across arbitrary CXR projection angles using only synthetic supervision. The method combines a Multi-stage Domain Randomization (MSDR) engine, which generates over 100,000 anatomically faithful and highly diverse synthetic radiographs from 3D CT volumes, with a Conditional Joint Annotation Regularization (CAR) learning strategy that leverages partial and imperfect labels by enforcing anatomical consistency in a latent space. Trained entirely on synthetic data, AnyCXR achieves strong zero-shot generalization on multiple real-world datasets, providing accurate delineation of 54 anatomical structures in PA, lateral, and oblique views. The resulting segmentation maps support downstream clinical tasks, including automated cardiothoracic ratio estimation, spine curvature assessment, and disease classification, where the incorporation of anatomical priors improves diagnostic performance. These results demonstrate that AnyCXR establishes a scalable and reliable foundation for anatomy-aware CXR analysis and offers a practical pathway toward reducing annotation burdens while improving robustness across diverse imaging conditions.

Zifei Dong, Wenjie Wu, Jinkui Hao +3

Department of Radiology, Northwestern University, Chicago, IL, USA · Data Science Institute, Vanderbilt University, Nashville, TN, USA · Department of Orthopedics, The Second Hospital of Shanxi Medical University, Taiyuan, Shanxi, P.R. China +2

Medical Image Segmentation Multi-Organ Segmentation Domain Generalization for Semantic Segmentation Weakly Supervised Image Segmentation

Ask PDF

Oct 15, 2026

SCCL-DT: Semantic-Conflict-Aware Contrastive Learning with Dual-Rule Teacher Distillation for Zero-Shot Chest X-Ray Diagnosis

Contrastive vision-language pre-training supports zero-shot chest X-ray diagnosis, but standard objectives treat every off-diagonal image-report pair in a batch as a negative. In radiology, recurring findings across patients mean that instance mismatch need not imply semantic incompatibility, creating false-negative supervision that we term a semantic conflict. A training-trace audit shows that pair-observations selected by the online mask used during training are enriched for report-derived finding agreement, although this proxy analysis does not establish clinical equivalence. Fixed thresholds can become miscalibrated as similarity distributions evolve, while soft reweighting leaves residual repulsive gradients. We therefore propose Semantic-Conflict-Aware Contrastive Learning (SCCL), which tracks modality-specific similarity distributions with exponential-moving-average (EMA) statistics, derives adaptive thresholds, and hard-masks flagged candidates before softmax normalization. Inclusive (OR) and consensus (AND) rules define two SCCL teachers, whose bidirectional cross-modal rankings are consolidated through standard distillation into a single deployable student, SCCL-DT. Across five public chest X-ray benchmarks, SCCL-DT achieves a mean zero-shot AUC of

0.850

, compared with

0.837

for the strongest evaluated baseline in our comparison. Code will be made publicly available soon.

Ting Liu, Haoran Lai, HaobinChen +2

School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, Anhui, 230026, P.R. China · Suzhou Institute for Advanced Research, University of Science and Technology of China, Suzhou, Jiangsu, 215123, P.R. China

Radiology VLMs Contrastive Learning Chest X-Ray Classification Zero-Shot Image Classification

Ask PDF

Abstract

Medical Image Representation Learning

Explore similar work

SparseContrast: Dynamic Sparse Attention for Efficient and Accurate Contrastive Learning in Medical Imaging

AnyCXR: Human Anatomy Segmentation of Chest X-ray at Any Acquisition Position using Multi-stage Domain Randomized Synthetic Data with Imperfect Annotations and Conditional Joint Annotation Regularization Learning

SCCL-DT: Semantic-Conflict-Aware Contrastive Learning with Dual-Rule Teacher Distillation for Zero-Shot Chest X-Ray Diagnosis