SCCL-DT: Semantic-Conflict-Aware Contrastive Learning with Dual-Rule Teacher Distillation for Zero-Shot Chest X-Ray Diagnosis

Ting Liu; Haoran Lai; HaobinChen; Ming Liu; S Kevin Zhou

cs.CVOct 15, 2026

SCCL-DT: Semantic-Conflict-Aware Contrastive Learning with Dual-Rule Teacher Distillation for Zero-Shot Chest X-Ray Diagnosis

Authors: Ting Liu, Haoran Lai, HaobinChen, Ming Liu, S Kevin Zhou

Organizations: School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, Anhui, 230026, P.R. China · Suzhou Institute for Advanced Research, University of Science and Technology of China, Suzhou, Jiangsu, 215123, P.R. China

Abstract

Contrastive vision-language pre-training supports zero-shot chest X-ray diagnosis, but standard objectives treat every off-diagonal image-report pair in a batch as a negative. In radiology, recurring findings across patients mean that instance mismatch need not imply semantic incompatibility, creating false-negative supervision that we term a semantic conflict. A training-trace audit shows that pair-observations selected by the online mask used during training are enriched for report-derived finding agreement, although this proxy analysis does not establish clinical equivalence. Fixed thresholds can become miscalibrated as similarity distributions evolve, while soft reweighting leaves residual repulsive gradients. We therefore propose Semantic-Conflict-Aware Contrastive Learning (SCCL), which tracks modality-specific similarity distributions with exponential-moving-average (EMA) statistics, derives adaptive thresholds, and hard-masks flagged candidates before softmax normalization. Inclusive (OR) and consensus (AND) rules define two SCCL teachers, whose bidirectional cross-modal rankings are consolidated through standard distillation into a single deployable student, SCCL-DT. Across five public chest X-ray benchmarks, SCCL-DT achieves a mean zero-shot AUC of $0.850$ , compared with $0.837$ for the strongest evaluated baseline in our comparison. Code will be made publicly available soon.

Figures & tables

Figure 1: Motivation and label-based analysis of candidate semantic conflicts. (a) Online inclusive masking excludes an off-diagonal candidate from the negative set when either its image–image or report–report similarity exceeds the adaptive threshold. (b) Anchor-level probability and expected count of cross-patient matches with identical present-label sets. (c) Exact present-label-set agreement for flagged versus all eligible pair-observations, overall and across training windows.

Figure 2: Overview of the two-stage SCCL-DT framework. Stage I trains inclusive and consensus teachers by hard-masking similarity-flagged off-diagonal candidates selected with modality-specific EMA thresholds. Stage II distills their image-to-text (i2t) and text-to-image (t2i) rankings into a single student using row- and column-wise Kullback–Leibler (KL) supervision. Only the student is retained for inference.

Method	Open-I	PadChest	ChestX-ray14	CheXpert	ChestXDet10	Mean
MedCLIP [ 30 ]	0.551	0.508	0.564	0.744	0.571	0.588
BiomedCLIP [ 34 ]	0.577	0.513	0.639	0.677	0.630	0.607
GLoRIA [ 12 ]	0.589	0.565	0.610	0.750	0.645	0.632
BioViL [ 2 ]	0.702	0.655	0.729	0.789	0.708	0.717
CheXzero [ 27 ]	0.726	0.648	0.712	0.889	0.640	0.723
MedKLIP [ 31 ]	0.759	0.629	0.726	0.879	0.713	0.741

Table 1: Zero-shot classification performance (AUC) on five chest X-ray benchmarks. Mean is the unweighted dataset average; higher is better. Bold and underline denote the best and second-best results in each column, respectively.

Figure 3: Qualitative zero-shot grounding on six ChestXDet10 cases. Rows show the unmasked Base model and SCCL-DT; green boxes are ground-truth lesions, white stars are Pointing Game maxima, and warmer colors indicate larger cross-attention weights. Each map is normalized independently.

Method	Mean	ATE	CALC	CONS	EFF	EMPH	FIB	FX	MASS	NOD	PTX
GLoRIA [ 12 ]	0.368	0.479	0.053	0.737	0.528	0.667	0.366	0.013	0.533	0.156	0.143
MedKLIP [ 31 ]	0.481	0.625	0.132	0.837	0.675	0.734	0.305	0.224	0.733	0.312	0.229
KAD [ 35 ]	0.391	0.646	0.132	0.699	0.618	0.644	0.244	0.199	0.267	0.316	0.143
CARZero [ 16 ]	0.543	0.604	0.184	0.824	0.782	0.846	0.561	0.184	0.700	0.286	0.457
RadZero (224 px) [ 24 ]	0.537	0.604	0.211	0.806	0.813	0.795	0.451	0.197	0.767	0.325	0.400
SCCL-DT (Ours)	0.583	0.667	0.184	0.855	0.802	0.821	0.524	0.263	0.767	0.351	0.600

Table 2: Zero-shot grounding on ChestXDet10, measured by per-finding and unweighted mean Pointing Game (PG) accuracy. ATE, CALC, CONS, EFF, EMPH, FIB, FX, MASS, NOD, and PTX denote Atelectasis, Calcification, Consolidation, Effusion, Emphysema, Fibrosis, Fracture, Mass, Nodule, and Pneumothorax, respectively. Bold and underline denote the best and second-best results in each column.

	Components			PadChest				Cross-data
#	OR	AND	KD	AUC $\uparrow$	MCC $\uparrow$	F1 $\uparrow$	ACC $\uparrow$	mAUC $\uparrow$
(a)				0.810	0.156	0.143	0.991	0.836
(b)	$\checkmark$			0.826	0.179	0.165	0.991	0.840
(c)		$\checkmark$		0.825	0.173	0.159	0.991	0.846
(d)	$\checkmark$		$\checkmark$	0.823	0.178	0.162	0.991	0.846
(e)		$\checkmark$	$\checkmark$	0.822	0.169	0.155	0.991	0.844

Table 3: Rule-specific teacher and knowledge-distillation analysis. OR/AND denote the teacher(s), KD denotes student distillation, and the empty setting is the unmasked Base. Cross-data mAUC averages five benchmarks; bold and underline mark best and second-best; ACC ties after rounding.

A. Threshold sensitivity
Design choice	AUC $\uparrow$	MCC $\uparrow$	F1 $\uparrow$	ACC $\uparrow$
Fixed $\tau{=}0.85$	0.807	0.147	0.130	0.991
Fixed $\tau{=}0.90$	0.812	0.155	0.136	0.991
Fixed $\tau{=}0.95$	0.814	0.156	0.138	0.991
Adaptive $k{=}1.5$	0.820	0.157	0.139	0.991
Adaptive $k{=}2$ (def.)	0.826	0.179	0.165	0.991

Table 4: Controlled PadChest analyses: (A) threshold sensitivity; (B) candidate-selection and logit-intervention controls. “def.” marks the prespecified default; bold and underline mark best and second-best within each block; ACC ties after rounding.

Similarity cue	AUC $\uparrow$	MCC $\uparrow$	F1 $\uparrow$	ACC $\uparrow$
Image only	0.814	0.158	0.141	0.991
Text only	0.816	0.157	0.138	0.991
Both (default)	0.826	0.179	0.165	0.991

Table 5: PadChest similarity-cue ablation under the default inclusive setting. Bold and underline mark the best and second-best AUC, MCC, and F1; ACC ties at

0.991

Figure 4: Adaptive image (solid) and text (dashed) thresholds (

k{=}2

) for SCCL and the unmasked Base. Base values are logged but not applied; shading marks the fixed range

0.85

–

0.95

and the dotted line the cosine ceiling.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Variant	Decision/sampling parameters	Logit intervention
Two-tier	$k_{H}^{v}=3.0$ , $k_{H}^{t}=2.0$ ; $k_{L}^{v}=1.5$ , $k_{L}^{t}=1.0$ ; $m=0.99$ ; $T_{w}=500$ training-call warm-up	Symmetric mask; i2t uses $M$ , t2i uses $M^{\top}$ ; selected cells replaced by $-10^{4}$
Count-matched Random	OR source: $k=2.0$ , $m=0.99$ ; $T_{w}=500$ training-call warm-up; full upper-triangular pool; exact per-call $K_{s}$ ; global seed $=1$ , mask base seed $=1$	Same hard replacement; matches only total unordered-pair count, not row/column degree
Soft weighting	$k=2.0$ , $m=0.99$ ; $T_{w}=500$ training-call warm-up; $\gamma=4.0$ ; std floor $10^{-6}$ ; pre-log clamp $10^{-30}$	i2t adds $\log\overline{W}$ , t2i its transpose, between fusion logits and CE

Appendix

Table S1: Implementation semantics of the three diagnostic controls.

	All anchors		Pathology-positive anchors
$B$	$\widehat{P}(\geq 1)$	$\widehat{E}[X]$	$\widehat{P}(\geq 1)$	$\widehat{E}[X]$
$64$	$62.46\%$	$6.37$	$38.47\%$	$0.69$
$128$	$72.05\%$	$12.83$	$53.41\%$	$1.38$
$256$	$80.21\%$	$25.77$	$66.78\%$	$2.78$

Appendix

Table S2: Exact cross-patient anchor-level collision statistics used in Fig. 1(b) in the main paper.

Pool / agreement	$N$	$F$	$G$	$FG$	Flagged	Baseline	Lift ( $95\%$ CI)	Capture
CheXpert / CheXpert	178,150,296	6,613,862	3,267,352	261,620	3.9556%	1.8340%	2.1568 (2.1471–2.1664)	8.0071%
NegBio / NegBio	170,566,834	6,323,181	3,306,661	255,078	4.0340%	1.9386%	2.0809 (2.0715–2.0902)	7.7141%
CheXpert / NegBio (fixed pool)	178,150,296	6,613,862	3,304,696	255,801	3.8676%	1.8550%	2.0850 (2.0755–2.0943)	7.7405%

Appendix

Table S3 : Best-checkpoint-aligned enrichment using the masks actually applied during training. Counts are repeated pair-observations along the trace.

Epoch window	Batch size	Batches	All $i<j$ observations / lift
0–1	64	6,600	13,305,600 / $1.93\times$
2–5	128	7,100	57,708,800 / $2.06\times$
6–10	256	4,435	144,758,400 / $2.13\times$
11–15	256	4,435	144,758,400 / $2.16\times$
16–20	256	4,435	144,758,400 / $2.22\times$

Appendix

Table S4: Five-window temporal analysis displayed in Fig. 1(c) in the main paper. All window-level

95\%

CI lower bounds exceed one.

Rule	$F$	$FG$	Agreement	Baseline	Lift	Trigger / capture
Image	1,348,802	95,719	7.0966%	1.8340%	3.8694	0.7571% / 2.9296%
Text	5,355,304	175,084	3.2694%	1.8340%	1.7826	3.0061% / 5.3586%
OR	6,613,862	261,620	3.9556%	1.8340%	2.1568	3.7125% / 8.0071%
AND	90,244	9,183	10.1757%	1.8340%	5.5483	0.0507% / 0.2811%

Appendix

Table S5: Four-rule exact CheXpert agreement on the fixed primary pool. Trigger is

F/N

and capture is

FG/G

Metric	Flagged	Eligible	Difference	Ratio
Shared any finding	57.4%	45.5%	$+11.9$ pp	$1.26\times$
Mean Jaccard	0.252	0.181	$+0.071$	$1.39\times$

Appendix

Table S6: Partial-overlap checks on the fixed CheXpert primary pool.

Pair pool	$N$	$F$	Flagged	Baseline	Lift ( $95\%$ CI)
All eligible $i<j$ pairs	505,289,600	27,476,966	35.6397%	10.1071%	3.5262 (3.5196–3.5329)
Exclude pairs where both are No Finding	444,327,034	17,348,162	2.0469%	1.0037%	2.0394 (2.0313–2.0474)
Both samples have No Finding $\neq 1$	215,233,750	7,968,418	4.4564%	2.0720%	2.1508 (2.1422–2.1593)
Both pathology-positive; no patient exclusion	178,166,296	6,619,910	3.9619%	1.8344%	2.1597 (2.1501–2.1693)

Appendix

Table S7: Sensitivity to the CheXpert-defined pair pool.

Radiology VLMs

Radiology vision-language models (VLMs) align medical images and text to support grounded diagnosis, reporting, retrieval, and clinical reasoning.

See all 109 papers

Concept-Guided Noisy Negative Suppression for Zero-Shot Classification and Grounding of Chest X-Ray Findings
May 19, 2026Chenyu Lian, Hong-Yu Zhou, Chun-Ka Wong +1Contrastive Learning Chest X-Ray Classification
The Center for Smart Health, School of Nursing, the Hong Kong Polytechnic University, Hong Kong, China · Research Institute for Smart Ageing, the Hong Kong Polytechnic University,2026 Hong Kong, China · School of Biomedical Engineering, Tsinghua Medicine, Tsinghua University, Beijing, China +1
ProtoCLIP: Prototype-Aligned Latent Refinement for Robust Zero-Shot Chest X-Ray Classification
Apr 20, 2026Florian Kittler, Sheethal Bhat, Andreas MaierVLM Robustness Chest X-Ray Classification
Friedrich-Alexander University Erlangen-Nuremberg, Erlangen 91054, Germany
SentZero: An Enhanced Sentence-Centric Vision-Language Pretraining for Multi-Task Zero-Shot Chest X-Ray Analysis
Sep 28, 2026Hangyul Yoon, Hyungyung Lee, Edward Choi +1Contrastive Learning Radiology VLMs
Kim Jaechul Graduate School of AI, Korea Advanced Institute of Science and Technology (KAIST)

May 19, 2026J/K move · Enter open · S save

Concept-Guided Noisy Negative Suppression for Zero-Shot Classification and Grounding of Chest X-Ray Findings

Chenyu Lian, Hong-Yu Zhou, Chun-Ka Wong, Jing Qin

The Center for Smart Health, School of Nursing, the Hong Kong Polytechnic University, Hong Kong, China · Research Institute for Smart Ageing, the Hong Kong Polytechnic University,2026 Hong Kong, China · School of Biomedical Engineering, Tsinghua Medicine, Tsinghua University, Beijing, China+1

Contrastive Learning Chest X-Ray Classification Medical VLMs Medical Image Grounding

Open paper AskPDF arXiv

Vision-language alignment using chest X-rays and radiology reports has emerged as an advanced paradigm for zero-shot classification and grounding of chest X-ray findings. However, standard contrastive learning typically treats radiographs and reports from different patients simply as negative pairs. This assumption introduces noisy negatives, as different patients frequently exhibit similar findings. Such noisy negatives cause semantic ambiguity and degrade performance in zero-shot understanding tasks. To address this challenge, we propose CoNNS, a concept-guided noisy-negative suppression framework. To support the negative suppression mechanism, unlike previous methods that use raw reports or templatized texts, we construct a hierarchical concept ontology using large language models. The ontology structures 41 key clinical concepts by explicitly modeling presence, attributes (location and characteristics), and texts (evidential segment and presence statement). Leveraging this ontology, we implement a cross-patient pair relabeling strategy comprising three steps: (1) Fine-Grained Breakdown to categorize pairs based on finding presence; (2) Noisy Negative Filtering to resolve semantic conflicts by removing false negatives; and (3) Hard Negative Mining to identify subtle attribute discrepancies using a lightweight language model. Finally, we propose a Concept-Aware NCE loss to align visual features with text while suppressing the identified noisy negatives. Extensive experiments across multi-granularity zero-shot grounding tasks and five zero-shot classification datasets validate that CoNNS outperforms existing state-of-the-art models. The code is available at https://github.com/DopamineLcy/conns.

Abstract

Radiology VLMs

Explore similar work

Concept-Guided Noisy Negative Suppression for Zero-Shot Classification and Grounding of Chest X-Ray Findings

ProtoCLIP: Prototype-Aligned Latent Refinement for Robust Zero-Shot Chest X-Ray Classification

SentZero: An Enhanced Sentence-Centric Vision-Language Pretraining for Multi-Task Zero-Shot Chest X-Ray Analysis