Diagnosis-Conditioned Spatial Gating and Decoder-Level Supervised Contrastive Learning for Radiology Report Generation

Md Mustafizur Rahman; Mylene C. Q. Farias

cs.CVOct 3, 2026

Diagnosis-Conditioned Spatial Gating and Decoder-Level Supervised Contrastive Learning for Radiology Report Generation

Authors: Md Mustafizur Rahman, Mylene C. Q. Farias

Organizations: Department of Computer Science, Texas State University, San Marcos, TX, USA

Abstract

Radiology report generation models can produce fluent text while still containing finding-level inaccuracies. Diagnosis-driven methods improve generation by conditioning on predicted findings, but these predictions do not directly modify the visual patch features provided to the decoder, and global gating applies the same modulation across spatial locations. We introduce a Position-Aware Gate (PAG) that uses predicted finding representations to modulate visual patches spatially without region supervision. We also propose a Decoder-level Supervised Contrastive Loss (DSCL) that structures decoder representations using shared positive findings rather than instance identity. On MIMIC-CXR, PAG+DSCL improves clinical efficacy (CE) F1 from 0.484 to 0.502 over a matched global-gate reference, while PAG and DSCL individually reach 0.491 and 0.495. Without additional fine-tuning, the combined model achieves 0.226 CE F1 on IU X-Ray, compared with 0.211 reported by PromptMRG.

Figures & tables

Figure 1: Example chest X-ray and reference report from MIMIC-CXR [ 7 ] .

Figure 2: Proposed framework. During training, ground-truth finding states provide diagnosis-driven prompt tokens, while classifier-predicted states drive PAG. At inference, classifier predictions serve both roles. PAG fuses state and finding-identity embeddings, attends to $S$ visual patches, and produces a spatial gate $\gamma_{s}$ for each location. The gated patches serve as decoder cross-attention memory. DSCL supervises pooled decoder representations using positive pairs defined by shared ground-truth positive findings.

Method	CE-P	CE-R	CE-F1	B-1	B-4	R-L
R2Gen † [ 1 ]	0.333	0.273	0.276	0.353	0.103	0.277
METrans. † [ 15 ]	0.364	0.309	0.311	0.386	0.124	0.291
KiUT † [ 4 ]	0.371	0.318	0.321	0.393	0.113	0.285
DCL † [ 9 ]	0.471	0.352	0.373	–	0.109	0.284
RGRG †# [ 14 ]	0.461	0.475	0.447	0.373	0.126	0.264
PromptMRG † [ 6 ]	0.501	0.509	0.476	0.398	0.112	0.268

Table 1: Results on MIMIC-CXR.

\dagger

indicates values taken from PromptMRG.

\#

indicates a different split and input resolution

>224\times 224

. Bold indicates the best comparable result shown.

Method	CE-P	CE-R	CE-F1	B-1	B-4	R-L
R2Gen † [ 1 ]	0.141	0.136	0.136	0.325	0.059	0.253
CVT2Dis. ∗† [ 11 ]	0.174	0.172	0.168	0.383	0.082	0.277
DCL † [ 9 ]	0.168	0.167	0.162	0.354	0.074	0.267
RGRG ∗† [ 14 ]	0.183	0.187	0.180	0.266	0.063	0.180
PromptMRG † [ 6 ]	0.213	0.229	0.211	0.401	0.098	0.281
PAG+DSCL	0.231	0.247	0.226	0.431	0.093	0.302

Table 2: Cross-dataset evaluation on IU X-Ray following PromptMRG.

\dagger

indicates values taken from PromptMRG.

*

indicates input resolution

>224\times 224

. Bold indicates the best comparable result shown.

#	Configuration	CE-P	CE-R	CE-F1	$\Delta$
1	Global+Align	0.504	0.511	0.484	0.000
2	PAG+Align	0.511	0.531	0.491	+0.007
3	Global+DSCL	0.514	0.536	0.495	+0.011
4	PAG+DSCL	0.527	0.542	0.502	+0.018
5	Global-PM+Align	0.502	0.522	0.483	-0.001

Table 3: Component ablation on MIMIC-CXR. PM denotes a parameter-matched global gate with hidden dimension 166.

\Delta

is relative to Global+Align.

Radiology Report Generation

Radiology report generation uses medical images and clinical context to produce structured, clinically accurate findings and impressions.

See all 159 papers

CWCD: Category-Wise Contrastive Decoding for Structured Medical Report Generation
Apr 12, 2026Shantam Srivastava, Mahesh Bhosale, David Doermann +1Radiology Report Generation Radiology VLMs
The Department of Computer Science and Engineering, University at Buffalo, The State University of New York, NY, USA
Graph-Supervised Hierarchical Clinical Alignment for Radiology Report Generation with Large Language Models
Aug 25, 2026Yingshu Li, Yunyi Liu, Zhanyu Wang +4Radiology Report Generation Radiology VLMs
University of Sydney Sydney, Australia · University of Wollongong Wollongong, Australia · University of Adelaide Adelaide, Australia
Learning to See Locally and Align Clinically with Pathology Semantics for Radiology Report Generation
Jul 31, 2026Xuan Cuong NgoRadiology Report Generation VLM Robustness

Apr 12, 2026J/K move · Enter open · S save

CWCD: Category-Wise Contrastive Decoding for Structured Medical Report Generation

Shantam Srivastava, Mahesh Bhosale, David Doermann, Mingchen Gao

The Department of Computer Science and Engineering, University at Buffalo, The State University of New York, NY, USA

Radiology Report Generation Radiology VLMs Contrastive Decoding Medical Imaging

Open paper AskPDF arXiv

Interpreting chest X-rays is inherently challenging due to the overlap between anatomical structures and the subtle presentation of many clinically significant pathologies, making accurate diagnosis time-consuming even for experienced radiologists. Recent radiology-focused foundation models, such as LLaVA-Rad and Maira-2, have positioned multi-modal large language models (MLLMs) at the forefront of automated radiology report generation (RRG). However, despite these advances, current foundation models generate reports in a single forward pass. This decoding strategy diminishes attention to visual tokens and increases reliance on language priors as generation proceeds, which in turn introduces spurious pathology co-occurrences in the generated reports. To mitigate these limitations, we propose Category-Wise Contrastive Decoding (CWCD), a novel and modular framework designed to enhance structured radiology report generation (SRRG). Our approach introduces category-specific parameterization and generates category-wise reports by contrasting normal X-rays with masked X-rays using category-specific visual prompts. Experimental results demonstrate that CWCD consistently outperforms baseline methods across both clinical efficacy and natural language generation metrics. An ablation study further elucidates the contribution of each architectural component to overall performance.

Abstract

Radiology Report Generation

Explore similar work

CWCD: Category-Wise Contrastive Decoding for Structured Medical Report Generation

Graph-Supervised Hierarchical Clinical Alignment for Radiology Report Generation with Large Language Models

Learning to See Locally and Align Clinically with Pathology Semantics for Radiology Report Generation