ICGF-CXRMate: Image-Conditioned Clinical Context Compression and Gated Fusion for Chest X-ray Report Generation

Vu Minh Tran; Thuc Huynh; Son Thanh Luu; Doanh C. Bui

cs.CVOct 22, 2026

ICGF-CXRMate: Image-Conditioned Clinical Context Compression and Gated Fusion for Chest X-ray Report Generation

Authors: Vu Minh Tran, Thuc Huynh, Son Thanh Luu, Doanh C. Bui

Organizations: University of Information Technology · Viet Nam National University Ho Chi Minh City

Abstract

Auxiliary patient data can improve chest X-ray report generation, but directly concatenating heterogeneous clinical information with visual tokens produces long, variable-length decoder prompts and may expose the model to redundant or image-irrelevant context. We propose ICGF-CXRMate, an image-conditioned clinical context compression and gated fusion framework. Learnable queries are first conditioned on the current chest X-ray representations and then attend to the patient-data sequence, compressing it into a fixed number of clinical context tokens. Gated cross-attention selectively integrates these tokens into the visual stream, while the compressed tokens are retained in the decoder prompt as a direct clinical pathway. Experiments on linked MIMIC-CXR and MIMIC-IV-ED examinations show that ICGF-CXRMate improves report quality while reducing the average prompt length. In Stage2, it improves RadGraph-F1 from 24.00 to 28.86, CXR-BERT similarity from 56.86 to 62.80, and BLEU-4 from 5.14 to 7.96 compared with CXRMate-ED, while reducing the average prompt length from 373.89 to 288.37 tokens, corresponding to a 22.9% reduction. After self-critical sequence training, ICGF-CXRMate outperforms CXRMate-ED on six of eight report-quality metrics in Stage3. Ablation studies confirm the complementary contributions of image-conditioned compression, gated fusion, and direct retention of the compressed clinical tokens.

Figures & tables

Figure 1: Overview of ICGF-CXRMate. The model encodes CXR images and heterogeneous auxiliary patient data into visual and clinical representations. Image-conditioned clinical context compression (Sec. 3.2 ) uses image-adapted queries to summarize the variable-length patient sequence into a fixed set of clinical tokens $Z$ . Gated cross-attention fusion (Sec. 3.3 ) selectively integrates the compressed context into the visual representations to obtain $\widetilde{E}_{\mathrm{img}}$ . The resulting multimodal prompt $P_{\mathrm{ICGF}}=[Z;\widetilde{E}_{\mathrm{img}}]$ is passed to the Llama-based decoder for report generation.

Figure 2: Example of a linked MIMIC-CXR examination and MIMIC-IV-ED stay. The left panel shows the CXR images, radiologist report, and image metadata, while the right panel shows the associated ED stay, triage, vital-sign, and medication records. The Findings and Impression sections are used as the report-generation targets.

Stage 1: Images only
Method	RG $\uparrow$	CX $\uparrow$	CB $\uparrow$	G $\uparrow$	BS $\uparrow$	R-L $\uparrow$	B4 $\uparrow$	ARN $\uparrow$	Prompt
CXRMate-ED	23.46	27.16	53.85	31.44	20.25	24.44	4.26	95.00	254.66
Stage 2: Images + triage, reconciled medicines, indication, and history
CXRMate-ED	24.00	27.32	56.86	32.75	21.47	25.46	5.14	96.18	373.89
ICGF-CXRMate (Ours)	28.86	30.90	62.80	35.25	25.82	28.43	7.96	95.69	288.37
Stage 3: Reinforcement learning from Stage 2

Table 1: Report-level performance on the combined Findings and Impression sections. Best results within each training stage are shown in bold .

Img.-cond.	Gated Fusion	Direct $Z$	RG $\uparrow$	CX $\uparrow$	CB $\uparrow$	G $\uparrow$	BS $\uparrow$	R-L $\uparrow$	B4 $\uparrow$	ARN $\uparrow$	Prompt
$\times$	$\times$	$\checkmark$	28.39	28.99	60.75	34.70	25.35	28.15	8.13	95.54	288.37
$\checkmark$	$\times$	$\checkmark$	24.58	26.39	55.82	32.99	21.97	25.81	5.19	95.09	288.37
$\times$	$\checkmark$	$\checkmark$	28.35	30.79	59.41	34.28	25.48	28.10	7.74	95.86	288.37
$\checkmark$	$\checkmark$	$\times$	25.84	28.84	53.06	34.94	22.62	26.52	5.54	95.42	272.37
$\checkmark$	$\checkmark$	$\checkmark$	28.86	30.90	62.80	35.25	25.82	28.43	7.96	95.69	288.37

Table 2: Component ablation on the combined Findings and Impression sections in Stage 2. Best and second-best results are shown in bold and underlined , respectively.

No. Tokens	RG $\uparrow$	CX $\uparrow$	CB $\uparrow$	G $\uparrow$	BS $\uparrow$	R-L $\uparrow$	B4 $\uparrow$	ARN $\uparrow$	Prompt
$K=8$	25.56	26.40	52.93	35.53	22.85	25.98	5.19	96.71	280.37
$K=16$	28.86	30.90	62.80	35.25	25.82	28.43	7.96	95.69	288.37
$K=32$	27.81	28.82	59.13	34.13	25.16	27.88	7.57	95.87	304.37
$K=48$	28.50	30.88	60.58	34.26	25.64	28.12	8.13	95.55	320.37

Table 3: Effect of the number of clinical context tokens

K

on the combined Findings and Impression sections in Stage 2. Prompt denotes the average number of input prompt tokens. Best and second-best results are shown in bold and underline , respectively.

Figure 3: Case study comparing CXRMate-ED and ICGF-CXRMate ( $K=16$ ).

Gated Multimodal Fusion

See all 14 papers

CXRMate-2: Structured Multimodal Temporal Embeddings and Tractable Reinforcement Learning for Clinically Acceptable Chest X-ray Radiology Report Generation
Apr 21, 2026Aaron Nicolson, Elizabeth J. Cooper, Hwan-Jin Yoon +7Radiology Report Generation Radiology VLMs
Australian e-Health Research Centre, CSIRO Health and Biosecurity, Brisbane & Melbourne, Australia · Princess Alexandra Hospital, Metro South Health, Brisbane, Australia · Queensland Digital Health Centre, University of Queensland, Brisbane, Australia +2
CPVB-RG: Cross-Prior Visual Bridging for Prior-Aware Chest X-Ray Report Generation
Oct 20, 2026Sike Xiang, Shuang Chen, Julie Cox +1Radiology Report Generation Medical VLMs
Department of Computer Science, Durham University, UK · Department of Radiology, County Durham and Darlington NHS Foundation Trust, UK
CARE-X: Towards Clinically Useful Radiology VLMs with Auxiliary Supervision, Reward-Aligned Learning, and Tool-Augmented Measurement
Aug 4, 2026Mercy Prasanna Ranjit, Anirban Porya, Sathvik Joel +8Radiology Report Generation Chest X-Ray Classification
Microsoft Research India · Medha AI, Narayana Health, India. · RTIICS, Narayana Health, India

Apr 21, 2026J/K move · Enter open · S save

CXRMate-2: Structured Multimodal Temporal Embeddings and Tractable Reinforcement Learning for Clinically Acceptable Chest X-ray Radiology Report Generation

Aaron Nicolson, Elizabeth J. Cooper, Hwan-Jin Yoon, Claire McCafferty+6

Australian e-Health Research Centre, CSIRO Health and Biosecurity, Brisbane & Melbourne, Australia · Princess Alexandra Hospital, Metro South Health, Brisbane, Australia · Queensland Digital Health Centre, University of Queensland, Brisbane, Australia+2

Radiology Report Generation Radiology VLMs Healthcare Radiology Report Evaluation

Open paper AskPDF arXiv

Chest X-ray (CXR) radiology report generation (RRG) models have shown rapid progress on automated metrics, yet their clinical utility remains uncertain due to limited qualitative evaluation by radiologists. We present CXRMate-2, a state-of-the-art CXR RRG model that enables tractable reinforcement learning (RL) through structured multimodal temporal embeddings and high-resolution visual feature compression, for efficient, unified conditioning of an LLM decoder on visual, textual, and temporal context from a study and its prior. This enables group relative policy optimisation (GRPO), where a proposed reward function is used to improve semantic alignment with radiologist reports. Across the MIMIC-CXR, CheXpert Plus, and ReXgradient datasets, CXRMate-2 achieves statistically significant improvements over strong benchmarks, including gains of 11.2% and 24.4% in GREEN and RadGraph-XL, respectively, on MIMIC-CXR relative to MedGemma 1.5 (4B). To directly compare CXRMate-2 against radiologist reporting, we conduct a blinded, randomised qualitative retrospective evaluation. Three consultant radiologists compare generated and radiologist reports across 120 studies from the MIMIC-CXR test set. Generated reports were deemed acceptable (defined as preferred or rated equally to radiologist reports) in 45% of ratings, with no statistically significant difference in preference rates for seven of the eight analysed findings. Preferences for radiologist reports were driven primarily by higher recall, while generated reports were consistently preferred for readability. Together, these results define a clear pathway to clinically acceptable CXR RRG. Improving recall and the detection of subtle findings represents the primary remaining barrier to non-inferiority with radiologist reporting, positioning CXR RRG for prospective evaluation in assistive, radiologist-led workflows.

Abstract

Gated Multimodal Fusion

Explore similar work

CXRMate-2: Structured Multimodal Temporal Embeddings and Tractable Reinforcement Learning for Clinically Acceptable Chest X-ray Radiology Report Generation

CPVB-RG: Cross-Prior Visual Bridging for Prior-Aware Chest X-Ray Report Generation

CARE-X: Towards Clinically Useful Radiology VLMs with Auxiliary Supervision, Reward-Aligned Learning, and Tool-Augmented Measurement