Organizations: National and Local Joint Engineering Laboratory of Computer Aided Design, School of Software Engineering, Dalian University, Dalian, China · Department of Surgery, Affiliated Zhongshan Hospital of Dalian University, Dalian, China · The Chinese University of Hong Kong, Shenzhen, China · University of California, San Francisco, USA · School of Computer Science and Technology, University of Science and Technology of China, Hefei, China · Yale University, New Haven, USA · The Hong Kong Polytechnic University, Hong Kong, China
Chest radiograph report generation is hampered by retrieved reports that resemble the current image but are clinically inconsistent with it, and by diagnostic-label imbalance. We present RG-MVR, a retrieval-grounded framework comprising Retrieval-Augmented Evidence Grounding, which combines dense memory summaries with visually gated sparse retrieved evidence; Adaptive Multi-view Evidential Reasoning, which weights image, memory, and retrieved evidence per case; and a Clinical Prompt Learner, which models finding mention separately from positive, negative, or uncertain status. On MIMIC-CXR, RG-MVR achieved a clinical-label F1 of 0.483 and a BLEU-4 of 0.118 (IU X-ray: 0.202 and 0.110). In an exploratory external cohort of 22 examinations, clinical-label agreement declined markedly (F1, 0.071) and unsupported statements remained common, although physicians found fewer unsupported temporal comparisons (9 vs. 20 cases) and device-related statements (7 vs. 15) than with the baseline. Evidence-aware retrieval and structured diagnostic prompting may reduce specific reporting errors, but broader external validation is required.
Figures & tables
Figure 1: Overview of the proposed RG-MVR framework. The model combines Retrieval-Augmented Evidence Grounding (RA-EG), which summarizes dense memory and denoises sparse retrieved evidence; Adaptive Multi-view Evidential Reasoning (AMER), which dynamically fuses visual, dense-memory, and sparse-evidence representations; and the Clinical Prompt Learner (CPL), which converts predicted clinical states into decoder prompt tokens.
MIMIC-CXR [ 10 ]
IU X-ray [ 11 ]
Patients
60,547 (this split)
Not reported
Studies
227,835
3,955 (reports)
Images
377,110
7,470
Site
Beth Israel Deaconess Medical Center (single center)
Indiana Network for Patient Care (multi-institutional)
Collection period
2011–2016
Not reported
Sex distribution (this split)
47.7% male / 52.3% female [n=57,270]
Not reported
Table 1: Cohort characteristics and evaluation splits. MIMIC-CXR [ 10 ] provides the primary training, validation, and test benchmark, whereas IU X-ray [ 11 ] is used only for external cross-dataset validation. Demographic fields are available only for the linked MIMIC-CXR subset and are not reported for IU X-ray.
Figure 2: Qualitative results on the MIMIC-CXR dataset. Matches with the ground truth are highlighted in the same colors, while inconsistencies are marked in black.
Data
Model
NLG Metrics
CE Metrics
MIMIC-CXR
BLEU-1
BLEU-2
BLEU-3
BLEU-4
ROUGE-L
METEOR
CIDEr
Precision
Recall
F1
Early Image Captioning Methods
ST [ 14 ]
0.299
0.184
0.121
0.084
0.263
0.124
-
0.249
0.203
0.204
Att2In [ 15 ]
0.325
0.203
0.136
0.096
0.276
0.134
-
0.322
0.239
0.249
AdaAtt [ 16 ]
0.299
0.185
0.124
0.088
0.266
0.118
-
0.268
0.186
0.181
TopDown [ 17 ]
0.317
0.195
0.130
0.092
0.267
-
-
0.320
0.231
0.238
Table 2: Comparison with published report-generation methods on MIMIC-CXR [ 10 ] and IU X-ray [ 11 ] . On MIMIC-CXR, RG-MVR achieves the best or tied-best results on most metrics, including all clinical-efficacy metrics. On IU X-ray, RG-MVR leads on BLEU-1 to BLEU-4 and ROUGE-L, whereas its clinical-efficacy scores remain below those of REVTAF and PromptMRG. NLG, natural language generation; CE, clinical efficacy (CheXbert-based precision, recall, and F1). Bold values indicate the best value within each dataset block; dashes indicate metrics not reported in the original publications.
Model
Samples
BLEU-1
BLEU-2
BLEU-3
BLEU-4
METEOR
ROUGE-L
Precision
Recall
F1
PromptMRG (re-evaluated)
3,858
0.387
0.230
0.147
0.100
0.147
0.261
0.506
0.466
0.455
RG-MVR
3,858
0.412
0.251
0.166
0.118
0.161
0.276
0.508
0.518
0.483
Table 3: Re-evaluation of the PromptMRG baseline and RG-MVR on the MIMIC-CXR test set (3,858 samples) using an identical metric pipeline. MIMIC-CXR served as the primary quantitative benchmark; the external DR cohort was reserved for generalization and safety stress testing.
Setting
RA-EG
AMER
CPL
NLG Metrics
CE Metrics
BLEU-1
BLEU-2
BLEU-3
BLEU-4
METEOR
ROUGE-L
Precision
Recall
F1
Hallucination
FPR
Base
0.399
0.243
0.161
0.114
0.160
0.273
0.487
0.490
0.460
41.5%
8.6%
(a)
✓
0.402
0.245
0.162
0.114
0.158
0.274
0.473
0.547
0.477
46.0%
10.8%
(b)
✓
0.405
0.247
0.164
0.115
0.158
0.274
0.480
0.550
0.482
44.7%
10.5%
(c)
✓
0.405
0.247
0.164
0.115
0.158
0.274
0.469
0.532
0.467
43.9%
10.3%
Ours
✓
✓
✓
0.412
0.251
0.166
0.118
0.161
0.276
0.508
0.518
0.483
39.1%
8.1%
Table 4: Single-component ablation on the MIMIC-CXR [ 10 ] test set. RA-EG and AMER each improve clinical-label recall and F1 over the base model but increase hallucination and false-positive rates when used alone; AMER gives the strongest single-component clinical-label result, and CPL a smaller improvement. Combining RA-EG, AMER, and CPL gives the best overall balance, improving clinical-label agreement while reducing hallucination and false-positive rates relative to the base model. FPR, false-positive rate.
Setting
e−ηlm
e−ηcls
NLG Metrics
CE Metrics
BLEU-1
BLEU-2
BLEU-3
BLEU-4
METEOR
ROUGE-L
Precision
Recall
F1
Hallucination
FPR
LLM+LCLS
0.404
0.246
0.164
0.116
0.156
0.270
0.469
0.527
0.467
48.1%
10.7%
(a)
✓
0.400
0.243
0.160
0.112
0.153
0.267
0.469
0.517
0.462
46.7%
10.4%
(b)
✓
0.402
0.245
0.162
0.114
0.158
0.271
0.440
0.489
0.434
46.8%
10.8%
Ours
✓
✓
0.412
0.251
0.166
0.118
0.161
0.276
0.508
0.518
0.483
39.1%
8.1%
Table 5: Ablation of the learnable precision-weighted multi-task objective on MIMIC-CXR [ 10 ] . Optimizing either task weight alone provides limited or uneven benefits, and classification-only weighting reduces clinical-label performance. Jointly learning the language-modeling and classification task weights yields the strongest overall language and clinical-label performance while reducing hallucination and false-positive burden.
Characteristic
Cases
Rate
Total cases
22
22/22 (100.0%)
Single image per case
11
11/22 (50.0%)
Multi-image per case
11
11/22 (50.0%)
Nodule or focal dense lesion
7
7/22 (31.8%)
Pleural or thoracic cavity abnormality
21
21/22 (95.5%)
Postoperative cases
4
4/22 (18.2%)
Table 6: Composition of the external DR cohort used for generalization and safety stress testing. The cohort includes both single-image and multi-image records, as well as cases with focal lesions, pleural or thoracic-cavity abnormalities, postoperative findings, support devices, and aortic calcification or tortuosity. These cohort descriptors are descriptive and are not population estimates.
Model
Samples
BLEU-1
BLEU-4
ROUGE-L
CIDEr
Precision
Recall
F1
FPR micro
Hallucination micro
Any FP
Base
22
0.2379
0.0000
0.1722
0.0305
0.0909
0.1212
0.0985
0.1719
0.9245
0.9091
RG-MVR
22
0.2266
0.0150
0.1664
0.0273
0.0833
0.0833
0.0712
0.0737
0.8750
0.4091
RG-MVR without retrieval
22
0.2134
0.0000
0.1696
0.0107
0.0682
0.0379
0.0485
0.0737
0.9130
0.6818
RG-MVR oracle states
22
0.2630
0.0223
0.1753
0.0397
0.5000
0.4697
0.4621
0.0175
0.2632
0.2273
Table 7: Automatic metrics on the external DR stress test. All standard-inference conditions show substantial cross-domain degradation and a high hallucination burden. Supplying oracle diagnostic states derived from the reference reports markedly improves clinical-label agreement and reduces hallucination, suggesting a bottleneck in disease-state prediction; this condition is a mechanistic upper bound rather than a deployable inference setting. FPR, false-positive rate; FP, false positive.
Model
Samples
Simple F1
Partial F1
Complete F1
Base
22
0.1115
0.0958
0.0579
RG-MVR
22
0.1424
0.1299
0.0819
RG-MVR without retrieval
22
0.1067
0.0917
0.0475
RG-MVR oracle states
22
0.1481
0.1331
0.0798
Table 8: RadGraph semantic robustness on the external DR cohort. Complete F1, which requires entity, label, and relation agreement, remains low across all inference conditions. RG-MVR improves strict semantic overlap relative to Base and no-retrieval inference, whereas oracle-state prompting improves CheXbert agreement without a comparable improvement in strict entity–relation overlap.
Error mode
Cases
Rate
Any hallucinated or wrong content
13
13/22 (59.1%)
Unsupported prior-comparison hallucination
8
8/22 (36.4%)
Device-related error
3
3/22 (13.6%)
Postoperative-status error
2
2/22 (9.1%)
Location or laterality error
3
3/22 (13.6%)
Missed major finding
15
15/22 (68.2%)
Table 9: Error modes identified in an earlier exploratory clinician annotation of the external DR cohort. This annotation set is separate from the paired two-physician assessment (Tables 10 and 11 ); the counts were obtained with different procedures and should not be compared directly.
Table 10: Descriptive results of the two-physician review of the 22-case external DR cohort. Counts represent non-empty annotation fields or keyword mentions, not adjudicated clinical error rates.
Endpoint
Base (n=22)
RG-MVR (n=22)
Difference (pp)
P value
Any incorrect or unsupported statement
22 (100.0%)
20 (90.9%)
−9.1
0.500
Unsupported prior/temporal comparison
20 (90.9%)
9 (40.9%)
−50.0
0.001
Unsupported device/procedure-related statement
15 (68.2%)
7 (31.8%)
−36.4
0.021
Table 11: Two-physician assessment of report-level error endpoints in the external DR cohort. Values are the number of cases with each endpoint (percentage); the difference is RG-MVR minus Base in percentage points.
Retrieval-augmented generation is an attractive way to improve chest X-ray reporting, because reports from similar prior studies supply clinical context a general vision-language model lacks. We show the same mechanism is a reliable source of clinical error. Over 100 MIMIC-CXR studies with a frozen LLaVA-1.5 generator and BioMedCLIP retrieval, CheXbert clinical F1 doubles under relevant retrieval (0.201 to 0.402) and collapses to 0.043 under clinically mismatched retrieval, a fifth of the image-only score; the retrieval-induced hallucination rate, counting only unsupported findings traceable to retrieved evidence, rises from 0.00 to 0.76 and 0.98. To show this is not an artefact of evidence-set coverage, we introduce a coincidental-overlap control that scores image-only generations against evidence they never saw, placing the chance base rate at 0.18, four to five times below the observed rates. The generator does not merely acquire findings, it transcribes text: 95% of reports produced under relevant retrieval contain an eight-word span occurring verbatim in the retrieved evidence but absent from the reference, against 0% without retrieval. We then explain the mechanism: a normal chest X-ray retrieves at least one abnormal precedent in 24 of 28 cases, because medical image-embedding similarity is dominated by anatomy and acquisition rather than by the presence of disease. This has a direct design consequence. Retrieval similarity does not predict harm (RIH rates 0.80/0.80/0.60/0.84 across similarity quartiles), so relevance gates conditioned on embedding similarity cannot work; gating on predicted pathology agreement removes 61% of unsupported evidence at no cost to useful coverage. We argue that retrieval-augmented clinical systems must be evaluated under retrieval failure, not only under retrieval success.
Emmanuel Idoko, Abdusshakur Olabisi, Shiloh Oni +1
University of Lagos · Machine Learning Collective · NITHUB, University of Lagos +2
Radiology reports are essential for diagnostic reasoning and patient care, yet their manual preparation is time-consuming and cognitively demanding. Automatic radiology report generation (RRG) offers a scalable alternative, but existing models often produce lengthy, unstructured narratives that overlook diagnostic cues and multi-view information. We present Clinical-Knowledge Guided Disease-aware Visual-Textual Alignment for QA-Style Radiology Report Generation (CliQ-RRG), a unified two-stage framework for interpretable and clinically grounded reporting. In Stage 1, CliQ-RRG employs a Disease-aware Visual-Textual Alignment module that aligns image and text representations using predicted disease embeddings, reinforced by a Prior-Guided Attention Module (PrAM) to capture multi-view dependencies across current and prior scans. In Stage 2, GraphRAG-guided retrieval augments intermediate textual representations with graph-grounded clinical evidence, and a large language model restructures them into concise, interpretable question-answer (QA) pairs with diagnostic summaries. Experiments on two public chest X-ray benchmarks demonstrate that CliQ-RRG consistently outperforms prior methods across both lexical and clinical metrics, generating accurate and clinically coherent QA-style radiology reports. Code is available at: https://github.com/chandravardhan-singh-raghaw/CliQ-RRG.
Automated radiology report generation has gained increasing attention with the rise of deep learning and large language models. However, fully generative approaches often suffer from hallucinations and lack clinical grounding, limiting their reliability in real-world workflows. In this study, we propose a multimodal retrieval-augmented generation (RAG) system for grounded drafting of chest radiograph impressions. The system combines contrastive image-text embeddings, case-based similarity retrieval, and citation-constrained draft generation to ensure factual alignment with historical radiology reports. A curated subset of the MIMIC-CXR dataset was used to construct a multimodal retrieval database. Image embeddings were generated using CLIP encoders, while textual embeddings were derived from structured impression sections. A fusion similarity framework was implemented using FAISS indexing for scalable nearest-neighbor retrieval. Retrieved cases were used to construct grounded prompts for draft impression generation, with safety mechanisms enforcing citation coverage and confidence-based refusal. Experimental results demonstrate that multimodal fusion significantly improves retrieval performance compared to image-only retrieval, achieving Recall@5 above 0.95 on clinically relevant findings. The grounded drafting pipeline produces interpretable outputs with explicit citation traceability, enabling improved trustworthiness compared to conventional generative approaches. This work highlights the potential of retrieval-augmented multimodal systems for reliable clinical decision support and radiology workflow augmentation