Organizations: School of Computer Science and Technology, Dalian University of Technology, No. 2 Linggong Road, Ganjingzi District, Dalian, 116024, Liaoning, China. · Department of Nuclear Medicine, The Second Hospital of Dalian Medical University, No. 467 Zhongshan Road, Shahekou District, Dalian, 116023, Liaoning, China.
Automated radiology report generation can draft clinically plausible reports, yet the pathway from visual evidence to reported findings remains opaque, leaving the diagnostic basis of these reports unexplained and the generated reports difficult to review or revise. We introduce ARIVE (Anatomy-guided Reasoning with Interleaved Visual Evidence), an explainable report generation framework that interleaves anatomical inspection with diagnostic reasoning prior to final report composition. To support this process, we constructed 159,416 multimodal chain-of-thought (MCoT) trajectories from MIMIC-CXR, wherein visual evidence from inspected anatomical regions informs subsequent clinical interpretation. Trained on these trajectories and further optimized using anatomical and clinical-content rewards, ARIVE achieved superior clinical-efficacy performance compared with strong baselines on MIMIC-CXR and the abnormality-centered MIMIC-ABN. ARIVE's reports were competitive with those of leading models in blinded expert grading, and its MCoT trajectories earned high ratings for diagnostic traceability. Furthermore, a paired review evaluation suggests that access to MCoT can aid in identifying errors within generated reports.
Figures & tables
Figure 1: ARIVE workflow for dataset construction, model training and report generation. Step 1 (bottom), radiologist-style diagnostic templates guide GPT-OSS-120B in converting MIMIC-CXR reports into textual reasoning trajectories. Chest ImaGenome annotations ground report-relevant steps in 29 anatomical regions to construct the multimodal chain-of-thought (MCoT) dataset. Step 2 (left), supervised fine-tuning (SFT) is followed by Anatomical and Clinical-Content Optimization (ACCO). Four rewards assess anatomical grounding, observation accuracy in intermediate reasoning and final reports, and clinical entity–relation agreement with reference reports. Step 3 (top and right), ARIVE interleaves anatomical inspection with diagnostic reasoning before generating the final report. CXB and RG denote CheXbert and RadGraph, respectively.
Figure 2: Clinical efficacy and anatomical grounding on MIMIC-CXR. (a)–(c), Comparison of ARIVE and six comparators using observation-wise CheXbert F1 (a), RadGraph relation and entity F1 (b), and GREEN (c). In (a), inner and outer tick labels show observation-specific F1 axis limits (polygon area does not indicate overall performance), and annotations give ARIVE’s scores. (d)–(f), Comparison of SFT, SFT w/ CoT, SFT w/ MCoT and ARIVE using CheXbert F1 changes from SFT in percentage points (d), RadGraph F1 (e), and GREEN (f). (g), Region-selection F1 (bars) and mean intersection over union (mIoU; lines) across all ground-truth regions in 636 Chest ImaGenome Gold examples. Error bars in (b,c,e,f) denote 95% confidence intervals. Asterisks indicate significantly higher scores for ARIVE in comparisons with the indicated models or training settings: ∗p<0.05 ; ∗∗p<0.01 .
MIMIC-ABN test set
Chest ImaGenome Gold subset
Training setting
F1,14ans (mic/mac)
F1-SRR-BERT
RG-F1
Region- selection F1
mIoU
F1,14rea (mic/mac)
F1,14ans (mic/mac)
F1,14con (mic/mac)
Supervised fine-tuning comparison
SFT w/ CoT
0.502/0.374
0.394
0.183
0.579
0.430
0.369/0.310
0.455/0.374
0.714/0.728
SFT w/ MCoT
0.514/0.377
0.410
0.188
0.605
0.474
0.463/0.413
0.472/0.411
0.902/0.894
ACCO ablations
ARIVE
0 . 5 4 5 / 0 . 4 0 8
0 . 4 4 1
0.207
0.603
0.474
0 . 5 1 3 / 0 . 4 5 1
0 . 5 0 5 / 0 . 4 4 4
0 . 9 4 5 /0.887
Table 3: MCoT supervision and ACCO component ablations on MIMIC-ABN and its 142-example Chest ImaGenome Gold subset. CheXbert F1,14 values are micro/macro averages across 14 observations; superscripts rea , ans and con denote reasoning-stage content, final-report content and reasoning–report consistency, respectively. Bold marks the highest value for each metric across all settings; micro/macro maxima are identified separately.
Figure 3: Report-quality grading and observation overlap with reference reports. (a), Blinded grade distributions assigned by the radiologist for 100 randomly selected cases from 71 patients. Scores range from 1 (unusable) to 5 (directly usable). (b), Mean report-quality scores with 95% confidence intervals. (c), Paired scores for ARIVE versus PriorRG, Qwen3.5 and reference reports, from left to right. Cells show case counts and dashed diagonals indicate equal scores. (d), Observation overlap between generated and reference reports. Left, ARIVE; center, Qwen3.5; right, PriorRG. Numbers indicate observations shared by the generated and reference reports or present in only one.
Figure 4: MCoT trajectory quality, reasoning–report consistency and report review. (a), Grade distributions assigned by the radiologist across four domains for 100 randomly selected cases from 80 patients. Scores range from 1 (lowest) to 5 (highest). (b), Mean MCoT scores with 95% confidence intervals. (c), Bidirectional reasoning–report consistency. Left, support from MCoT trajectories for report claims ( n=597 ; positive, negative or uncertain). Right, retention of MCoT findings in the final reports ( n=449 ; positive or uncertain). (d), Number of reports judged to contain errors during paired review without and with MCoT (8 and 11 of 100 reports, respectively). (e), Review time per report under each condition. Points show individual review times and shaded curves show kernel density estimates. Boxes span the interquartile range (IQR), with medians marked. Whiskers extend to the most extreme observations within 1.5 IQR of the box edges.
Figure 5: Final-report comparison in three selected MIMIC-CXR cases. Each column presents the radiograph, reference report and reports generated by ARIVE, PriorRG and Qwen3.5. Blue and vermilion highlights indicate concordant and unsupported findings, respectively. In these examples, ARIVE captures the principal findings and key negative observations while avoiding several unsupported or overstated findings produced by the comparators.
Figure 6: Inspectable ARIVE trajectories in two selected MIMIC-CXR cases. Each case pairs the input radiograph and predicted anatomical boxes with the requested regions, MCoT trajectory, generated report and reference report. Case 1 shows a trajectory broadly concordant with the reference report. Case 2 traces an unsupported claim of mild cardiomegaly from intermediate reasoning into the final report.
Figure 7: Interleaved anatomical inspection and diagnostic reasoning in ARIVE. During generation, ARIVE inspects selected anatomical regions and inserts their visual representations into the model context to guide subsequent reasoning.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Statistic
Train
Validation
Test
Samples
159,416
1,286
2,463
Studies
142,946
1,151
2,211
MIMIC-CXR
Patients
57,729
448
285
Samples
34,259
302
432
Studies
34,152
263
378
MIMIC-ABN
Patients
23,039
181
134
Appendix
Supplementary Table S3: Sample, study and patient counts after frontal-view filtering and restriction of the training splits to Chest ImaGenome Silver overlap.
MIMIC-CXR
MIMIC-ABN
Anatomical Region
Train
Gold subset
Train
Gold subset
Right lung
152,734
546
33,492
116
Right upper lung zone
4,757
46
1,015
12
Right mid lung zone
7,654
56
2,072
8
Right lower lung zone
31,442
174
6,706
35
Right hilar structures
48,140
343
11,108
81
Appendix
Supplementary Table S4: Anatomical bounding-box annotation counts in the training splits and Chest ImaGenome Gold subsets of MIMIC-CXR and MIMIC-ABN.
MIMIC-CXR
MIMIC-ABN
Observation
Train
Validation
Test
Train
Validation
Test
Enlarged Cardiomediastinum
29,449
253
687
2,624
27
54
Cardiomegaly
43,528
333
1,047
7,077
58
131
Lung Opacity
42,613
322
949
10,854
91
205
Lung Lesion
6,577
74
165
2,130
25
53
Edema
21,952
186
638
2,488
27
68
Appendix
Supplementary Table S5: CheXbert positive-label counts in the training, validation and test splits of MIMIC-CXR and MIMIC-ABN.
Characteristic
Mean
Min
Max
Reasoning text length (words)
399.98
118
823
Region-grounding instances per trajectory
5.75
0
15
Appendix
Supplementary Table S6: Reasoning length and anatomical grounding of MCoT training trajectories. 0 0 footnotetext: Statistics summarize n=159,416 training trajectories. Word counts include only prose within <think> , after removing complete anatomical grounding blocks (anatomical labels, bounding-box coordinates and <image> markers) and other special markers. MCoT, multimodal chain-of-thought.
Average
Precision
Recall
F1
Micro
0.9160
0.9194
0.9177
Macro
0.9110
0.9009
0.9056
Appendix
Supplementary Table S7: Reasoning CheXbert performance on the held-out split of the MCoT training set. Uncertain labels are treated as positive for metric computation.
Model
Parameters (B)
Decoding
Output
Mean time (s/sample)
MLRG
0.30
Beam search (3)
Report
0.51
PriorRG
0.34
Beam search (3)
Report
0.32
MedGemma-1.5
4.30
Greedy
Report
1.30
ARIVE
4.85
Sampling
MCoT + report
1.13
MAIRA-2
6.88
Greedy
Report
1.20
Libra
6.89
Beam search (10)
Report
15.03
Appendix
Supplementary Table S8: Model size, decoding strategy and mean inference time on the MIMIC-CXR test set.
Textual CoT construction
You are an expert chest X-ray radiologist. Produce a complete, optimized first-person reasoning narrative for the current CXR.
The examples define only the desired narrative voice, regional progression, paragraph development, and reasoning depth. Never transfer their clinical findings or routine normal-structure checklist to the current case.
Supplementary Table S9: Prompt for textual CoT construction. The source report supplies the current findings; few-shot examples specify the narrative style.
Bounding-box insertion
You are an expert radiologist. Insert relevant bounding boxes from the Anatomical Information into the Generated Reasoning.
Rules: - Preserve the original content and wording of the Generated Reasoning.
- Insert only bounding boxes whose anatomical regions are explicitly mentioned or directly implied in the reasoning.
- Insert each bounding box at most once, immediately before the sentence describing its corresponding region.
- Never add boxes for irrelevant or unmentioned regions.
Appendix
Supplementary Table S10: Prompt for inserting anatomical bounding boxes into textual CoT trajectories to construct MCoT.
MIMIC-CXR report generation
You are an assistant in radiology, responsible for analyzing medical imaging studies and generating detailed, structured, and accurate radiology reports.
[Indication: {INDICATION} ]
[[ Prior chest X-ray: <image> Prior findings: {PRIOR_CXR_REPORT} ]] Current chest X-ray: <image> Before answering, think step by step internally. In your internal thinking, output the bounding box of the key diagnostic region in the format <|object_ref_start|> anatomical object name <|object_ref_end|><|box_start|> x1 y1 x2 y2 <|box_end|> . Generate a comprehensive and detailed description of findings based on the current chest X-ray image and the indication. [[ Include a thorough comparison with a prior chest X-ray, emphasizing any significant changes, progression, or improvement. ]]
Appendix
Supplementary Table S11: Prompt for MIMIC-CXR report generation, covering the full findings description. The [...] block is included only when an indication is available; [[...]] blocks are included only when a prior study is available.
MIMIC-ABN report generation
You are an assistant in radiology, responsible for analyzing medical imaging studies and generating detailed, structured, and accurate radiology reports.
[Indication: {INDICATION} ]
[[ Prior chest X-ray: <image> Prior abnormal findings: {PRIOR_ABNORMAL_FINDINGS} ]] Current chest X-ray: <image> Before answering, think step by step internally. In your internal thinking, output the bounding box of the key diagnostic region in the format <|object_ref_start|> anatomical object name <|object_ref_end|><|box_start|> x1 y1 x2 y2 <|box_end|> . Generate a comprehensive and detailed description of abnormal findings only based on the current chest X-ray image and the indication. [[ Include a thorough comparison with a prior chest X-ray, emphasizing any significant changes, progression, or improvement. ]]
Appendix
Supplementary Table S12: Prompt for MIMIC-ABN report generation, restricted to abnormal findings. The [...] block is included only when an indication is available; [[...]] blocks are included only when a prior study is available.
Radiology report generation (RRG) streamlines clinical workflows and alleviates the burden of radiologists' documentation. Standard vision-language models capture global visual cues and lack fine-grained alignment between anatomical regions and diagnostic text. Most retrieval-augmented methods retrieve complete historical reports, introducing redundant noise and severe vision-text mismatches that trigger clinically risky hallucinations and missed lesions. In this paper, we introduce AR-RRG, an anatomy-guided retrieval-driven method for the generation of radiology reports. AR-RRG consists of two core modules: the anatomy-aware self-training lesion detection network (ASLDNet) and the retrieval-driven radiology report generation (RRRG). ASLDNet leverages anatomy-aware spatial verification (ASV) and class-adaptive dynamic thresholding (CDT) via self-training, delivering accurate lesion localization to support subsequent generation. As the generative branch of AR-RRG, RRRG eliminates clinical hallucinations through dual-granularity retrieval (DR) and complements missing lesions with a neighborhood consensus (NC) mechanism. A parallel continuous visual perception (PCVP) stream converts radiographs into visual tokens through fixed medical encoding and resampling. Multi-scale text and visual features are fused, followed by LoRA-efficient large language model tuning to generate clinically consistent, visually grounded radiology reports. Extensive evaluations on MS-CXR, MIMIC-CXR, and IU X-Ray demonstrate that AR-RRG consistently outperforms state-of-the-art methods across mainstream metrics. Our code will be made publicly available.
Xiang Zhang, Shijie Song, Wanqing Zhao +8
the School of Computer Science, Northwest University, Shaanxi, China · the School of Electronic Information, Northwest University, Shaanxi, China · the School of Art, Northwest University, Shaanxi, China
Automated radiology report generation holds immense potential to alleviate the heavy workload of radiologists. Despite the formidable vision-language capabilities of recent Multimodal Large Language Models (MLLMs), their clinical deployment is severely constrained by inherent limitations: their "black-box" decision-making renders the generated reports untraceable due to the lack of explicit visual evidence to support the diagnosis, and they struggle to access external domain knowledge. To address these challenges, we propose the Evidence-driven Radiology Report Generation Agent (EviAgent). Unlike opaque end-to-end paradigms, EviAgent coordinates a transparent reasoning trajectory by breaking down the complex generation process into granular operational units. We integrate multi-dimensional visual experts and retrieval mechanisms as external support modules, endowing the system with explicit visual evidence and high-quality clinical priors. Extensive experiments on MIMIC-CXR, CheXpert Plus, and IU-Xray datasets demonstrate that EviAgent outperforms both large-scale generalist models and specialized medical models, providing a robust and trustworthy solution for automated radiology report generation.
Tuoshi Qi, Shenshen Bu, Yingfei Xiang +1
School of Computer Science and Engineering, Sun Yat-sen University · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences · Sangfor Technologies, Shenzhen, Guangdong, China
Recent advancements in Large Vision-Language Models (LVLM) enhance X-ray report generation, but often neglect clinical context, overlook workflow integration, and lack explainability, which hinders clinical adoption due to interpretability and reliability concerns. This paper addresses the dual challenges of performance and expert explainability in automated X-ray report generation for clinical AI applications. Our proposed approach, TransXplainRRG, utilises an off-the-shelf Swin Transformer model along with a transformer-based text encoder that incorporates patient medical history to generate a radiology report. Further, we explore an expert-guided `inside-out' approach and extract only abnormal findings for radiology report refinement. Thus, this study bridges the gap between high-performance automation and interpretability critical for clinical practice by combining state-of-the-art transformer-based vision encoders, text encoders, and LLMs. TransXplainRRG is trained and tested on the large-scale MIMIC-CXR dataset and further evaluated on the unseen IU X-Ray dataset to demonstrate its generalizability and robustness. We employed the GREEN metric as a complementary measure of clinical consistency, achieving scores of 0.327 on MIMIC-CXR and 0.605 on IU X-Ray, both higher than those achieved by MAIRA-2 under the same evaluation setting. It also performed competitively across standard evaluation metrics, frequently matching or surpassing state-of-the-art models. Additionally, we introduce qualitative evaluation metrics developed from radiologists' viewpoints and conduct a preliminary radiologist evaluation on 100 generated reports to assess their clinical relevance. Quantitative and qualitative analyses further demonstrate the potential of our TransXplainRRG framework for reliable and explainable radiology report generation.
Chayan Mondal, Duc-Son Pham, Tele Tan +4
Curtin University, Bentley, Western Australia · Fiona Stanley Hospital, Murdoch, Western Australia