HERO: Hierarchical Evidential Reasoning Optimization for Radiology Report Generation via Reason-then-Summarize

Kun Zhao; Guodong Liu; Hui Ji; Siyuan Dai; Pan Wang; Jifeng Song; Heng Huang; Chenghua Lin

cs.LGJan 6, 2026

HERO: Hierarchical Evidential Reasoning Optimization for Radiology Report Generation via Reason-then-Summarize

Authors: Kun Zhao, Guodong Liu, Hui Ji, Siyuan Dai, Pan Wang, Jifeng Song, Heng Huang, Chenghua Lin, +2 more

Organizations: University of Pittsburgh · Eli and Lilly Company · The University of Manchester · University of Texas Rio Grande Valley

Abstract

Multimodal Large Language Models (MLLMs) have substantially advanced Radiology Report Generation (RRG), yet aligning them through reinforcement learning (RL) remains challenging due to heterogeneous medical supervision. Vanilla Group Relative Policy Optimization (GRPO) assigns uniform credit across the entire generation, leading to segment interference, token dilution, and rationale-diagnosis decoupling, which exacerbates clinical hallucinations. We propose HERO (Hierarchical rationalE Reasoning Optimization), a factorized policy optimization framework that aligns heterogeneous supervision with three optimization granularities. HERO separately optimizes reasoning, diagnosis, and rationale faithfulness through complementary segment-, token-, and completion-level optimization with a heterogeneous reward formulation covering diagnostic accuracy, reasoning quality, and think-answer consistency. Experiments on MIMIC-CXR and IU-Xray show that HERO outperforms strong supervised and reinforcement learning baselines, achieving state-of-the-art clinical efficacy while producing more rationale-supported and think-answer-consistent reports, thereby substantially mitigating clinical hallucinations.

Figures & tables

Figure 1: Overview of the HERO framework. During policy rollout, completions are generated as <think> and <answer> segments, with conditional GT injection to break exploration deadlocks. The model is evaluated by a heterogeneous Reward Bank ( $R_{1}$ - $R_{9}$ ). To optimize the policy, HERO factorizes the GRPO objective across three granularities: (1) Segment-level: routes rewards to estimate separate advantages for reasoning and diagnosis; (2) Token-level: allocates larger gradient weights to diagnostically salient tokens; and (3) Completion-level: applies an rationale-supported gate to scale the surrogate loss based on think-answer consistency. These factors are jointly optimized in a unified training objective.

	MIMIC-CXR
	NLG Metrics					Clinical Efficacy
	B-4	MTR	RG-L	BERTScore	SRR-BERT	RadGraph	RadCliQ $\downarrow$	RateScore	GREEN	Micro-F1(14)	avg-Macro-F1(14)	w-Macro-F1(14)	Micro-F1(5)	avg-Macro-F1(5)	w-Macro-F1(5)
Llava-1.5	0.102	0.323	0.353	0.611	0.501	0.320	1.358	0.637	0.498	0.550	0.408	0.533	0.593	0.516	0.583
Llava-Next(vicuna)	0.054	0.165	0.263	0.438	0.463	0.265	1.245	0.617	0.503	0.408	0.330	0.413	0.419	0.392	0.445
Llava-Next(mistral)	0.102	0.325	0.350	0.609	0.505	0.314	1.326	0.636	0.493	0.544	0.412	0.525	0.574	0.512	0.567
Llava-Rad	0.104	0.322	0.357	0.612	0.494	0.320	1.369	0.640	0.495	0.553	0.411	0.527	0.588	0.512	0.576

Table 1: Report-level MIMIC-CXR results, all methods re-scored under one unified protocol. NLG (left) and clinical-efficacy (right) metrics; RadCliQ (

\downarrow

) is lower-is-better, all others higher-is-better. Bold best, underline runner-up per column. DPO (Our rewards) is an optimizer-swap ablation under our reward.

	IU-Xray
	NLG Metrics					Clinical Efficacy
	B-4	MTR	RG-L	BERTScore	SRR-BERT	RadGraph	RadCliQ $\downarrow$	RateScore	GREEN	Micro-F1(14)	avg-Macro-F1(14)	w-Macro-F1(14)	Micro-F1(5)	avg-Macro-F1(5)	w-Macro-F1(5)
Llava-1.5	0.072	0.293	0.332	0.597	0.483	0.310	1.073	0.613	0.628	0.387	0.076	0.247	0.033	0.019	0.030
Llava-Next(vicuna)	0.039	0.151	0.167	0.388	0.494	0.315	0.647	0.646	0.622	0.164	0.091	0.217	0.113	0.099	0.120
Llava-Next(mistral)	0.078	0.294	0.344	0.548	0.463	0.308	1.167	0.577	0.482	0.329	0.047	0.127	0.104	0.012	0.078
Llava-Rad	0.096	0.320	0.365	0.612	0.495	0.363	1.462	0.655	0.587	0.388	0.089	0.260	0.147	0.100	0.139

Table 2: Report-level IU-Xray results, all methods re-scored under one unified protocol. NLG (left) and clinical-efficacy (right) metrics; RadCliQ (

\downarrow

) is lower-is-better, all others higher-is-better. Bold best, underline runner-up per column. DPO (Our rewards) is an optimizer-swap ablation under our reward.

Answer-Level: Clinical Efficacy
	L15	LNv	LNm	Q25	Q35	InVL	ESC	MPO	Edit	DPO	Ours
Micro-F1(14)	0.580	0.584	0.592	0.535	0.497	0.584	0.554	0.577	0.572	0.390	0.607
avg-Macro-F1(14)	0.407	0.398	0.420	0.363	0.347	0.364	0.382	0.361	0.367	0.275	0.412
w-Macro-F1(14)	0.527	0.565	0.521	0.549	0.488	0.545	0.557	0.544	0.542	0.381	0.575
Micro-F1(5)	0.596	0.585	0.583	0.559	0.473	0.481	0.571	0.564	0.560	0.469	0.618
avg-Macro-F1(5)	0.520	0.496	0.518	0.473	0.451	0.466	0.481	0.502	0.478	0.448	0.525

Table 3: Answer-level clinical efficacy (answer JSON vs. GT, ignore convention), think–answer self-consistency (SCS, GT-free), and evidence metrics on MIMIC-CXR (

N{=}2047

, greedy). SCS and

R_{6}^{\text{gt}}

coincide with our reward terms

R_{5}

and

R_{6}

and are thus optimized by our method. L15 = LLaVA-1.5, LNv/LNm = LLaVA-Next (Vicuna/Mistral), Q25 = Qwen-2.5-VL-7B (SFT on our data), Q35 = Qwen-3.5-122B, InVL = InternVL-241B, ESC = ESC-RL, Edit = EditGRPO, DPO = DPO under our reward (answer JSON parseable for 36% of studies). Bold best, underline runner-up per row.

	Cumulative				Ablation
Metric	Base	+L1	+L2	Full	$-$ L1	$-$ L2	$-$ Gate	L3o	L2o
NLG Metrics
B-4	0.106	0.111	0.107	0.102	0.119	0.103	0.097	0.105	0.105
METEOR	0.329	0.353	0.342	0.334	0.356	0.343	0.329	0.328	0.327
ROUGE-L	0.357	0.347	0.335	0.340	0.369	0.338	0.327	0.356	0.355
BERTScore	0.610	0.606	0.602	0.617	0.618	0.589	0.579	0.612	0.607

Table 4: Ablation of the three-layer framework on MIMIC-CXR (transposed; metrics in rows, variants in columns). Bold best, underline runner-up per metric. Cumulative : Base

\rightarrow

+L1

\rightarrow

+L2

\rightarrow

Full (

=

+L3). Ablations :

-

L1/

-

L2/

-

Gate are leave-one-out from Full; L3o/L2o are single-layer variants.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Answer $a_{k}$	Think mention $\psi_{k}$	GT $g_{k}$	Score $s_{k}$	Interpretation
1	1	1	$+1.0$	True positive correctly identified and explained.
1	0	1	$+0.5$	True positive, but the supporting narrative is missing.
1	1	0	$-0.5$	False positive accompanied by a fabricated explanation.
1	0	0	$-1.0$	Pure false positive without a supporting narrative.
0	0	0	$+1.0$	True negative correctly left unmentioned.
0	1	0	$-1.0$	Mentioned in the reasoning but not claimed in the answer.

Appendix

Table 5: Scoring Matrix: Pathology-level scoring rules.

$C(y_{i})$	$-1.0$	$-0.5$	$-0.25$	$0.0$	$0.25$	$0.5$	$1.0$
$\phi_{\text{evidence}}$	$0.007$	$0.076$	$0.223$	$0.500$	$0.777$	$0.924$	$0.993$

Appendix

Table 6: Gate response

\phi_{\text{evidence}}=\sigma(\gamma(C-\delta))

\gamma{=}5.0

\delta{=}0.0

. The gate is smooth and unsaturated across the attainable score range

C\in[-1,1]

, suppressing completions whose positive findings lack support in think without hard-masking them.

	Clinical Efficacy
	Micro-F1(14)	avg-Macro-F1(14)	w-Macro-F1(14)	Micro-F1(5)	avg-Macro-F1(5)	w-Macro-F1(5)
Llava-1.5 (report)	0.476	0.354	0.462	0.548	0.481	0.557
Llava-1.5 (answer)	0.470	0.341	0.456	0.541	0.473	0.552
Llava-Next(vicuna) (report)	0.362	0.299	0.378	0.391	0.389	0.443
Llava-Next(vicuna) (answer)	0.450	0.332	0.441	0.518	0.475	0.541
Llava-Next(mistral) (report)	0.499	0.384	0.484	0.557	0.456	0.556

Appendix

Table 7: Clinical efficacy on the human-annotation test set (

n{=}500

, greedy). “(report)” = CheXbert labels of the generated report (the <think> section; the full report for Llava-Rad, R2Gen and R2GenCMN) vs GT (ignore-uncertain); “(answer)” = parsed <answer> -JSON labels vs GT (label_f1, ignore-uncertain).

Figure 2: Qualitative comparison on MIMIC-CXR. For each study, rows are models and columns are the disease labels that are positive in the ground truth (bold) or predicted by any model. Cell colour encodes the <answer> label against the ground truth (green: correct positive, red: missed, orange: false positive, grey “?”: uncertain answer, not scored); $\times$ marks a positive answer whose own <think> negates the finding. SDP / SDR = correct diagnoses supported by the think divided by the model’s positive diagnoses / by the ground-truth disease positives (– : empty denominator); support is judged by the CheXpert rule labeler with its negation errors on these studies corrected by manual reading. (a,b) Ours recovers every co-occurring finding; (c,d) baselines contradict their own reasoning; (e) baselines add spurious positives. Labels: CM = Cardiomegaly, LO = Lung Opacity, ED = Edema, AT = Atelectasis, EF = Pleural Effusion, PNA = Pneumonia, PTX = Pneumothorax. Q25 = Qwen-2.5-VL-7B (SFT), L15 = LLaVA-1.5.

Explore similar work

Enhancing Visual Reasoning in Chest X-Ray Report Generation Using Reinforcement Learning
Sep 25, 2026Denis Musinguzi, Andrew Katumba, Prasenjit MitraRadiology Report Generation VLM Hallucination Mitigation
Carnegie Mellon University · Makerere University
Multi-Modal Multi-Agent Reinforcement Learning for Radiology Report Generation
Feb 17, 2026Kaito Baba, Risa Kishikawa, Satoshi KoderaRadiology Report Generation Multimodal Agents
Department of Cardiovascular Medicine The University of Tokyo Hospital, Tokyo, Japan
Graph-Supervised Hierarchical Clinical Alignment for Radiology Report Generation with Large Language Models
Aug 25, 2026Yingshu Li, Yunyi Liu, Zhanyu Wang +4Radiology Report Generation Radiology VLMs
University of Sydney Sydney, Australia · University of Wollongong Wollongong, Australia · University of Adelaide Adelaide, Australia

Sep 25, 2026J/K move · Enter open · S save

Enhancing Visual Reasoning in Chest X-Ray Report Generation Using Reinforcement Learning

Denis Musinguzi, Andrew Katumba, Prasenjit Mitra

Carnegie Mellon University · Makerere University

Radiology Report Generation VLM Hallucination Mitigation VLM Reasoning RL for VLMs Visual Grounding

Open paper AskPDF arXiv

Medical report generation has made significant progress with the rise of modern vision-language models and the growing availability of large-scale medical datasets. However, hallucinations remain a major challenge, largely due to the limitations of supervised fine-tuning (SFT), which prioritizes lexical similarity to reference reports rather than clinical correctness. While reinforcement learning has shown strong performance in domains with verifiable rewards such as mathematics and code generation, its application to open-ended medical tasks remains limited. Existing work focuses on evaluating final answers, overlooking the model's reasoning, despite evidence that flawed reasoning can degrade overall performance. In this study, we propose a framework that verifies the model's reasoning process by integrating anatomical regions, bounding boxes, and region-level textual descriptions. We design spatial and factual reward mechanisms to ensure that the model's reasoning is both visually grounded and factually accurate. Starting from Qwen3-VL-8B-Instruct as our base model, we adapt it to the medical domain using supervised fine-tuning, introduce reasoning capability through a cold-start SFT stage, and refine it with reinforcement learning. We find that RL provides performance gains beyond those achievable through SFT alone, and that jointly verifying both reasoning steps and final outputs yields larger improvements than verifying either in isolation. We further identify multiple modes of reward hacking in the RL stage. Finally, the model's structured think traces enhance interpretability, making its outputs easier to audit for clinical use.

Abstract

Explore similar work

Enhancing Visual Reasoning in Chest X-Ray Report Generation Using Reinforcement Learning

Multi-Modal Multi-Agent Reinforcement Learning for Radiology Report Generation

Graph-Supervised Hierarchical Clinical Alignment for Radiology Report Generation with Large Language Models