Medical report generation has made significant progress with the rise of modern vision-language models and the growing availability of large-scale medical datasets. However, hallucinations remain a major challenge, largely due to the limitations of supervised fine-tuning (SFT), which prioritizes lexical similarity to reference reports rather than clinical correctness. While reinforcement learning has shown strong performance in domains with verifiable rewards such as mathematics and code generation, its application to open-ended medical tasks remains limited. Existing work focuses on evaluating final answers, overlooking the model's reasoning, despite evidence that flawed reasoning can degrade overall performance. In this study, we propose a framework that verifies the model's reasoning process by integrating anatomical regions, bounding boxes, and region-level textual descriptions. We design spatial and factual reward mechanisms to ensure that the model's reasoning is both visually grounded and factually accurate. Starting from Qwen3-VL-8B-Instruct as our base model, we adapt it to the medical domain using supervised fine-tuning, introduce reasoning capability through a cold-start SFT stage, and refine it with reinforcement learning. We find that RL provides performance gains beyond those achievable through SFT alone, and that jointly verifying both reasoning steps and final outputs yields larger improvements than verifying either in isolation. We further identify multiple modes of reward hacking in the RL stage. Finally, the model's structured think traces enhance interpretability, making its outputs easier to audit for clinical use.
Figures & tables
Model
BLEU-2
ROUGE-L
METEOR
RADGRAPH-F1
BERTSCORE
Qwen3-VL-8B-Instruct ( Bai et al. 2025 )
4.4 ± 1.10
14.0 ± 1.00
21.3 ± 1.10
10.8 ± 1.50
42.0 ± 1.90
MedGemma ( Sellergren et al. 2025 )
9.6 ± 1.45
20.7 ± 0.25
26.1 ± 0.35
15.5 ± 0.80
47.7 ± 1.35
DeepMedix-R1 ( Lin et al. 2025 )
21.5 ± 0.56
22.3 ± 0.33
29.0 ± 0.57
18.6 ± 0.22
52.8 ± 1.46
MAIRA-2 ( Bannur et al. 2024a )
14.2 ± 1.00
17.7 ± 0.60
21.5 ± 0.50
12.9 ± 0.30
46.6 ± 1.50
UniRG-CXR ( Liu et al. 2026 )
26.2 ± 1.20
26.4 ± 0.45
31.5 ± 0.85
26.7 ± 0.46
47.8 ± 0.75
RadVLM-GRPO ( Gundersen et al. 2025 )
22.5 ± 0.20
30.0 ± 0.10
30.2 ± 1.00
25.7 ± 0.10
59.0 ± 0.70
Table 1: Results of our model and other baselines on the MIMIC-CXR dataset test set. It includes the results after each training stage. Bold values indicate the best performance, while underlined values indicate the model with the second-best performance.
Model
BLEU-2
ROUGE-L
METEOR
RADGRAPH-F1
BERTSCORE
Our model
27.51 ± 0.39
34.47 ± 0.26
39.92 ± 0.21
36.53 ± 0.23
61.49 ± 0.05
MAIRA-2
21.90 ± 0.34
27.40 ± 0.26
33.10 ± 0.26
27.10 ± 0.24
47.70 ± 0.04
DeepMedix-R1
23.78 ± 0.37
28.53 ± 0.28
33.16 ± 0.27
27.66 ± 0.23
52.97 ± 0.04
MedGemma
3.24 ± 0.38
9.07 ± 0.42
22.24 ± 0.86
24.18 ± 0.22
17.25 ± 0.08
RadVLM-GRPO
20.50 ± 0.48
27.26 ± 0.37
29.40 ± 0.74
24.70 ± 0.35
53.14 ± 0.10
UniRG-CXR
27.46 ± 0.52
33.89 ± 0.25
40.12 ± 1.00
36.12 ± 0.18
60.70 ± 0.06
Table 2: Results of various models on the test split of the IU X-ray dataset, considering only frontal chest X-ray images. Bold values indicate the best performance while underlined values indicate the second-best performance.
Stage
BL-2
RG-L
MR
RG-F1
BS
SFT
22.0
25.3
27.0
16.9
52.0
C-S SFT
14.9
23.2
26.1
20.4
54.2
RL
23.4
28.5
32.5
27.9
56.3
Table 3: Bleu-2 (BL-2), Rouge-L (RG-L), RadGraph-F1 (RG-F1), Meteor, and BertScore (BS) results after different stages of the training pipeline. SFT denotes the model after the SFT stage, C-S SFT denotes the model after the cold-start SFT stage and RL denotes the model after the RL stage. Bold values indicate the best performance while underlined values indicate the second-best performance.
Figure 2: Qualitative evaluation on the MIMIC-CXR test set. Predicted region boxes, the corresponding reasoning trace, and the generated report.
Model
RG-L
BL-2
Meteor
RG-F1
BS
Ans Verif
26.2
23.2
23.1
21.0
53.8
Reas Verif
25.5
22.1
33.5
23.2
55.2
Joint Verif
26.8
23.3
32.5
27.9
55.6
Table 4: Results showing Rouge-L (RG-L), Bleu-2 (BL-2), Meteor, RadGraph-F1 (RG-F1) and Bertscore (BS) results for verification of the answer (Ans Verif), the reasoning process (Reas Verif), and the joint verification (Joint Verif).
IoU Thres.
BL-4
RG-L
RG-F1
Meteor
BS
0.00
4.69
25.17
20.27
26.87
54.27
0.25
7.89
27.12
23.73
32.25
55.25
0.50
4.98
24.78
20.21
27.62
54.74
Table 5: Bleu-4 (BL-4), Rouge-L (RG-L), RadGraph-F1 (RG-F1), Meteor, and BertScore (BS) results of the model at different IoU thresholds for gating the regional factual reward.
Spat
Fac
BL-4
RG-L
RG-F1
Meteor
BS
✗
✓
4.69
25.17
20.27
26.87
54.27
✓
✗
6.63
25.27
20.68
28.46
52.30
✗
✗
5.36
24.74
21.18
28.85
55.22
✓
✓
7.89
27.12
23.73
32.25
55.25
Table 6: Bleu-4 (BL-4), Rouge-L (RG-L), RadGraph-F1 (RG-F1), Meteor, and BertScore (BS) results of the ablation on factual and spatial rewards. Spat stands for spatial reward and Fac stands for factual reward.
Multimodal Large Language Models (MLLMs) have substantially advanced Radiology Report Generation (RRG), yet aligning them through reinforcement learning (RL) remains challenging due to heterogeneous medical supervision. Vanilla Group Relative Policy Optimization (GRPO) assigns uniform credit across the entire generation, leading to segment interference, token dilution, and rationale-diagnosis decoupling, which exacerbates clinical hallucinations. We propose HERO (Hierarchical rationalE Reasoning Optimization), a factorized policy optimization framework that aligns heterogeneous supervision with three optimization granularities. HERO separately optimizes reasoning, diagnosis, and rationale faithfulness through complementary segment-, token-, and completion-level optimization with a heterogeneous reward formulation covering diagnostic accuracy, reasoning quality, and think-answer consistency. Experiments on MIMIC-CXR and IU-Xray show that HERO outperforms strong supervised and reinforcement learning baselines, achieving state-of-the-art clinical efficacy while producing more rationale-supported and think-answer-consistent reports, thereby substantially mitigating clinical hallucinations.
Kun Zhao, Guodong Liu, Hui Ji +7
University of Pittsburgh · Eli and Lilly Company · The University of Manchester +1
Medical imaging is a cornerstone of diagnostics, yet automated chest X-ray report generation struggles with structural adherence, anatomical completeness, and semantic faithfulness. We introduce DobicVLM, a vision-language model combining supervised fine-tuning on MedGemma-4B with Group Relative Policy Optimization (GRPO) and clinically-grounded programmatic rewards. Our approach uses interpretable, rule-based reward components; structural verification, anatomical checklist, semantic similarity, and length constraints to enforce clinical standards without neural reward models. Trained on 1,000 de-identified image-report pairs from a private clinical dataset (with ethics approval and compliance to local regulations), DobicVLM is evaluated via blinded expert review on 69 held-out cases. DobicVLM outperforms Gemini 2.5 Flash across the majority of criteria, achieving the highest impression accuracy (27.2%) and medical terminology (86.5%) compared to both Gemini 2.5 Flash and MedGemma 4B baselines, with minor trade-offs in completeness and referrals. This demonstrates GRPO's value for transparent alignment in resource-limited settings. Keywords: Vision-Language Models, Radiology Report Generation, Reinforcement Learning, Medical AI, GRPO
Thanni Adewuyi, Angelica Obayi, Andem Aniekan +10
Dobic Health · HelpMum Africa · University of Ibadan
Medical Vision-Language Models (VLMs) hold immense promise for complex clinical tasks, but their reasoning capabilities are often constrained by text-only paradigms that fail to ground inferences in visual evidence. This limitation not only curtails performance on tasks requiring fine-grained visual analysis but also introduces risks of visual hallucination in safety-critical applications. Thus, we introduce MedVR, a novel reinforcement learning framework that enables annotation-free visual reasoning for medical VLMs. Its core innovation lies in two synergistic mechanisms: Entropy-guided Visual Regrounding (EVR) uses model uncertainty to direct exploration, while Consensus-based Credit Assignment (CCA) distills pseudo-supervision from rollout agreement. Without any human annotations for intermediate steps, MedVR achieves state-of-the-art performance on diverse public medical VQA benchmarks, significantly outperforming existing models. By learning to reason directly with visual evidence, MedVR promotes the robustness and transparency essential for accelerating the clinical deployment of medical AI.
Zheng Jiang, Heng Guo, Chengyu Fang +4
Tsinghua University · DAMO Academy, Alibaba Group · Hupan Lab, 310023, Hangzhou, China +2