Organizations: DAMO Academy, Alibaba Group · Ningbo No.2 Hospital, China · Hupan Lab, 310023, China · Fudan University, China · Zhejiang University, China
Radiologists highly desire fully automated AI for radiology report generation (R2G), yet existing approaches fall short in clinical utility. Reinforcement learning (RL) holds potential to address these shortcomings, but its adoption in this task remains underexplored. In this paper, we revisit RL in terms of data efficiency and optimization effectiveness for R2G tasks. First, we explore the impact of data quantity and quality on the performance of RL in medical contexts, revealing that data quality plays a more critical role than quantity. To this end, we propose a diagnostic diversity-based data sampling strategy that enables comparable performance with fewer samples. Second, we observe that the majority of tokens in radiology reports are template-like and diagnostically uninformative, whereas the low frequency of clinically critical tokens heightens the risk of being overlooked during optimization. To tackle this, we introduce Diagnostic Token-weighted Policy Optimization (DiTPO), which directly optimizes for clinical accuracy by using a diagnostic F1 score as the reward signal. Unlike standard RL approaches that treat all tokens equally, DiTPO explicitly models the varying importance of different tokens through rule- or gradient-based mechanisms to prioritize clinically relevant content. Extensive experiments on the MIMIC-CXR, IU-Xray, and CheXpert Plus datasets demonstrate that our framework achieves state-of-the-art (SOTA) performance while requiring substantially fewer training samples in RL. Notably, on MIMIC-CXR, our framework attains an F1 score of 0.516 using only 20% of the RL training samples.
Figures & tables
Figure 1: (a) Diagnostic F1 scores of GRPO and DEER with different proportions of RL training data. The results indicate that with only about 20% of the RL training data, the models’performance is comparable to that achieved with 100% of the data. (b) Comparison between uniform GRPO and our DiTPO. GRPO assigns equal advantages to all tokens regardless of their clinical importance, while DiTPO assigns significantly higher advantages to diagnostically critical tokens.
Figure 2: The DEER framework consists of three stages: (1) SFT for cold - start initialization to provide foundational R2G capabilities; (2) Data Selection via DDSampling to retain high - quality diverse samples; and (3) DiTPO, where rule - based (TF - IDF) and gradient - based token weighting are integrated to produce diagnosis - aware token - level advantages for policy optimization.
Paradigm
Method
RL Data
CE Metrics ↑
NLG Metrics ↑
Model Metrics ↑
P
R
F1
B-4
MTR
R-L
GR
RG
SFT
R2Gen [ 5 ]
-
0.333
0.273
0.276
0.103
0.142
0.277
-
-
CvT2Dis. [ 19 ]
-
0.360
0.412
0.384
0.124
0.153
0.285
-
-
DCL [ 12 ]
-
0.471
0.352
0.373
0.109
0.150
0.284
-
-
R2-LLM
-
0.465
0.482
0.473
0.128
0.175
0.291
-
-
RGRG ♭ [ 26 ]
-
0.461
0.475
0.447
0.126
0.168
0.264
0.268
0.224
Table 1: Comparison with SOTA methods on the MIMIC-CXR dataset. Results are grouped by paradigm (SFT vs. SFT+RL). "20%" and "100%" indicate the percentage of training samples used during the RL stage. ♭ signifies results from reproduced codes. Best and second-best values are in bold and underlined . The GRPO approach in the table refer to our basic GRPO framework described in Section 3.1. Notably, DiTPO refer to DEER framework without data selection stage, and both DiTPO and DEER in main results section use gradient-based weighting.
Table 4Table 5Table 6
Figure 3: Diagnostic token weighting case study. Sentences describing the same clinical finding in both the reference and generated reports are highlighted with matching colors. The generated report uses red color intensity to visualize token-level weights.
Recent reinforcement learning (RL) approaches have advanced radiology report generation (RRG), yet two core limitations persist: (1) report-level rewards offer limited evidence-grounded guidance for clinical faithfulness; and (2) current methods lack an explicit self-improving mechanism to align with clinical preference. We introduce clinically aligned Evidence-aware Self-Correcting Reinforcement Learning (ESC-RL), comprising two key components. First, a Group-wise Evidence-aware Alignment Reward (GEAR) delivers group-wise, evidence-aware feedback. GEAR reinforces consistent grounding for true positives, recovers missed findings for false negatives, and suppresses unsupported content for false positives. Second, a Self-correcting Preference Learning (SPL) strategy automatically constructs a reliable, disease-aware preference dataset from multiple noisy observations and leverages an LLM to synthesize refined reports without human supervision. ESC-RL promotes clinically faithful, disease-aligned reward and supports continual self-improvement during training. Extensive experiments on two public chest X-ray datasets demonstrate consistent gains and state-of-the-art performance.
Qin Zhou, Guoyan Liang, Qianyi Yang +4
Department of Computer Science and Engineering, ECUST, China · Key Laboratory of Smart Manufacturing in Energy Chemical Process, Ministry of Education, P. R. China · Zhejiang University, Hangzhou, China +1
Automated radiology report generation (RRG) has gained increasing attention because it can reduce the heavy workload of clinical report writing. However, most existing methods mainly optimize for natural language generation (NLG) metrics that focus on language fluency, while providing little control over clinically important factors such as precision and recall. As consequence, generated reports may be fluent but not well aligned with different clinical needs. To address this challenge, we propose a reinforcement learning framework for precision recall controllable RRG, where a control parameter explicitly adjusts the trade-off between clinical precision and recall during inference. This design allows the model to flexibly generate reports according to different clinical requirements. To ensure clinical correctness, we introduce a clinical reward into the training objective, which helps improve clinical efficacy (CE) beyond standard language-based optimization. In addition, we apply a group-relative training strategy that normalizes rewards within each training group, reducing reward variance and improving training stability. Extensive experiments on the MIMIC-CXR dataset show that our method consistently outperforms state-of-the-art approaches in both NLG and CE evaluation metrics, while providing reliable control over the CE precision recall trade-off.
Ling Chen, Ruinan Jin, Jun Luo +6
The Ohio State University, Columbus, OH 43221, USA · University Medical Center Regensburg, Regensburg, Germany
Medical imaging interpretation is a foundational pillar of modern clinical diagnostics, yet the manual generation of radiology reports remains a time-consuming process prone to interpretation inconsistencies. Within the field of medical AI, automating these descriptions through deep learning promises to streamline clinical workflows and standardise diagnostic output. However, accurate disease detection and precise report generation remain significant challenges due to limitations in capturing fine-grained visual features and ensuring clinical coherence. To address these issues, we propose RL-ACRGNet, an improved encoder-decoder model that integrates a pre-trained DenseNet encoder with a multilevel LSTM decoder within an off-policy reinforcement learning framework. Using a dual-network approach to refine visual-semantic embeddings through a metric-based reward mechanism, we demonstrate that RL-ACRGNet consistently outperforms state-of-the-art baselines on the IU-Xray dataset, achieving quantitative improvements in BLEU-4 (0.47%), METEOR (0.17%) and ROUGE-L (0.518). Furthermore, comprehensive evaluations on the large-scale MIMIC-CXR data set confirm the robust generalisation of the model and its ability to generate high-quality, clinically relevant reports
Yogesh Kumar Meena, Saurabh Agarwal, K. V. Arya
Human-AI Interaction (HAIx) Lab, Indian Institute of Technology Gandhinagar, India · Department of Computer Science and Engineering, Madhav Institute of Technology and Science Deemed University (MITS-DU), Gwalior, India · Multimedia and Information Security Research Group, Department of Computer Science and Engineering, ABV-Indian Institute of Information Technology and Management, Gwalior 474015, India