Reinforcement Learning Improves LLM Accuracy and Reasoning in Disease Classification from Radiology Reports

Yishu Wei; Yi Lin; Adam Flanders; George Shih; Yifan Peng

cs.AIApr 21, 2026

Reinforcement Learning Improves LLM Accuracy and Reasoning in Disease Classification from Radiology Reports

Authors: Yishu Wei, Yi Lin, Adam Flanders, George Shih, Yifan Peng

Organizations: Department of Population Health Sciences, Weill Cornell Medicine, New York, NY · Department of Radiology, Thomas Jefferson University, Philadelphia, PA · Department of Radiology, Weill Cornell Medicine, New York, NY

Abstract

Accurate disease classification from radiology reports is essential for many applications. While supervised fine-tuning (SFT) of lightweight LLMs improves accuracy, it can degrade reasoning. We propose a two-stage approach: SFT on disease labels followed by Group Relative Policy Optimization (GRPO) to refine predictions by optimizing accuracy and format without reasoning supervision. Across three radiologist-annotated datasets, SFT outperformed baselines and GRPO further improved classification and enhanced reasoning recall and comprehensiveness.

Explore similar work

ReST-RL: Reinforcing LLM Reasoning through Unified Self-Training and Value-Guided Search
Aug 27, 2025Sining Zhoubian, Dan Zhang, Jie TangLarge Language Model Reasoning Reinforcement Learning With Verifiable Reward
Learning to Refine Hidden States for Reliable LLM Reasoning
Jun 16, 2026Chia-Hsuan Hsu, Jui-Ming YaoLarge Language Model Reasoning Efficient Latent Reasoning
Fathom-Vaidya: Advancing Medical Reasoning with Rubric-Based Rewards
Sep 21, 2026Kalash Shah, Kunal Singh, Snehan J +1

Aug 27, 2025 · cs.AIJ/K move · Enter open · S save

ReST-RL: Reinforcing LLM Reasoning through Unified Self-Training and Value-Guided Search

Sining Zhoubian, Dan Zhang, Jie Tang

Department of Computer Science and Technology, Tsinghua University · Work done when interned at Z.ai. · School of Electronics and Computer Scinence, Univeristy of Southampton

With respect to improving the reasoning accuracy of LLMs, the representative reinforcement learning (RL) method - Group Relative Policy Optimization (GRPO) - has achieved critical success, yet it still suffers from the issue of insignificant reward signals. This paper introduces ReST-RL, a unified Reinforced Self-Training (ReST) policy-value framework that reconnects policy optimization and value-guided search to improve LLM reasoning ability. Firstly, ReST-GRPO adopts an optimized ReST-style algorithm to reshape the policy-induced trajectory distribution by increasing the reward variance of GRPO sampling and exposing the policy to more informative partial states, thereby improving training efficiency and effectiveness. Then, we further introduce a decoding optimization method, VM-MCTS, which trains a Value Model (VM) from self-collected Monte-Carlo Tree Search (MCTS) targets and deploys it through an adapted MCTS algorithm to provide precise process signals and verification scores, further enhancing reasoning accuracy. These two stages are internally dependent - ReST-GRPO yields higher-quality trajectories for value learning with VM-MCTS, which in turn enables more effective inference-time search. We validate our framework on multiple coding benchmarks (e.g., APPS, BigCodeBench, and HumanEval), where it significantly outperforms other reinforcement training baselines (naive GRPO, DAPO, and ReST-DPO), as well as decoding and verification baselines (e.g., PRM-BoN and ORM-MCTS), indicating its power to strengthen LLM reasoning capability. Moreover, we further evaluate ReST-RL on out-of-domain math and science reasoning tasks, where it achieves improved performance without target-domain tuning and favorable end-to-end efficiency trade-offs, providing preliminary transfer evidence beyond our primary coding domain.

Large Language Model Reasoning Reinforcement Learning With Verifiable Reward Deepmind Control Suite Self-Training Deep Reinforcement Learning Flow-Grpo Monte Carlo Tree Search

Open paper Ask about it PDF

Abstract

Explore similar work

ReST-RL: Reinforcing LLM Reasoning through Unified Self-Training and Value-Guided Search

Learning to Refine Hidden States for Reliable LLM Reasoning

Fathom-Vaidya: Advancing Medical Reasoning with Rubric-Based Rewards