RL for Language Model Reasoning

RL: Reinforcement Learning

This field studies how reward feedback can teach language models to solve multi-step problems, use tools, and decide how much reasoning a task needs. A common approach is reinforcement learning from verifiable rewards (RLVR), often using group-relative policy optimization, where sampled solutions are scored for correctness; other methods add step-level rewards, process verifiers, search, or teacher guidance to address sparse feedback. Work also explores curricula, adaptive rollout budgets, self-distillation, and controls for reasoning length and diversity. Key open tensions include balancing exploration with stable learning, ensuring rewards and verifiers do not encourage shortcuts, and improving generalization without excessive computation or overthinking.

Latest papers

1,103 papers170 papers in the last four weeks, up 124% on the four weeks before. 1.0% of all new papers.

PIVOT: Progressive Intervention via Online Teacher Correction
Oct 29, 2026Zhihao Yang, Boyuan Zheng, Puhui Tan +7RL for Language Model Reasoning LLM-Guided RL
University of Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · Tsinghua University +1
SCARL: Sequence Confidence-Aligned Reinforcement Learning
Oct 29, 2026Zhenghong Ke, Jingbo Zhou, Hua WuReinforcement Learning RL for Language Model Reasoning
Baidu Frontier Research Department · Nanjing University
Incentivizing Self-Learning in Large Language Models via Reinforcement Learning
Oct 29, 2026Lingkai Kong, Yuzhe Gu, Haiteng Zhao +6Language Model Self-Improvement RL for Language Model Reasoning
Shanghai AI Laboratory · Shanghai Jiao Tong University · MMLab, The Chinese University of Hong Kong +2
There Is No Free Lunch in RLVR: Every pass@1 Gain Has a pass@k Price
Oct 29, 2026Yinuo LiuRL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Reasoning or Pattern Exploitation? Mechanistic Insights from RL-Trained Language Models
Oct 29, 2026Aishik Nagar, Cheston TanRL for Language Model Reasoning LLM Interpretability
Carnegie Mellon University · Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR) · Centre for Frontier AI Research (CFAR), Agency for Science, Technology and Research (A*STAR)
When Does Closed-Loop Curriculum Adaptation Help GRPO? An Analysis on Countdown
Oct 28, 2026Yuv Raj Pant, Moshiur Rahman Tonmoy, Kaiqun FuRL for Language Model Reasoning Curriculum RL
Department of Computer Science, Texas Christian University · Department of Electrical Engineering and Computer Science, Texas A&M University - Kingsville
Learning to Reason from Reference Samples Alone: Reinforced Adversarial Learning for Modern LLM Post-Training
Oct 28, 2026Albert Bou, Javier Valdes-Aleman, Jon M. Laurent +4RL for Language Model Reasoning LLM Post-Training
Edison Scientific Inc. San Francisco, CA, USA
Latent Decision Flow Rewards for Clinical Guideline Reasoning
Oct 28, 2026Gaurav Bhatt, Marinka ZitnikRL for Language Model Reasoning Clinical Reasoning
Harvard University
Fu-TCM: a multimodal large language model that learns traditional Chinese medicine diagnostic reasoning
Oct 28, 2026Jiahang Xie, Tong Jin, Lingfei Wu +9Medical LLMs Traditional Chinese Medicine
School of Data Science, Fudan University, Shanghai, China · School of Life Sciences, Beijing University of Chinese Medicine, Beijing, China · The Affiliated People’s Hospital of Fujian University of Traditional Chinese Medicine, Fuzhou, Fujian, China +3
Bayes-Adaptive MDPs for Rollout Allocation to Enhance GRPO Training
Oct 27, 2026Faeze Moradi Kalarde, Armin Toroghi, Nam Ho Koh +4RL Post-Training Group Relative Policy Optimization
University of Toronto · Vector Institute of AI · Korea Advanced Institute of Science & Technology
Guided-RLVR: Guided Exploration in Reinforcement Learning with Verifiable Rewards
Oct 27, 2026Bo Yang, Zishuai Zhang, Weihua Li +3RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, Beihang University · School of Artificial Intelligence, Beihang University
When All Rollouts Fail: Policy-Compatible Joint Decoding for Hard-Problem Reasoning
Oct 27, 2026Zehao Wang, Hanzhu Chen, Runquan Gui +2RL for Language Models RL for Language Model Reasoning
University of Science and Technology of China · Tencent · Shanghai Artificial Intelligence Laboratory
AudioRSI: Recursively Self-Improving Audio Reasoners
Oct 27, 2026Jiajun Fan, Roger Ren, Jingyuan Li +6Audio Reasoning Audio-Language Models
UIUC
S3AC: Stabilized, Selective, and Scalable Actor-Critic for LLM Reasoning
Oct 26, 2026Shuo Shen, Guangran Cheng, Chengqi Lyu +3RL for Language Model Reasoning RL for Language Models
Shanghai AI Laboratory · Independent Researcher
Trainable Boundary-Keeping Where No Formal Answer Exists: A Controlled Knock-In Study of Reward Training in Evidence-Sufficiency Judgment
Oct 26, 2026YaJie YinLLM Abstention Refusal Behavior in Language Models
Improving Reasoning with Rollout Agreement for Group-Based Policy Optimization
Oct 26, 2026Sohyun Son, Yerim Hong, Jaeyun Park +1Reinforcement Learning RL for Language Model Reasoning
Kyung Hee University, Yongin, Republic of Korea
Correlation-Aware Exploration for Long-Trace Reasoning
Oct 26, 2026Junxing Hu, Zicheng Zhang, Zhiqian Zhang +4RL for Language Model Reasoning RL for Language Models
Microsoft
What Outcome RL Buys: Finishing, Reliability, Rarely Reach
Oct 26, 2026Shan YangReinforcement Learning RL for Language Model Reasoning
Individual Researcher.
Uncovering the Turing Test's Latent Signal: Indistinguishability as a Reward
Oct 26, 2026Emanuele Rimoldi, Riccardo Neumarker, Tomaso Poggio +1Adversarial Imitation Learning RL for Language Models
MIT · EPFL · ETH Zurich +1
Self-Evolution Reasoner: Continuous Optimization via Policy-Intrinsic Exploration and Exploitation
Oct 25, 2026Wenhang Shi, Yiren Chen, Zhe Zhao +3Language Model Self-Improvement RL for Language Model Reasoning
Renmin University of China · Peking University · Tencent
Intervention-Certified Amplification: A Validity Gate for $σ$ -Weighted Post-Training
Oct 24, 2026Leiyue Chen, Meng Wei, Xuefeng Hu +1RL Post-Training RL for Language Model Reasoning
ByteDance
More: Mode-Optimized Reasoning with Self-Adaptive Effort for Self-Evolving Agents
Oct 24, 2026Dianxing Shi, Dingjie Fu, Tianyu Lei +1Adaptive Reasoning in Language Models Efficient Language Model Reasoning
Beihang University · Huazhong University of Technology
MS-TTRL: Breaking the Frequency Conflict in Multiple-Solution Test-Time Reinforcement Learning
Oct 23, 2026Zhixia Zhang, Binqian Xu, Huaiyang Wang +2Test-Time RL RL for Language Model Reasoning
Beihang University · National University of Singapore
Inherited, Not Invented: Tracing the Provenance of Wait in Reasoning Models
Oct 22, 2026Zhizhuo Kou, Sizai Hou, Long Chen +6RL for Language Model Reasoning Data Provenance in Language Models
The Hong Kong University of Science and Technology · Huawei Noah’s Ark Lab · Peking University
MOER: Mix-Policy Optimization under Expert Prefix Guidance with Selective Rollback
Oct 22, 2026Xuyang Zhong, Jiuding Yang, Yiwen Guo +1RL for Language Model Reasoning Policy Optimization
City University of Hong Kong · LIGHTSPEED · Independent Researcher
GIFT: Gradient Informed Filtering for Distillation Augmented Reasoning RL
Oct 22, 2026Shijie Li, Yilin Gao, Tieyuan Chen +3RL for Language Model Reasoning LLM Reasoning
Shanghai Jiao Tong University · Tencent
Route-and-Generate: KL-Regularized Router Improvement for Reasoning Routing in LLMs
Oct 21, 2026Tien-Phat Nguyen, Truong Nguyen, Thanh-Thien Le +4Adaptive Reasoning in Language Models Efficient Language Model Reasoning
Hanoi University of Science and Technology, Vietnam · Monash University, Australia · New York University, USA +1
Consistency Is Not Validity: Self-Rewarded Moral Argumentation Makes Language Models Decisive but Pliable
Oct 21, 2026Marius De Kuthy Meurers, Aaron Parisi, Bernd BohnetMoral Reasoning in Language Models Reasoning Consistency in Language Models
Google DeepMind
LongZero: Enhancing Long-Context Reasoning in LLMs from Zero Data
Oct 21, 2026Zhennan Wan, Jianfei ChenLong-Context Reasoning in Language Models RL for Language Model Reasoning
Dept. of Comp. Sci. and Tech., Institute for AI, BNRist Center, THBI Lab, Tsinghua-Bosch Joint ML Center, Tsinghua University
EPIC: Rethinking Trust Regions for Autoregressive LLM Policies via Exact-KL Prefix Importance-Ratio Clipping
Oct 21, 2026Chengxi Li, Hanlin Wu, Zhicheng Cai +4RL for Language Model Reasoning Proximal Policy Optimization
Institute for AI Industry Research (AIR), Tsinghua University · Department of Computer Science and Technology, Tsinghua University · SIA-Lab of Tsinghua AIR and ByteDance Seed

Oct 29, 2026J/K move · Enter open · S save

PIVOT: Progressive Intervention via Online Teacher Correction

Zhihao Yang, Boyuan Zheng, Puhui Tan, Yukun Chen+6

University of Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · Tsinghua University+1

RL for Language Model Reasoning LLM-Guided RL Reinforcement Learning with Verifiable Rewards

Open paper AskPDF arXiv

Reinforcement learning from verifiable rewards (RLVR) is limited by sparse outcome-level feedback on hard reasoning tasks: when all rollouts in a group fail, rewards provide no learning signal. Stronger teachers can provide additional guidance, but this raises a central question: how can such signals be incorporated into RLVR to support successful rollouts while remaining learnable for the student? We distinguish two complementary mechanisms: trajectory-level intervention, which improves access to reward-bearing trajectories, and token-level supervision, which provides dense targets to guide student learning. Our controlled experiments show that either mechanism alone can remain insufficient, motivating their combination. We therefore introduce PIVOT, which inserts teacher-generated corrections into student rollouts and applies token-level KL supervision to these segments alongside the RLVR objective. Beyond their combination, we systematically analyze where to intervene and how to supervise, yielding a recipe for effective guidance. Across hard logic games and mathematical reasoning benchmarks, PIVOT consistently outperforms existing methods, nearly doubling mean avg@8 on logic games from 19.5% for the best-performing baseline to 35.6%.