RL for Language Model Reasoning

RL: Reinforcement Learning

This field studies how reward feedback can teach language models to solve multi-step problems, use tools, and decide how much reasoning a task needs. A common approach is reinforcement learning from verifiable rewards (RLVR), often using group-relative policy optimization, where sampled solutions are scored for correctness; other methods add step-level rewards, process verifiers, search, or teacher guidance to address sparse feedback. Work also explores curricula, adaptive rollout budgets, self-distillation, and controls for reasoning length and diversity. Key open tensions include balancing exploration with stable learning, ensuring rewards and verifiers do not encourage shortcuts, and improving generalization without excessive computation or overthinking.

Latest papers

1,103 papers170 papers in the last four weeks, up 124% on the four weeks before. 1.0% of all new papers.

Probing to Refine: Reinforcement Distillation of LLMs via Explanatory Inversion
Feb 26, 2026Zhen Tan, Chengshuai Zhao, Song Wang +3Language Model Distillation Reasoning Distillation
Arizona State University · University of Virginia · University of North Carolina at Chapel Hill
Truncated Step-Level Sampling with Process Rewards for Retrieval-Augmented Reasoning
Feb 26, 2026Chris Samarinas, Haw-Shiuan Chang, Hamed ZamaniReinforcement Learning Process Reward Models
Center for Intelligent Information Retrieval University of Massachusetts Amherst Amherst, MA, United States
Towards Better RL Training Data Utilization via Second-Order Rollout
Feb 26, 2026Zhe Yang, Yudong Wang, Rang Li +1RL for Language Model Reasoning RL Post-Training
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · ByteDance BandAI
Know What You Know: Metacognitive Entropy Calibration for Verifiable RL Reasoning
Feb 26, 2026Qiannian Zhao, Chen Yang, Jinhao Jing +5RL for Language Model Reasoning Large Reasoning Models
Compress the Easy, Explore the Hard: Difficulty-Aware Entropy Regularization for Efficient LLM Reasoning
Feb 26, 2026Qin-Wen Luo, Sheng Ren, Xiang Chen +4Efficient Language Model Reasoning CoT Compression
Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu, China · College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu, China · Didichuxing Co. Ltd, Beijing, China
Stable Adaptive Thinking via Advantage Shaping and Length-Aware Gradient Regulation
Feb 26, 2026Zihang Xu, Haozhi Xie, Ziqi Miao +3Overthinking in Language Models Efficient Language Model Reasoning
Renmin University of China
Reinforcement-aware Knowledge Distillation for LLM Reasoning
Feb 26, 2026Zhaoyang Zhang, Shuli Jiang, Yantao Shen +6Knowledge Distillation RL for Knowledge Distillation
AWS Agentic AI
Improving Parametric Knowledge Access in Reasoning Language Models
Feb 25, 2026Melody Ma, John HewittFactual Knowledge in Language Models Knowledge-Intensive QA
Columbia University
UpSkill: Mutual Information Skill Learning for Structured Response Diversity in LLMs
Feb 25, 2026Devan Shah, Owen Yang, Daniel Yang +2RL for Language Model Reasoning Reasoning Diversity
Princeton University
ExpLang: Improved Exploration and Exploitation in LLM Reasoning with On-Policy Thinking Language Selection
Feb 25, 2026Changjiang Gao, Zixian Huang, Kaichen Yang +3RL for Language Model Reasoning Cross-Lingual Reasoning in Language Models
National Key Laboratory for Novel Software Technology, Nanjing University, Jiangsu, China · Department of Linguistics and Translation, City University of Hong Kong, Hong Kong, China · Shanghai Artificial Intelligence Laboratory, Shanghai, China +1
Distill and Align Decomposition for Enhanced Claim Verification
Feb 25, 2026Jabez Magomere, Elena Kochkina, Samuel Mensah +6Claim Verification RL for Language Model Reasoning
University of Oxford · JPMorgan AI Research
Explore-on-Graph: Incentivizing Autonomous Exploration of Large Language Models on Knowledge Graphs with Path-refined Reward Modeling
Feb 25, 2026Shiqi Yan, Yubo Chen, Ruiqi Zhou +8LLM Reasoning with Graphs Knowledge Graph Question Answering
Zhongguancun Laboratory, Beijing, China · Department of Electronic Engineering, Tsinghua University, Beijing, China · Ant International, Ant Group, Hangzhou, Zhejiang, China +1
Overconfident Errors Need Stronger Correction: Asymmetric Confidence Penalties for Reinforcement Learning
Feb 24, 2026Yuanda Xu, Hejian Sang, Zhengze Zhou +2RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
LinkedIn Corporation, CA, USA
Prompt Interference in LLM Post-Training: Why Pass@k Optimization Can Degrade Pass@1
Feb 24, 2026Anas Barakat, Souradip Chakraborty, Khushbu Pahwa +1RL for Language Model Reasoning
Singapore University of Technology and Design · University of Maryland, College Park · University of Central Florida
The Art of Efficient Reasoning: Data, Reward, and Optimization
Feb 24, 2026Taiqiang Wu, Zenan Xu, Bo Zhou +1Efficient Language Model Reasoning RL for Language Model Reasoning
The University of Hong Kong · LLM Department, Tencent
Group Orthogonalized Policy Optimization:Group Policy Optimization as Orthogonal Projection in Hilbert Space
Feb 24, 2026Wang ZixianLLM Alignment RL for Language Model Reasoning
China Mobile Communications Group Shandong Co., Ltd. Tai’an Branch
Buffer Matters: Unleashing the Power of Off-Policy Reinforcement Learning in Large Language Model Reasoning
Feb 24, 2026Xu Wan, Yansheng Wang, Wenqi Huang +1RL for Language Model Reasoning RL Post-Training
Zhejiang University · Bytedance Seed Robotics · Peking University +1
Counterfactual Simulation Training for Chain-of-Thought Faithfulness
Feb 24, 2026Peter Hase, Christopher PottsCoT Faithfulness CoT Monitoring
Stanford University
Actor-Curator: Co-adaptive Curriculum Learning via Policy-Improvement Bandits for RL Post-Training
Feb 24, 2026Zhengyao Gu, Jonathan Light, Raul Astudillo +7Curriculum RL Curriculum Learning
University of Illinois Chicago · Caltech · RPI +2
KairosVL: Orchestrating Time Series and Semantics for Unified Reasoning
Feb 24, 2026Haotian Si, Changhua Pei, Xiao He +9Time Series Reasoning RL for Language Model Reasoning
CNIC, CAS, Beijing, China · University of Chinese Academy of Sciences · Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences +4
LAD: Learning Advantage Distribution for Reasoning
Feb 23, 2026Wendi Li, Sharon LiRL for Language Model Reasoning RL Post-Training
Department of Computer Sciences, University of Wisconsin–Madison
ReSyn: Autonomously Scaling Synthetic Environments for Reasoning Models
Feb 23, 2026Andre He, Nathaniel Weir, Kaj Bostrom +4Synthetic Data Generation for Language Models RL for Language Model Reasoning
Carnegie Mellon University · Amazon Web Services
DSDR: Dual-Scale Diversity Regularization for Exploration in LLM Reasoning
Feb 23, 2026Zhongwei Wan, Yun Shen, Zhihao Dou +9Reinforcement Learning RL for Language Model Reasoning
The Ohio State University, USA · Case Western Reserve University, USA · The Chinese University of Hong Kong, Hong Kong +4
Ada-RS: Adaptive Rejection Sampling for Selective Thinking
Feb 23, 2026Yirou Ge, Yixi Li, Alec Chiu +8Efficient Language Model Reasoning RL for Language Model Reasoning
PayPal AI
LLMs Can Learn to Reason Via Off-Policy RL
Feb 22, 2026Daniel Ritter, Owen Oertell, Bradley Guo +3RL for Language Model Reasoning RL Post-Training
Cornell University · Databricks · Harvard University
Smooth Gate Functions for Soft Advantage Policy Optimization
Feb 22, 2026Egor Denisov, Svetlana Glazyrina, Maksim Kryzhanovskiy +1Policy Optimization RL for Language Model Reasoning
Lomonosov Moscow State University, Moscow, Russia · Institute for Artificial Intelligence, Lomonosov Moscow State University, Moscow, Russia
Soft Sequence Policy Optimization
Feb 22, 2026Svetlana Glazyrina, Maksim Kryzhanovskiy, Roman IschenkoLLM Alignment RL Post-Training
Lomonosov Moscow State University, Moscow, Russia · Institute for Artificial Intelligence, Lomonosov Moscow State University, Moscow, Russia
Learning to Reason for Multi-Step Retrieval of Personal Context in Personalized Question Answering
Feb 22, 2026Maryam Amirizaniani, Alireza Salemi, Hamed ZamaniQuestion Answering Retrieval-Augmented Generation
University of Washington Seattle, WA, USA · University of Massachusetts Amherst Amherst, MA, USA
How to Allocate, How to Learn? Dynamic Rollout Allocation and Advantage Modulation for Policy Optimization
Feb 22, 2026Yangyi Fang, Jiaye Lin, Xiaoliang Fu +6Reinforcement Learning RL for Language Model Reasoning
Meituan · Tsinghua University · Fudan University +1
IAPO: Information-Aware Policy Optimization for Token-Efficient Reasoning
Feb 22, 2026Yinhan He, Yaochen Zhu, Mingjia Shi +5Efficient Language Model Reasoning Language Model Post-Training
University of Virginia, Charlottesville, VA, USA · LinkedIn Inc., Sunnyvale, CA, USA

Feb 26, 2026J/K move · Enter open · S save

Probing to Refine: Reinforcement Distillation of LLMs via Explanatory Inversion

Zhen Tan, Chengshuai Zhao, Song Wang, Jundong Li+2

Arizona State University · University of Virginia · University of North Carolina at Chapel Hill

Language Model Distillation Reasoning Distillation Knowledge Distillation RL for Language Model Reasoning Group Relative Policy Optimization

Open paper AskPDF arXiv

Distilling robust reasoning capabilities from large language models (LLMs) into smaller, computationally efficient student models remains an unresolved challenge. Despite recent advances, distilled models frequently suffer from superficial pattern memorization and subpar generalization. To overcome these limitations, we introduce a novel distillation framework that moves beyond simple mimicry to instill a deeper conceptual understanding. Our framework features two key innovations. First, to address pattern memorization, Explanatory Inversion (EI) generates targeted ``explanatory probes'' that compel the student to articulate the underlying logic behind an answer, rather than just memorizing it. Second, to improve generalization, Explanatory GRPO (EXGRPO) uses a reinforcement learning algorithm with a novel Dialogue Structure Utility Bonus, which explicitly rewards the student for maintaining a coherent reasoning process across these probes. Extensive evaluations on 12 datasets demonstrate significant improvements. Using Gemma-7b as the student model, our method yields an average 20.39% increase over zero-shot performance and a 6.02% improvement over the state-of-the-art distillation baselines. Moreover, models distilled with our method show remarkable training efficiency (e.g., surpassing vanilla fine-tuning with 10-25% training data) and strong generalization to out-of-distribution tasks. Implementation is released at https://github.com/Zhen-Tan-dmml/ExGRPO.git.