RL for Language Model Reasoning

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

91 papers in the last four weeks, up 176% on the four weeks before. 0.9% of all new papers.

Jul 13Week of Sep 28

Latest papers 707

All topics

Distilled Reinforcement Learning for LLM Post-training
Jul 19, 2026Chen Wang, Zhaochun Li, Jionghao Bai +4RL for Language Model Reasoning Cross-Architecture Knowledge Distillation
College of Elite Engineers, Nankai University · Zhongguancun Academy · Beijing Institute of Technology +4
LenGuard-GPC: Length Guarding with Guided-Prompt Consistency for Spatial Reasoning Reinforce Learning
Jul 19, 2026Xingjian Tao, Yiwei Wang, Yujun Cai +1Visual Spatial Reasoning Reward Shaping
HKUST(GZ) · UC Merced · University of Queensland +1
Understanding Reasoning from Pretraining to Post-Training
Jul 17, 2026Jingyan Shen, Ang Li, Salman Rahman +4Language Model Pretraining RL for Language Model Reasoning
New York University · Columbia University · University of California, Los Angeles +2
Mask-Aware Policy Gradients for Diffusion Language Models
Jul 16, 2026Haran Raajesh, Kulin Shah, Adam Klivans +1Masked Diffusion Models RL for Language Model Reasoning
Department of Computer Science The University of Texas at Austin
On-Policy Delta Distillation
Jul 16, 2026Byeongho Heo, Jaehui Hwang, Sangdoo Yun +1RL for Language Model Reasoning Language Model Distillation
NAVER AI Lab
Beyond Entropy: Correctness-Aware Advantage Shaping via Contrastive Policy Optimization
Jul 16, 2026Weiwen Xu, Jia Liu, Hou Pong Chan +4RL for Language Model Reasoning Policy Optimization
1The Chinese University of Hong Kong · 2South China University of Technology · 3Nanyang Technological University
A Continuous-Time Reinforcement Learning Framework for Fine-Tuning Discrete Diffusion Models
Jul 16, 2026Zikun Zhang, Jiayuan Sheng, David D. Yao +1Masked Diffusion Models RL for Language Model Reasoning
Department of Industrial Engineering and Operations Research, Columbia University, New York, NY 10027
LOTAPO: Leave-One-Turn Attribution for Self-Generated Process Rewards in Multi-Turn Search Reasoning
Jul 15, 2026Qiang Zhu, Jiajun Wu, Longyi WangRL for Language Model Reasoning RL for Tool Use
Zhejiang University
Demystifying On-Policy Distillation: Roles, Pathologies, and Regulations
Jul 15, 2026Rui Wang, Hongru Wang, Yi Chen +4RL for Language Model Reasoning Language Model Distillation
†The Chinese University of Hong Kong · ‡Tencent AI Lab
GFlowRL: Scaling Distribution-Matching RL to Large Language Models
Jul 15, 2026Xiaodong Liu, Michael Xu, Jack W. Stokes +3RL for Code Generation Reinforcement Learning
Microsoft Research
Verifier-Based Reinforcement Fine-Tuning of Reasoning Models for Thermal Energy Storage Control
Jul 14, 2026Takumi Shioda, Kohei Terashima, Tatsuo NagaiRL for Language Model Reasoning Building Energy Management
The University of Tokyo · Tokyo University of Science
Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning
Jul 14, 2026Xinyu Tang, Qianggang Cao, Yurou Liu +13RL for Language Models RL for Language Model Reasoning
1Gaoling School of Artificial Intelligence, Renmin University of China · 2Ant Group · 3Tsinghua University +1
SCOPE-RL: Optimizing Reasoning Paths Before and After Success
Jul 13, 2026Xiaojian Liu, Han Xu, Jianqiang Xia +6Reward Shaping RL for Language Model Reasoning
Baidu Inc. · Shandong University
ARMOR: Stabilizing On-Policy LLM RL with Off-Policy Anchor Samples
Jul 11, 2026Kexin Huang, Junkang Wu, Jinda Lu +7Reinforcement Learning KL-Regularized RL
University of Science and Technology of China · Peking University · Dartmouth College
Beyond Euclidean Clipping: Overcoming Exploration Collapse in LLM RL via Riemannian Isometric Policy Optimization
Jul 11, 2026Zhicheng Cai, Xinyuan Guo, Hanlin Wu +3Reinforcement Learning RL for Language Model Reasoning
Institute for AI Industry Research (AIR), Tsinghua University · ByteDance Seed · SIA-Lab of Tsinghua AIR and ByteDance Seed.
Cost of Reasoning in non-English Languages: A Case Study on Japanese
Jul 11, 2026Yuu JinnaiMultilingual Language Models Multilingual Language Model Evaluation
CyberAgent / Tokyo, Japan
Switch-Reasoner: Learn When to Think in Multitask Mixtures via Reinforcement Learning
Jul 9, 2026Yiyang Fang, Pei Fu, Jinjie Li +7Efficient Multimodal Inference RL for Language Model Reasoning
Wuhan University · The Hong Kong University of Science and Technology · Xiaomi Inc +2
Agon: Competitive Cross-Model RL with Implicit Rival Grading of Reasoning
Jul 8, 2026Vladislav BeliaevSequential MARL RL for Language Model Reasoning
Independent Researcher
Max Out GRPO Signal: Adaptive Trace Prefix Control for Hard Reasoning Problems
Jul 8, 2026Vladislav BeliaevRL for Language Model Reasoning Group Relative Policy Optimization
Independent Researcher
RL Post-Training Builds Compositional Reasoning Strategies
Jul 8, 2026Azwar Abdulsalam, Nishil Patel, Andrew SaxeCompositional Reasoning RL for Language Model Reasoning
Gatsby Computational Neuroscience Unit, UCL, London, United Kingdom.
UP: Unbounded Positive Asymmetric Optimization for Breaking the Exploration-Stability Dilemma
Jul 8, 2026Chongyu Fan, Pengfei Liu, Jingjia Huang +2RL for Language Model Reasoning Policy Optimization
ByteDance Seed · Michigan State University
Information Gain-based Rollout Policy Optimization: An Adaptive Tree-Structured Rollout Approach for Multi-Turn Search Agents
Jul 7, 2026Yijun Zhang, Fan Xu, Jiaxin Ding +3Tree Search RL for Language Model Reasoning
Shanghai Jiao Tong University
Mitigating Factual Hallucination in Large Reasoning Models via Mixed-Mode Advantage Regularization
Jul 7, 2026Kaishen Wang, Tong Zheng, Xuehao Cui +3LLM Hallucination Mitigation RL for Language Model Reasoning
Department of Computer Science, University of Maryland, College Park
Self-Review Reinforcement Learning (SRRL) with Cross-Episode Memory and Policy Distillation
Jul 6, 2026Muhammad Zain Amin, Kibele Sebnem YildirimReinforcement Learning RL for Language Model Reasoning
Department of Software Engineering and IT, ´Ecole de Technologie Sup´erieure, Montreal, Canada · Centre int´egr´e de traumatologie, Hˆopital du Sacr´e-Cœur de Montr´eal, Universit´e de Montr´eal, Montreal, Canada · Desautels Faculty of Management, McGill University, Montreal, Canada.
Weak-to-Strong Generalization via Direct On-Policy Distillation
Jul 6, 2026Shiyuan Feng, Huan-ang Gao, Haohan Chi +7RL for Language Model Reasoning Weak-to-Strong Generalization
1SIA-Lab of Tsinghua AIR and ByteDance Seed · Institute for AI Industry Research (AIR), Tsinghua University · 4Peking University +1
TREK: Distill to Explore, Reinforce to Refine
Jul 6, 2026Yuanda Xu, Zhengze Zhou, Kayhan Behdin +10RL for Language Model Reasoning LLM-Guided RL
LinkedIn Corporation · Harvard University · Georgia Institute of Technology
Reason, Reward, Refine: Step-Level Errors Corrections with Structured Feedback for Physics Reasoning in Small Language Models
Jul 6, 2026Raj Jaiswal, Dhruv Jain, Rishabh Dhawan +4LLM Self-Correction Physical Reasoning
IIIT Delhi · IIT BHU · NII Tokyo +2
On the effectiveness of reward functions in reinforcement learning for confidence calibration of large language models
Jul 5, 2026Chee Heng Tan, Zhuoyi Lin, Mehul Motani +1Reward Hacking Confidence Estimation in Language Models
School of Computing National University of Singapore · Institute of Advanced Intelligence and Computing Agency for Science, Technology and Research · Department of Electrical & Computer Engineering National University of Singapore
NormWorlds-CF: Solver-Verified Counterfactual Normative Reasoning with Metamorphic-Relation GRPO
Jul 4, 2026Xinqi ZhangCounterfactual Evaluation Moral Reasoning in Language Models
Tsinghua University
ACPO: Adaptive Credit Policy Optimization via Fine-Grained Surrogate Entropy
Jul 3, 2026Zijun Xie, Yuyang You, Yongzhi Li +6Token-Level Credit Assignment RL for Language Model Reasoning
Peking University · Baidu Inc. · Kuaishou Inc.

Jul 19, 2026, cs.LGJ/K move · Enter open · S save

Distilled Reinforcement Learning for LLM Post-training

Chen Wang, Zhaochun Li, Jionghao Bai, Yining Zhang+3

College of Elite Engineers, Nankai University · Zhongguancun Academy · Beijing Institute of Technology+4

RL for Language Model Reasoning Cross-Architecture Knowledge Distillation On-Policy Distillation RL Post-Training Knowledge Distillation

Open paper AskPDF arXiv

Large language model (LLM) post-training is essential for improving reasoning, adaptation, and alignment. Existing methods mainly follow two paradigms: reinforcement learning (RL) and on-policy distillation (OPD). However, RL relies on coarse-grained outcome supervision, resulting in difficult credit assignment and limited capability to acquire new knowledge. OPD, meanwhile, unconditionally matches teacher logits through KL divergence, which creates a dilemma: similar teachers provide little new knowledge, while substantially different teachers often yield ineffective guidance, largely restricting OPD to within-family distillation. We propose Distilled Reinforcement Learning (Distilled RL), which integrates teacher supervision into the RL objective to provide fine-grained guidance, selectively transfer new knowledge and avoid unconditional imitation. Distilled RL contains three components: reverse importance sampling with clipping, negative sample reset, and sequence-level geometric normalization. Through a concise and interpretable case study, we demonstrate that Distilled RL can effectively transfer previously unavailable knowledge from a teacher model to a student model. Extensive experiments across both within-family and cross-family distillation settings show that Distilled RL substantially outperforms standard RL and OPD in terms of both pass@1 and pass@k. Our code is available at https://github.com/597358816/Distilled-RL.

Momentum

Related topics

Latest papers 707