RL for Language Model Reasoning

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

91 papers in the last four weeks, up 176% on the four weeks before. 0.9% of all new papers.

Jul 13Week of Sep 28

Latest papers 707

All topics

Inducing Process Supervision from Outcome-Only Reinforcement Learning
Sep 29, 2026Shengda Fan, Xin Cong, Zhong Zhang +2Reinforcement Learning Process Reward Models
Renmin University of China · Tsinghua University · University of Electronic Science and Technology of China +1
SERA: Scale-Equalized Rollout Allocation for Maximum Likelihood Reinforcement Learning
Sep 29, 2026Zihao Chen, Fanxiang Xiong, Hongran Ren +6Reinforcement Learning RL for Language Model Reasoning
Harbin Institute of Technology, Shenzhen · The Chinese University of Hong Kong, Shenzhen · Nanyang Technological University
Towards Mitigating Deceptive Safety Alignment in Large Reasoning Models
Sep 28, 2026Xiangyu Zhou, Saleh Zare Zade, Rafi Ibn Sultan +2Language Model Safety Evaluation Deception in Language Models
Wayne State University
An RL View of OPD: Least Square Policy Distillation for Sample-Efficient LLM Reasoning
Sep 28, 2026Shangzhe Li, Yuxiao Yang, Tianrun Yu +4RL for Language Model Reasoning Language Model Distillation
University of North Carolina at Chapel Hill · Brigham Young University · NVIDIA
Frontier Learning: Training LLM Reasoners at the Edge of Capability
Sep 28, 2026Robin Faro, Shyam Sundhar Ramesh, Ilija Bogunovic +1RL for Language Model Reasoning LLM Post-Training
University College London London, UK · University of Basel Basel, Switzerland
MaPP: A Unified Marginalized Posterior-Predictive Framework for Data-Efficient RLVR
Sep 28, 2026Yangyang Ren, Haodong Zhu, Sheng Xu +4Bayesian RL RL for Language Model Reasoning
Beihang University · Zhongguancun Academy · Communication University of China +4
Teach to Learn: Hint Annealing for Self-improving LLM Reasoning
Sep 28, 2026Zile Wang, Zijian Li, Haodong Wang +5Reinforcement Learning Self-Training for Language Models
The Hong Kong University of Science and Technology · Yuanbao Team, Tencent
Beyond Verbalized Confidence: Calibrating Reasoners with Differentiable Readouts
Sep 28, 2026Chenxiao Fan, Chongming Gao, Gangyi Zhang +8Confidence Estimation in Language Models RL for Language Model Reasoning
University of Science and Technology of China · Qwen Business Unit of Alibaba · National University of Singapore
Can We Trust the Teacher? Decoupled Credit Direction-Magnitude for Self-Distillation
Sep 28, 2026Yugu Li, Zehong Cao, Peizhen Li +3Token-Level Credit Assignment RL for Language Model Reasoning
School of CSIT, Adelaide University, Adelaide, SA 5000, Australia · CAIAA, University of North Texas, Denton, TX 76203, USA · School of EECMS, Curtin University, Bentley, WA 6102, Australia +1
Rewarding Novel Deductions: Solver-guided Process Supervision for Logical Reasoning
Sep 28, 2026Muhammad Asif Ali, Wenqing Wang, Huan Wang +1RL for Language Model Reasoning Process Supervision
FORTE Lab, Faculty of Science, Information Technology University, Lahore, Pakistan · College of Informatics, Huazhong Agricultural University, Wuhan, China · Qatar Computing Research Institute, Hamad Bin Khalifa University, Doha, Qatar
GlyphBench: A Playground for Language-Model Reinforcement Learning
Sep 28, 2026Roger Creus Castanyer, Marc-Alexandre Côté, Matthew James Sargent +3RL Benchmarks RL for Language Models
Mila - Quebec Artificial Intelligence Institute · Université de Montréal · Vmax
On the Token Value Inequality in Efficient Reasoning
Sep 27, 2026Runjia Zeng, Hang Hua, Yiyang Liu +5RL for Language Model Reasoning Efficient Language Model Reasoning
Rochester Institute of Technology · Purdue University · MIT-IBM Watson AI Lab +3
RAISE: Reinforcing Access Control Policy Synthesis in LLMs via Symbolic Evaluation
Sep 27, 2026Yingming Zhou, Adarsh Vatsa, William EiersRL for Language Model Reasoning LLM-Based Program Synthesis
Stevens Institute of Technology
Surprising Success, Repeated Failure: Entropy-Guided Credit Assignment for Exploration in LLM Reasoning
Sep 27, 2026Woongyeong Yeo, Minki Kang, Chanuk Lee +3RL for Language Model Reasoning Credit Assignment in RL
KAIST
Selecting Diverse SFT Traces Improves Post-RL Generalization
Sep 27, 2026Dylan Zhang, Mingyuan Wu, Jinning LiReasoning Diversity RL for Language Model Reasoning
University of Illinois Urbana-Champaign, work done at Google · Google
TGRL: Temperature-Grouped Reinforcement Learning for Efficient Exploration in LLMs
Sep 27, 2026Zihan Lin, Xiaohan Wang, Jie Cao +4RL for Language Models RL for Language Model Reasoning
University of Chinese Academy of Sciences · Meituan · MAIS&NLPR, Institute of Automation, Chinese Academy of Sciences
OpenFC: Learning Verification Policies towards Open-Search Fact Checking
Sep 27, 2026Xinming Wang, Kaixiang Qiu, Yansong Lin +5RL for Language Model Reasoning Automated Fact-Checking
Institute of Automation, Chinese Academy of Sciences · University of Electronic Science and Technology of China · Beijing Institute of Technology +1
TeacherGRPO: Closing the Capacity Gap in Reasoning Distillation via Teacher Alignment
Sep 27, 2026Zhenyu Lei, Zihan Chen, Yaochen Zhu +5Reasoning Distillation RL for Language Model Reasoning
University of Virginia · Netflix · University of Washington +2
Rufus-Air: An Open LLM Post-Training Recipe
Sep 24, 2026Chia-Yuan Chang, Renyuan Cheng, Rui Feng +19LLM Fine-Tuning RL for Language Model Reasoning
Amazon
CounterRoute: Self-Routed Reasoning via Hierarchical Counterfactual Credit Assignment
Sep 24, 2026Ruochen Jiao, Besnik Fetahu, Zhenyu Shi +1RL for Language Model Reasoning Efficient Language Model Reasoning
When and Where to Trust the Teacher: Unifying On-Policy Distillation and GRPO through Entropy-Calibrated Credit Assignment
Sep 23, 2026Jie Zhang, Jingxiao Yang, Zhehao Huang +2RL for Language Model Reasoning Group Relative Policy Optimization
Shanghai Jiao Tong University · Zhejiang University
RL Starts before RL: On Policy Distillation for Better Reinforcement Learning
Sep 23, 2026Shuai Dong, Yongfu Zhu, Yuqi Xu +26Reinforcement Learning RL for Language Model Reasoning
Fudan University · SII · Peking University +2
DCRL: Decoupling and Coupling Reinforcement Learning via Policy-Reward Manifold Alignment
Sep 23, 2026Henan Sun, Zehua Li, Haitao Hu +4Reinforcement Learning RL for Language Model Reasoning
The Hong Kong University of Science and Technology (Guangzhou) · Huawei Noah’s Ark Lab · Tencent HY +1
Planned Test-Time Scaling with Coordinated Reasoning Paths
Sep 23, 2026Xueqing Wu, Langxing Bai, Hritik Bansal +5RL for Language Model Reasoning Test-Time Scaling
University of California, Los Angeles · Amazon
Giving Credit Where It's Due: Redundancy-Aware Learning for Efficient Reasoning
Sep 22, 2026Yuqing Zhou, Hong Wang, Manqing Mao +8RL for Language Model Reasoning Step-Level Credit Assignment in RL
George Mason University · Amazon, Inc
Beyond Repeated Sampling: Learning Search Policies for LLM Reasoning
Sep 22, 2026Ismail Labiad, Matthieu Kowalski, Marc Schoenauer +2RL for Language Model Reasoning LLM Reasoning
Meta FAIR · Université Paris-Saclay, LISN, Inria, CNRS · NYU Courant Institute and CDS
PACT: From Credit Assignment to Critic Alignment
Sep 22, 2026Jiayan Fu, Hang Xu, Yong Zhang +4Reinforcement Learning Token-Level Credit Assignment
AllSpark Team
Rollout Efficiency in Reinforcement Learning for Reasoning Large Language Models: A Taxonomy and Future Directions
Sep 21, 2026Niloofar Gholipour, Marcos Assuncao, Gursimran Singh +8LLM Inference Efficiency Reinforcement Learning
École de technologie supérieure, Univ. of Québec, Canada · Huawei Technologies, Canada · The Univ. of Melbourne, Australia +1
TelecomGPT-R1: Unified Post-Training for Reasoning Across Heterogeneous Telecom Tasks
Sep 21, 2026Bohao Wang, Chenwei Wu, Hang Zou +7LLM Grounding RL for Language Model Reasoning
College of Information Science and Electronic Engineering, Zhejiang University, 310027 Hangzhou, China · Research Institute for Digital Future, Khalifa University, P O Box 127788, Abu Dhabi, UAE · Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI 48109-2122, USA +1
Fathom-Vaidya: Advancing Medical Reasoning with Rubric-Based Rewards
Sep 21, 2026Kalash Shah, Kunal Singh, Snehan J +1RL for Language Model Reasoning Rubric-Based RL
Fractal AI Research

Sep 29, 2026, cs.LGJ/K move · Enter open · S save

Inducing Process Supervision from Outcome-Only Reinforcement Learning

Shengda Fan, Xin Cong, Zhong Zhang, Haotian Chen+1

Renmin University of China · Tsinghua University · University of Electronic Science and Technology of China+1

Reinforcement Learning Process Reward Models RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards

Open paper AskPDF arXiv

Process reward models (PRMs) have become a key component for LLMs, as their step-level feedback supports both post-training and test-time reasoning. However, training strong PRMs remains costly: human step annotation is difficult to scale, while Monte Carlo estimation is computationally expensive and can drift from the intrinsic correctness of steps. To get effective PRMs at low cost, we introduce TIPS (Thinking-Induced Process Supervision), an outcome-only reinforcement learning (RL) framework for training generative PRMs. In TIPS, the model generates a chain-of-thought (CoT) followed by step-level labels and an outcome label. The reward depends solely on whether the predicted outcome matches the ground truth, and the resulting group-relative advantage is used to optimize the entire generated response. Intuitively, when checking intermediate steps helps determine the outcome, more accurate checks can lead to better outcome judgments and higher rewards. Outcome-only RL can therefore reinforce step-level verification without explicit process supervision. We validate the effectiveness of TIPS across math and agent benchmarks and four backbone families. Notably, TIPS-Qwen3-4B-Thinking-2507 reaches 85.2 F1 on ProcessBench with only 3.2K outcome-labeled trajectories, surpassing all evaluated trained PRMs and strong prompt-only judges such as GPT-5.4-Instruct and Claude-4.7-Opus, while still trailing o1-mini. Code and data are available at https://github.com/RUCBM/TIPS.

Momentum

Related topics

Latest papers 707