RL for Language Model Reasoning

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

91 papers in the last four weeks, up 176% on the four weeks before. 0.9% of all new papers.

Jul 13Week of Sep 28

Latest papers 707

All topics

ReST-RL: Reinforcing LLM Reasoning through Unified Self-Training and Value-Guided Search
Aug 27, 2025Sining Zhoubian, Dan Zhang, Jie TangRL for Code Generation RL for Language Model Reasoning
Department of Computer Science and Technology, Tsinghua University · Work done when interned at Z.ai. · School of Electronics and Computer Scinence, Univeristy of Southampton
Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts
Aug 13, 2025Maxime Heuillet, Yufei Cui, Boxing Chen +2RL for Language Model Reasoning Efficient Language Model Training
Universit´e Laval (IID), Canada · Mila - Qu´ebec AI Institute, Canada · Huawei Noah’s Ark Lab (Montreal Research Center), Canada +1
From "Aha Moments" to Controllable Thinking: Toward Meta-Cognitive Reasoning in Large Reasoning Models via Decoupled Reasoning and Control
Aug 6, 2025Rui Ha, Rui Pu, Chaozhuo Li +2RL for Language Model Reasoning LLM Inference Acceleration
Beijing University of Posts and Telecommunications, China · 2Chongqing University of Posts and Telecommunications, China
Reconsidering Overthinking: Penalizing Internal and External Redundancy in CoT Reasoning
Aug 4, 2025Taihang Zhen, Jialiang Hong, Kai Chen +12RL for Language Model Reasoning CoT Compression
School of Electronic Science and Engineering, Nanjing University, Nanjing 210023, China · China Mobile (Suzhou) Software Technology Co., Ltd., Suzhou 215000, China · State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210023, China +2
Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applications
Aug 1, 2025Zizhan Ma, Wenxuan Wang, Meidan Ding +7LLM Prompting RL for Language Model Reasoning
The Chinese University of Hong Kong · Renmin University of China · Shenzhen University +3
Post-Training Large Language Models via Reinforcement Learning from Self-Feedback
Jul 29, 2025Carel van Niekerk, Renato Vukovic, Benjamin Ruppik +3RL for Language Models RL for Language Model Reasoning
Heinrich Heine University Düsseldorf, Germany
Re:Form -- Reducing Human Annotations in Scalable Formal Software Verification with RL in LLMs: A Preliminary Study on Dafny
Jul 22, 2025Chuanhao Yan, Fengdi Che, Xuhan Huang +12RL for Language Models RL for Language Model Reasoning
Shanghai AI Lab · Tsinghua University · University of Alberta +5
BOW: Training Language Models to Reason Over Plausible Next Words
Jun 16, 2025Ming Shen, Zhikun Xu, Jacob Dineen +2RL for Language Model Reasoning Next-Token Prediction
Arizona State University
Evaluating the Scalability and Adversarial Generalization of GRPO-Trained NLI Models
Apr 25, 2025Pablo Miralles-González, Javier Huertas-Tato, Alejandro Martín +1LLM Fine-Tuning RL for Language Model Reasoning
Department of Computer Systems, Technical University of Madrid, Madrid, Spain · United Arab Emirates University, Al Ain, United Arab Emirates
Efficient Reinforcement Finetuning via Adaptive Curriculum Learning
Apr 7, 2025Taiwei Shi, Yiyang Wu, Linxin Song +2RL for Language Model Reasoning Mathematical Reasoning
University of Southern California · Carnegie Mellon University · Mohamed Bin Zayed University of Artificial Intelligence
LearNAT: Learning NL2SQL with AST-guided Task Decomposition for Large Language Models
Apr 3, 2025Weibin Liao, Xin Gao, Tianyu Jia +6LLM Planning RL for Language Model Reasoning
School of Computer Science, Peking University · 2Key Laboratory of High Confidence Software Technologies, Ministry of Education · School of Computer Sciences, Beijing University of Posts and Telecommunications +5
RL-STaR: Theoretical Analysis of Reinforcement Learning Frameworks for Self-Taught Reasoner
Oct 31, 2024Fu-Chieh Chang, Yu-Ting Lee, Hui-Ying Shih +2Reinforcement Learning RL for Language Model Reasoning
MediaTek Research, Taipei, Taiwan · Graduate Institute of Communication Engineering, National Taiwan University, Taipei, Taiwan · Department of Mathematical Sciences, National Chengchi University, Taipei, Taiwan +2
Amortizing intractable inference in large language models
Oct 6, 2023Edward J. Hu, Moksh Jain, Eric Elmoznino +4Amortized Inference LLM Fine-Tuning
Mila – Quebec AI Institute, Universit´e de Montr´eal · University of Oxford
Countdown-Code: A Testbed for Studying The Emergence and Generalization of Reward Hacking in RLVR
Date pendingMuhammad Khalifa, Zohaib Khan, Omer Tafveez +2RL Benchmarks Reward Hacking
NVIDIA · University of Michigan · University of Illinois Urbana-Champaign
A Recipe for Long-Context Reasoning in Large Language Models via On-Policy Optimization and Distillation
Date pendingMiguel Moura Ramos, Duarte M. Alves, André F. T. MartinsLLM Alignment RL for Language Model Reasoning
Instituto Superior Técnico, Universidade de Lisboa · Instituto de Telecomunicações · TransPerfect +1
Complementing reinforcement learning with SFT through logit averaging in the post training of LLMs
Date pendingXingwei Gan, Ying ZhuRL for Language Model Reasoning Group Relative Policy Optimization
UC San Diego
Using Reward Uncertainty to Induce Diverse Behaviour in Reinforcement Learning
Date pendingAnthony GX-Chen, Ankit Anand, Gheorghe Comanici +7Reinforcement Learning RL for Language Model Reasoning
New York University · Google DeepMind

Aug 27, 2025, cs.AIJ/K move · Enter open · S save

ReST-RL: Reinforcing LLM Reasoning through Unified Self-Training and Value-Guided Search

Sining Zhoubian, Dan Zhang, Jie Tang

Department of Computer Science and Technology, Tsinghua University · Work done when interned at Z.ai. · School of Electronics and Computer Scinence, Univeristy of Southampton

RL for Code Generation RL for Language Model Reasoning Monte Carlo Tree Search RL Post-Training Language Model Self-Improvement

Open paper AskPDF arXiv

With respect to improving the reasoning accuracy of LLMs, the representative reinforcement learning (RL) method - Group Relative Policy Optimization (GRPO) - has achieved critical success, yet it still suffers from the issue of insignificant reward signals. This paper introduces ReST-RL, a unified Reinforced Self-Training (ReST) policy-value framework that reconnects policy optimization and value-guided search to improve LLM reasoning ability. Firstly, ReST-GRPO adopts an optimized ReST-style algorithm to reshape the policy-induced trajectory distribution by increasing the reward variance of GRPO sampling and exposing the policy to more informative partial states, thereby improving training efficiency and effectiveness. Then, we further introduce a decoding optimization method, VM-MCTS, which trains a Value Model (VM) from self-collected Monte-Carlo Tree Search (MCTS) targets and deploys it through an adapted MCTS algorithm to provide precise process signals and verification scores, further enhancing reasoning accuracy. These two stages are internally dependent - ReST-GRPO yields higher-quality trajectories for value learning with VM-MCTS, which in turn enables more effective inference-time search. We validate our framework on multiple coding benchmarks (e.g., APPS, BigCodeBench, and HumanEval), where it significantly outperforms other reinforcement training baselines (naive GRPO, DAPO, and ReST-DPO), as well as decoding and verification baselines (e.g., PRM-BoN and ORM-MCTS), indicating its power to strengthen LLM reasoning capability. Moreover, we further evaluate ReST-RL on out-of-domain math and science reasoning tasks, where it achieves improved performance without target-domain tuning and favorable end-to-end efficiency trade-offs, providing preliminary transfer evidence beyond our primary coding domain.

Momentum

Related topics

Latest papers 707