RL for Language Model Reasoning

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

91 papers in the last four weeks, up 176% on the four weeks before. 0.9% of all new papers.

Jul 13Week of Sep 28

Latest papers 707

All topics

EP-GRPO: Entropy-Progress Aligned Group Relative Policy Optimization with Implicit Process Guidance
May 6, 2026Song Yu, Li Li, Wenwen Zhao +1RL for Language Model Reasoning Group Relative Policy Optimization
School of Computer and Information Science, Southwest University, Chongqing 400715, China
Strat-Reasoner: Reinforcing Strategic Reasoning of LLMs in Multi-Agent Games
May 6, 2026Yidong He, Yutao Lai, Pengxu Yang +4Game-Playing Agents RL for Language Model Reasoning
School of Software Engineering, South China University of Technology · Department of Computer Science, University of Oxford
Power Distribution Bridges Sampling, Self-Reward RL, and Self-Distillation
May 6, 2026Akiyoshi Tomihari, Issei SatoReinforcement Learning KL-Regularized RL
Department of Computer Science, The University of Tokyo
Correct Is Not Enough: Training Reasoning Planners with Executor-Grounded Rewards
May 5, 2026Tianyang Han, Hengyu Shi, Junjie Hu +3RL for Code Generation Process Reward Models
1D4 Lab · 2Independent Researcher
DGPO: Distribution Guided Policy Optimization for Fine Grained Credit Assignment
May 5, 2026Hongbo Jin, Rongpeng Zhu, Zhongjing Du +4Token-Level Credit Assignment RL for Language Model Reasoning
Peking University · SJTU · Tsinghua University
HeavySkill: Heavy Thinking as the Inner Skill in Agentic Harness
May 4, 2026Jianing Wang, Linsen Guo, Zhengyu Chen +8RL for Language Model Reasoning Test-Time Scaling
Meituan LongCat Team · National Engineering Research Center for Software Engineering, Peking University, Beijing, China
Binary Rewards and Reinforcement Learning: Fundamental Challenges
May 4, 2026Marc DymetmanReinforcement Learning KL-Regularized RL
∗Scientific Consultant
Towards Understanding Specification Gaming in Reasoning Models
May 4, 2026Kei Nishimura-Gasparian, Robert McCarthy, David LindnerReward Hacking RL for Language Model Reasoning
1MATS · University College London, London, UK · 3Google Deep- Mind.
Enhanced LLM Reasoning by Optimizing Reward Functions with Search-Driven Reinforcement Learning
May 3, 2026Arash Ahmadi, Sarah Sharif, Yaser +1Reinforcement Learning Large Language Model-Guided Optimization
Mike · School of Electrical and Computer Engineering, University of Oklahoma, Norman, OK 73019 USA · Intelligent Neuromorphic and Quantum Understanding for Innovative Research and Engineering (INQUIRE) Laboratory, University of Oklahoma, Norman, OK, USA
Selector-Guided Autonomous Curriculum for One-Shot Reinforcement Learning from Verifiable Rewards
May 3, 2026Rudray Dave, Vedang Dubey, Smit Deoghare +1Reinforcement Learning RL for Language Model Reasoning
Department of Artificial Intelligence Sardar Vallabhbhai National Institute of Technology Surat, India
Grounding Multi-Hop Reasoning in Structural Causal Models via Group Relative Policy Optimization
May 2, 2026Yunhan Bu, Quan Zhang, Huaping Zhang +9Structural Causal Models RL for Language Model Reasoning
Beijing Institute of Technology, Beijing, China · School of Computer Science and Technology, Xinjiang University, Urumqi, China · Military Science Information Research Center, Academy of Military Science, Beijing, China
Segment-Aligned Policy Optimization for Multi-Modal Reasoning
May 2, 2026Lei Gao, Zhuoming Li, Mengxi Jia +4RL for Language Model Reasoning Policy Optimization
Fudan University · China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd. · Southeast University +1
ResRL: Boosting LLM Reasoning via Negative Sample Projection Residual Reinforcement Learning
May 1, 2026Zihan Lin, Xiaohan Wang, Jie Cao +6RL for Language Models Reinforcement Learning
MAIS&NLPR, Institute of Automation, Chinese Academy of Sciences, Beijing, China · Meituan, Beijing, China · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences, Beijing, China.
Uniform-Correct Policy Optimization: Breaking RLVR's Indifference to Diversity
May 1, 2026Anamika Lochab, Bolian Li, Ruqi ZhangRL for Language Model Reasoning Group Relative Policy Optimization
Department of Computer Science Purdue University, West Lafayette
RSAT: Structured Attribution Makes Small Language Models Faithful Table Reasoners
Apr 30, 2026Jugal Gajjar, Kamalasankari SubramaniakuppusamyTable QA LLM Grounding
Department of Computer Science, The George Washington University, USA
Kernelized Advantage Estimation: From Nonparametric Statistics to LLM Reasoning
Apr 30, 2026Shijin Gong, Kai Ye, Jin Zhu +3Value Function Estimation RL for Language Model Reasoning
School of Management, University of Science and Technology of China · Department of Statistics, London School of Economics and Political Science · School of Mathematics, University of Birmingham +2
Latent-GRPO: Group Relative Policy Optimization for Latent Reasoning
Apr 30, 2026Jingcheng Deng, Zihao Wei, Liang Pang +4Reinforcement Learning RL for Language Model Reasoning
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Institute of Automation, Chinese Academy of Sciences
ANCORA: Learning to Question via Manifold-Anchored Self-Play for Verifiable Reasoning
Apr 30, 2026Chengcao YangSelf-Play RL RL for Language Model Reasoning
Wuhan University Wuhan, China
Distributional Alignment Games for Answer-Level Fine-Tuning
Apr 29, 2026Mehryar Mohri, Jon Schneider, Yifan WuFine-Tuning RL for Language Model Reasoning
Google Research · Microsoft Research
Addressing Performance Saturation for LLM RL via Precise Entropy Curve Control
Apr 29, 2026Bolian Li, Yifan Wang, Yi Ding +3RL for Language Models Reinforcement Learning
Purdue University, West Lafayette, IN, USA
How Fast Should a Model Commit to Supervision? Training Reasoning Models on the Tsallis Loss Continuum
Apr 28, 2026Chu-Cheng Lin, Eugene IeSupervised Fine-Tuning Reinforcement Learning
Google
One Refiner to Unlock Them All: Inference-Time Reasoning Elicitation via Reinforcement Query Refinement
Apr 28, 2026Yixiao Zhou, Dongzhou Cheng, zhiliang wu +3LLM Alignment RL for Language Model Reasoning
Zhejiang University · Shanghai Innovation Institute · 1Zhejiang University, 2Shanghai Innovation Institute +9
JURY-RL: Votes Propose, Proofs Dispose for Label-Free RLVR
Apr 28, 2026Xinjie Chen, Biao Fu, Jing Wu +4RL for Language Model Reasoning Mathematical Reasoning
Tongyi Lab, Alibaba Group, Hangzhou, China
Why Does Reinforcement Learning Generalize? A Feature-Level Mechanistic Study of Post-Training in Large Language Models
Apr 27, 2026Dan Shi, Zhuowen Han, Simon Ostermann +3Reinforcement Learning RL for Language Model Reasoning
TJUNLP Lab, School of Computer Science and Technology, Tianjin University, China · German Research Center for Artificial Intelligence (DFKI), Saarbrücken, Germany · Saarland University, Saarbrücken, Germany
Thoughts-as-Planning: Latent World Models for Chain-of-Thoughts Optimization via Reinforcement Planning
Apr 27, 2026Dong Liu, Yanxuan Yu, Ying Nian WuCoT Reasoning Large Language Model-Guided Optimization
University of California, Los Angeles · Columbia University
IRIS: Interleaved Reinforcement with Incremental Staged Curriculum for Cross-Lingual Mathematical Reasoning
Apr 27, 2026Navya Gupta, Rishitej Reddy Vyalla, Avinash Anand +8Cross-Lingual Reasoning in Language Models RL for Language Model Reasoning
1Singapore Institute of Technology, Singapore · 2IIIT Delhi, New Delhi, India · University of California, San Diego, USA +2
Stabilizing Efficient Reasoning with Step-Level Advantage Selection
Apr 27, 2026Han Wang, Xiaodong Yu, Jialian Wu +4RL for Language Model Reasoning Efficient Language Model Reasoning
UNC Chapel Hill · Advanced Micro Devices, Inc.
SFT-then-RL Outperforms Mixed-Policy Methods for LLM Reasoning
Apr 26, 2026Alexis Limozin, Eduard Durech, Torsten Hoefler +2RL for Language Model Reasoning Policy Optimization
ETH AI Center, ETH Zürich · EPFL · Allen Institute for AI
Process Supervision of Confidence Margin for Calibrated LLM Reasoning
Apr 25, 2026Liaoyaqi Wang, Chunsheng Zuo, William Jurayj +2Confidence Estimation in Language Models RL for Language Model Reasoning
Johns Hopkins University
Language as a Latent Variable for Reasoning Optimization
Apr 23, 2026Linjuan Wu, Haoran Wei, Jialong Tang +4Multilingual Language Models Cross-Lingual Reasoning in Language Models
Zhejiang University · Tongyi Lab, Alibaba Grou.

May 6, 2026, cs.LGJ/K move · Enter open · S save

EP-GRPO: Entropy-Progress Aligned Group Relative Policy Optimization with Implicit Process Guidance

Song Yu, Li Li, Wenwen Zhao, Zhisheng Yang

School of Computer and Information Science, Southwest University, Chongqing 400715, China

RL for Language Model Reasoning Group Relative Policy Optimization Credit Assignment in RL Reinforcement Learning with Verifiable Rewards

Open paper AskPDF arXiv

Reinforcement learning with verifiable rewards (RLVR), particularly Group Relative Policy Optimization (GRPO), has advanced LLM reasoning. However, GRPO suffers from three credit assignment failures: uniform token-level granularity that ignores heterogeneous informational value, uniform polarity that penalizes correct steps and rewards incorrect ones, and zero-variance collapse that erases outcome-driven gradients. We systematically quantify these failures, revealing highly non-uniform token informativeness, widespread step-level polarity misalignment, and substantial training waste. To address these limitations, we propose Entropy-Progress Aligned GRPO (EP-GRPO), a framework that mines the model's intrinsic information flow for dense, self-supervised guidance. EP-GRPO integrates entropy-gated modulation to prioritize high entropy decision pivots, implicit process signals from policy divergence anchored to outcome advantages for directional token-level feedback without external reward models, and cumulative entropy mapping that enables progress-aligned advantage normalization, naturally maintaining gradient flow under zero reward variance. Extensive experiments on mathematical reasoning benchmarks demonstrate that EP-GRPO achieves superior accuracy and efficiency compared to GRPO and its variants. The code will be available.

Momentum

Related topics

Latest papers 707