Reinforcement Learning with Verifiable Rewards

Also known as RLVR

Follow this topic Ask about this topic

Momentum

59 papers in the last four weeks, up 119% on the four weeks before. 0.6% of all new papers.

Jul 13Week of Sep 28

Latest papers 438

All topics

Gradient Starvation in Binary-Reward GRPO: Why Group-Mean Centering Fails and Why the Simplest Fix Works
May 8, 2026Wenhua Nie, Jianan Wu, Junlin Liu +6Reinforcement Learning Group Relative Policy Optimization
National Taiwan University
Your Language Model is Its Own Critic: Reinforcement Learning with Value Estimation from Actor's Internal States
May 8, 2026Yunho Choi, Jongwon Lim, Woojin Ahn +3Value Function Estimation RL for Language Model Reasoning
Graduate School of Data Science Seoul National University · Computer Science and Engineering Seoul National University
HTPO: Towards Exploration-Exploitation Balanced Policy Optimization via Hierarchical Token-level Objective Control
May 8, 2026Xincheng Yao, Ruoqi Li, Cheng Chen +4RL for Language Models Exploration-Exploitation Tradeoff
School of Information Science and Electronic Engineering, Shanghai Jiao Tong University · Xiaohongshu Inc. · MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University.
Implicit Compression Regularization: Concise Reasoning via Internal Shorter Distributions in RL Post-Training
May 8, 2026Chen Wang, Hexuan Deng, Yining Zhang +5Overthinking in Language Models Reinforcement Learning with Verifiable Rewards
College of Software, Nankai University · 2Zhongguancun Academy · 3Harbin Institute of Technology +4
Structured Role-Aware Policy Optimization for Multimodal Reasoning
May 8, 2026Bingqing Jiang, Difan ZouToken-Level Credit Assignment Multimodal Reasoning
School of Computing & Data Science, The University of Hong Kong · School of Computing & Data Science and Institute of Data Science, The University of Hong Kong
Can You Break RLVER? Probing Adversarial Robustness of RL-Trained Empathetic Agents
May 8, 2026Deeraj S K, Sadhana Devarajan, Krishna Mehra +1Emotion Recognition in Conversations Adversarial Robustness of Language Models
Department of Artificial Intelligence, Sardar Vallabhbhai National Institute of Technology, Surat, India
Adaptive Negative Reinforcement for LLM Reasoning:Dynamically Balancing Correction and Diversity in RLVR
May 8, 2026Yash Ingle, Jaival Chauhan, Ankit Yadav +1Reinforcement Learning RL for Language Model Reasoning
Sardar Vallabhbhai National Institute of Technology (SVNIT), Surat, India
Where to Spend Rollouts: Hit-Utility Optimal Rollout Allocation for Group-Based RLVR
May 8, 2026Tao Wang, Shuo Li, Yan Sun +2RL for Language Model Reasoning Group Relative Policy Optimization
University of Pennsylvania · New Jersey Institute of Technology · University of Tennessee, Knoxville
Verifier-Backed Hard Problem Generation for Mathematical Reasoning
May 7, 2026Yuhang Lai, Jiazhan Feng, Yee Whye Teh +1Reinforcement Learning with Verifiable Rewards Multi-Agent LLMs
Department of Data Science, City University of Hong Kong · School of Intelligence Science and Technology, Peking University · Work done during Jiazhan’s visiting at the Department of Statistics, University of Oxford. +2
Beyond Negative Rollouts: Positive-Only Policy Optimization with Implicit Negative Gradients
May 7, 2026Mingwei Xu, Hao FangReinforcement Learning RL for Language Model Reasoning
University of Washington, Seattle, USA
On the Implicit Reward Overfitting and the Low-rank Dynamics in RLVR
May 7, 2026Hao Ye, Jisheng Dang, Junfeng Fang +7Reinforcement Learning Reinforcement Learning with Verifiable Rewards
Lanzhou University · National University of Singapore · Beijing Institute of Technology
Skip What You Can Predict: Predictive Repositioning for Policy Optimization for Efficient LLM Training
May 7, 2026Ismam Nur Swapnil, Aranya Saha, Tasneea Zahra +3RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Bangladesh University of Engineering and Technology · University of Maryland, College Park · Illinois Institute of Technology +1
OPSD Compresses What RLVR Teaches: A Post-RL Compaction Stage for Reasoning Models
May 7, 2026Jaehoon Kim, Dongha LeeReinforcement Learning with Verifiable Rewards On-Policy Distillation
Yonsei University
Listwise Policy Optimization: Group-based RLVR as Target-Projection on the LLM Response Simplex
May 7, 2026Yun Qu, Qi Wang, Yixiu Mao +11RL for Language Model Reasoning Group Relative Policy Optimization
Department of Automation, Tsinghua University · 2LLM Department, Tencent
Beyond Uniform Credit Assignment: Selective Eligibility Traces for RLVR
May 7, 2026Chaoli Mou, Zhan Zhuang, Xinning Chen +1RL for Language Model Reasoning Credit Assignment in RL
Southern University of Science and Technology · City University of Hong Kong
AGPO: Asymmetric Group Policy Optimization for Verifiable Reasoning and Search Ads Relevance at JD
May 7, 2026Yang Xu, Kun Yao, Yiming Deng +3RL for Language Model Reasoning Policy Optimization
Nanjing University, Nanjing, China · Peking University, Beijing, China · JD.com, Beijing, China
Nonsense Helps: Prompt Space Perturbation Broadens Reasoning Exploration
May 7, 2026Langlin Huang, Chengsong Huang, Jinyuan Li +3LLM Prompting RL for Language Model Reasoning
Washington University in St. Louis
SPARK: Self-Play with Asymmetric Reward from Knowledge Graphs
May 7, 2026Hyobin Park, Taeseop Kim, Dong-Geol ChoiAI for Science Self-Play RL
ANTLAB, South Korea · Hanbat National University, South Korea
Rollout Pass-Rate Control: Steering Binary-Reward RL Toward Its Most Informative Regime
May 6, 2026Tianshu Zhu, Wenyu Zhang, Xiaoying Zuo +8RL for Code Generation Reinforcement Learning
Baidu
EP-GRPO: Entropy-Progress Aligned Group Relative Policy Optimization with Implicit Process Guidance
May 6, 2026Song Yu, Li Li, Wenwen Zhao +1RL for Language Model Reasoning Group Relative Policy Optimization
School of Computer and Information Science, Southwest University, Chongqing 400715, China
Correct Is Not Enough: Training Reasoning Planners with Executor-Grounded Rewards
May 5, 2026Tianyang Han, Hengyu Shi, Junjie Hu +3RL for Code Generation Process Reward Models
1D4 Lab · 2Independent Researcher
Reference-Sampled Boltzmann Projection for KL-Regularized RLVR: Target-Matched Weighted SFT, Finite One-Shot Gaps, and Policy Mirror Descent
May 4, 2026Yao Shu, Chenxing Wei, Hongbin Lin +2Supervised Fine-Tuning Reinforcement Learning
Binary Rewards and Reinforcement Learning: Fundamental Challenges
May 4, 2026Marc DymetmanReinforcement Learning KL-Regularized RL
∗Scientific Consultant
Selector-Guided Autonomous Curriculum for One-Shot Reinforcement Learning from Verifiable Rewards
May 3, 2026Rudray Dave, Vedang Dubey, Smit Deoghare +1Reinforcement Learning RL for Language Model Reasoning
Department of Artificial Intelligence Sardar Vallabhbhai National Institute of Technology Surat, India
MIRL: Mutual Information-Guided Reinforcement Learning for Vision-Language Models
May 2, 2026Yin Zhang, Jiaxuan Zhao, Zonghan Wu +5Vision-Language Models Reinforcement Learning
School of Mathematics, Tianjin University, Tianjin, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · Shanghai Advanced Institute of Finance (SAIFS), East China Normal University, Shanghai, China +4
Exploring Pass-Rate Reward in Reinforcement Learning for Code Generation
May 1, 2026Xin-Ye Li, Ren-Biao Liu, Yun-Ji Zhang +3RL for Code Generation Reinforcement Learning
State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing 210023, China · School of Artificial Intelligence, Nanjing University, Nanjing 210023, China
ResRL: Boosting LLM Reasoning via Negative Sample Projection Residual Reinforcement Learning
May 1, 2026Zihan Lin, Xiaohan Wang, Jie Cao +6RL for Language Models Reinforcement Learning
MAIS&NLPR, Institute of Automation, Chinese Academy of Sciences, Beijing, China · Meituan, Beijing, China · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences, Beijing, China.
Uniform-Correct Policy Optimization: Breaking RLVR's Indifference to Diversity
May 1, 2026Anamika Lochab, Bolian Li, Ruqi ZhangRL for Language Model Reasoning Group Relative Policy Optimization
Department of Computer Science Purdue University, West Lafayette
ANCORA: Learning to Question via Manifold-Anchored Self-Play for Verifiable Reasoning
Apr 30, 2026Chengcao YangSelf-Play RL RL for Language Model Reasoning
Wuhan University Wuhan, China
Co-Evolving Policy Distillation
Apr 29, 2026Naibin Gu, Chenxu Yang, Qingyi Si +7Reinforcement Learning with Verifiable Rewards Multi-Teacher Knowledge Distillation
Institute of Information Engineering, CAS, 2School of Cyber Security, UCAS, 3JD.COM

May 8, 2026, cs.LGJ/K move · Enter open · S save

Gradient Starvation in Binary-Reward GRPO: Why Group-Mean Centering Fails and Why the Simplest Fix Works

Wenhua Nie, Jianan Wu, Junlin Liu, Ziwei Li+5

National Taiwan University

Reinforcement Learning Group Relative Policy Optimization Reinforcement Learning with Verifiable Rewards

Open paper AskPDF arXiv

Group Relative Policy Optimization (GRPO) is a standard algorithm for reinforcement learning from verifiable rewards, but its group-mean-centered advantage can fail under binary rewards. The failure mode is gradient starvation: when every response in a group is correct or every response is wrong, the centered advantage is exactly zero and the policy receives no learning signal. We prove that the true degeneracy rate always exceeds the i.i.d. Bernoulli prediction by Jensen's inequality, and observe a 0.69 degeneracy rate at group size four in logged Qwen3.5-9B GSM8K training. We then show that the fixed-reference Sign advantage,

A=2r-1

, performs pass@

G

failure descent by increasing the probability that at least one sample in the group succeeds. On the full GSM8K test set across seven seeds, Sign reaches 73.8% accuracy versus 28.4% for standard normalized group-mean DrGRPO at group size four, a 45.4 point gain with

p<0.0001

. The effect is directionally consistent on Llama-3.1-8B and positive but underpowered on a MATH-500 transfer check. Pass@

k

analysis indicates that the main benefit is search compression rather than large capacity expansion, aligning the empirical gains with recent RLVR ceiling observations.

Momentum

Related topics

Latest papers 438