Reinforcement Learning with Verifiable Rewards

Also known as RLVR

This field studies how to improve language and multimodal models by rewarding outputs whose correctness or task progress can be checked, for example with unit tests, mathematical answers, formal proofs, or simulated environments. Common approaches use policy-gradient methods such as PPO and GRPO, with outcome rewards supplemented by step-level feedback, rollout exploration, or replay to improve credit assignment and data efficiency. Work spans math and coding benchmarks as well as search, GUI, and other interactive agents. A central tension is that rewards must be scalable yet faithful: imperfect or narrow verifiers can be exploited, and a correct final answer does not by itself establish sound reasoning, visual grounding, or robust generalization.

Latest papers

692 papers137 papers in the last four weeks, up 111% on the four weeks before. 0.8% of all new papers.

Breaking the Impasse: Dual-Scale Evolutionary Policy Training for Social Language Agents
May 9, 2026Minzheng Wang, Run Luo, Yanbo Wang +6Reinforcement Learning Self-Play RL
MAIS, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · National University of Singapore +1
The Cancellation Hypothesis in Critic-Free RL: From Outcome Rewards to Token Credits
May 9, 2026Tianhao Cheng, Zeyu Huang, Zihan Qiu +5Reinforcement Learning Token-Level Credit Assignment
1Fudan University · 3The University of Edinburgh · 6Qwen Team, Alibaba Group +4
PYTHALAB-MERA: Validation-Grounded Memory, Retrieval, and Acceptance Control for Frozen-LLM Coding Agents
May 8, 2026Mehmet IscanRL for Code Generation Execution-Guided Code Generation
PythaLab, Yildiz Technical University
DUET: Optimize Token-Budget Allocation for Reinforcement Learning with Verifiable Rewards
May 8, 2026Haoyu Hu, Xuandong Zhao, Xuhai "Orson'' Xu +1Reinforcement Learning Group Relative Policy Optimization
Cornell University · University of California, Berkeley · Columbia University
AIPO: Learning to Reason from Active Interaction
May 8, 2026Junnan Liu, Linhao Luo, Thuy-Trang Vu +1RL for Language Model Reasoning LLM-Guided RL
Department of Data Science and AI, Faculty of Information Technology, Monash University, Australia
Rubric-Grounded RL: Structured Judge Rewards for Generalizable Reasoning
May 8, 2026Manish Bhattarai, Ismael Boureima, Nishath Rajiv Ranasinghe +2Reinforcement Learning RL for Language Model Reasoning
Los Alamos National Laboratory, Los Alamos, NM, USA.
Interactive Critique-Revision Training for Reliable Structured LLM Generation
May 8, 2026Fei Xu Yu, Zuyuan Zhang, Mahdi Imani +2LLM Self-Refinement Group Relative Policy Optimization
The George Washington University · Northeastern University · United States Military Academy
Gradient Starvation in Binary-Reward GRPO: Why Group-Mean Centering Fails and Why the Simplest Fix Works
May 8, 2026Wenhua Nie, Jianan Wu, Junlin Liu +6Reinforcement Learning Group Relative Policy Optimization
National Taiwan University
Your Language Model is Its Own Critic: Reinforcement Learning with Value Estimation from Actor's Internal States
May 8, 2026Yunho Choi, Jongwon Lim, Woojin Ahn +3Value Function Estimation RL for Language Model Reasoning
Graduate School of Data Science Seoul National University · Computer Science and Engineering Seoul National University
HTPO: Towards Exploration-Exploitation Balanced Policy Optimization via Hierarchical Token-level Objective Control
May 8, 2026Xincheng Yao, Ruoqi Li, Cheng Chen +4RL for Language Models Exploration-Exploitation Tradeoff
School of Information Science and Electronic Engineering, Shanghai Jiao Tong University · Xiaohongshu Inc. · MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University.
Implicit Compression Regularization: Concise Reasoning via Internal Shorter Distributions in RL Post-Training
May 8, 2026Chen Wang, Hexuan Deng, Yining Zhang +5Overthinking in Language Models Reinforcement Learning with Verifiable Rewards
College of Software, Nankai University · 2Zhongguancun Academy · 3Harbin Institute of Technology +4
Structured Role-Aware Policy Optimization for Multimodal Reasoning
May 8, 2026Bingqing Jiang, Difan ZouToken-Level Credit Assignment Multimodal Reasoning
School of Computing & Data Science, The University of Hong Kong · School of Computing & Data Science and Institute of Data Science, The University of Hong Kong
Can You Break RLVER? Probing Adversarial Robustness of RL-Trained Empathetic Agents
May 8, 2026Deeraj S K, Sadhana Devarajan, Krishna Mehra +1Emotion Recognition in Conversations Adversarial Robustness of Language Models
Department of Artificial Intelligence, Sardar Vallabhbhai National Institute of Technology, Surat, India
Adaptive Negative Reinforcement for LLM Reasoning:Dynamically Balancing Correction and Diversity in RLVR
May 8, 2026Yash Ingle, Jaival Chauhan, Ankit Yadav +1Reinforcement Learning RL for Language Model Reasoning
Sardar Vallabhbhai National Institute of Technology (SVNIT), Surat, India
Where to Spend Rollouts: Hit-Utility Optimal Rollout Allocation for Group-Based RLVR
May 8, 2026Tao Wang, Shuo Li, Yan Sun +2RL for Language Model Reasoning Group Relative Policy Optimization
University of Pennsylvania · Now at Google DeepMind · New Jersey Institute of Technology +1
Verifier-Backed Hard Problem Generation for Mathematical Reasoning
May 7, 2026Yuhang Lai, Jiazhan Feng, Yee Whye Teh +1Mathematical Problem Generation Reinforcement Learning with Verifiable Rewards
Department of Data Science, City University of Hong Kong · School of Intelligence Science and Technology, Peking University · Work done during Jiazhan’s visiting at the Department of Statistics, University of Oxford. +2
Beyond Negative Rollouts: Positive-Only Policy Optimization with Implicit Negative Gradients
May 7, 2026Mingwei Xu, Hao FangReinforcement Learning RL for Language Model Reasoning
University of Washington, Seattle, USA
On the Implicit Reward Overfitting and the Low-rank Dynamics in RLVR
May 7, 2026Hao Ye, Jisheng Dang, Junfeng Fang +7Reward Over-Optimization Reinforcement Learning
Lanzhou University · National University of Singapore · Beijing Institute of Technology
Skip What You Can Predict: Predictive Repositioning for Policy Optimization for Efficient LLM Training
May 7, 2026Ismam Nur Swapnil, Aranya Saha, Tasneea Zahra +3RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Bangladesh University of Engineering and Technology · University of Maryland, College Park · Illinois Institute of Technology +1
OPSD Compresses What RLVR Teaches: A Post-RL Compaction Stage for Reasoning Models
May 7, 2026Jaehoon Kim, Dongha LeeReinforcement Learning with Verifiable Rewards On-Policy Distillation
Yonsei University
Listwise Policy Optimization: Group-based RLVR as Target-Projection on the LLM Response Simplex
May 7, 2026Yun Qu, Qi Wang, Yixiu Mao +11RL for Language Model Reasoning Group Relative Policy Optimization
Department of Automation, Tsinghua University · 2LLM Department, Tencent
Selective Successor Eligibility Traces for RLVR
May 7, 2026Chaoli Mou, Xinning Chen, Zhan Zhuang +3RL for Language Model Reasoning Credit Assignment in RL
Southern University of Science and Technology · City University of Hong Kong
AGPO: Asymmetric Group Policy Optimization for Verifiable Reasoning and Search Ads Relevance at JD
May 7, 2026Yang Xu, Kun Yao, Yiming Deng +3RL for Language Model Reasoning Policy Optimization
Nanjing University, Nanjing, China · Peking University, Beijing, China · JD.com, Beijing, China
Nonsense Helps: Prompt Space Perturbation Broadens Reasoning Exploration
May 7, 2026Langlin Huang, Chengsong Huang, Jinyuan Li +3LLM Prompting RL for Language Model Reasoning
Washington University in St. Louis
SPARK: Self-Play with Asymmetric Reward from Knowledge Graphs
May 7, 2026Hyobin Park, Taeseop Kim, Dong-Geol ChoiAI for Science Self-Play RL
ANTLAB, South Korea · Hanbat National University, South Korea
Rollout Pass-Rate Control: Steering Binary-Reward RL Toward Its Most Informative Regime
May 6, 2026Tianshu Zhu, Wenyu Zhang, Xiaoying Zuo +8RL for Code Generation Reinforcement Learning
Baidu
EP-GRPO: Entropy-Progress Aligned Group Relative Policy Optimization with Implicit Process Guidance
May 6, 2026Song Yu, Li Li, Wenwen Zhao +1RL for Language Model Reasoning Group Relative Policy Optimization
School of Computer and Information Science, Southwest University, Chongqing 400715, China
Correct Is Not Enough: Training Reasoning Planners with Executor-Grounded Rewards
May 5, 2026Tianyang Han, Hengyu Shi, Junjie Hu +3RL for Code Generation Process Reward Models
1D4 Lab · 2Independent Researcher
Reference-Sampled Boltzmann Projection for KL-Regularized RLVR: Target-Matched Weighted SFT, Finite One-Shot Gaps, and Policy Mirror Descent
May 4, 2026Yao Shu, Chenxing Wei, Hongbin Lin +2Supervised Fine-Tuning Reinforcement Learning
Binary Rewards and Reinforcement Learning: Fundamental Challenges
May 4, 2026Marc DymetmanReinforcement Learning KL-Regularized RL
∗Scientific Consultant

May 9, 2026J/K move · Enter open · S save

Breaking the Impasse: Dual-Scale Evolutionary Policy Training for Social Language Agents

Minzheng Wang, Run Luo, Yanbo Wang, Zichen Liu+5

MAIS, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · National University of Singapore+1

Reinforcement Learning Self-Play RL Reinforcement Learning with Verifiable Rewards Conversational Agents Evolutionary RL

Open paper AskPDF arXiv

While Reinforcement Learning with Verifiable Rewards (RLVR) has proven effective for closed-ended tasks, extending it to open-ended social language games via self-play reveals a critical issue: evolution impasse. Due to the vast strategy space, language agents frequently converge to homogenized behaviors, leading to deterministic match outcomes that eliminate the gradient signals necessary for policy evolution. To tackle this issue, we propose Dual-scale Evolutionary Policy Training (DEPT) for social language games. DEPT introduces a time-scaled evolutionary perception mechanism that detects impasse by quantifying dual-scale value baseline divergence alongside match entropy. Upon perceiving the collapse, it then activates asymmetric advantage reshaping to dynamically modulate the optimization landscape for intervention. Thus, our method effectively restores gradient signals and enforces sustained strategic exploration. Extensive experiments on multiple social language games demonstrate that DEPT outperforms strong baselines, avoiding policy degeneration and driving the continuous evolution of social language agents.