Reinforcement Learning with Verifiable Rewards

Also known as RLVR

This field studies how to improve language and multimodal models by rewarding outputs whose correctness or task progress can be checked, for example with unit tests, mathematical answers, formal proofs, or simulated environments. Common approaches use policy-gradient methods such as PPO and GRPO, with outcome rewards supplemented by step-level feedback, rollout exploration, or replay to improve credit assignment and data efficiency. Work spans math and coding benchmarks as well as search, GUI, and other interactive agents. A central tension is that rewards must be scalable yet faithful: imperfect or narrow verifiers can be exploited, and a correct final answer does not by itself establish sound reasoning, visual grounding, or robust generalization.

Latest papers

692 papers137 papers in the last four weeks, up 111% on the four weeks before. 0.8% of all new papers.

Hide to Guide: Learning via Semantic Masking
May 24, 2026Ruitao Liu, Qinghao Hu, Alex Hu +6Reward Hacking RL for Language Model Reasoning
1MIT · 2NVIDIA
RL with Learnable Textual Feedback: A Bilevel Approach
May 23, 2026Utsav Singh, Sidhaarth Sredharan, Souradip Chakraborty +1Reinforcement Learning RL for Language Model Reasoning
University of Central Florida · 2Carnegie Mellon University · 3Universtiy of Maryland
CurveRL: Principled Distribution-Aware Context Reweighting for LLM Reasoning
May 23, 2026Ke Sun, Yizhou Zhao, Jiayi Xin +2Reinforcement Learning RL for Language Model Reasoning
University of Pennsylvania
Clipping Bottleneck: Stabilizing RLVR via Stochastic Recovery of Near-Boundary Signals
May 21, 2026Shuo Yang, Jinda Lu, Chiyu Ma +8RL for Language Model Reasoning Group Relative Policy Optimization
Qwen Pilot Team, Alibaba Group
One-Way Policy Optimization for Self-Evolving LLMs
May 21, 2026Shuo Yang, Jinda Lu, Kexin Huang +6Policy Optimization Reinforcement Learning with Verifiable Rewards
1Shenzhen Graduate School, Peking Uni · 2Qwen Pilot Team · 2Qwen Pilot Team 3Dartmouth College +1
From Reasoning Chains to Verifiable Subproblems: Curriculum Reinforcement Learning Enables Credit Assignment for LLM Reasoning
May 21, 2026Xitai Jiang, Zihan Tang, Wenze Lin +3RL for Language Model Reasoning Credit Assignment in RL
LeapLab, Tsinghua University · Qiuzhen College, Tsinghua University · B Gao Huang
ECPO: Evidence-Coupled Policy Optimization for Verifier-Audited Candidate Ranking
May 21, 2026Miaobo Hu, Shuhao Hu, BoKun Wang +4Data Provenance Learning to Rank
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China
OPPO: Bayesian Value Recursion for Token-Level Credit Assignment in LLM Reasoning
May 21, 2026Yu Li, Rui Miao, Tian Lan +1Value Function Estimation Token-Level Credit Assignment
1George Washington University · 2The University of Texas at Dallas
You Only Need Minimal RLVR Training: Extrapolating LLMs via Rank-1 Trajectories
May 20, 2026Zhepei Wei, Xinyu Zhu, Wei-Lin Chen +3RL for Language Model Reasoning Low-Rank Approximation
University of Virginia · Washington University in St. Louis
DelTA: Discriminative Token Credit Assignment for Reinforcement Learning from Verifiable Rewards
May 20, 2026Kaiyi Zhang, Wei Wu, Yankai LinPolicy Gradient Token-Level Credit Assignment
Gaoling School of Artificial Intelligence, Renmin University of China · Ant International
How Much Online RL is Enough? Informative Rollouts for Offline Preference Optimization in RLVR
May 20, 2026Richa Verma, Balaraman RavindranDirect Preference Optimization RL for Language Model Reasoning
TCS Research Department of CSE IIT Madras Chennai, India · Department of Data Science & AI, Wadhwani School of Data Science & AI IIT Madras Chennai, India
LamPO: A Lambda Style Policy Optimization for Reasoning Language Models
May 20, 2026Redacted by arXivRL for Language Model Reasoning Group Relative Policy Optimization
Pinterest, San Francisco, CA 94107, USA
Domain-Adaptable Reinforcement Learning for Code Generation with Dense Rewards
May 20, 2026Erfan Aghadavoodi Jolfaei, Daniel Maninger, Abhinav Anand +2RL for Code Generation Reinforcement Learning
Technische Universität Darmstadt, Darmstadt, Germany · Hessian Center for Artificial Intelligence (hessian.AI), Darmstadt, Germany · National Research Center for Applied Cybersecurity ATHENE, Darmstadt, Germany.
Advantage Collapse in Group Relative Policy Optimization: Diagnosis and Mitigation
May 20, 2026Xixiang He, Qiyao Sun, Ao Cheng +5RL for Language Model Reasoning Group Relative Policy Optimization
National University of Defense Technology, Changsha, Hunan, China · Intelligent Game and Decision Lab, Beijing, China · The Chinese University of Hong Kong, Shenzhen, Guangdong, China
Multi-Step Likelihood-Ratio Correction for Reinforcement Learning with Verifiable Rewards
May 20, 2026Deokgyu Yoon, Hyungkyu Kang, Joongkyu Lee +4RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
1Seoul National University · 2Upstage
PlexRL: Cluster-Level Orchestration of Serviceized LLM Execution for RLVR
May 20, 2026Yiqi Zhang, Fangzheng Jiao, Tian Tang +13LLM Inference Efficiency LLM Serving
National University of Singapore · Shanghai Qiji Zhifeng Co., Ltd. · Beihang University +3
Not Every Rubric Teaches Equally: Policy-Aware Rubric Rewards for RLVR
May 19, 2026Utkarsh Tyagi, Xingang Guo, MohammadHossein Rezaei +5Reward Modeling Rubric-Based RL
Scale AI · Persona
Rewarding Beliefs, Not Actions: Consistency-Guided Credit Assignment for Long-Horizon Agents
May 19, 2026Wenjie Tang, Minne Li, Sijie Huang +2Reinforcement Learning Partially Observable RL
College of Computer, National University of Defense Technology · 2Intelligent Game and Decision Lab (IGDL) · Institute of Artificial Intelligence, Xiamen University
GeoX: Mastering Geospatial Reasoning Through Self-Play and Verifiable Rewards
May 19, 2026Kyeongjin Ahn, Seungeon Lee, Krishna P. Gummadi +1Spatial Reasoning Self-Play RL
KAIST · MPI-SWS · MPI-SP
GoLongRL: Capability-Oriented Long Context Reinforcement Learning with Multitask Alignment
May 19, 2026Minxuan Lv, Tiehua Mei, Tanlong Du +9Reinforcement Learning Long-Context Modeling
Kuaishou Technology · University of Chinese Academy of Sciences
CEPO: RLVR Self-Distillation using Contrastive Evidence Policy Optimization
May 19, 2026Ahmed Heakl, Abdelrahman M. Shaker, Youssef Mohamed +4Token-Level Credit Assignment RL for Language Model Reasoning
MBZUAI · Linköping University · Australian National University
When to Stop Reusing: Dynamic Gradient Gating for Sample-Efficient RLVR
May 19, 2026Yuchun Miao, Sen Zhang, Yuqi Zhang +4RL for Language Model Reasoning Policy Optimization
National Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University · Meituan Longcat Team · The University of Sydney +1
Rethinking Muon Beyond Pretraining: Spectral Failures and High-Pass Remedies for VLA and RLVR
May 19, 2026Chongyu Fan, Gaowen Liu, Mingyi Hong +2Muon Optimizer Vision-Language-Action Models
Michigan State University · Cisco · University of Minnesota +1
AMR-SD: Asymmetric Meta-Reflective Self-Distillation for Token-Level Credit Assignment
May 18, 2026Zhenlin Wei, Pu Jian, Yingzhuo Deng +6Token-Level Credit Assignment RL for Language Model Reasoning
Meituan Beijing, China · University of Chinese Academy of Sciences
Knowledge-to-Verification: Exploring RLVR for LLMs in Knowledge-Intensive Domains
May 18, 2026Zhonghang Yuan, Zhefan Wang, Fang Hu +7RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Shanghai Artificial Intelligence Laboratory · Shanghai Innovation Institute
How Off-Policy Can GRPO Be? Mu-GRPO for Efficient LLM Reinforcement Learning
May 17, 2026Minghao Tian, Yunfei Xie, Chen WeiRL for Language Model Reasoning Group Relative Policy Optimization
Rice University
Leveraging Error Diversity in Group Rollouts for Reinforcement Learning
May 17, 2026Wenpu Liu, Yuqi Xu, Weichu Xie +8Reward Shaping RL for Language Model Reasoning
Peking University · Shanghai Innovation Institute
Step-wise Rubric Rewards for LLM Reasoning
May 17, 2026Weichu Xie, Haozhe Zhao, Wenpu Liu +15Reward Modeling Credit Assignment in RL
Peking University · JD Explore Academy · Shanghai Jiao Tong University +2
TIER: Trajectory-Invariant Execution Rewards for Multi-Step Tool Composition
May 16, 2026Anay Kulkarni, ChiaEn Lu, Dheeraj Mekala +3Reward Modeling Reinforcement Learning with Verifiable Rewards
UC San Diego · Cisco Research
The Unlearnability Phenomenon in RLVR for Language Models
May 16, 2026Yulin Chen, He He, Chen ZhaoReinforcement Learning RL for Language Model Reasoning
1New York University · 2NYU Shanghai

May 24, 2026J/K move · Enter open · S save

Hide to Guide: Learning via Semantic Masking

Ruitao Liu, Qinghao Hu, Alex Hu, Yecheng Wu+5

1MIT · 2NVIDIA

Reward Hacking RL for Language Model Reasoning LLM-Guided RL Reinforcement Learning with Verifiable Rewards

Open paper AskPDF arXiv

Reinforcement learning with verifiable rewards (RLVR) has become a powerful paradigm for improving language models on reasoning-intensive tasks, but its effectiveness is often limited by exploration. For example, models often fail on hard problems, leaving little useful reward signal. External expert traces offer a natural source of guidance, yet they may also expose reward-relevant content along the critical path to the verifier target, such as final answers, intermediate values, executable implementations, or answer-related entities. This content can create an unintended reward hacking channel, allowing the policy to obtain reward by copying the trace rather than learning the underlying reasoning or agentic behavior. Existing guided-RL methods reduce this risk by using partial trajectories, but they mainly control how much expert information is shown heuristically rather than which parts should be hidden. To this end, we propose Semantic Masked Expert Policy Optimization (SMEPO), a fine-grained semantic masking strategy for expert-guided RLVR. Instead of truncating traces coarsely or revealing them unchanged, SMEPO masks reward-relevant semantic spans along the critical path while preserving the expert's decomposition, plan, and procedural structure. This turns hard problems from reasoning from scratch into a fill-in-the-blank process: the policy can follow the expert's problem-solving route, but must still reconstruct the missing values, code, or entities by itself. SMEPO is simple to apply and requires no changes to the reward function or RL objective. Across diverse domains, including math, code, and agentic search, SMEPO improves accuracy by up to 3.2 points over GRPO and reduces training time by up to 4.2x. The code is available at https://github.com/mit-han-lab/SMEPO.