Reinforcement Learning with Verifiable Rewards

Also known as RLVR

This field studies how to improve language and multimodal models by rewarding outputs whose correctness or task progress can be checked, for example with unit tests, mathematical answers, formal proofs, or simulated environments. Common approaches use policy-gradient methods such as PPO and GRPO, with outcome rewards supplemented by step-level feedback, rollout exploration, or replay to improve credit assignment and data efficiency. Work spans math and coding benchmarks as well as search, GUI, and other interactive agents. A central tension is that rewards must be scalable yet faithful: imperfect or narrow verifiers can be exploited, and a correct final answer does not by itself establish sound reasoning, visual grounding, or robust generalization.

Latest papers

692 papers137 papers in the last four weeks, up 111% on the four weeks before. 0.8% of all new papers.

PIVOT: Progressive Intervention via Online Teacher Correction
Oct 29, 2026Zhihao Yang, Boyuan Zheng, Puhui Tan +7RL for Language Model Reasoning LLM-Guided RL
University of Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · Tsinghua University +1
SCARL: Sequence Confidence-Aligned Reinforcement Learning
Oct 29, 2026Zhenghong Ke, Jingbo Zhou, Hua WuReinforcement Learning RL for Language Model Reasoning
Baidu Frontier Research Department · Nanjing University
Distributional Reward Calibration and Budget-Neutral Credit Assignment for Multimodal RLVR
Oct 29, 2026Lang Chen, Zongxin Xu, Borui Wang +2Reinforcement Learning Token-Level Credit Assignment
Shenzhen University
H $^2$ R: Hierarchical Hint-Based Reward for Mathematical Reasoning via Reinforcement Learning
Oct 29, 2026He Tu, Siye Li, Hongyi WenLLM Mathematical Reasoning Reward Shaping
New York University Shanghai
There Is No Free Lunch in RLVR: Every pass@1 Gain Has a pass@k Price
Oct 29, 2026Yinuo LiuRL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Grounding Multimodal Reasoning with Evidence-Ablated Negatives
Oct 29, 2026Chenxi Liu, Tianyi Xiong, Yanshuo Chen +1Multimodal Reasoning RL for VLMs
Department of Computer Science, University of Maryland, College Park, College Park, MD 20742
Collective Dynamics of Self-Reinforcement: How Cross-Task Coupling Drives Policy Collapse in Unsupervised RLVR
Oct 27, 2026Peize DingRL for Language Models Reinforcement Learning
Columbia University, New York, NY, USA
Uncovering Shortcut Amplification in Video RLVR through the Lens of Perceptual Uncertainty Collapse
Oct 27, 2026Jiafeng Liang, Xianhao Ou, Tao Ren +6RL for Video Reasoning Reinforcement Learning with Verifiable Rewards
Harbin Institute of Technology · Peking University · Huawei Technologies Co., Ltd +2
Guided-RLVR: Guided Exploration in Reinforcement Learning with Verifiable Rewards
Oct 27, 2026Bo Yang, Zishuai Zhang, Weihua Li +3RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, Beihang University · School of Artificial Intelligence, Beihang University
When All Rollouts Fail: Policy-Compatible Joint Decoding for Hard-Problem Reasoning
Oct 27, 2026Zehao Wang, Hanzhu Chen, Runquan Gui +2RL for Language Models RL for Language Model Reasoning
University of Science and Technology of China · Tencent · Shanghai Artificial Intelligence Laboratory
MVRCap: Towards Comprehensive Video Captioning via Hierarchical Multi-Dimensional Verifiable Rewards
Oct 27, 2026Weihuang Lin, Xianing Chen, Zhibin Wang +9Video Captioning Reinforcement Learning with Verifiable Rewards
Xiamen University · Alibaba Group
Audit Where Hacks Pay: Budgeted Strong-Verifier Audits Against Reward Hacking in GRPO
Oct 26, 2026Shurun Li, Michael Hua Wang, Haibo Zeng +1Reward Hacking Reinforcement Learning with Verifiable Rewards
REASONS Lab, Institute for Network Sciences and Cyberspace, Tsinghua University · Virginia Tech
S3AC: Stabilized, Selective, and Scalable Actor-Critic for LLM Reasoning
Oct 26, 2026Shuo Shen, Guangran Cheng, Chengqi Lyu +3RL for Language Model Reasoning RL for Language Models
Shanghai AI Laboratory · Independent Researcher
Improving Reasoning with Rollout Agreement for Group-Based Policy Optimization
Oct 26, 2026Sohyun Son, Yerim Hong, Jaeyun Park +1Reinforcement Learning RL for Language Model Reasoning
Kyung Hee University, Yongin, Republic of Korea
Audio-R1: Audio Reasoning through Reinforcement Learning on Self-Supervised Tasks
Oct 25, 2026Jihoon Jeong, Mirco Ravanelli, Zhepei Wang +1Audio Reasoning Audio-Language Models
Mila-Québec AI Institute · Université Laval · Concordia University +1
Self-Evolution Reasoner: Continuous Optimization via Policy-Intrinsic Exploration and Exploitation
Oct 25, 2026Wenhang Shi, Yiren Chen, Zhe Zhao +3Language Model Self-Improvement RL for Language Model Reasoning
Renmin University of China · Peking University · Tencent
A Sign Law for Entropy Drift under Sampler Mismatch in RLVR
Oct 25, 2026Zhiyuan Wang, Jacob Steeves, Chang LiuReinforcement Learning Reinforcement Learning with Verifiable Rewards
Of Chains and Gains: What Reinforcement Learning changes on Reasoning-based Image Quality Assessment
Oct 23, 2026Yixuan Li, Yuxuan Jiang, Hanwei Zhu +4Image Quality Assessment Reinforcement Learning with Verifiable Rewards
University of Bristol, UK · Jiangxi University of Finance and Economics, China · China University of Mining and Technology, China +1
From Visual Affinity to Learning Credit in Multimodal Reinforcement Learning
Oct 23, 2026Yuhang Han, Dingning Liu, Yujie Chen +7RL for VLMs Step-Level Credit Assignment in RL
The Hong Kong University of Science and Technology (Guangzhou) · Shanghai Jiao Tong University · Shadoweave +2
MS-TTRL: Breaking the Frequency Conflict in Multiple-Solution Test-Time Reinforcement Learning
Oct 23, 2026Zhixia Zhang, Binqian Xu, Huaiyang Wang +2Test-Time RL RL for Language Model Reasoning
Beihang University · National University of Singapore
SOPO: Socratic Guided Policy Optimization for Span-Level Hallucination Detection
Oct 23, 2026Jiawei Dong, Zilong Bai, Pengtian zhu +1Hallucination Detection LLM Hallucination Detection
Patsnap Pte. Ltd., Suzhou, China
Questioning Instrumental Convergence in RLVR: Checkpoint Evidence from OLMo-3
Oct 22, 2026Barış Alparslan, Sandy Tanwisuth, Caleb PettersonRL Post-Training Reinforcement Learning with Verifiable Rewards
ASELSAN · BlueDot Impact · University of Minnesota - Twin Cities
Flat-RLVR: Sharpness-Aware Reinforcement Learning with Verifiable Rewards for Reasoning Capacity in LLMs
Oct 21, 2026Yeonbum Kim, Hyun Kyu Lee, Sung Whan YoonRL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Graduate School of Artificial Intelligence, Ulsan National Institute of Science and Technology, Ulsan, Republic of Korea · Department of Electrical Engineering, Ulsan National Institute of Science and Technology (UNIST), Ulsan, Republic of Korea.
ReAPO: Retry-Aware Policy Optimization for Efficient Exploration in LLM Reasoning
Oct 20, 2026Nghi Dao, Calder Morton-Ferguson, Noah Golowich +1RL for Language Model Reasoning Efficient Language Model Reasoning
Columbia University · Yale University · UT Austin
Self-Credit: Credit Assignment for Long-Horizon Agentic RL via Intra-group Rescaling
Oct 20, 2026Fangcong Yin, Ming Yin, Tanvi Bhandarkar +7Agentic RL Step-Level Credit Assignment in RL
Meta · New York University · Duke University
Reinforcement Learning with Visually Attributable On-Policy Self-Distillation
Oct 19, 2026Shiao Xie, Siyu Chen, Jianwei Lv +2RL for VLMs On-Policy Self-Distillation
Baidu Inc., China
Co-Evolving Policy and Reward for Code Generation: A Mechanistic Study of Single-Adapter Drift
Oct 19, 2026Amarsaikhan Batjargal, Noriko TomuroRL for Code Generation Reinforcement Learning
Jarvis College of Computing and Digital Media, DePaul University, Chicago, IL, USA
Policy-Adaptive Credit Assignment for Spatial Reasoning with Verifiable Rewards
Oct 19, 2026Hailiang Yang, Shengchao ChenProcess Reward Models Step-Level Credit Assignment in RL
School of Computer Science and Engineering, University of New South Wales · Australian AI Institute, University of Technology Sydney
Exploration via Activation-Space Steering for Reinforcement Learning with Verifiable Rewards
Oct 19, 2026Yunan Zeng, Jinhao Jing, Xi Cheng +1RL for Language Model Reasoning RL Exploration
Fudan University · The Chinese University of Hong Kong, Shenzhen · Kunlun Inc. +1
Replan for Error Recovery: Training Embodied VLMs through Verifiable Interaction with Action-Grouped Turn-GSPO
Oct 19, 2026Zhuoran Wang, Keqing Fan, Yang Li +6VLMs for Robotics Robot Task Planning
School of Information, Renmin University of China · Key Laboratory of Data Engineering and Knowledge Engineering, Beijing, China · Beihang University +2

Oct 29, 2026J/K move · Enter open · S save

PIVOT: Progressive Intervention via Online Teacher Correction

Zhihao Yang, Boyuan Zheng, Puhui Tan, Yukun Chen+6

University of Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · Tsinghua University+1

RL for Language Model Reasoning LLM-Guided RL Reinforcement Learning with Verifiable Rewards

Open paper AskPDF arXiv

Reinforcement learning from verifiable rewards (RLVR) is limited by sparse outcome-level feedback on hard reasoning tasks: when all rollouts in a group fail, rewards provide no learning signal. Stronger teachers can provide additional guidance, but this raises a central question: how can such signals be incorporated into RLVR to support successful rollouts while remaining learnable for the student? We distinguish two complementary mechanisms: trajectory-level intervention, which improves access to reward-bearing trajectories, and token-level supervision, which provides dense targets to guide student learning. Our controlled experiments show that either mechanism alone can remain insufficient, motivating their combination. We therefore introduce PIVOT, which inserts teacher-generated corrections into student rollouts and applies token-level KL supervision to these segments alongside the RLVR objective. Beyond their combination, we systematically analyze where to intervene and how to supervise, yielding a recipe for effective guidance. Across hard logic games and mathematical reasoning benchmarks, PIVOT consistently outperforms existing methods, nearly doubling mean avg@8 on logic games from 19.5% for the best-performing baseline to 35.6%.