Reinforcement Learning with Verifiable Rewards

Also known as RLVR

This field studies how to improve language and multimodal models by rewarding outputs whose correctness or task progress can be checked, for example with unit tests, mathematical answers, formal proofs, or simulated environments. Common approaches use policy-gradient methods such as PPO and GRPO, with outcome rewards supplemented by step-level feedback, rollout exploration, or replay to improve credit assignment and data efficiency. Work spans math and coding benchmarks as well as search, GUI, and other interactive agents. A central tension is that rewards must be scalable yet faithful: imperfect or narrow verifiers can be exploited, and a correct final answer does not by itself establish sound reasoning, visual grounding, or robust generalization.

Latest papers

692 papers137 papers in the last four weeks, up 111% on the four weeks before. 0.8% of all new papers.

DyJR: Preserving Diversity in Reinforcement Learning with Verifiable Rewards via Dynamic Jensen-Shannon Replay
Mar 17, 2026Long Li, Zhijian Zhou, Tianyi Wang +7Behavioral Diversity in RL Reinforcement Learning
Griffith University, Australia · Fudan University, China · Shanghai Innovation Institute, China +3
Noisy Data is Destructive to Reinforcement Learning with Verifiable Rewards
Mar 17, 2026Yuxuan Zhu, Daniel KangReinforcement Learning with Verifiable Rewards Noisy-Label Learning
UIUC · Bridgewater AIA Labs
Molecular Identifier Visual Prompt and Verifiable Reinforcement Learning for Chemical Reaction Diagram Parsing
Mar 16, 2026Jiahe Song, Chuang Wang, Yinfan Wang +11Visual Prompt Learning Reinforcement Learning with Verifiable Rewards
Shanghai Jiao Tong University · Shanghai AI Laboratory · Beihang University +2
From $\log π$ to $π$ : Taming Divergence in Soft Clipping via Bilateral Decoupled Decay of Probability Gradient Weight
Mar 15, 2026Xiaoliang Fu, Jiaye Lin, Yangyi Fang +6RL for Language Model Reasoning Policy Gradient Methods
Meituan · Fudan University · Tsinghua University +2
Rethinking Multiple-Choice Questions for RLVR: Unlocking Potential via Distractor Design
Mar 13, 2026Xu Guo, Qiming Ge, Jian Tong +8RL for Language Model Reasoning Multiple-Choice Question Answering
Shanghai AI Laboratory · Shanghai Innovation Institute · Fudan University
EvolveCoder: Evolving Test Cases via Adversarial Verification for Code Reinforcement Learning
Mar 13, 2026Chi Ruan, Dongfu Jiang, Huaye Zeng +2RL for Code Generation Reinforcement Learning with Verifiable Rewards
University of Waterloo · Vector Institute · Harvard University
Hindsight-Anchored Policy Optimization: Learning Through Hindsight with Thompson Sampling-Inspired Adaptive Gating
Mar 11, 2026Yuning Wu, Ke Wang, Haoran Liu +3Reinforcement Learning RL for Language Model Reasoning
Amazon
$V_{0.5}$ : Generalist Value Model as a Prior for Sparse RL Rollouts
Mar 11, 2026Yi-Kai Zhang, Yueqing Sun, Hongyan Hao +4Reinforcement Learning with Verifiable Rewards Value Function Estimation
School of Artificial Intelligence, Nanjing University · National Key Laboratory for Novel Software Technology, Nanjing University · Meituan, China
Reinforcement Learning with Conditional Expectation Reward
Mar 11, 2026Changyi Xiao, Caijun Xu, Yixin CaoRL for Language Model Reasoning LLM Answer Verification
Fudan University
Does LLM Alignment Really Need Diversity? An Empirical Study of Adapting RLVR Methods for Moral Reasoning
Mar 11, 2026Zhaowei Zhang, Xiaohan Liu, Xuekai Zhu +6LLM Alignment Distribution Matching
Institute for Artificial Intelligence, Peking University · University of Michigan · Shanghai Jiao Tong University +3
Safe and Scalable Web Agent Learning via Recreated Websites
Mar 11, 2026Hyungjoo Chae, Jungsoo Park, Alan RitterWeb Agents LLM Agent Training
School of Interactive Computing, Georgia Institute of Technology
CLIPO: Contrastive Learning in Policy Optimization Generalizes RLVR
Mar 10, 2026Sijia Cui, Pengyu Cheng, Jiajun Song +6RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Qwen Large Model Application Team, Alibaba · Institute of Automation, Chinese Academy of Sciences
Good Reasoning Makes Good Demonstrations: Implicit Reasoning Quality Supervision via In-Context Reinforcement Learning
Mar 10, 2026Tiehua Mei, Minxuan Lv, Leiyu Pan +5RL for Language Model Reasoning LLM Reasoning
School of Data Science, Fudan University · University of Chinese Academy of Sciences · College of Intelligence and Computing, Tianjin University
Decoupling Reasoning and Confidence: Resurrecting Calibration in Reinforcement Learning from Verifiable Rewards
Mar 10, 2026Zhengzhao Ma, Xueru Wen, Boxi Cao +6Reinforcement Learning RL for Language Model Reasoning
Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China +2
How Far Can Unsupervised RLVR Scale LLM Training?
Mar 9, 2026Bingxiang He, Yuxin Zuo, Zeyuan Liu +18RL for Language Models Reinforcement Learning with Verifiable Rewards
Tsinghua University · Shanghai AI Lab · Xi’an Jiaotong University +3
SWE-Fuse: Empowering Software Agents via Issue-free Trajectory Learning and Entropy-aware RLVR Training
Mar 9, 2026Xin-Cheng Wen, Binbin Chen, Haoxuan Lan +3Software Engineering Agents LLM Agent Training
Ant Group · The Chinese University of Hong Kong
Countdown-Code: A Testbed for Studying The Emergence and Generalization of Reward Hacking in RLVR
Mar 7, 2026Muhammad Khalifa, Zohaib Khan, Omer Tafveez +2RL Benchmarks Reward Hacking
NVIDIA · University of Michigan · University of Illinois Urbana-Champaign
Chart-RL: Generalized Chart Comprehension via Reinforcement Learning with Verifiable Rewards
Mar 7, 2026Xin Zhang, Xingyu Li, Rongguang Wang +4Chart QA RL for VLMs
Oracle AI
Reference-guided Policy Optimization for Molecular Optimization via LLM Reasoning
Mar 6, 2026Xuan Li, Zhanke Zhou, Zongze Li +4Molecular Optimization Large Language Model-Guided Optimization
Hong Kong Baptist University · CMIC, Shanghai Jiao Tong University · DAMO Academy, Alibaba Group +1
BeamPERL: Parameter-Efficient RL with Verifiable Rewards Specializes Compact LLMs for Structured Beam Mechanics Reasoning
Mar 4, 2026Tarjei Paule Hage, Markus J. BuehlerStructural Engineering RL for Language Model Reasoning
Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA · Department of Civil and Environmental Engineering, Department of Mechanical Engineering, Schwarzman College of Computing, Massachusetts Institute of Technology, Cambridge, MA, USA
Heterogeneous Agent Collaborative Reinforcement Learning
Mar 3, 2026Zhixia Zhang, Zixuan Huang, Gongxun Li +10Multi-Agent Collaboration Heterogeneous MARL
Beihang University · Bytedance China · Apple +2
Tool Verification for Test-Time Reinforcement Learning
Mar 2, 2026Ruotong Liao, Nikolai Röhrich, Xiaohan Wang +4Test-Time RL RL for Language Model Reasoning
Ludwig-Maximilians University of Munich · MCML · Stanford University
LongRLVR: Long-Context Reinforcement Learning Requires Verifiable Context Rewards
Mar 2, 2026Guanzheng Chen, Michael Qizhe Shieh, Lidong BingRL for Language Models Reinforcement Learning with Verifiable Rewards
National University of Singapore · MiroMind AI · absolute AI
Efficient RLVR Training via Weighted Mutual Information Data Selection
Mar 2, 2026Xinyu Zhou, Boyu Zhu, Haotian Zhang +2Reinforcement Learning RL for Language Model Reasoning
HKUST(GZ) · UCL · Kuaishou Technology +1
Efficient Token-Level Policy Optimization for Masked Diffusion Models via Velocity Alignment
Mar 2, 2026Chenxing Wei, Jiazhen Kang, Hong Wang +8RL for Diffusion Models Policy Optimization
Shenzhen University, China · Guangming Laboratory, China · ByteDance, China +5
RLAR: An Agentic Reward System for Multi-task Reinforcement Learning on Large Language Models
Feb 28, 2026Andrew Zhuoer Feng, Cunxiang Wang, Bosi Wen +4RL for Language Models Reward Design for RL
Tsinghua Univeristy · Peking University
Recycling Failures: Salvaging Exploration in RLVR via Fine-Grained Off-Policy Guidance
Feb 27, 2026Yanwei Ren, Haotian Zhang, Likang Xiao +6RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
School of Artificial Intelligence, Beihang University, Beijing, China · Hangzhou International Innovation Institute, Beihang University, Hangzhou, China · The Hong Kong Polytechnic University, Hong Kong SAR, China +3
Know What You Know: Metacognitive Entropy Calibration for Verifiable RL Reasoning
Feb 26, 2026Qiannian Zhao, Chen Yang, Jinhao Jing +5RL for Language Model Reasoning Large Reasoning Models
ContextRL: Enhancing MLLM's Knowledge Discovery Efficiency with Context-Augmented RL
Feb 26, 2026Xingyu Lu, Jinpeng Wang, YiFan Zhang +12RL for VLMs Reinforcement Learning with Verifiable Rewards
Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Harbin Institute of Technology, Shenzhen, Shenzhen, China · Chinese Academy of Sciences, Beijing, China +2
GUI-Libra: Training Native GUI Agents to Reason and Act with Action-aware Supervision and Partially Verifiable RL
Feb 25, 2026Rui Yang, Qianhui Wu, Zhaoyang Wang +8GUI Agents RL for GUI Agents
UIUC · Microsoft · UNC-Chapel Hill

Mar 17, 2026J/K move · Enter open · S save

DyJR: Preserving Diversity in Reinforcement Learning with Verifiable Rewards via Dynamic Jensen-Shannon Replay

Long Li, Zhijian Zhou, Tianyi Wang, Weidi Xu+6

Griffith University, Australia · Fudan University, China · Shanghai Innovation Institute, China+3

Behavioral Diversity in RL Reinforcement Learning RL for Language Model Reasoning Experience Replay Reinforcement Learning with Verifiable Rewards

Open paper AskPDF arXiv

While Reinforcement Learning (RL) enhances Large Language Model reasoning, on-policy algorithms like GRPO are sample-inefficient as they discard past rollouts. Existing experience replay methods address this by reusing accurate samples for direct policy updates, but this can incur additional storage costs and concentrate the policy on a narrow set of solutions. We argue that historical data should prioritize sustaining local policy plasticity rather than simply reinforcing accuracy. We define local policy plasticity as a persistent ability to remain backtrackable and switchable with respect to recent successful trajectories: throughout training, the policy maintains non-negligible support on recently reachable alternatives, so it can smoothly reallocate probability mass when needed without being locked into a single dominant path. To this end, we propose Dynamic Jensen-Shannon Replay (DyJR), a simple yet effective regularization framework using a dynamic reference distribution from recent trajectories. DyJR introduces two innovations: (1) A Time-Sensitive Dynamic Buffer that uses FIFO and adaptive sizing to retain only temporally proximal samples, synchronizing with model evolution; and (2) Jensen-Shannon Divergence Regularization, which replaces direct replay policy-gradient updates with a regularization signal that discourages premature over-commitment to a single Rank-1 trajectory. Experiments on mathematical reasoning, Text-to-SQL, and coding benchmarks demonstrate that DyJR improves over GRPO as well as baselines such as RLEP and Ex-GRPO, while maintaining training efficiency comparable to the original GRPO. Furthermore, Rank-k token probability evolution and semantic-mode analyses show reduced concentration and greater retention of alternative continuations. Matched admission-objective comparisons and repeated runs provide additional evidence for the roles of DyJR's components.