Reinforcement Learning with Verifiable Rewards

Also known as RLVR

Follow this topic Ask about this topic

Momentum

59 papers in the last four weeks, up 119% on the four weeks before. 0.6% of all new papers.

Jul 13Week of Sep 28

Latest papers 438

All topics

Hindsight-Anchored Policy Optimization: Learning Through Hindsight with Thompson Sampling-Inspired Adaptive Gating
Mar 11, 2026Yuning Wu, Ke Wang, Haoran Liu +3Reinforcement Learning RL for Language Model Reasoning
Amazon
Gradient Regularization Mitigates Reward Hacking in Reinforcement Learning from Human Feedback and Verifiable Rewards
Feb 20, 2026Johannes Ackermann, Michael Noukhovitch, Takashi Ishida +1Reward Hacking Reinforcement Learning with Verifiable Rewards
The University of Tokyo · Mila · RIKEN AIP
On the Emergence of Implicit Curriculum in RLVR Learning Dynamics
Feb 16, 2026Yu Huang, Zixin Wen, Yuejie Chi +4RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Department of Statistics and Data Science, Wharton School, University of Pennsylvania · Machine Learning Department, Carnegie Mellon University · Department of Statistics and Data Science, Yale University +1
To Mix or To Merge: Toward Multi-Domain Reinforcement Learning for Large Language Models
Feb 13, 2026Haoqing Wang, Xiang Long, Ziheng Li +3RL for Language Models Gradient Interference
Samsung Research, Beijing, China · Peking University
Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation
Feb 10, 2026Pei-Chi Pan, Yingbin Liang, Sen LinReward Modeling Reward Hacking
Department of Computer Science University of Houston · The Ohio State University · University of Houston
F-GRPO: Don't Let Your Policy Learn the Obvious and Forget the Rare
Feb 6, 2026Daniil Plyusov, Alexey Gorbatovski, Boris Shaposhnikov +4Reinforcement Learning RL for Language Model Reasoning
T-Tech · Saint Petersburg Electrotechnical University “LETI”
Unveiling Implicit Advantage Symmetry: Why GRPO Struggles with Exploration and Difficulty Adaptation
Feb 5, 2026Zhiqi Yu, Zhangquan Chen, Mengting Liu +2RL for Language Model Reasoning Group Relative Policy Optimization
University of Hong Kong · Tsinghua University · Sun Yat-sen University
CodeScaler: Scaling Code LLM Training and Test-Time Inference via Reward Models
Feb 4, 2026Xiao Zhu, Xinyu Zhou, Boyu Zhu +5RL for Code Generation Reward Modeling
LARK, HKUST(GZ) · Kuaishou Technology · UCL +3
HeaPA: Difficulty-Aware Heap Sampling and On-Policy Query Augmentation for LLM Reinforcement Learning
Jan 30, 2026Weiqi Wang, Xin Liu, Binxuan Huang +13RL for Language Models RL for Language Model Reasoning
Stores Foundational AI, Amazon Inc., Palo Alto, CA, USA · Department of Computer Science and Engineering, HKUST, Hong Kong SAR, China
Spurious Rewards Paradox: Mechanistically Understanding How RLVR Activates Memorization Shortcuts in LLMs
Jan 16, 2026Lecheng Yan, Ruizhe Li, Guanhua Chen +5Reward Hacking Memorization in Language Models
Southern University of Science and Technology · University of Science and Technology of China · University of Aberdeen +5
X-Coder: Advancing Competitive Programming with Synthetic Tasks, Solutions, and Tests
Jan 11, 2026Jie Wu, Haoling Li, Xin Zhang +8RL for Code Generation Synthetic Data Generation
Tsinghua University · Microsoft
ScalePRM: Training Process Reward Models by Scaling Verification Compute Without Ground Truth
Dec 2, 2025Salman Rahman, Sruthi Gorantla, Arpit Gupta +3Process Reward Models Reinforcement Learning with Verifiable Rewards
2UCLA · Work done while as an intern at Amazon AGI. · 1Amazon AGI
Honesty over Accuracy: Trustworthy Language Models through Reinforced Hesitation
Nov 14, 2025Mohamad Amin Mohamadi, Tianhao Wang, Zhiyuan LiRL for Language Models LLM Hallucination Mitigation
Toyota Technological Institute at Chicago · University of California, San Diego
Selective Expert Guidance for Effective and Diverse Exploration in Reinforcement Learning of LLMs
Oct 5, 2025Zishang Jiang, Jinyi Han, Tingyun Li +7RL for Language Model Reasoning LLM-Guided RL
School of Data Science, Fudan University · Shanghai Institute of Artificial Intelligence for Education, East China Normal University · College of Computer Science and Artificial Intelligence, Fudan University +1
Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers
Oct 1, 2025Xin-Qiang Cai, Wei Wang, Feng Liu +3Reinforcement Learning RL for Language Model Reasoning
RIKEN AIP, Tokyo, Japan · The University of Tokyo, Tokyo, Japan · The University of Melbourne, Melbourne, Australia
Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts
Aug 13, 2025Maxime Heuillet, Yufei Cui, Boxing Chen +2RL for Language Model Reasoning Efficient Language Model Training
Universit´e Laval (IID), Canada · Mila - Qu´ebec AI Institute, Canada · Huawei Noah’s Ark Lab (Montreal Research Center), Canada +1
Countdown-Code: A Testbed for Studying The Emergence and Generalization of Reward Hacking in RLVR
Date pendingMuhammad Khalifa, Zohaib Khan, Omer Tafveez +2RL Benchmarks Reward Hacking
NVIDIA · University of Michigan · University of Illinois Urbana-Champaign
VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning
Date pendingJunxiang Xu, Ruisi Wang, Fanyi Pu +49VLM Evaluation Visual Reasoning

Mar 11, 2026, cs.LGJ/K move · Enter open · S save

Hindsight-Anchored Policy Optimization: Learning Through Hindsight with Thompson Sampling-Inspired Adaptive Gating

Yuning Wu, Ke Wang, Haoran Liu, Chaoqun Jia+2

Amazon

Reinforcement Learning RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards Sparse-Reward RL Off-Policy RL

Open paper AskPDF arXiv

Reinforcement Learning with Verifiable Rewards improves reasoning in large language models, yet on-policy learning often suffers from cold-start challenges in sparse-reward settings. Recent mixed-policy approaches address this by combining off-policy teacher data with on-policy training. However, simply combining these introduce a persistent off-policy gradient mass that risks training collapse and instability. To address this challenge, we propose Hindsight-Anchored Policy Optimization (HAPO), a framework that allows teacher intervention to act as a temporary support. HAPO employs Beta-Binomial confidence gating, an adaptive gating mechanism that decides when to open the gate for teacher intervention. The intervention operates with Synthetic Success Injection, which replaces the group's lowest-reward rollout with a verified teacher trajectory. We also introduce adaptive threshold annealing, which gradually retracts the support and restores on-policy training within a finite horizon to mitigate persistent off-policy drift. We demonstrate that HAPO can be layered on top of existing mixed-policy methods in a generalizable manner. Across six math reasoning benchmarks and two model scales, HAPO improves the average accuracy of three major mixed-policy methods while maintaining training stability.

Momentum

Related topics

Latest papers 438