Reinforcement Learning with Verifiable Rewards

Also known as RLVR

Follow this topic Ask about this topic

Momentum

59 papers in the last four weeks, up 119% on the four weeks before. 0.6% of all new papers.

Jul 13Week of Sep 28

Latest papers 438

All topics

OPPO: Bayesian Value Recursion for Token-Level Credit Assignment in LLM Reasoning
May 21, 2026Yu Li, Rui Miao, Tian Lan +1Value Function Estimation Token-Level Credit Assignment
1George Washington University · 2The University of Texas at Dallas
You Only Need Minimal RLVR Training: Extrapolating LLMs via Rank-1 Trajectories
May 20, 2026Zhepei Wei, Xinyu Zhu, Wei-Lin Chen +3RL for Language Model Reasoning Low-Rank Approximation
University of Virginia · Washington University in St. Louis
DelTA: Discriminative Token Credit Assignment for Reinforcement Learning from Verifiable Rewards
May 20, 2026Kaiyi Zhang, Wei Wu, Yankai LinPolicy Gradient Token-Level Credit Assignment
Gaoling School of Artificial Intelligence, Renmin University of China · Ant International
How Much Online RL is Enough? Informative Rollouts for Offline Preference Optimization in RLVR
May 20, 2026Richa Verma, Balaraman RavindranDirect Preference Optimization RL for Language Model Reasoning
TCS Research Department of CSE IIT Madras Chennai, India · Department of Data Science & AI, Wadhwani School of Data Science & AI IIT Madras Chennai, India
LamPO: A Lambda Style Policy Optimization for Reasoning Language Models
May 20, 2026Redacted by arXivRL for Language Model Reasoning Group Relative Policy Optimization
Pinterest, San Francisco, CA 94107, USA
Domain-Adaptable Reinforcement Learning for Code Generation with Dense Rewards
May 20, 2026Erfan Aghadavoodi Jolfaei, Daniel Maninger, Abhinav Anand +2RL for Code Generation Reinforcement Learning
Technische Universität Darmstadt, Darmstadt, Germany · Hessian Center for Artificial Intelligence (hessian.AI), Darmstadt, Germany · National Research Center for Applied Cybersecurity ATHENE, Darmstadt, Germany.
Advantage Collapse in Group Relative Policy Optimization: Diagnosis and Mitigation
May 20, 2026Xixiang He, Qiyao Sun, Ao Cheng +5RL for Language Model Reasoning Group Relative Policy Optimization
National University of Defense Technology, Changsha, Hunan, China · Intelligent Game and Decision Lab, Beijing, China · The Chinese University of Hong Kong, Shenzhen, Guangdong, China
Multi-Step Likelihood-Ratio Correction for Reinforcement Learning with Verifiable Rewards
May 20, 2026Deokgyu Yoon, Hyungkyu Kang, Joongkyu Lee +4RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
1Seoul National University · 2Upstage
PlexRL: Cluster-Level Orchestration of Serviceized LLM Execution for RLVR
May 20, 2026Yiqi Zhang, Fangzheng Jiao, Tian Tang +13LLM Inference Efficiency LLM Serving
National University of Singapore · Shanghai Qiji Zhifeng Co., Ltd. · Beihang University +3
Not Every Rubric Teaches Equally: Policy-Aware Rubric Rewards for RLVR
May 19, 2026Utkarsh Tyagi, Xingang Guo, MohammadHossein Rezaei +5Reward Modeling Rubric-Based RL
Scale AI · Persona
Rewarding Beliefs, Not Actions: Consistency-Guided Credit Assignment for Long-Horizon Agents
May 19, 2026Wenjie Tang, Minne Li, Sijie Huang +2Reinforcement Learning Partially Observable RL
College of Computer, National University of Defense Technology · 2Intelligent Game and Decision Lab (IGDL) · Institute of Artificial Intelligence, Xiamen University
GeoX: Mastering Geospatial Reasoning Through Self-Play and Verifiable Rewards
May 19, 2026Kyeongjin Ahn, Seungeon Lee, Krishna P. Gummadi +1Self-Play RL Geospatial Reasoning
KAIST · MPI-SWS · MPI-SP
GoLongRL: Capability-Oriented Long Context Reinforcement Learning with Multitask Alignment
May 19, 2026Minxuan Lv, Tiehua Mei, Tanlong Du +9Reinforcement Learning Long-Context Modeling
Kuaishou Technology · University of Chinese Academy of Sciences
CEPO: RLVR Self-Distillation using Contrastive Evidence Policy Optimization
May 19, 2026Ahmed Heakl, Abdelrahman M. Shaker, Youssef Mohamed +4Token-Level Credit Assignment RL for Language Model Reasoning
MBZUAI · Linköping University · Australian National University
When to Stop Reusing: Dynamic Gradient Gating for Sample-Efficient RLVR
May 19, 2026Yuchun Miao, Sen Zhang, Yuqi Zhang +4RL for Language Model Reasoning Policy Optimization
National Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University · Meituan Longcat Team · The University of Sydney +1
Rethinking Muon Beyond Pretraining: Spectral Failures and High-Pass Remedies for VLA and RLVR
May 19, 2026Chongyu Fan, Gaowen Liu, Mingyi Hong +2Muon Optimizer Vision-Language-Action Models
Michigan State University · Cisco · University of Minnesota +1
AMR-SD: Asymmetric Meta-Reflective Self-Distillation for Token-Level Credit Assignment
May 18, 2026Zhenlin Wei, Pu Jian, Yingzhuo Deng +6Token-Level Credit Assignment RL for Language Model Reasoning
Meituan Beijing, China · University of Chinese Academy of Sciences
Knowledge-to-Verification: Exploring RLVR for LLMs in Knowledge-Intensive Domains
May 18, 2026Zhonghang Yuan, Zhefan Wang, Fang Hu +7RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Shanghai Artificial Intelligence Laboratory · Shanghai Innovation Institute
How Off-Policy Can GRPO Be? Mu-GRPO for Efficient LLM Reinforcement Learning
May 17, 2026Minghao Tian, Yunfei Xie, Chen WeiRL for Language Model Reasoning Group Relative Policy Optimization
Rice University
Leveraging Error Diversity in Group Rollouts for Reinforcement Learning
May 17, 2026Wenpu Liu, Yuqi Xu, Weichu Xie +8Reward Shaping RL for Language Model Reasoning
Peking University · Shanghai Innovation Institute
Step-wise Rubric Rewards for LLM Reasoning
May 17, 2026Weichu Xie, Haozhe Zhao, Wenpu Liu +15Reward Modeling Credit Assignment in RL
Peking University · JD Explore Academy · Shanghai Jiao Tong University +2
TIER: Trajectory-Invariant Execution Rewards for Multi-Step Tool Composition
May 16, 2026Anay Kulkarni, ChiaEn Lu, Dheeraj Mekala +3Reward Modeling Reinforcement Learning with Verifiable Rewards
UC San Diego · Cisco Research
The Unlearnability Phenomenon in RLVR for Language Models
May 16, 2026Yulin Chen, He He, Chen ZhaoReinforcement Learning RL for Language Model Reasoning
1New York University · 2NYU Shanghai
PopuLoRA: Co-Evolving LLM Populations for Reasoning Self-Play
May 16, 2026Roger Creus Castanyer, Geoffrey Bradway, Lorenz Wolf +3Self-Play RL RL for Language Model Reasoning
Vmax
GRID: Graph Representation of Intelligence Data for Security Text Knowledge Graph Construction
May 15, 2026Liangyi Huang, Zichen Liu, Fei Shao +5KG Construction Cyber Threat Intelligence
Arizona State University · Case Western Reserve University · University of Notre Dame +2
Nudging Beyond the Comfort Zone: Efficient Strategy-Guided Exploration for RLVR
May 15, 2026Chanuk Lee, Sangwoo Park, Minki Kang +1RL for Language Model Reasoning RL Exploration
KAIST
SAGE: Shaping Anchors for Guided Exploration in RLVR of LLMs
May 15, 2026Chanuk Lee, Minki Kang, Sung Ju HwangKL-Regularized RL RL for Language Model Reasoning
KAIST
Calibrating LLMs with Semantic-level Reward
May 15, 2026Fengfei Yu, Ruijia Niu, Dongxia Wu +2Language Model Calibration LLM Reliability
Department of Computer Science and Engineering, University of California San Diego, La Jolla, California, USA · Halıcıoğlu Data Science Institute, University of California San Diego, La Jolla, California, USA · Department of Statistics, Stanford University, Stanford, California, USA
Video Models Can Reason with Verifiable Rewards
May 14, 2026Tinghui Zhu, Sheng Zhang, James Y. Huang +5Video Diffusion Models RL for Video Generation
University of California, Davis · Microsoft Research · University of Southern California +1
Learning from Language Feedback via Variational Policy Distillation
May 14, 2026Yang Li, Erik Nijkamp, Semih Yavuz +1On-Policy Self-Distillation RL for Language Model Reasoning
Salesforce AI Research

May 21, 2026, cs.LGJ/K move · Enter open · S save

OPPO: Bayesian Value Recursion for Token-Level Credit Assignment in LLM Reasoning

Yu Li, Rui Miao, Tian Lan, Zhengling Qi

1George Washington University · 2The University of Texas at Dallas

Value Function Estimation Token-Level Credit Assignment RL for Language Model Reasoning Credit Assignment in RL Step-Level Credit Assignment in RL Reinforcement Learning with Verifiable Rewards+3

Open paper AskPDF arXiv

Reinforcement learning with verifiable rewards has become the standard recipe for improving LLM reasoning, but the dominant algorithm GRPO assigns a single trajectory-level advantage to every token, diluting the signal at pivotal reasoning steps and injecting noise at uninformative ones. Critic-free alternatives derived from on-policy distillation supply per-token signals through oracle-conditioned likelihood ratios, yet apply each signal in isolation from the trajectory-level evidence accumulated up to that position. We propose Oracle-Prompted Policy Optimization (OPPO), which rests on a single observation: the oracle signal used by prior distillation-style methods for local discrimination is also the natural Bayesian update of the model's belief about eventual success. Accumulating the signal along a trajectory yields, in closed form and at the cost of one extra forward pass, a running estimate of the success probability at every position, together with a token-level advantage that requires no learned value network and no additional rollouts. A first-order analysis factorizes the advantage into the per-token discrimination signal used by distillation methods modulated by a state weight that concentrates credit on genuinely pivotal tokens, with a directional variance-reduction guarantee. The framework admits two estimators differing only in which model scores the evidence: a \textit{self-oracle} that reuses the student and recovers the on-policy distillation reward as a strict special case, and a \textit{teacher-oracle} that delegates scoring to a stronger frozen model. On two base LLMs across seven mathematics, science, and code reasoning benchmarks, OPPO improves over GRPO, DAPO, and SDPO by up to

+6.0

points on AMC'23 and

+5.2

points on AIME'24, with gains that widen monotonically with response length.

Momentum

Related topics

Latest papers 438