Reinforcement Learning with Verifiable Rewards

Also known as RLVR

This field studies how to improve language and multimodal models by rewarding outputs whose correctness or task progress can be checked, for example with unit tests, mathematical answers, formal proofs, or simulated environments. Common approaches use policy-gradient methods such as PPO and GRPO, with outcome rewards supplemented by step-level feedback, rollout exploration, or replay to improve credit assignment and data efficiency. Work spans math and coding benchmarks as well as search, GUI, and other interactive agents. A central tension is that rewards must be scalable yet faithful: imperfect or narrow verifiers can be exploited, and a correct final answer does not by itself establish sound reasoning, visual grounding, or robust generalization.

Latest papers

692 papers137 papers in the last four weeks, up 111% on the four weeks before. 0.8% of all new papers.

PopuLoRA: Co-Evolving LLM Populations for Reasoning Self-Play
May 16, 2026Roger Creus Castanyer, Geoffrey Bradway, Lorenz Wolf +3Self-Play RL RL for Language Model Reasoning
Vmax
GRID: Graph Representation of Intelligence Data for Security Text Knowledge Graph Construction
May 15, 2026Liangyi Huang, Zichen Liu, Fei Shao +5KG Construction Cyber Threat Intelligence
Arizona State University · Case Western Reserve University · University of Notre Dame +2
Nudging Beyond the Comfort Zone: Efficient Strategy-Guided Exploration for RLVR
May 15, 2026Chanuk Lee, Sangwoo Park, Minki Kang +1RL for Language Model Reasoning RL Exploration
KAIST
SAGE: Shaping Anchors for Guided Exploration in RLVR of LLMs
May 15, 2026Chanuk Lee, Minki Kang, Sung Ju HwangKL-Regularized RL RL for Language Model Reasoning
KAIST
Calibrating LLMs with Semantic-level Reward
May 15, 2026Fengfei Yu, Ruijia Niu, Dongxia Wu +2Language Model Calibration LLM Reliability
Department of Computer Science and Engineering, University of California San Diego, La Jolla, California, USA · Halıcıoğlu Data Science Institute, University of California San Diego, La Jolla, California, USA · Department of Statistics, Stanford University, Stanford, California, USA
Video Models Can Reason with Verifiable Rewards
May 14, 2026Tinghui Zhu, Sheng Zhang, James Y. Huang +5Video Diffusion Models RL for Video Generation
University of California, Davis · Microsoft Research · University of Southern California +1
Learning from Language Feedback via Variational Policy Distillation
May 14, 2026Yang Li, Erik Nijkamp, Semih Yavuz +1On-Policy Self-Distillation RL for Language Model Reasoning
Salesforce AI Research
Boosting Reinforcement Learning with Verifiable Rewards via Randomly Selected Few-Shot Guidance
May 14, 2026Kai Yan, Alexander G. Schwing, Yu-Xiong WangRL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
University of Illinois Urbana-Champaign
Learning from Failures: Correction-Oriented Policy Optimization with Verifiable Rewards
May 14, 2026Mengjie Ren, Jie Lou, Boxi Cao +6RL for Code Generation LLM Self-Correction
Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Xiaohongshu Inc.
Learning to Build the Environment: Self-Evolving Reasoning RL via Verifiable Environment Synthesis
May 14, 2026Yucheng Shi, Zhenwen Liang, Kishan Panaganti +3Reinforcement Learning RL for Language Model Reasoning
Tencent HY LLM
GenCircuit-RL: Reinforcement Learning from Hierarchical Verification for Genetic Circuit Design
May 14, 2026Noah FlynnRL for Code Generation Program Synthesis
University of California, Berkeley, CA, USA.
PDCR: Perception-Decomposed Confidence Reward for Vision-Language Reasoning
May 13, 2026Hee Suk Yoon, Eunseop Yoon, Ji Woo Hong +6VLM Reasoning Reinforcement Learning with Verifiable Rewards
Korea Advanced Institute of Science and Technology (KAIST) · Microsoft Research Asia (MSRA) · University of Illinois at Urbana-Champaign (UIUC)
Revisiting Reinforcement Learning with Verifiable Rewards from a Contrastive Perspective
May 13, 2026Feng Zhang, Xinhong Ma, Ziqiang Dong +5RL for Language Model Reasoning Group Relative Policy Optimization
Beijing Institute of Technology · Qwen Business Unit of Alibaba · The Chinese University of Hong Kong, Shenzhen +1
Multi-Rollout On-Policy Distillation via Peer Successes and Failures
May 12, 2026Weichen Yu, Xiaomin Li, Yizhou Zhao +8RL for Language Model Reasoning Language Model Distillation
1Carnegie Mellon University · 2Microsoft · 3Purdue University
AlphaGRPO: Unlocking Self-Reflective Multimodal Generation in UMMs via Decompositional Verifiable Reward
May 12, 2026Runhui Huang, Jie Wu, Rui Yang +2Unified Multimodal Models Multimodal Generation
The University of Hong Kong · Bytedance Seed
Reward Hacking in Rubric-Based Reinforcement Learning
May 12, 2026Anas Mahmoud, MohammadHossein Rezaei, Zihao Wang +3Reward Hacking Reinforcement Learning
Scale AI
StepCodeReasoner: Aligning Code Reasoning with Stepwise Execution Traces via Reinforcement Learning
May 12, 2026Hao Wang, Rui Li, Lei Sha +1RL for Language Model Reasoning Execution-Guided Code Generation
Beihang University, Beijing, China · Peking University, Beijing, China · Zhongguancun Laboratory, Beijing, China +1
Entropy Polarity in Reinforcement Fine-Tuning: Direction, Asymmetry, and Control
May 12, 2026Jiazheng Zhang, Ziche Fu, Junrui Shen +17RL for Language Model Reasoning Policy Optimization
Fudan NLP Group, Honor Device Co., Ltd · University of Hong Kong, Shanghai Jiao Tong University and Tencent Hunyuan
CuSearch: Curriculum Rollout Sampling via Search Depth for Agentic RAG
May 12, 2026Jianghan Shen, Siqi Luo, Xinyu Cheng +6RL for Language Models Reinforcement Learning
1Nanjing University · 2Shanghai Artificial Intelligence Laboratory · University of Hong Kong +1
Selective Off-Policy Reference Tuning with Plan Guidance
May 12, 2026Duc Anh Le, Tien-Phat Nguyen, Thien Huu Nguyen +2Reinforcement Learning RL for Language Model Reasoning
Independent Author · Hanoi University of Science and Technology, Hanoi, Vietnam · University of Oregon, Eugene, OR, USA +1
Understanding and Preventing Entropy Collapse in RLVR with On-Policy Entropy Flow Optimization
May 12, 2026Huimin Xu, Shuai Zhao, Xiaobao Wu +1Reinforcement Learning RL for Language Model Reasoning
Nanyang Technological University, Singapore · Shanghai Jiao Tong University, China · 3VinUniversity, Vietnam
Breaking $\textit{Winner-Takes-All}$ : Cooperative Policy Optimization Improves Diverse LLM Reasoning
May 12, 2026Haoxuan Chen, Tianming Liang, Wei-Shi Zheng +1Reinforcement Learning RL for Language Model Reasoning
1ISEE Lab, Sun Yat-sen University
fg-expo: Frontier-guided exploration-prioritized policy optimization via adaptive kl and gaussian curriculum
May 12, 2026Mingxiong Lin, Zhangquan Gong, Maowen Tang +6Reinforcement Learning RL for Language Model Reasoning
OPPO AI Center
Rebellious Student: Reversing Teacher Signals for Reasoning Exploration with Self-Distilled RLVR
May 11, 2026Jeonghye Kim, Jiwon Jeon, Dongsheng Li +1RL for Language Model Reasoning Language Model Distillation
Microsoft Research · KAIST
Verifiable Process Rewards for Agentic Reasoning
May 11, 2026Huining Yuan, Zelai Xu, Huaijie Wang +6Process Reward Models Credit Assignment in RL
Tsinghua University
Relative Score Policy Optimization for Diffusion Language Models
May 11, 2026Zichao Yu, Shengze Xu, Bingqing Jiang +2RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
University of Science and Technology of China · The Chinese University of Hong Kong · The University of Hong Kong
expo: Exploration-prioritized policy optimization via adaptive kl regulation and gaussian curriculum sampling
May 11, 2026Mingxiong Lin, Zhangquan Gong, Maowen Tang +6Reinforcement Learning KL-Regularized RL
OPPO AI Center
BoostAPR: Boosting Automated Program Repair via Execution-Grounded Reinforcement Learning with Dual Reward Models
May 9, 2026Yuanhao Li, Hongbo Wang, Xiaotang Shang +3Automated Program Repair Process Reward Models
State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China · University of Luxembourg
Forge: Quality-Aware Reinforcement Learning for NP-Hard Optimization in LLMs
May 9, 2026Xiaozhe Li, Xinyu Fang, Shengyuan Ding +5RL Benchmarks Reinforcement Learning
Tongji University1 · Shanghai AI Lab2 · Zhejiang University3 +3
How You Begin is How You Reason: Driving Exploration in RLVR via Prefix-Tuned Priors
May 9, 2026Yifan Xu, Junren Chen, Yifan ChenRL for Language Model Reasoning Prompt Tuning
Hong Kong Baptist University · University of Maryland, College Park

May 16, 2026J/K move · Enter open · S save

PopuLoRA: Co-Evolving LLM Populations for Reasoning Self-Play

Roger Creus Castanyer, Geoffrey Bradway, Lorenz Wolf, Maxwill Lin+2

Vmax

Self-Play RL RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards Teacher-Student Learning

Open paper AskPDF arXiv

We introduce PopuLoRA, a population-based asymmetric self-play framework for reinforcement learning with verifiable rewards (RLVR) post-training of LLMs. Teachers and students are specialised LoRA adapters on a shared frozen base: teachers propose problems, matched students solve them under a programmatic verifier, and cross-evaluation between sub-populations replaces the self-calibration that limits single-agent self-play. A family of LoRA weight-space evolution operators (mutations and crossovers that produce same-rank population members in seconds) serves as the replacement step of a population-based training loop at 7B scale. We instantiate PopuLoRA on top of Absolute Zero Reasoner and compare it against a per-adapter compute-matched single-agent baseline. Where the single agent self-calibrates to generating easy problems it can reliably solve, the population enters a co-evolutionary arms race: teachers produce increasingly complex problems, student solve rates oscillate, and problem-space coverage keeps expanding throughout training. Despite lower training-time reward, the population mean outperforms the baseline on three code benchmarks (HumanEval+, MBPP+, LiveCodeBench) and seven math benchmarks (AIME 24/25, AMC 23, MATH-500, Minerva, GSM8K, OlympiadBench), and even the weakest member of the population beats the baseline on aggregate.