Reinforcement Learning with Verifiable Rewards

Also known as RLVR

This field studies how to improve language and multimodal models by rewarding outputs whose correctness or task progress can be checked, for example with unit tests, mathematical answers, formal proofs, or simulated environments. Common approaches use policy-gradient methods such as PPO and GRPO, with outcome rewards supplemented by step-level feedback, rollout exploration, or replay to improve credit assignment and data efficiency. Work spans math and coding benchmarks as well as search, GUI, and other interactive agents. A central tension is that rewards must be scalable yet faithful: imperfect or narrow verifiers can be exploited, and a correct final answer does not by itself establish sound reasoning, visual grounding, or robust generalization.

Latest papers

692 papers137 papers in the last four weeks, up 111% on the four weeks before. 0.8% of all new papers.

Apriel-Reasoner: RL Post-Training for General-Purpose and Efficient Reasoning
Apr 2, 2026Rafael Pardinas, Ehsan Kamalloo, David Vazquez +1LLM Inference Efficiency RL for Language Model Reasoning
ServiceNow AI Research
ThinkTwice: Jointly Optimizing Large Language Models for Reasoning and Self-Refinement
Apr 2, 2026Difan Jiao, Qianfeng Wen, Blair Yang +2RL for Language Model Reasoning LLM Reasoning
Department of Computer Science, University of Toronto · Coolwei AI Lab
Learning to Hint for Reinforcement Learning
Apr 1, 2026Yu Xia, Canwen Xu, Zhewei Yao +2Reinforcement Learning RL for Language Model Reasoning
University of California, San Diego · Snowflake AI Research
Learning to Model Optimization Problems from Solver Outcomes
Apr 1, 2026Runda Guan, Xiangqing Shen, Jiajun Zhang +3Natural Language to Optimization Modeling Learning to Optimize
School of Computer Science and Engineering, Nanjing University of Science and Technology · School of Intelligence Science and Technology, Nanjing University · Institute of Automation, Chinese Academy of Sciences
Learning to Generate Formally Verifiable Step-by-Step Logic Reasoning via Structured Formal Intermediaries
Mar 31, 2026Luoxin Chen, Yichi Zhou, Huishuai ZhangLLM Reasoning LLM Reliability
Wangxuan Institue of Computer Technology, Peking University. · ByteDance.
SOLE-R1: Video-Language Reasoning as the Sole Reward for On-Robot Reinforcement Learning
Mar 30, 2026Philip Schroeder, Thomas Weng, Karl Schmeckpeper +3RL for Robotics Multimodal Reward Modeling
MIT · RAI Institute
Seeing with You: Perception-Reasoning Coevolution for Multimodal Reasoning
Mar 30, 2026Ziqi Miao, Haonan Jia, Lijun Li +4Multimodal Reasoning Multimodal Large Language Models
SARL: Label-Free Reinforcement Learning by Rewarding Reasoning Topology
Mar 30, 2026Yifan Wang, Bolian Li, David Cho +3RL for Language Model Reasoning Reinforcement Learning
Purdue University · Texas Instruments
Wan-R1: Verifiable-Reinforcement Learning for Video Reasoning
Mar 29, 2026Ming Liu, Yunbei Zhang, Shilong Liu +2RL for Video Reasoning Video Reasoning
Iowa State University, Ames, IA 50011, USA · Tulane University, New Orleans, LA 70118, USA · Princeton University, Princeton, NJ 08544, USA
Bridging Visual Representation and Reinforcement Learning from Verifiable Rewards in Large Vision-Language Models
Mar 28, 2026Yuhang Han, Yuyang Wu, Zhengbo Jiao +6VLM Reasoning RL for VLMs
EPIC Lab, SJTU · HKUST (GZ) · Huawei
Incentivizing Temporal-Awareness in Egocentric Video Understanding Models
Mar 28, 2026Zhiyang Xu, Tian Qin, Bowen Jin +4Egocentric Video Understanding RL for VLMs
Virginia Tech · Harvard University · UIUC +2
Beyond Where to Look: Trajectory-Guided Reinforcement Learning for Multimodal RLVR
Mar 27, 2026Jinda Lu, Junkang Wu, Jinghan Li +6RL for VLMs Multimodal Reasoning
University of Science and Technology of China · Pecking University · Institute of Software, Chinese Academy of Sciences
MSRL: Scaling Generative Multimodal Reward Modeling via Multi-Stage Reinforcement Learning
Mar 26, 2026Chenglong Wang, Yifu Huo, Yang Gan +8Multimodal Reward Modeling RL for Generative Models
Northeastern University, Shenyang, China · ByteDance
Bridging Perception and Reasoning: Token Reweighting for RLVR in Multimodal LLMs
Mar 26, 2026Jinda Lu, Junkang Wu, Jinghan Li +6RL for VLMs VLM Reasoning
University of Science and Technology of China · Peking University · Independent Researcher
Prune as You Generate: Online Rollout Pruning for Faster and Better RLVR
Mar 25, 2026Haobo Xu, Sirui Chen, Ruizhong Qiu +5RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
University of Illinois at Urbana-Champaign · Amazon
Towards Effective Experiential Learning: Dual Guidance for Utilization and Internalization
Mar 25, 2026Fei Bai, Zhipeng Chen, Chuan Hao +6RL for Language Model Reasoning Reinforcement Learning
Gaoling School of Artificial Intelligence, Renmin University of China · IQuest Research · Beihang University
Rethinking Token-Level Policy Optimization for Multimodal Chain-of-Thought
Mar 24, 2026Yunheng Li, Hangyi Kuang, Hengrui Zhang +4VLM Reasoning Multimodal CoT Reasoning
VCIP, School of Computer Science, Nankai University · Kuaishou Technology
Sparse but Critical: A Token-Level Analysis of Distributional Shifts in RLVR Fine-Tuning of LLMs
Mar 23, 2026Haoming Meng, Kexin Huang, Shaohang Wei +6RL for Language Model Reasoning RL Fine-Tuning
Alibaba Group
On the Direction of RLVR Updates for LLM Reasoning: Identification and Exploitation
Mar 23, 2026Kexin Huang, Haoming Meng, Junkang Wu +10RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Alibaba Group
P^2O: Joint Policy and Prompt Optimization
Mar 23, 2026Xinyu Lu, Kaiqi Zhang, Jinglin Yang +6RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Institute of Information Engineering, Chinese Academy of Sciences +2
Prompt replay: speeding up grpo with on-policy reuse of high-signal prompts
Mar 22, 2026Andrei Baroian, Rutger BergerPrompt Selection for RL RL Post-Training
Leiden Institute of Advanced Computer Science Leiden University
WIST: Web-Grounded Iterative Self-Play Tree for Domain-Targeted Reasoning Improvement
Mar 22, 2026Fangyuan Li, Pengfei Li, Shijie Wang +4Language Model Self-Improvement RL for Language Model Reasoning
Harbin Institute of Technology · Shanghai Innovation Institute · Shanghai Artificial Intelligence Laboratory
RLVR Training of LLMs Does Not Improve Thinking Ability for General QA: Evaluation Method and a Simple Solution
Mar 21, 2026Kaiyuan Li, Jing-Cheng Pang, Yang YuLanguage Model Reasoning Evaluation RL for Language Model Reasoning
National Key Laboratory for Novel Software Technology & School of Artificial Intelligence, Nanjing University, China · Polixir.ai, China · Huawei Technologies Co., Ltd., China
Human-Level Text-to-SQL via Reinforcement Learning on Verified Data, Without Pipeline Engineering
Mar 20, 2026Yuxuan Zhu, Tengjun Jin, Yoojin Choi +1Text-to-SQL Reinforcement Learning with Verifiable Rewards
University of Illinois · Bridgewater AIA Labs
OS-Themis: A Scalable Critic Framework for Generalist GUI Rewards
Mar 19, 2026Zehao Li, Zhenyu Wu, Yibo Zhao +11Reward Modeling RL for GUI Agents
University of Science and Technology of China · Shanghai AI Laboratory · CUHK MMLab +2
Context Bootstrapped Reinforcement Learning
Mar 19, 2026Saaket Agashe, Jayanth Srinivasa, Gaowen Liu +2RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
University of California, Santa Barbara · Cisco Research
Discounted Beta-Bernoulli Reward Estimation for Sample-Efficient Reinforcement Learning with Verifiable Rewards
Mar 19, 2026Haechan Kim, Soohyun Ryu, Gyouk Chu +2Reinforcement Learning RL for Language Model Reasoning
Kim Jaechul Graduate School of AI, KAIST, Daejeon, South Korea · KRAFTON, Seoul, South Korea · AITRICS, Seoul, South Korea
RAWR: Reward Assignment Without Rollouts in Verifiable Domains
Mar 18, 2026Corentin Royer, Anna Hedström, Debarun Bhattacharjya +3Process Reward Models Reinforcement Learning with Verifiable Rewards
International Business Machines · ETH AI Center · Lirio +1
Dual Consensus: Escaping from Spurious Majority in Unsupervised RLVR via Two-Stage Vote Mechanism
Mar 17, 2026Kaixuan Du, Meng Cao, Hang Zhang +3RL for Language Model Reasoning Self-Training for Language Models
School of Automation Science and Electrical Engineering, Beihang University
Execution-Grounded Credit Assignment for GRPO in Code Generation
Mar 17, 2026Abhijit Kumar, Natalya Kumar, Shikhar GuptaRL for Code Generation Credit Assignment

Apr 2, 2026J/K move · Enter open · S save

Apriel-Reasoner: RL Post-Training for General-Purpose and Efficient Reasoning

Rafael Pardinas, Ehsan Kamalloo, David Vazquez, Alexandre Drouin

ServiceNow AI Research

LLM Inference Efficiency RL for Language Model Reasoning Efficient Language Model Reasoning Reinforcement Learning with Verifiable Rewards RL Post-Training Large Reasoning Models

Open paper AskPDF arXiv

Building general-purpose reasoning models using reinforcement learning with verifiable rewards (RLVR) across diverse domains has been widely adopted by frontier open-weight models. However, their training recipes and domain mixtures are often not disclosed. Joint optimization across domains poses significant challenges: domains vary widely in rollout length, problem difficulty, and sample efficiency. Further, models with long chain-of-thought traces increase inference cost and latency, making efficiency critical for practical deployment. We present Apriel-Reasoner, trained with a reproducible multi-domain RL post-training recipe on Apriel-Base, a 15B-parameter open-weight LLM, across five domains using public datasets: mathematics, code generation, instruction following, logical puzzles, and function calling. We introduce adaptive domain sampling that preserves target completed-rollout ratios despite heterogeneous rollout dynamics, and a difficulty-aware length penalty that, at no additional training overhead, encourages longer reasoning for difficult problems and shorter traces for easy ones. Trained with a strict 16K-token output budget, Apriel-Reasoner remains effective at a 32K output budget and improves over Apriel-Base on AIME 2025, GPQA, MMLU-Pro, and LiveCodeBench while producing 30-50% shorter reasoning traces. Among the evaluated open-weight models of similar scale, it improves the accuracy-token tradeoff using a fully public-data recipe.