RL for Language Model Reasoning

RL: Reinforcement Learning

This field studies how reward feedback can teach language models to solve multi-step problems, use tools, and decide how much reasoning a task needs. A common approach is reinforcement learning from verifiable rewards (RLVR), often using group-relative policy optimization, where sampled solutions are scored for correctness; other methods add step-level rewards, process verifiers, search, or teacher guidance to address sparse feedback. Work also explores curricula, adaptive rollout budgets, self-distillation, and controls for reasoning length and diversity. Key open tensions include balancing exploration with stable learning, ensuring rewards and verifiers do not encourage shortcuts, and improving generalization without excessive computation or overthinking.

Latest papers

1,103 papers170 papers in the last four weeks, up 124% on the four weeks before. 1.0% of all new papers.

CODA: Difficulty-Aware Compute Allocation for Adaptive Reasoning
Mar 9, 2026Siye Wu, Jian Xie, Yikai Zhang +1Adaptive Reasoning in Language Models Adaptive Computation
♠Fudan University
Reasoning as Compression: Unifying Budget Forcing via the Conditional Information Bottleneck
Mar 9, 2026Fabio Valerio Massoli, Andrey Kuzmin, Arash BehboodiEfficient Language Model Reasoning CoT Compression
Qualcomm AI Research
Revealing Behavioral Plasticity in Large Language Models: A Token-Conditional Perspective
Mar 9, 2026Liyuan Mao, Le Yu, Jing Zhou +7Inference-Time Language Model Alignment RL for Language Models
Qwen Team, Alibaba Group · Shanghai Jiao Tong University
In-Context Reinforcement Learning for Tool Use in Large Language Models
Mar 9, 2026Yaoqi Ye, Yiran Zhao, Keyu Duan +4RL for Tool Use In-Context RL
National University of Singapore · Salesforce AI Research · University of California Berkeley +1
SmartThinker: Progressive Chain-of-Thought Length Calibration for Efficient Large Language Model Reasoning
Mar 9, 2026Chenzhi Hu, Qinzhe Hu, Yuhang Xu +6Efficient Language Model Reasoning Large Reasoning Models
Shanghai Jiao Tong University · Beihang University
$\textbf{Re}^{2}$ : Unlocking LLM Reasoning via Reinforcement Learning with Re-solving
Mar 7, 2026Pinzheng Wang, Shuli Xu, Juntao Li +4RL for Language Model Reasoning Reinforcement Learning
School of Computer Science and Technology, Soochow University · Department of Foundation Model, 2012 Labs, Huawei · College of Intelligence and Computing, Tianjin University
AutoTool: Automatic Scaling of Tool-Use Capabilities in RL via Decoupled Entropy Constraints
Mar 7, 2026Yirong Zeng, Xiao Ding, Yufei Liu +9RL for Tool Use Efficient Language Model Reasoning
Harbin Institute of Technology, SCIR Lib · Peking University · Huawei Technologies Ltd.
Countdown-Code: A Testbed for Studying The Emergence and Generalization of Reward Hacking in RLVR
Mar 7, 2026Muhammad Khalifa, Zohaib Khan, Omer Tafveez +2RL Benchmarks Reward Hacking
NVIDIA · University of Michigan · University of Illinois Urbana-Champaign
DiSCTT: Consensus-Guided Self-Curriculum for Efficient Test-Time Adaptation in Reasoning
Mar 5, 2026Mohammad Mahdi Moradi, Sudhir MudurTest-Time Adaptation Self-Training for Language Models
Department of Compute Science, Concordia University, Montreal, Canada.
SarcasmMiner: A Dual-Track Post-Training Framework for Robust Audio-Visual Sarcasm Reasoning
Mar 5, 2026Zhu Li, Yongjian Chen, Huiyuan Lai +3Sarcasm Detection Multimodal Reasoning
Speech Technology Lab, University of Groningen, The Netherlands · Center for Language and Cognition, University of Groningen, The Netherlands
Breaking Contextual Inertia: Reinforcement Learning with Single-Turn Anchors for Stable Multi-Turn Interaction
Mar 5, 2026Xingwu Chen, Zhanqiu Zhang, Yiwen Guo +1Multi-Turn Language Model Reasoning RL for Language Model Reasoning
School of Computing and Data Science, The University of Hong Kong · LIGHTSPEED · Independent Researcher
$V_1$ : Unifying Generation and Self-Verification for Parallel Reasoners
Mar 4, 2026Harman Singh, Xiuyu Li, Kusha Sareen +14Language Model Self-Verification Test-Time Scaling
UC Berkeley · Mila · Together AI +1
Unbiased Dynamic Pruning for Efficient Group-Based Policy Optimization
Mar 4, 2026Haodong Zhu, Yangyang Ren, Yanjing Li +6Reinforcement Learning RL for Language Model Reasoning
Beihang University · Zhongguancun Academy · Rakuten Singapore +3
BeamPERL: Parameter-Efficient RL with Verifiable Rewards Specializes Compact LLMs for Structured Beam Mechanics Reasoning
Mar 4, 2026Tarjei Paule Hage, Markus J. BuehlerStructural Engineering RL for Language Model Reasoning
Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA · Department of Civil and Environmental Engineering, Department of Mechanical Engineering, Schwarzman College of Computing, Massachusetts Institute of Technology, Cambridge, MA, USA
Confidence-Calibrated Small-Large Language Model Collaboration for Cost-Efficient Reasoning
Mar 4, 2026Chuang Zhang, Zizhen Zhu, Yihao Wei +5Language Model Cascading Efficient Language Model Reasoning
Amazon Web Services · Tsinghua University
MIND: Unified Inquiry and Diagnosis RL with Criteria Grounded Clinical Supports for Psychiatric Consultation
Mar 4, 2026Guoyi Li, Shihao Xu, Jiatong Ma +4Healthcare Evidence-Grounded Reasoning
EverMind AI Inc., California, USA · Venus Team, Ant Group · Tianqiao and Chrissy Chen Institute, Shanghai, China +3
RAPO: Expanding Exploration for LLM Agents via Retrieval-Augmented Policy Optimization
Mar 3, 2026Siwei Zhang, Yun Xiong, Xi Chen +4Agentic RL RL for Language Model Reasoning
Fudan University · Zhejiang University · UC Davis
Chart Deep Research in LVLMs via Parallel Relative Policy Optimization
Mar 3, 2026Jiajin Tang, Gaoyang, Wenjie Wang +2Chart Understanding Large Vision-Language Models
ByteDance · ShanghaiTech University · School of Computer Science and Engineering, Sun Yat-sen University
Beyond One-Size-Fits-All: Adaptive Subgraph Denoising for Zero-Shot Graph Learning with Large Language Models
Mar 3, 2026Fengzhi Li, Liang Zhang, Yuan Zuo +5LLM Reasoning with Graphs Zero-Shot Learning
JIUTIAN Research, Beijing, China · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · MIIT Key Laboratory of Data and Decision Intelligence, Beihang University, Beijing, China +1
Tool Verification for Test-Time Reinforcement Learning
Mar 2, 2026Ruotong Liao, Nikolai Röhrich, Xiaohan Wang +4Test-Time RL RL for Language Model Reasoning
Ludwig-Maximilians University of Munich · MCML · Stanford University
Learning from Synthetic Data Improves Multi-hop Reasoning
Mar 2, 2026Anmol Kabra, Yilun Yin, Albert Gong +6RL for Language Model Reasoning Multi-Hop Reasoning
Cornell University · University of Cambridge · Stanford University
Efficient RLVR Training via Weighted Mutual Information Data Selection
Mar 2, 2026Xinyu Zhou, Boyu Zhu, Haotian Zhang +2Reinforcement Learning RL for Language Model Reasoning
HKUST(GZ) · UCL · Kuaishou Technology +1
Learning Structured Reasoning via Tractable Trajectory Control
Mar 2, 2026Po-Nien Kung, Zhen Yang, Jeffrey Luo +7Structured Reasoning in Language Models RL for Language Model Reasoning
University of California, Los Angeles · Apple
ProtRLSearch: A Multi-Round Multimodal Protein Search Agent with Large Language Models Trained via Reinforcement Learning
Mar 2, 2026Congying Liu, Taihao Li, Ming Huang +5Multimodal Search Agents Biomedical QA
University of Chinese Academy of Sciences, Beijing, China · Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou, China · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China +3
Provable and Practical In-Context Policy Optimization for Self-Improvement
Mar 2, 2026Tianrun Yu, Yuxiao Yang, Zhaoyang Wang +6Language Model Self-Improvement Test-Time Scaling
Brigham Young University · University of North Carolina at Chapel Hill · Microsoft
Can Thinking Models Think to Detect Hateful Memes?
Mar 1, 2026Mohamed Bayan Kmainasi, Mucahid Kutlu, Ali Ezzat Shahroor +2Harmful Meme Detection Multimodal Large Language Models
Qatar University Doha, Qatar · Qatar Computing Research Institute Doha, Qatar · APAVI.AI France +1
Learn Hard Problems During RL with Reference Guided Fine-tuning
Mar 1, 2026Yangzhen Wu, Shanda Li, Zixin Wen +5LLM Mathematical Reasoning RL for Language Model Reasoning
UC Berkeley · Carnegie Mellon University · ByteDance Seed
How RL Unlocks the Aha Moment in Geometric Interleaved Reasoning
Mar 1, 2026Xiangxiang Zhang, Caijun Jia, Siyuan Li +9Visually Grounded Reasoning RL for Language Model Reasoning
ByteDance · Shenyang Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Science +2
Stepwise Penalization for Length-Efficient Chain-of-Thought Reasoning
Feb 27, 2026Xintong Li, Sha Li, Rongmei Lin +10Efficient Language Model Reasoning RL for Language Model Reasoning
University of California, San Diego · Amazon
Recycling Failures: Salvaging Exploration in RLVR via Fine-Grained Off-Policy Guidance
Feb 27, 2026Yanwei Ren, Haotian Zhang, Likang Xiao +6RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
School of Artificial Intelligence, Beihang University, Beijing, China · Hangzhou International Innovation Institute, Beihang University, Hangzhou, China · The Hong Kong Polytechnic University, Hong Kong SAR, China +3

Mar 9, 2026J/K move · Enter open · S save

CODA: Difficulty-Aware Compute Allocation for Adaptive Reasoning

Siye Wu, Jian Xie, Yikai Zhang, Yanghua Xiao

♠Fudan University

Adaptive Reasoning in Language Models Adaptive Computation Cost-Aware Inference Large Reasoning Models RL for Language Model Reasoning Reward Design for RL

Open paper AskPDF arXiv

The emergence of large reasoning models demonstrates that scaling inference-time compute significantly enhances performance on complex tasks. However, it often falls into another trap: overthinking simple problems, where repetitive rationales yield minimal accuracy gains at a disproportionately high cost. This motivates adaptive reasoning: dynamically aligning reasoning depth with instance difficulty. In this paper, we study adaptive reasoning from an optimality perspective, formalizing it as a utility maximization problem where tokens are allocated until the marginal accuracy gain falls below the incremental cost. Based on this, we propose CODA (Compute Allocation by Difficulty Awareness), a method that operationalizes this principle by allocating tokens via a policy-internal difficulty signal. Specifically, CODA estimates difficulty via group-based rollouts and maps it to two non-negative gates that modulate a length-dependent shaping term on top of the binary base reward. The easy-side gate penalizes verbosity on simple instances, whereas the hard-side gate encourages more deliberative rollouts on challenging ones. Across model scales and benchmarks, CODA achieves adaptive reasoning without external annotations or user-provided budgets: on easy tasks, CODA reduces token costs by over 60% while maintaining strong accuracy, whereas on hard tasks it incentivizes more deliberative rollouts to maximize performance.