RL for Language Model Reasoning

RL: Reinforcement Learning

This field studies how reward feedback can teach language models to solve multi-step problems, use tools, and decide how much reasoning a task needs. A common approach is reinforcement learning from verifiable rewards (RLVR), often using group-relative policy optimization, where sampled solutions are scored for correctness; other methods add step-level rewards, process verifiers, search, or teacher guidance to address sparse feedback. Work also explores curricula, adaptive rollout budgets, self-distillation, and controls for reasoning length and diversity. Key open tensions include balancing exploration with stable learning, ensuring rewards and verifiers do not encourage shortcuts, and improving generalization without excessive computation or overthinking.

Latest papers

1,105 papers170 papers in the last four weeks, up 124% on the four weeks before. 1.0% of all new papers.

AgenticQwen: Training Small Agentic Language Models with Dual Data Flywheels for Industrial-Scale Tool Use
Apr 23, 2026Yuanjie Lyu, Chengyu Wang, Haonan Zheng +4Tool-Augmented Language Model Agents Agentic RL
Alibaba Group, Hangzhou, China
Understanding and Mitigating Spurious Signal Amplification in Test-Time Reinforcement Learning for Math Reasoning
Apr 23, 2026Yongcan Yu, Lingxiao He, Jian Liang +5Reinforcement Learning RL for Language Model Reasoning
1NLPR & MAIS, Institute of Automation, Chinese Academy of Sciences · *Work done during an internship at Meituan · School of Artificial Intelligence, University of Chinese Academy of Sciences +2
GRPO-VPS: Enhancing Group Relative Policy Optimization with Verifiable Process Supervision for Effective Reasoning
Apr 22, 2026Jingyi Wang, Lei Zhu, Tengjin Weng +8RL for Language Model Reasoning Group Relative Policy Optimization
Tsinghua University · Huawei Noah’s Ark Lab · Shenzhen University
Where Reasoning Breaks: Logic-Aware Path Selection by Controlling Logical Connectives in LLMs Reasoning Chains
Apr 22, 2026Seunghyun Park, Yuanyuan LeiInference-Time Search RL for Language Model Reasoning
Independent Researcher · University of Florida Gainesville, FL, United States
Reinforcing privacy reasoning in LLMs via normative simulacra from fiction
Apr 21, 2026Matt Franchi, Madiha Zahrah Choksi, Harold Triedman +1Contextual Integrity Privacy-Preserving Language Models
Cornell Tech New York, NY 10044, USA
Pause or Fabricate? Training Language Models for Grounded Reasoning
Apr 21, 2026Yiwen Qiu, Linjuan Wu, Yizhou Liu +9LLM Grounding RL for Language Model Reasoning
1Zhejiang University · 2Tencent · 3Xiaohongshu Inc.
TEMPO: Scaling Test-time Training for Large Reasoning Models
Apr 21, 2026Qingyang Zhang, Xinke Kong, Haitao Wu +7Reinforcement Learning Inference-Time Optimization
Tianjin University · Tongyi Lab, Alibaba Group · The Chinese University of Hong Kong +1
DR-MMSearchAgent: Deepening Reasoning in Multimodal Search Agents
Apr 21, 2026Shengqin Wang, Wentao Yan, Huichi Zhou +4Reward Shaping RL for Language Model Reasoning
East China Normal University · University College London · Huawei Noah’s Ark Lab +2
How Much Thinking is Enough? Quantifying and Understanding Redundancy in LLM Reasoning
Apr 21, 2026Zhiyuan Zhai, Xinkai You, Wenjing Yan +1LLM Inference Efficiency RL for Language Model Reasoning
Fudan University · The Chinese University of Hong Kong
OLLM: Options-based Large Language Models
Apr 21, 2026Shashank Sharma, Janina Hoffmann, Vinay NamboodiriLLM Alignment RL for Language Model Reasoning
Department of Computer Science University of Bath · Department of Psychology University of Bath
TRN-R1-Zero: Text-rich Network Reasoning via LLMs with Reinforcement Learning Only
Apr 21, 2026Yilun Liu, Ruihong Qiu, Zi HuangZero-Shot Learning LLM Reasoning with Graphs
School of Electrical Engineering and Computer Science The University of Queensland Brisbane, Queensland, Australia
Fine-Tuning Small Reasoning Models for Quantum Field Theory
Apr 21, 2026Nathaniel S. Woodward, Zhiqi Gao, Yurii Kvasiuk +3Physical Reasoning LLM Fine-Tuning
Department of Physics, University of Wisconsin-Madison · Department of Computer Science, University of Wisconsin-Madison · Perimeter Institute for Theoretical Physics
Discrete Tilt Matching
Apr 20, 2026Yuyuan Chen, Shiyi Wang, Peter Potaptchik +2Masked Diffusion Models RL for Language Model Reasoning
1Harvard University · University of Oxford · 3IAIFI +1
When Can LLMs Learn to Reason with Weak Supervision?
Apr 20, 2026Salman Rahman, Jingyan Shen, Anna Mordvina +3Supervised Fine-Tuning RL for Language Model Reasoning
University of California, Los Angeles · New York University · Google
OGER: A Robust Offline-Guided Exploration Reward for Hybrid Reinforcement Learning
Apr 20, 2026Xinyu Ma, Mingzhou Xu, Xuebo Liu +4Reward Shaping RL for Language Model Reasoning
Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen, China · Hithink RoyalFlush Information Network, Hangzhou, China · Computer and Information Science, University of Macau, Macau, China
Too Correct to Learn: Reinforcement Learning on Saturated Reasoning Data
Apr 20, 2026Zhenwen Liang, Yujun Zhou, Sidi Lu +3Reinforcement Learning RL for Language Model Reasoning
Tencent AI Lab · University of Notre Dame
Knowing When to Quit: A Principled Framework for Dynamic Abstention in LLM Reasoning
Apr 20, 2026Hen Davidov, Nachshon Cohen, Oren Kalinsky +4RL for Language Model Reasoning LLM Abstention
University of Oxford. Work done partly during an internship at amazon. · 2Amazon.
Learning from Less: Measuring the Effectiveness of RLVR in Low Data and Compute Regimes
Apr 20, 2026Justin Bauer, Thomas Walshe, Derek Pham +4RL for Language Model Reasoning Small Language Models
Snorkel AI · University of Wisconsin-Madison · University of Oxford
Reducing Credit Assignment Variance via Counterfactual Reasoning Paths
Apr 20, 2026Fei Ding, Yongkang Zhang, Youwei Wang +1RL for Language Model Reasoning Credit Assignment in RL
Alibaba Group · Tsinghua University
Negative Advantages Is a Double-Edged Sword: Calibrating advantages in GRPO for Search Agents
Apr 20, 2026Jiayi Wu, Ruobing Xie, Zeqian Huang +6Reinforcement Learning RL for Language Model Reasoning
School of Data Science and Engineering, East China Normal University · Tencent · Tsinghua University
Audio-DeepThinker: Progressive Reasoning-Aware Reinforcement Learning for High-Quality Chain-of-Thought Emergence in Audio Language Models
Apr 20, 2026Xiang He, Chenxing Li, Jinting Wang +5RL for Language Model Reasoning Audio Reasoning
Tencent AI Lab, Beijing, China · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China
Neural Garbage Collection: Learning to Forget while Learning to Reason
Apr 20, 2026Michael Y. Li, Jubayer Ibn Hamid, Emily B. Fox +1LLM Compression Memory-Efficient Optimization
Stanford University
HEALing Entropy Collapse: Enhancing Exploration in Few-Shot RLVR via Hybrid-Domain Entropy Dynamics Alignment
Apr 20, 2026Zhanyu Liu, Qingguo Hu, Ante Wang +5RL for Language Model Reasoning Few-Shot Learning
Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan (Xiamen University), Ministry of Culture and Tourism, China · Tsinghua University · Xiamen Unisound Intelligence Technology Co., Ltd
Learning to Correct: Calibrated Reinforcement Learning for Multi-Attempt Chain-of-Thought
Apr 20, 2026Muhammed Emrullah Ildiz, Halil Alperen Gozeten, Ege Onur Taga +1CoT Reasoning RL for Language Model Reasoning
University of Michigan, Ann Arbor · Google Research
LEPO: Latent Reasoning Policy Optimization for Large Language Models
Apr 20, 2026Yuyan Zhou, Jiarui Yu, Hande Dong +4Reinforcement Learning RL for Language Model Reasoning
Tencent · Zhejiang University · Shanghai Jiao Tong University
Stratagem: Learning Transferable Reasoning via Trajectory-Modulated Game Self-Play
Apr 20, 2026Xiachong Feng, Deyi Yin, Xiaocheng Feng +9Self-Play RL RL for Language Model Reasoning
The University of Hong Kong · Harbin Institute of Technology · Harbin Institute of Technology, Shenzhen
Poly-EPO: Training Exploratory Reasoning Models
Apr 19, 2026Ifdita Hasan Orney, Jubayer Ibn Hamid, Shreya S Ramanujam +5Reinforcement Learning RL for Language Model Reasoning
Stanford University
Internalizing Outcome Supervision into Process Supervision: A New Paradigm for Reinforcement Learning for Reasoning
Apr 19, 2026Fei Ding, Yongkang Zhang, Runhao Liu +4RL for Language Model Reasoning Credit Assignment in RL
Alibaba Group · Tsinghua University
Easy Samples Are All You Need: Self-Evolving LLMs via Data-Efficient Reinforcement Learning
Apr 19, 2026Zhiyin Yu, Bo Zhang, Qibin Hou +3Self-Training for Language Models RL for Language Model Reasoning
Peking University · Shanghai Artificial Intelligence Laboratory · Nankai University +1
Abstain-R1: Calibrated Abstention and Post-Refusal Clarification via Verifiable RL
Apr 18, 2026Skylar Zhai, Jingcheng Liang, Dongyeop KangLLM Hallucination Mitigation RL for Language Model Reasoning
University of Minnesota

Apr 23, 2026J/K move · Enter open · S save

AgenticQwen: Training Small Agentic Language Models with Dual Data Flywheels for Industrial-Scale Tool Use

Yuanjie Lyu, Chengyu Wang, Haonan Zheng, Yuanhao Yue+3

Alibaba Group, Hangzhou, China

Tool-Augmented Language Model Agents Agentic RL Synthetic Data Generation RL for Language Model Reasoning LLM Tool Use LLM Agent Training+1

Open paper AskPDF arXiv

Modern industrial applications increasingly demand language models that act as agents, capable of multi-step reasoning and tool use in real-world settings. These tasks are typically performed under strict cost and latency constraints, making small agentic models highly desirable. In this paper, we introduce the AgenticQwen family of models, trained via multi-round reinforcement learning (RL) on synthetic data and a limited amount of open-source data. Our training framework combines reasoning RL and agentic RL with dual data flywheels that automatically generate increasingly challenging tasks. The reasoning flywheel increases task difficulty by learning from errors, while the agentic flywheel expands linear workflows into multi-branch behavior trees that better reflect the decision complexity of real-world applications. We validate AgenticQwen on public benchmarks and in an industrial agent system. The models achieve strong performance on multiple agentic benchmarks, and in our industrial agent system, close the gap with much larger models on search and data analysis tasks. Model checkpoints and part of the synthetic data: https://huggingface.co/collections/alibaba-pai/agenticqwen. Data synthesis and RL training code: https://github.com/haruhi-sudo/data_synth_and_rl. The data synthesis pipeline is also integrated into EasyDistill: https://github.com/modelscope/easydistill.