Multi-Turn RL

RL: Reinforcement Learning

Latest papers

27 papers10 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

What Does Multi-Turn Reinforcement Learning Really Improve in LLM Agents?
Nov 4, 2026Kejian Zhu, Zhuoran Jin, Shangqing Tu +5RL for LLM Agents Reinforcement Learning
The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences · Tsinghua University
TSBPO: Tree-Structured Branching for Policy Optimization in Interactive Language Agents
Oct 30, 2026Rui Yu, Chong Mu, Jinchuan ZhangRL for LLM Agents Multi-Turn RL
University of Electronic Science and Technology of China
InterQ-RL: Interaction-Aware Correction of Execution Mismatch in Quantized Agentic RL
Oct 26, 2026Yingbo Hao, Song Liu, Guanghua Yu +3Reinforcement Learning Agentic RL
South China University of Technology · Tencent
Multiscale Policy Optimization for Long-Horizon LLM Agents
Oct 26, 2026Tao Ren, Hui Yang, Jia Deng +16Policy Optimization RL for LLM Agents
Peking University · IQuest Research · Tsinghua University +3
One Act, Many Masks: Effect-Marginal Policy Optimization for Long-Horizon Agents
Oct 21, 2026Shaojin Ma, Ruitao Wang, Pengyi Li +6Policy Optimization RL for LLM Agents
Tianjin University · Fudan University · Renmin University of China
Rewards for Budget-Aware Multi-Turn Agents That Adapt to Arbitrary Test-Time Constraints
Oct 20, 2026Tom A. Lamb, Adam Golinski, Eleonora Gualdoni +2LLM Agent Efficiency Reward Design for RL
Apple · University of Oxford
Productive State-Space Exploration for Multi-turn RL in LLM Agents
Oct 19, 2026Mohammed Haroon Dupty, Yong Liang Goh, Song Guo +1RL for LLM Agents RL Exploration
Institute of Advanced Computing and Intelligence, A*STAR · School of Computing, National University of Singapore
Grounded Credit Assignment in Reinforcement Learning for Multi-Turn Search Agents
Oct 16, 2026Qirui Liu, Hao Chen, Yi Wen +7Credit Assignment in RL Reinforcement Learning with Verifiable Rewards
South China University of Technology · FiT, Tencent · City University of Hong Kong +5
C3: Rethinking Process Supervision via Confidence-Guided Credit Correction in Multi-Turn Reinforcement Learning
Oct 14, 2026Ai Jian, Haoran Pan, Eryu Guo +5Multi-Turn RL Reinforcement Learning
Beijing University of Posts and Telecommunications · Peking University · Institute of Automation, Chinese Academy of Sciences +2
DeltaTTG: Training Multi-Turn Agents with a Token-to-Go Progress Meter
Oct 13, 2026Xuyan Ye, Xiyan Fu, Shengda Fan +3RL for LLM Agents Reinforcement Learning
College of Computing and Data Science, Nanyang Technological University · Gaoling School of Artificial Intelligence, Renmin University of China
PEARL: Process-Experience Aware Reinforcement Learning for Multi-Turn Agents
Oct 10, 2026Jie Yang, Mianqiu Huang, Yiwei Zhao +3Multi-Turn RL Reinforcement Learning
Fudan University · Shanghai Innovation Institute
T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks
Sep 10, 2026Junyao Yang, Yucheng Shi, Zhongzhi Li +4RL for Tool Use Long-Horizon Agent Evaluation
Tencent Hy Foundation Model · National University of Singapore · University of Georgia +2
SocialRL: Refining LLMs' Social Intelligence through Multi-turn Reinforcement Learning and Reward Design
Sep 9, 2026Jianing Wang, Xintao Wang, Aili Chen +7RL for Language Models Reinforcement Learning
Fudan University · Hello Group
AHEAD: Adaptive Hindsight with Environment-Augmented Distillation for Agentic RL
Aug 25, 2026Xiaolong Jin, Dingmin Wang, Vijay Lingam +1RL for LLM Agents LLM-Guided RL
AWS AI Labs · Purdue University
RTPO: Reverse-Turn Policy Optimization for Stabilizing Agentic RL Training
Aug 19, 2026Yugu Li, Zehong Cao, Jianglin Qiao +1Agentic RL Reinforcement Learning
School of CSIT · ACFR · School of EECMS
Drowning in Routine: Signal Dilution in Multi-Turn Agent Training
Jun 20, 2026Yann Pernot, Vi RetaultReinforcement Learning Credit Assignment in RL
Mila - Québec AI Institute · Polytechnique Montréal
Progress-SQL: Improving Reinforcement Learning for Text-to-SQL via Progressive Rewards
Jun 5, 2026Shihao Zhang, Xiaoman Wang, Yuan Liu +2RL for Language Models Reinforcement Learning
East China Normal University
DRIFT: Decoupled Rollouts and Importance-Weighted Fine-Tuning for Efficient Multi-Turn Optimization
May 29, 2026Jian Mu, Tianyi Lin, Chengwei Qin +2Supervised Fine-Tuning RL for Language Models
The Hong Kong University of Science and Technology (Guangzhou) · The Chinese University of Hong Kong, Shenzhen.
What and When to Distill: Selective Hindsight Distillation for Multi-Turn Agents
May 19, 2026Xiaozhe Li, Tianyi Lyu, Yang Li +6Reinforcement Learning Credit Assignment in RL
1Tongji University · 2Shanghai AI Laboratory · 5Independent +2
GAGPO: Generalized Advantage Grouped Policy Optimization
May 13, 2026Siyuan Zhu, Chao Yu, Rongxin Yang +4Reinforcement Learning Group Relative Policy Optimization
School of Computer Science and Engineering, Sun Yat-sen University · Meituan
T $^2$ PO: Uncertainty-Guided Exploration Control for Stable Multi-Turn Agentic Reinforcement Learning
May 4, 2026Haixin Wang, Hejie Cui, Chenwei Zhang +7RL for LLM Agents Reinforcement Learning
University of California, Los Angeles · 2Amazon.com Inc.
MM-Doc-R1: Training Agents for Long Document Visual Question Answering through Multi-turn Reinforcement Learning
Apr 15, 2026Jiahang Lin, Kai Hu, Binghai Wang +12Long-Document Visual Question Answering Multi-Turn RL
Fudan University · Shanghai Qiji Zhifeng Co., Ltd
Multi-Turn Reinforcement Learning for Tool-Calling Agents with Iterative Reward Calibration
Apr 3, 2026Wachiravit Modecrua, Krittanon Kaewtawee, Krittin Pachtrachai +1Multi-Turn RL RL for Tool Use
Amity Research and Application Center (ARAC)
SLEA-RL: Step-Level Experience Augmented Reinforcement Learning for Multi-Turn Agentic Training
Mar 18, 2026Prince Zizhuang Wang, Shuli JiangRL for LLM Agents Agentic RL
Carnegie Mellon University
TSR: Trajectory-Search Rollouts for Multi-Turn RL of LLM Agents
Feb 12, 2026Aladin Djuhera, Swanand Kadhe, Farhan Ahmed +3Tree Search LLM Agent Training
Technical University Munich · IBM Research
Iterative Policy Optimization: Batch-Online Multi-Turn RL via Single-Turn RL
Nov 26, 2025Daniel R. Jiang, Ankur Samanta, Yukai Yang +3Reinforcement Learning Policy Iteration
Meta AI · Columbia University · FAIR at Meta
Test-Time Policy Adaptation for Enhanced Multi-Turn Interactions with LLMs
Sep 27, 2025Chenxing Wei, Hong Wang, Ying He +2Inference-Time Adaptation of Language Models Multi-Turn Dialogue Management
School of Computing and Data Science, The University of Hong Kong · Shenzhen Loop Area Institute · University of Science and Technology of China +2

Nov 4, 2026J/K move · Enter open · S save

What Does Multi-Turn Reinforcement Learning Really Improve in LLM Agents?

Kejian Zhu, Zhuoran Jin, Shangqing Tu, Tianyi Men+4

The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences · Tsinghua University

RL for LLM Agents Reinforcement Learning Multi-Turn RL Long-Horizon RL

Open paper AskPDF arXiv

While multi-turn agentic reinforcement learning (RL) excels on long-horizon tasks, its performance gains remain highly sensitive to interaction constraints. Evaluating across sampling breadth

K

and interaction depth

T

via

Pass@(K,T)

, we show that trajectory-level RL advantages depend on sufficient turn budgets, diminishing significantly when budgets tighten. Besides, agentic RL does not uniformly elevate single-step decision quality; rather, its strength concentrates in error recovery and long-horizon resilience. To mitigate the deployment risks caused by this budget dependence, we provide a preliminary discussion on the benefits and limitations of process supervision signals and introduce BudgetMix RL as an alternative that varies interaction budgets during training. Experiments demonstrate that BudgetMix RL achieves robust performance across diverse execution settings. Project Page: https://rethinking-agentic-rl.github.io/