Multi-Turn RL

RL: Reinforcement Learning

Follow this topic Ask about this topic

Latest papers

20 papers4 papers in the last four weeks, up 33% on the four weeks before. 0.0% of all new papers.

Productive State-Space Exploration for Multi-turn RL in LLM Agents
Oct 19, 2026Mohammed Haroon Dupty, Yong Liang Goh, Song Guo +1RL for LLM Agents RL Exploration
Institute of Advanced Computing and Intelligence, A*STAR · School of Computing, National University of Singapore
Grounded Credit Assignment in Reinforcement Learning for Multi-Turn Search Agents
Oct 16, 2026Qirui Liu, Hao Chen, Yi Wen +7Credit Assignment in RL Reinforcement Learning with Verifiable Rewards
South China University of Technology · FiT, Tencent · City University of Hong Kong +5
C3: Rethinking Process Supervision via Confidence-Guided Credit Correction in Multi-Turn Reinforcement Learning
Oct 14, 2026Ai Jian, Haoran Pan, Eryu Guo +5Multi-Turn RL Reinforcement Learning
Beijing University of Posts and Telecommunications · Peking University · Institute of Automation, Chinese Academy of Sciences +2
DeltaTTG: Training Multi-Turn Agents with a Token-to-Go Progress Meter
Oct 13, 2026Xuyan Ye, Xiyan Fu, Shengda Fan +3RL for LLM Agents Reinforcement Learning
College of Computing and Data Science, Nanyang Technological University · Gaoling School of Artificial Intelligence, Renmin University of China
PEARL: Process-Experience Aware Reinforcement Learning for Multi-Turn Agents
Oct 10, 2026Jie Yang, Mianqiu Huang, Yiwei Zhao +3Multi-Turn RL Reinforcement Learning
Fudan University · Shanghai Innovation Institute
T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks
Sep 10, 2026Junyao Yang, Yucheng Shi, Zhongzhi Li +4Reinforcement Learning with Verifiable Rewards Long-Horizon Agent Evaluation
Tencent Hy Foundation Model · National University of Singapore · University of Georgia +2
SocialRL: Refining LLMs' Social Intelligence through Multi-turn Reinforcement Learning and Reward Design
Sep 9, 2026Jianing Wang, Xintao Wang, Aili Chen +7RL for Language Models Reinforcement Learning
Fudan University · Hello Group
AHEAD: Adaptive Hindsight with Environment-Augmented Distillation for Agentic RL
Aug 25, 2026Xiaolong Jin, Dingmin Wang, Vijay Lingam +1RL for LLM Agents LLM-Guided RL
AWS AI Labs · Purdue University
RTPO: Reverse-Turn Policy Optimization for Stabilizing Agentic RL Training
Aug 19, 2026Yugu Li, Zehong Cao, Jianglin Qiao +1Agentic RL Reinforcement Learning
School of CSIT · ACFR · School of EECMS
Drowning in Routine: Signal Dilution in Multi-Turn Agent Training
Jun 20, 2026Yann Pernot, Vi RetaultReinforcement Learning Credit Assignment in RL
Mila - Québec AI Institute · Polytechnique Montréal
Progress-SQL: Improving Reinforcement Learning for Text-to-SQL via Progressive Rewards
Jun 5, 2026Shihao Zhang, Xiaoman Wang, Yuan Liu +2RL for Language Models Reinforcement Learning
East China Normal University
DRIFT: Decoupled Rollouts and Importance-Weighted Fine-Tuning for Efficient Multi-Turn Optimization
May 29, 2026Jian Mu, Tianyi Lin, Chengwei Qin +2Supervised Fine-Tuning RL for Language Models
The Hong Kong University of Science and Technology (Guangzhou) · The Chinese University of Hong Kong, Shenzhen.
What and When to Distill: Selective Hindsight Distillation for Multi-Turn Agents
May 19, 2026Xiaozhe Li, Tianyi Lyu, Yang Li +6Reinforcement Learning Credit Assignment in RL
1Tongji University · 2Shanghai AI Laboratory · 5Independent +2
GAGPO: Generalized Advantage Grouped Policy Optimization
May 13, 2026Siyuan Zhu, Chao Yu, Rongxin Yang +4Reinforcement Learning Group Relative Policy Optimization
School of Computer Science and Engineering, Sun Yat-sen University · Meituan
T $^2$ PO: Uncertainty-Guided Exploration Control for Stable Multi-Turn Agentic Reinforcement Learning
May 4, 2026Haixin Wang, Hejie Cui, Chenwei Zhang +7RL for LLM Agents Reinforcement Learning
University of California, Los Angeles · 2Amazon.com Inc.
MM-Doc-R1: Training Agents for Long Document Visual Question Answering through Multi-turn Reinforcement Learning
Apr 15, 2026Jiahang Lin, Kai Hu, Binghai Wang +12Long-Document VQA Multi-Turn RL
Fudan University · Shanghai Qiji Zhifeng Co., Ltd
Multi-Turn Reinforcement Learning for Tool-Calling Agents with Iterative Reward Calibration
Apr 3, 2026Wachiravit Modecrua, Krittanon Kaewtawee, Krittin Pachtrachai +1Multi-Turn RL RL for Tool Use
Amity Research and Application Center (ARAC)
SLEA-RL: Step-Level Experience Augmented Reinforcement Learning for Multi-Turn Agentic Training
Mar 18, 2026Prince Zizhuang Wang, Shuli JiangRL for LLM Agents Agentic RL
Carnegie Mellon University
TSR: Trajectory-Search Rollouts for Multi-Turn RL of LLM Agents
Feb 12, 2026Aladin Djuhera, Swanand Kadhe, Farhan Ahmed +3Tree Search LLM Agent Training
Technical University Munich · IBM Research
Iterative GRPO: Batch-Online Policy Iteration for Multi-Turn RL via Single-Turn RLHF
Nov 26, 2025Daniel R. Jiang, Ankur Samanta, Yukai Yang +3Reinforcement Learning Policy Iteration
Meta · FAIR at Meta

Oct 19, 2026J/K move · Enter open · S save

Productive State-Space Exploration for Multi-turn RL in LLM Agents

Mohammed Haroon Dupty, Yong Liang Goh, Song Guo, Wee Sun Lee

Institute of Advanced Computing and Intelligence, A*STAR · School of Computing, National University of Singapore

RL for LLM Agents RL Exploration Multi-Turn RL Intrinsic Motivation

Open paper AskPDF arXiv

Reinforcement Learning (RL) for Large Language Model (LLM) agents in multi-turn, long-horizon environments remains challenging due to inefficient action selection and repetitive behavioral patterns. In particular, agents often become trapped in high-confidence yet unproductive behaviors, repeatedly executing invalid actions or entering action loops that hinder task completion. We observe that these frequently violate environment preconditions, revealing a mismatch between the agent's representation of the internal state and the dynamics of the underlying environment. In addition, repetitive high-confidence actions lead to cyclical behaviors that provide little to no progress towards the goal. We argue that addressing these challenges requires more productive task-aligned exploration to improve the agent's understanding of environment dynamics. Motivated by intrinsic curiosity-driven exploration, we propose a conditional state-space exploration framework that balances task completion with intrinsic information-seeking behavior. Specifically, we introduce an intrinsic reward that penalizes actions whose observation duplicates prior state memory. This provides the benefit of reducing redundant behaviors and encouraging state-space coverage without relying on a general information-theoretic measure. Strong experimental results demonstrate that our approach substantially improves policy learning in multi-turn environments, mitigating invalid actions and repetitive loops.