Long-Horizon RL

RL: Reinforcement Learning

Follow this topic Ask about this topic

Latest papers

32 papers12 papers in the last four weeks, up 100% on the four weeks before. 0.1% of all new papers.

MiRL: Error-Targeted Reinforcement Learning for Correcting Terminal Agent Misbehaviors
Oct 26, 2026Runchu Tian, Zaid Khan, Zun Wang +2RL for LLM Agents Reinforcement Learning
Department of Computer Science, University of North Carolina at Chapel Hill
ReflectPO: Reflection-Guided Policy Optimization for Long-Horizon LLM Agents
Oct 22, 2026Rumei Li, Xunliang CaiLong-Horizon LLM Agents RL for LLM Agents
Meituan, China
Short-to-Long: Detached Features for Long-Horizon Value Learning
Oct 21, 2026Tong Che, Boris Ivanovic, Marco PavoneReinforcement Learning Long-Horizon RL
NVIDIA Research
VaDar: Implicit Goal Conditioning via Value Disaggregation
Oct 19, 2026Shashwat Saxena, Mehul Goel, Sreyas Venkataraman +2Long-Horizon RL Reinforcement Learning
Carnegie Mellon University
Arctic-SWE: Revisiting the Long Horizon Agent Training by Modeling-System Co-Design
Oct 17, 2026Boyi Liu, Yite Wang, Xinyu Lian +6Long-Horizon RL RL for LLM Agents
Snowflake · University of Illinois at Urbana-Champaign · University of Houston +1
Tree-Aware Posterior Distillation for Long-Horizon Agentic Reinforcement Learning
Oct 16, 2026Heming Zou, Qi Wang, Yixiu Mao +7RL for LLM Agents Long-Horizon RL
Tsinghua University · Foundation Model Department, Tencent
Long-Horizon RL Needs Muon
Oct 14, 2026Kai Ruan, Jinghao Lin, Tao Chen +3Long-Horizon RL Long-Horizon LLM Agents
Gaoling School of Artificial Intelligence, Renmin University of China · Duke University · Institute of Automation, Chinese Academy of Sciences +1
Tev: Scaling Long-Horizon Agentic Environments with Dense Progress Rewards
Oct 13, 2026Wenbo Hu, Shraman Pramanick, Krishna Kumar Singh +3LLM Agent Training Long-Horizon LLM Agents
Adobe Research · University of California, Los Angeles
Travel Back To Move Forward: Retrospective Exploration and Evolving for Long Horizon Agents
Oct 9, 2026Ziang Ye, Wentao Shi, Lei Chen +7Long-Horizon LLM Agents LLM Agent Planning
University of Science and Technology of China · Shanghai Innovation Institute · Meituan +4
Trajectory-Aware Policy Optimization: Closing the State Distribution Gap in Long-Horizon LLM RL
Oct 9, 2026Zhicheng Cai, Weinan Dai, Yuyan Zhou +3Long-Horizon RL Reinforcement Learning
Institute for AI Industry Research (AIR), Tsinghua University SIA-Lab of Tsinghua AIR and ByteDance Seed
SCAD: Structured Credit Assignment and Distillation for Long-Horizon Agents
Oct 2, 2026Shangyang Wu, Shuai Zhao, Ziyue Zhu +3Hierarchical Planning Step-Level Credit Assignment in RL
Beijing University of Posts and Telecommunications · Tsinghua University
Learning Multiple Timescales for Goal-Conditioned Reinforcement Learning
Oct 1, 2026Pedro Robles Dutenhefner, Dikshant Shehmar, Wagner Meira +1Reinforcement Learning Offline RL
Universidade Federal de Minas Gerais · University of Alberta · Alberta Machine Intelligence Institute (Amii) +1
Cross-Rollout Bellman Closure for Long-Horizon Agentic Reinforcement Learning
Sep 28, 2026Yangyang Ren, Haodong Zhu, Linlin Yang +3Critic-Free RL Step-Level Credit Assignment in RL
Beihang University · Zhongguancun Academy · Communication University of China +2
QwenGyre: An Elastic Reinforcement Learning Framework for Training xLong-Horizon Agents
Sep 27, 2026Weiqi Wang, Yuxin Zhou, Mouxiang Chen +9Reinforcement Learning Long-Horizon Agent Tasks
Alibaba Token Hub, Alibaba Group · University of Science and Technology of China · Tsinghua University
T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks
Sep 10, 2026Junyao Yang, Yucheng Shi, Zhongzhi Li +4RL for Tool Use Long-Horizon Agent Evaluation
Tencent Hy Foundation Model · National University of Singapore · University of Georgia +2
Long-Horizon Language Model Reinforcement Learning via Progressive Point Matching
Sep 7, 2026Preston Fu, Kevin Frans, Oleh Rybkin +2Dense Reward RL Reward Shaping
UC Berkeley · Carnegie Mellon University
Elastic Horizon: Discovering the Effective Interaction Frontier in Agentic Reinforcement Learning
Sep 7, 2026Gangyi Zhang, Junjie Meng, Letian Zhang +6Agentic RL Curriculum RL
Qwen Business Unit of Alibaba
Tracking the Moving Frontier: Long-Short Term Advantage Estimator
Sep 6, 2026Xinhao Yao, Lu Yu, Changhao Wang +5Long-Horizon RL Reinforcement Learning
Renmin University of China · Ant Group
Recursive Value Learning for Long-Horizon Offline Goal-Conditioned RL
Sep 2, 2026Hyeonseong Jeon, Youngwoon LeeLong-Horizon RL Reinforcement Learning
1Yonsei University · 2Seoul National University
Hierarchical Soft Actor-Critic for Sparse-Reward Long-Horizon Reinforcement Learning
Jul 26, 2026Zahra Abdalla Elashaal, Afef Hfaiedh, Nahla Khraief +2Reinforcement Learning Hierarchical RL
Lab. of Robotics, Informatics and Complex Systems, National Engineering School of Tunis, University of Tunis El Manar, Tunis, Tunisia · Faculty of Information Technology, University of Tripoli, Tripoli, Libya · Lab. of Robotics, Informaticsand Complex Systems, National Engineering School of Tunis, University of Tunis El Manar, Tunis, Tunisia +3
Horizon Adaptive Offline Policy Learning via Value Stitching
Jun 19, 2026Kexin Zheng, Xianyuan Zhan, Xintao YanValue Function Estimation Offline RL
The University of Hong Kong · Tsinghua University
Reinforcement Learning with Pairwise Preferences in Long-Term Decision Problems
May 29, 2026Jonathan Colaço Carr, Prakash Panangaden, Doina Precup +1Pairwise Preference Learning Markov Decision Processes
Stanford University · School of Computer Science, McGill University, Montreal, Quebec, Canada · Mila - Quebec AI Institute, Montreal, Quebec, Canada +1
Rewarding Beliefs, Not Actions: Consistency-Guided Credit Assignment for Long-Horizon Agents
May 19, 2026Wenjie Tang, Minne Li, Sijie Huang +2Reinforcement Learning Partially Observable RL
College of Computer, National University of Defense Technology · 2Intelligent Game and Decision Lab (IGDL) · Institute of Artificial Intelligence, Xiamen University
HINT-SD: Targeted Hindsight Self-Distillation for Long-Horizon Agents
May 18, 2026Woongyeng Yeo, Yumin Choi, Taekyung Ki +1Credit Assignment in RL LLM Agent Training
KAIST
Offline Reinforcement Learning with Universal Horizon Models
May 15, 2026Hojun Chung, Junseo Lee, Songhwai OhOffline RL Model-Based RL
Interdisciplinary Program in Artificial Intelligence and ASRI, Seoul National University · Department of Electrical and Computer Engineering, Seoul National University
Train Short, Act Long: Multistability and the Horizon Independence of Recurrent Policies
May 12, 2026Asad Bakija, Florent De Geeter, Julien Brandoit +2Dynamical Systems Reinforcement Learning
Dept. of Electrical Engineering and Computer Science University of Liège Liège, Belgium · University of Liège
Milestone-Guided Policy Learning for Long-Horizon Language Agents
May 7, 2026Zixuan Wang, Yuchen Yan, Hongxing Li +7Credit Assignment in RL Long-Horizon Agent Tasks
Zhejiang University · Baidu Inc.
Long-Horizon Q-Learning: Accurate Value Learning via n-Step Inequalities
May 7, 2026Armaan A. Abraham, Lucy Xiaoyang Shi, Chelsea FinnQ-Learning Temporal-Difference Learning
Stanford University
HiMAC: Hierarchical Macro-Micro Learning for Long-Horizon LLM Agents
Mar 1, 2026Hongbo Jin, Rongpeng Zhu, Jiayu Ding +2Long-Horizon LLM Agents Long-Horizon Agent Tasks
School of Electronic and Computer Engineering, Peking University
Chain-of-Goals Hierarchical Policy for Long-Horizon Offline Goal-Conditioned RL
Feb 3, 2026Jinwoo Choi, Sang-Hyun Lee, Seung-Woo SeoReinforcement Learning Hierarchical RL
Department of Electrical and Computer Engineering, Seoul National University, Seoul, South Korea · Department of Automotive Engineering, Ajou University, Gyeonggi-do, South Korea

Oct 26, 2026J/K move · Enter open · S save

MiRL: Error-Targeted Reinforcement Learning for Correcting Terminal Agent Misbehaviors

Runchu Tian, Zaid Khan, Zun Wang, Justin Chih-Yao Chen+1

Department of Computer Science, University of North Carolina at Chapel Hill

RL for LLM Agents Reinforcement Learning Terminal Agents Long-Horizon RL

Open paper AskPDF arXiv

Error-targeted learning gives a legible way to improve a model: identify specific weaknesses of the model, train the model to fix these weaknesses, and check whether the weaknesses are fixed. We study how to apply error-targeted learning to terminal agents, which solve long-horizon tasks specified in natural language by interacting with a command shell. Existing error-targeted methods are designed for single-turn tasks, and existing analyses of agent failures are used to evaluate and debug agents rather than to train them. We propose Misbehavior-driven RL (MiRL), which first profiles the errors of a policy to find generalizable misbehaviors: kinds of wrong steps that recur across many environments. MiRL then trains the policy with RL, starting rollouts at the steps where these misbehaviors occur and rewarding a rollout only if it passes the tests and avoids the misbehavior. On Qwen3.5-4B, eight misbehaviors account for 52.1% of wrong steps. After training with MiRL, the policy repeats the measured misbehaviors 43% to 57% less often on average on unseen prefixes. MiRL outperforms TMax-4B on Terminal-Bench Lite (51.8% vs. 47.4%) and matches it on Terminal-Bench 2.1 (21.3% vs. 20.6%), while using 35% of its rollouts. These results show that error-targeted learning can be applied to realistic, long-horizon terminal tasks.