Long-Horizon LLM Agents

LLM: Large Language Model

Long-horizon LLM agents are systems designed to pursue goals through many connected actions, often while using tools and responding to changing information. Common approaches include hierarchical planning and subgoals, reinforcement learning with step-level or milestone rewards, persistent memory and context compression, and harnesses that verify actions and recover from errors. Researchers evaluate them in interactive settings such as ALFWorld, WebShop, ScienceWorld, WebArena, software-engineering benchmarks, and deep-research tasks. A central challenge is making performance reliable over time: errors and lost constraints can compound, while memory, verification, and extra computation can improve success but add cost or introduce new failure modes.

Latest papers

451 papers92 papers in the last four weeks, up 56% on the four weeks before. 0.5% of all new papers.

HAGE: Harnessing Agentic Memory via RL-Driven Weighted Graph Evolution
May 11, 2026Dongming Jiang, Yi Li, Guanpeng Li +2Agent Memory Graph-Based Agent Memory
Department of Computer Science, The University of Texas at Dallas · Department of Electrical and Computer Engineering, University of Florida · University of California, Davis
EvoMAS: Learning Execution-Time Workflows for Multi-Agent Systems
May 9, 2026Chengdong Xu, Kaiqiang Ke, Ziheng Liu +4Multi-Agent LLM Systems Multi-Agent System Optimization
Sun Yat-Sen University · ShanghaiTech University · Zhejiang University +1
Slipstream: Trajectory-Grounded Compaction Validation for Long-Horizon Agents
May 9, 2026Zhuofu Chen, Rui Pan, Yinwei Dai +1LLM Agent Memory Long-Horizon LLM Agents
Princeton University
NARRA-Gym for Evaluating Interactive Narrative Agents
May 8, 2026Yue Huang, Yuchen Ma, Jiayi Ye +14Language Model Generation Evaluation Interactive Storytelling
University of Notre Dame · 2LMU Munich · 11Munich Center for Machine Learning +8
Tools as Continuous Flow for Evolving Agentic Reasoning
May 8, 2026Tairan Huang, Siyu Shang, Qiang Chen +2Tool-Use Planning AI Agent Benchmarks
Central South University · The Hong Kong University of Science and Technology
StraTA: Incentivizing Agentic Reinforcement Learning with Strategic Trajectory Abstraction
May 7, 2026Xiangyuan Xue, Yifan Zhou, Zidong Wang +5Agentic RL Hierarchical RL
1The Chinese University of Hong Kong · 2Shanghai Artificial Intelligence Laboratory · University of Georgia +2
Milestone-Guided Policy Learning for Long-Horizon Language Agents
May 7, 2026Zixuan Wang, Yuchen Yan, Hongxing Li +7Credit Assignment in RL Long-Horizon Agent Tasks
Zhejiang University · Baidu Inc.
TACT: Mitigating Overthinking and Overacting in Coding Agents via Activation Steering
May 7, 2026Yuan Sui, Yulin Chen, Yibo Li +6AI Coding Agents Long-Horizon LLM Agents
National University of Singapore · Peking University · Meta
From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanisms
May 7, 2026Jinghao Luo, Yuchen Tian, Chuxue Cao +6Continual Learning for LLM Agents LLM Agent Memory
2South China Normal University · 1Hong Kong Baptist University · 3Hong Kong University of Science and Technology +2
LongSeeker: Elastic Context Orchestration for Long-Horizon Search Agents
May 6, 2026Yijun Lu, Rui Ye, Yuwen Du +3Web Search Agents Tool-Using Agents
Shanghai Jiao Tong University
Tree-based Credit Assignment for Multi-Agent Memory System
May 6, 2026Marina Mao, Alexandr Liu, Pengbo Li +3Credit Assignment Long-Horizon LLM Agents
University of Science and Technology of China · 2LLM Department, Tencent · 3The Hong Kong University of Science and Technology +1
MEMTIER: Tiered Memory Architecture and Retrieval Bottleneck Analysis for Long-Running Autonomous AI Agents
May 5, 2026Bronislav Sidik, Lior RokachLLM Agent Memory Hierarchical Memory
Institute for Applied AI Research · Faculty of Computer and Information Science · Ben-Gurion University of the Negev, Beer Sheva, Israel
MemFlow: Intent-Driven Memory Orchestration for Small Language Model Agents
May 5, 2026Jiayi Chen, Yingcong Li, Guiling WangLLM Agent Workflow Optimization Persistent Memory for Language Models
New Jersey Institute of Technology, Newark, USA
Beyond State Machines: Executing Network Procedures with Agentic Tool-Calling Sequences
May 4, 2026Purna Sai Garigipati, Onur Ayan, Kishor Chandra Joshi +1LLM Agent Orchestration Tool-Using Agents
Heisenberg Research Center, Huawei Technologies Duesseldorf GmbH, 80992 Munich, Germany · Eindhoven University of Technology, Eindhoven, The Netherlands
On Training Large Language Models for Long-Horizon Tasks: An Empirical Study of Horizon Length
May 4, 2026Sunghwan Kim, Junhee Cho, Beong-woo Kwak +6Long-Horizon Planning Length Generalization
Department of Artificial Intelligence, Yonsei University · †Work done during internship at Microsoft Research. · 2Microsoft Research.
The Conversations Beneath the Code: Triadic Data for Long-Horizon Software Engineering Agents
May 4, 2026Yelin KimSoftware Engineering Agents Human-AI Interaction
Independent Researcher
Planner Matters! An Efficient and Unbalanced Multi-agent Collaboration Framework for Long-horizon Planning
May 4, 2026Wenyi Wu, Sibo Zhu, Kun Zhou +1Reinforcement Learning Multi-Agent Collaboration
University of California, San Diego
Beyond Compaction: Structured Context Eviction for Long-Horizon Agents
May 1, 2026Andrew Semenov, Svyatoslav DorofeevLong-Horizon LLM Agents LLM Context Management
Kiz8
Synthetic Computers at Scale for Long-Horizon Productivity Simulation
Apr 30, 2026Tao Ge, Baolin Peng, Hao Cheng +1Computer-Use Agents Synthetic Data Generation
Microsoft
Bridging Values and Behavior: A Hierarchical Framework for Proactive Embodied Agents
Apr 30, 2026Chunhui Zhang, Yuxuan Wang, Aoyang Qin +4Hierarchical Planning Agent Evaluation
State Key Laboratory of General Artificial Intelligence, BIGAI · School of Computer Science, Peking University · Tsinghua University +3
End-to-end autonomous scientific discovery on a real optical platform
Apr 29, 2026Shuxing Yang, Fujia Chen, Rui Zhao +10Autonomous Scientific Discovery Long-Horizon LLM Agents
State Key Laboratory of Extreme Photonics and Instrumentation, College of Information Science and Electronic Engineering, ZJU-Hangzhou Global Scientific and Technological Innovation Center, The Electromagnetics Academy at Zhejiang University, Zhejiang University, Hangzhou 310027, China · Key Laboratory of Advanced Micro/Nano Electronic Devices & Smart Systems of Zhejiang, Jinhua Institute of Zhejiang University, Zhejiang University, Jinhua 321099, China · Shaoxing Institute of Zhejiang University, Zhejiang University, Shaoxing 312000, China +3
Operating-Layer Controls for Onchain Language-Model Agents Under Real Capital
Apr 28, 2026T. J. Barton, Chris Constantakis, Patti Hauseman +4AI Agent Reliability LLM Agent Evaluation
DX Research Group (DXRG)
Cutscene Agent: An LLM Agent Framework for Automated 3D Cutscene Generation
Apr 28, 2026Lanshan He, Haozhou Pang, Qi Gan +12LLM Agent Orchestration 3D Scene Generation
Kuaishou GameMind Lab
Bridging Reasoning and Action: Hybrid LLM-RL Framework for Efficient Cross-Domain Task-Oriented Dialogue
Apr 25, 2026Yangyang Zhao, Linfan Dai, Li Cai +2Constrained RL LLM-Guided RL
School of Computer Science and Technology, Changsha University of Science and Technology · School of Computer Science and Engineering, Central South University · Text Computing and Cognitive Intelligence MOE Engineering Research Center, Guizhou University +2
Co-Evolving LLM Decision and Skill Bank Agents for Long-Horizon Tasks
Apr 22, 2026Xiyang Wu, Zongxia Li, Guangyao Shi +5Continual Learning for LLM Agents Game-Playing Agents
University of Maryland · University of Southern California · Good Start Labs +2
Stateless Decision Memory for Enterprise AI Agents
Apr 22, 2026Vasundra SrinivasanLLM Agent Memory Long-Horizon LLM Agents
Data Engineering for Multimodal AI (O’Reilly) · Stanford School of Engineering
DR-Venus: Towards Frontier Edge-Scale Deep Research Agents with Only 10K Open Data
Apr 21, 2026Venus Team, Sunhao Dai, Yong Deng +10Agentic RL Deep Research Agents
Ant Group
GenericAgent: A Token-Efficient Self-Evolving LLM Agent via Contextual Information Density Maximization (V1.0)
Apr 18, 2026Jiaqing Liang, Jinyi Han, Weijia Li +15Tool-Augmented Language Model Agents LLM Agent Memory
Advantage AI Agent Lab (A3 Lab)
Experience Compression Spectrum: Unifying Memory, Skills, and Rules in LLM Agents
Apr 17, 2026Xing Zhang, Guanghui Wang, Yanwei Cui +4LLM Agent Memory LLM Agent Skill Learning
AWS Generative AI Innovation Center · HSBC Holdings Plc., HSBC Technology Center, China.
LLMs Corrupt Your Documents When You Delegate
Apr 17, 2026Philippe Laban, Tobias Schnabel, Jennifer NevilleLLM Reliability Long-Horizon LLM Agents
Microsoft Research

May 11, 2026J/K move · Enter open · S save

HAGE: Harnessing Agentic Memory via RL-Driven Weighted Graph Evolution

Dongming Jiang, Yi Li, Guanpeng Li, Qiannan Li+1

Department of Computer Science, The University of Texas at Dallas · Department of Electrical and Computer Engineering, University of Florida · University of California, Davis

Agent Memory Graph-Based Agent Memory Long-Horizon LLM Agents Agent Memory Management

Open paper AskPDF arXiv

Memory retrieval in agentic large language model (LLM) systems is often treated as a static lookup problem, relying on flat vector search or fixed binary relational graphs. However, fixed graph structures cannot capture the varying strength, confidence, and query-dependent relevance of relationships between events. In this paper, we propose HAGE, a weighted multi-relational memory framework that reconceptualizes retrieval as sequential, query-conditioned traversal over a unified relational memory graph. Memory is organized as relation-specific graph views over shared memory nodes, where each edge is associated with a trainable relation feature vector encoding multiple relational signals. Given a query, an LLM-based classifier identifies the relational intent, and a routing network dynamically modulates the corresponding dimensions of the edge embedding. Traversal scores are computed via a learned combination of semantic similarity and these query-conditioned edge representations. This allows memory traversal to prioritize high-utility relational paths while softly suppressing noisy or weakly relevant connections. Beyond adaptive traversal, HAGE further introduces a reinforcement learning-based training framework that jointly optimizes routing behavior and edge representations using downstream tasks. Finally, empirical results demonstrate improved long-horizon reasoning accuracy and a favorable accuracy-efficiency trade-off compared to state-of-the-art agentic memory systems. Our code is available at https://github.com/FredJiang0324/HAGE_MVPReview.