When to Forget: Risk-Controlled Context Adaptation for Long-Horizon Agents

Chenle Chen; Yangbo Wei; Zhijie Zheng; Zhenyu Su; Qifan Wang; Junhong Qian; Chao Yao; Chen Wu

cs.AIOct 16, 2026

When to Forget: Risk-Controlled Context Adaptation for Long-Horizon Agents

Authors: Chenle Chen, Yangbo Wei, Zhijie Zheng, Zhenyu Su, Qifan Wang, Junhong Qian, Chao Yao, Chen Wu,

Organizations: University of California, Los Angeles · Shanghai Jiao Tong University · Arizona State University · Eastern Institute of Technology, Ningbo

Abstract

Long-horizon agents learn from failure, but the value of a failure record can change as execution progresses. We study when retaining, compressing, or removing the same history block improves subsequent execution relative to full-history KEEP. We introduce RECAP, a framework that combines matched context interventions, execution-relational prediction, and independent policy certification. Matched environment states and rollout seeds measure success and continuation prompt-cost differences; prefix-only relations between a failure block and the current execution propose promising edits. Independent task bundles then select a policy subject to harmful-intervention and non-positive-benefit risk constraints under a fixed measurement protocol. A five-seed longitudinal case holds both the failure block and its summary fixed: compression fails at an early state and becomes a cost-saving intervention at a later state. Across five backbones, RECAP produces positive paired-success saving estimates with backbone-specific reliability-efficiency tradeoffs. On Hunyuan-A13B, it edits 43.24% of retained checkpoints while matching KEEP's observed 91.89% continuation success and reducing unconditional mean continuation prompt cost by 4,619.8 tokens (2.13%). The framework provides an execution-level reference for deciding when to forget: retain history by its value to what comes next.

LLM Context Management

Large language model context management studies how to select, compress, retrieve, and organize information within bounded model inputs.

See all 247 papers

AMBER: Training Long-Horizon Web Agents through Append-Only Memory
Oct 5, 2026Chinmay Savadikar, Zhaoyu Zhang, Mingyu Zhao +4Long-Horizon LLM Agents Agent Memory Management
North Carolina State University · Shopify
StateComp: Learning When to Compress History in Long Horizon Agents
Sep 23, 2026Mingxuan Wang, Hongyue Chen, Yinglong Guo +6Context Compression Long-Horizon LLM Agents
TierFlow Team · Gaoling School of Artificial Intelligence, Renmin University of China · Tsinghua University
AgentRewind: Recoverable Execution for Long-Horizon LLM Agents
Aug 14, 2026Yu Zhuang, Kefei Chen, Yitong Duan +3Long-Horizon LLM Agents Long-Horizon Agent Tasks
University of Chinese Academy of Sciences · Zhongguancun Academy, Beijing, China · Institute of Automation, Chinese Academy of Sciences +1

Oct 5, 2026J/K move · Enter open · S save

AMBER: Training Long-Horizon Web Agents through Append-Only Memory

Chinmay Savadikar, Zhaoyu Zhang, Mingyu Zhao, Shuang Xie+3

North Carolina State University · Shopify

Long-Horizon LLM Agents Agent Memory Management RL for LLM Agents Web Agents

Open paper AskPDF arXiv

Modern language-model agents increasingly interact with external environments over long-horizon, multi-step trajectories, where the accumulated interaction history can quickly exceed practical context budgets. To ensure reliability, agents must maintain factual information over long horizons, remember execution errors and corrective feedback, and track progress across actions. Several approaches have been proposed to achieve this without the need for maintaining the entire execution history in context, such as using the reasoning and action history, learning to maintain a fixed-size memory through an overwrite mechanism, and periodic summarization. Although overwrite memory can in principle retain anything an append-only memory can, it must learn to carry each fact through every subsequent rewrite, which is difficult to learn from sparse outcome rewards; for interactive applications like web agents, we find that trained overwrite memories delete key information required by the trajectory, as well as corrective feedback received from the environment. We introduce AMBER (Append-only Memory Bank for Evidence Retention) - a simple and scalable framework where an agent jointly learns to reason, act, and write free-form memory, while an append-only rule guarantees retention by construction. This allows AMBER to be trained end-to-end with reinforcement learning from outcome rewards without the need for extensive curated SFT data. On WebArena Lite, AMBER improves average success over overwrite-based memory by 4.09 percentage points, increases the fraction of tasks solved in five repeated runs by 4.8 percentage points, and matches an overwrite baseline trained on substantially more expensive curated supervision. AMBER achieves these improvements while maintaining a practical token budget, providing a strong balance between context efficiency, task performance, and reliable long-horizon execution.

Abstract

LLM Context Management

Explore similar work

AMBER: Training Long-Horizon Web Agents through Append-Only Memory

StateComp: Learning When to Compress History in Long Horizon Agents

AgentRewind: Recoverable Execution for Long-Horizon LLM Agents