Context as an Environment: Programmatic Context Management for Long-Horizon Agents

Yin Lin; Elaine Ang; Erkang Zhu; Bolin Ding; Jingren Zhou

cs.AIAug 21, 2026

Context as an Environment: Programmatic Context Management for Long-Horizon Agents

Authors: Yin Lin, Elaine Ang, Erkang Zhu, Bolin Ding, Jingren Zhou

Organizations: Alibaba Group · Columbia University

Abstract

LLM agents increasingly take on long-running tasks whose history grows far beyond a single model context window. Existing approaches compress earlier interactions or extract selected information into fixed memory representations, committing to what to preserve before future needs are known. We present Scroll, a context manager that treats each agent session as an executable Session Environment. The environment is backed by an append-only Event Log and a sandboxed, persistent Python kernel. The kernel maintains a typed namespace across model calls, allowing tool outputs, retrieved history, and derived state to be bound to variables rather than serialized into the prompt at each call. Model-written code searches, materializes, and transforms session state through exec; only explicitly printed projections enter the model's working view for the next call. Context management thus becomes a programming task that inherits the improving coding abilities of LLMs, while the Event Log preserves lossless historical ground truth. As the working view approaches its budget, stale spans are evicted but remain recoverable: an eviction index keeps compact landmarks tied to exact Event Log addresses, so that the agent navigates directly to evicted regions instead of searching the full log. With Qwen3.8-Max as the backbone, Scroll achieves 94.8% on LongMemEval_S; 73.1% on BEAM_10M, surpassing the best published memory system by 5.1 points; and 86.7% on LOCA_256K, exceeding the best published long-horizon agent by 37.4 points.

Figures & tables

Figure 1: Overview of Scroll. Scroll keeps the full session in a persistent, executable Session Environment; model-written code retrieves and computes over it via exec , and print selects the working view exposed to the next model call.

Operation	Interface	Role
Locate	ms.search(query, k, ...)	Find candidate Event Log records via BM25-ranked full-text search and structured scope, kind, and time filters; each hit carries its stable seq address.
Materialize	ms.expand(seq) ms.expand(seq_lo, seq_hi)	Recover exact turns or sequence spans, and load externalized payload contents behind lazy handles.
Compute	Ordinary Python and permitted database, filesystem, and tool interfaces	Filter, join, aggregate, resolve updates, inspect artifacts, or construct derived state.
Expose	print(value)	Return the selected projection as a bounded observation; everything else stays in the kernel.

Table 1: The model-facing interface factorizes context construction into location, materialization, computation, and exposure.

Figure 2: Programmatic context construction. Three exec turns compute over resident state in the kernel; only print output crosses into the next model context/working view.

Method	LongMemEval S	BEAM 10M
RAG ( 27 )	–	24.9
LIGHT ( 27 )	–	26.6
Zep ( 37 )	90.2	–
Mem0 ( 6 ; 19 )	94.4	48.6
Hindsight ( 5 )	94.6	64.1
EmergenceMem ( 8 )	86.0	–

Table 2: Comparison with existing long-term memory systems. For each baseline, we report the best publicly available result under that system’s own setup as of August 15, 2026; “–” denotes no publicly reported result. These are reference points from the literature rather than a controlled comparison: reader models differ across rows and can substantially affect scores—EmergenceMem (GPT-4o), Zep (GPT-5.4), Mastra OM (GPT-5 mini), Mem0 (GPT-5), Hindsight (Gemini 3 Pro), Exabase M-1 (Gemini 3 Flash), RAG and LIGHT (Llama-4-Maverick); Honcho uses a multi-model pipeline and Cognee does not report its reader.

Agent loop	128K	256K	$\Delta$
Summarization Agent	86.7	65.3	-21.4
Retrieval Agent	88.0	66.7	-21.3
CodeAct Agent	89.3	85.3	-4.0
Scroll (ours)	89.3	86.7	-2.6

Table 3: LOCA accuracy (%) of different context-management strategies at the two largest environment description lengths. All agent loops use Qwen3.8-Max as the backbone;

\Delta

denotes the absolute drop from 128K to 256K.

Backbone	LongMemEval S	BEAM 10M	LOCA 128K	LOCA 256K
Qwen3.8-Max	94.8	73.1	89.3	86.7
Qwen3.7-Max	92.8	66.6	78.7	60.0
Deepseek-v4-pro	93.2	70.2	69.3	58.7
GLM-5.2	93.6	70.7	66.7	62.7
Kimi-K2.7	92.0	67.2	30.7	32.0
Qwen3.6-35B-A3B	88.8	58.1	37.3	22.7

Table 4: Scroll across backbones. Only the foundation model changes; harness, tools, prompts, and context-management rules are identical.

Figure 3: Ablating Scroll’s components on BEAM

{}_{\text{10M}}

(judge scores; hatched bars are Scroll variants; Qwen3.8-Max, thinking on). Lossy summarization: summaries replace the originals at ingestion. Scroll w/o REPL: ms exposed as ordinary tool calls, with no persistent kernel. Scroll w/o index: the eviction index is removed, leaving keyword search only. Scroll: the full system.

Figure 4: Per-task cost of Scroll (backbone: Qwen3.8-Max): (a) input tokens, (b) output tokens, (c) agent turns. Boxes span the IQR with the median marked; red diamonds are means. Log scale in (a) and (b).

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Question type	LongMemEval S	LongMemEval M
Single-session (user)	98.6	100.0
Single-session (assistant)	100.0	100.0
Single-session (preference)	100.0	83.3
Multi-session	88.0	81.2
Temporal reasoning	94.0	93.6
Knowledge update	98.7	87.2

Appendix

Table 5: Per-question-type accuracy (%) of Scroll on LongMemEval S and LongMemEval M (backbone: Qwen3.8-Max). Question types follow the benchmark taxonomy [ 31 ] .

Category	Mem0	Hindsight	Exabase M-1	Scroll (ours)
Information extraction	56.2	51.2	66.3	75.0
Multi-session reasoning	26.1	16.6	9.6	21.9
Event ordering	20.2	61.6	67.5	64.1
Temporal reasoning	16.3	41.3	58.8	47.5
Knowledge update	75.0	65.0	45.0	92.5
Contradiction resolution	32.5	56.9	58.8	88.1

Appendix

Table 6: Per-category judge scores on BEAM 10M . Categories follow the benchmark’s ten memory abilities [ 27 ] . Scroll uses Qwen3.8-Max with thinking on; baseline breakdowns are taken from their published evaluations and use different configurations, so cross-system comparison is indicative rather than controlled.

System	128K	256K
GPT-5.2-Medium + ReAct [ 33 ]	38.7	21.3
GPT-5.2-Medium + PTC [ 3 ]	49.3	–
Claude-4.5-Opus + ReAct [ 35 ]	34.0	14.7
MiniMax M3 + ReAct [ 20 ]	–	49.3
Scroll (Qwen3.8-Max)	89.3	86.7

Appendix

Table 7: System-level comparison on LOCA. Results for prior systems are taken from the LOCA paper [ 35 ] or its public leaderboard and use different backbone models; “–” denotes no publicly reported result.

LLM Context Management

Large language model context management studies how to select, compress, retrieve, and organize information within bounded model inputs.

See all 251 papers

PRO-LONG: Programmatic Memory Enables Long-Horizon Reasoning
Jul 22, 2026Alexis Fox, Junlin Wang, Paul Rosu +1Memory-Augmented Language Model Agents Persistent Memory for Language Models
Duke University
Continuous Context Management
Sep 28, 2026William Hoy, Jingxuan Fan, Nurcin Celik +1Context Distillation Long-Horizon LLM Agents
University of Miami · Harvard University
ContextPilot: Teaching Agents for Proactive Context Management via Fine-grained RL
Aug 28, 2026Zhuoshi Pan, Qizhi Pei, Junru Lu +4LLM Context Management Long-Horizon LLM Agents
Tsinghua University · Tencent Youtu Lab · Shanghai AI Lab

Jul 22, 2026J/K move · Enter open · S save

PRO-LONG: Programmatic Memory Enables Long-Horizon Reasoning

Alexis Fox, Junlin Wang, Paul Rosu, Bhuwan Dhingra

Duke University

Memory-Augmented Language Model Agents Persistent Memory for Language Models Long-Horizon Agent Tasks LLM Context Management Long-Horizon LLM Reasoning Long-Horizon Reasoning+1

Open paper AskPDF arXiv

Long-horizon tasks require sustained perception, reasoning, and exploration, and are a persistent challenge for large language model (LLM) agents. This gap is reflected in their limited performance on continual learning benchmarks such as ARC-AGI-3 and NetHack, especially when evaluated without specialized harnesses. Various agent harnesses have been proposed to close the gap, and each commits to a strategy for handling long sequences of observations, i.e., what information to save from the environment and how to load it into model context, a choice we argue is particularly consequential. Existing methods for context management have typically been designed around a tradeoff, where preserving more information makes relevant details harder to retrieve. Recent progress in coding agents, however, makes new approaches to memory viable. We propose PRO-LONG, a minimal context management framework built around programmatic memory for LLM agents in long-horizon, exploratory settings. PRO-LONG addresses the tradeoff by keeping a complete, structured interaction log and using programmatic tools to search this history efficiently. On NetHack, we find increasing gains from using PRO-LONG for cross-episodic memory, with scores reaching up to 2.2x those of a base coding agent. On the full ARC-AGI-3 public game set, PRO-LONG improves over this baseline agent by an average of 18.0 percentage points across frontier models, and matches or exceeds state-of-the-art specialized harnesses (up to 97.4% best@2) while using 4.2-5.8x fewer tokens. Relevant code and logs are available at https://github.com/alexisfox7/PRO-LONG.

Abstract

LLM Context Management

Explore similar work

PRO-LONG: Programmatic Memory Enables Long-Horizon Reasoning

Continuous Context Management

ContextPilot: Teaching Agents for Proactive Context Management via Fine-grained RL