Contextual Memory Virtualisation: DAG-Based State Management and Structurally Lossless Trimming for LLM Agents

Cosmo Santoni

cs.SEFeb 25, 2026

Contextual Memory Virtualisation: DAG-Based State Management and Structurally Lossless Trimming for LLM Agents

Authors: Cosmo Santoni

Organizations: Imperial College London

Abstract

As large language models engage in extended reasoning tasks, they accumulate significant state -- architectural mappings, trade-off decisions, codebase conventions -- within the context window. This understanding is lost when sessions reach context limits and undergo lossy compaction. We propose Contextual Memory Virtualisation (CMV), a system that treats accumulated LLM understanding as version-controlled state. Borrowing from operating system virtual memory, CMV models session history as a Directed Acyclic Graph (DAG) with formally defined snapshot, branch, and trim primitives that enable context reuse across independent parallel sessions. We introduce a three-pass structurally lossless trimming algorithm that preserves every user message and assistant response verbatim while reducing token counts by a mean of 20% and up to 86% for sessions with significant overhead by stripping mechanical bloat such as raw tool outputs, base64 images, and metadata. A single-user case-study evaluation across 76 real-world coding sessions demonstrates that trimming remains economically viable under prompt caching, with the strongest gains in mixed tool-use sessions, which average 39% reduction and reach break-even within 10 turns. A reference implementation is available at https://github.com/IngressionLabs/contextual-memory-virtualisation.

Figures & tables

Panel of Figure 1 — Figure 1: Context window before (left, 132k message tokens, 76% capacity) and after (right, 2.3k message tokens, 12% capacity) native autocompaction (e.g., Claude Code). Autocompaction summarises 98% of accumulated session state into a brief summary to reclaim window space.

Model	Base Input	Cache Write	Cache Read
Opus 4.6	$5.00	$6.25	$0.50

Table 1: API pricing per million tokens as of February 2026. Cache write cost is

1.25\times

base input. Break-even results scale linearly across pricing tiers.

Metric	Min	Median	Mean	Max
Token reduction (%)	0	12	20	86
Cache miss penalty ($)	$-$ 0.02	0.32	0.30	0.82
Break-even (turns, Opus 4.6)	1	38	35	60

Table 2: Trimming results across 76 sessions from a single API-key user (Opus 4.6,

h=0.9

). Column extrema are independent; the 0% minimum reduction yields the 60-turn cap, while the 1-turn minimum break-even corresponds to the session with maximum reduction. The negative minimum penalty arises when the trimmed session is small enough that its cold-cache cost is lower than the pre-trim steady-state cost.

Bloat Profile	Sessions	Mean Red.	Median Red.	Mean Break-even	Mean Context
Mixed ( $\geq$ 15%)	12	39%	33%	10 turns	97k
Conversational ( $<$ 15%)	64	17%	10%	40 turns	82k
All sessions	76	20%	12%	35 turns	84k

Table 3: Trimming results segmented by session bloat profile (tool result bytes as % of total JSONL bytes).

Figure 2: Distribution of token reduction across 76 sessions, segmented by bloat profile. The median reduction is 12%; the mean is pulled higher (20%) by a tail of sessions with significant trimmable overhead.

Figure 3: Break-even turns vs. token reduction. Sessions with $>$ 30% reduction reach break-even within 15 turns. Sessions with minimal overhead cluster at the 60-turn cap, correctly indicating trimming is unnecessary.

Figure 4: Cumulative input cost with and without trimming. The highlighted session (46% reduction) reaches break-even at turn 6. Faint lines show other sessions; sessions with greater reduction diverge earlier, while sessions with minimal reduction show negligible separation.

Figure 5: Context composition by session. Green represents conversation content (preserved by trimming); red, orange, purple, and blue represent trimmable overhead. Sessions with more overhead see larger reductions.

LLM Context Management

Large language model context management studies how to select, compress, retrieve, and organize information within bounded model inputs.

See all 248 papers

Persistent Context Graphs for Efficient Memory Compaction in LLM Agents
Sep 30, 2026Jingbo Yang, Kwei-Herng Lai, Xiaowen Wang +6Context Compaction for LLM Agents LLM Context Management
University of California, Santa Barbara · Microsoft · University of Notre Dame
Memori: A Persistent Memory Layer for Efficient, Context-Aware LLM Agents
Mar 20, 2026Luiz C. Borro, Luiz A. B. Macarini, Gordon Tindall +2Persistent Memory for Language Models LLM Memory
Memori Labs Inc.
KVMem: Virtualizing Million-Token Agent Workspaces on a Consumer GPU
Sep 4, 2026Di Chai, Leye Wang, Zeshen Su +2Memory Management for LLM Agents LLM Context Management
Shanghai University of Finance and Economics · Peking University · Northwestern Polytechnical University +1

Sep 30, 2026J/K move · Enter open · S save

Persistent Context Graphs for Efficient Memory Compaction in LLM Agents

Jingbo Yang, Kwei-Herng Lai, Xiaowen Wang, Zhaoxuan Tan+5

University of California, Santa Barbara · Microsoft · University of Notre Dame

Context Compaction for LLM Agents LLM Context Management Agent Memory Management Persistent Memory for Language Models Graph-Based Agent Memory

Open paper AskPDF arXiv

As LLM capabilities advance, agents are tackling increasingly complex tasks over longer horizons. Their growing interaction histories make memory compaction essential for staying within context windows and reducing prefill cost. Existing methods summarize the history or compress its KV cache, often adding model computation to preserve information for future requests. A new user request can change which history matters, but reassessing that history with the model requires re-encoding it if the KV cache has expired. Past attention provides signals of historical importance and dependencies between messages, while relevance to the current task must be assessed using the new user request. We introduce ReCAP, a memory compaction method that stores attention-derived importance scores and dependency links in a lightweight, persistent context graph. For each new request, ReCAP combines stored importance with relevance cues from the request and follows dependency links to select messages and their supporting context, without additional model calls for selection. Compared with Codex's default summarization-based compaction, ReCAP reduces estimated latency for compaction and cold restoration by approximately 95% on both Qwen3-Coder and gpt-oss. It also roughly halves the historical context per call on SWE-Together at comparable task quality and improves accuracy on the code tasks of Lost-in-Conversation over full history by 19.8 and 41.2 points.

Abstract

LLM Context Management

Explore similar work

Persistent Context Graphs for Efficient Memory Compaction in LLM Agents

Memori: A Persistent Memory Layer for Efficient, Context-Aware LLM Agents

KVMem: Virtualizing Million-Token Agent Workspaces on a Consumer GPU