Bud ContextCompact 2.0: Context Compaction for Agent Success and Accuracy with Small Language Models and Constrained Context Windows

Jithin VG; Ditto PS

cs.AIOct 18, 2026

Bud ContextCompact 2.0: Context Compaction for Agent Success and Accuracy with Small Language Models and Constrained Context Windows

Authors: Jithin VG, Ditto PS

Organizations: Bud Ecosystem

Abstract

Agents that work for many turns outgrow the context window of the model that serves them, and on small language models (SLMs) and constrained inference systems that window is small. Context compaction replaces older turns with a summary so a session can continue, but it is lossy, costs a model call, and can help or hurt depending on when it runs. We describe Bud ContextCompact 2.0, the compaction built into the Bud agent runtime, and evaluate it in two settings. On recall questions over recorded agent sessions, compaction to a 16K budget raised a 1.2B-parameter model's accuracy to 0.335, from 0.247 with truncation and 0.176 with its full 32K history (p < 0.001), and raised a 9B model from 0.515 (truncation) to 0.577 (p = 0.04). On 100 SWE-bench Verified tasks run end to end by a 27B reasoning model on a 49K-token window (556 graded sessions), compaction raised the resolve rate on long sessions from 41% to 53% (+12.3 points, 95% CI +1.6 to +23.0, McNemar p = 0.043), left short sessions unchanged, and used 13% fewer tokens per solved task at equal total cost. The main failure without compaction was not the history overflowing but a reasoning turn exhausting the room the history left. The gain has limits: it vanished under a 45-minute session budget, because each summary took a median of 161 s, and in a threshold study it appeared only when compaction triggered early enough to leave room. An 8B model submitted after about ten turns at 5-8K tokens and never needed compaction. We argue that the accuracy results transfer across hardware, including CPU inference, for an equivalent model and window, while the time cost scales with the hardware and must be budgeted for.

Figures & tables

Figure 1: Recall-probe accuracy (ContextCompact 1.x). DeepSeek: 358 probes, mean of three runs. Small models: 239 probes, one run each, served locally on an NVIDIA GB10. The full-history arm uses a larger window than the compacted arm has: 1M for DeepSeek, 64K for Qwen 3.5 9B, 32K for LFM2.5 1.2B.

Model	Budget	Full	Bound	Compacted	vs full	vs bound
DeepSeek V4 Flash	32K	0.592	0.559	0.581	n.s. ( $p\geq 0.27$ )	$+$ , n.s. ( $p\geq 0.13$ )
Qwen 3.5 9B	32K (full: 64K)	0.544	0.515	0.577	$+8$ , $p=0.32$	$+15$ , $p=0.04$
LFM2.5 1.2B	16K (full: 32K)	0.176	0.247	0.335	$+38$ , $p<0.001$	$+21$ , $p=0.006$

Table 1: Recall-probe accuracy and net answers gained (paired McNemar). DeepSeek: per-run

p

over three runs; small models: one run each. Compacted prompts were 0.52 (DeepSeek), 0.34 (9B) and 0.40 (1.2B) of the full history’s tokens.

Figure 2: Resolve rate by arm with Wilson 95% intervals. Long pairs: either arm’s prompt crossed the compaction threshold. Without a session budget, compaction raised the resolve rate on long sessions and left short ones unchanged; under a 45-minute budget the gain vanished.

No budget	Pairs	Off	Auto	Only off	Only auto	Auto $-$ off (95% CI), $p$
Long, first 60 tasks	60	27	35	3	11	$+13.5$ ( $+2.7$ , $+25.7$ ), 0.057
Long, 40 added tasks	42	15	19	6	10	$+10.4$ ( $-10.4$ , $+31.2$ ), 0.45
Long, all 100 tasks	102	42	54	9	21	$\mathbf{+12.3}$ ( $+1.6$ , $+23.0$ ), 0.043
Short, all 100 tasks	98	44	45	4	5	$+1.7$ ( $-5.1$ , $+8.5$ ), 1.0
All pairs, 100 tasks	200	86	99	13	26	$+6.5$ ( $+0.5$ , $+13.0$ ), 0.053

Table 2: Paired resolution without a session budget. Differences in percentage points; intervals from a task-level bootstrap;

p

from the exact McNemar test.

Figure 5: Prompt size per turn for one task and repeat. Each drop is a compaction taking effect. Background compaction applies from the turn after it finishes, so the prompt can run past the threshold before it drops.

Figure 6: Resolve rate by compaction threshold on a 56,320-token window (25 tasks, 2 repeats, 50 sessions per point, Wilson intervals).

Qwen3.8-27B, 56K window	Off	24K	32K	48K
Resolved (of 50)	18	21	19	16
Sessions that compacted	–	20	16	4
Reasoning exhausted the window	29	26	24	31
Long pairs: only off / only threshold		2 / 7	3 / 3	6 / 6
Long pairs: threshold $-$ off (pp)		$+13.9$	$0.0$	$+2.4$
Long pairs: McNemar $p$		0.18	1.0	1.0

Table 3: Threshold study. Long pairs: either arm’s prompt crossed 24K (33–36 pairs per threshold).

No budget, 100 tasks, 200 sessions per arm	Off	Auto
Prompt tokens	98.9M	95.0M
Output tokens	9.54M	11.30M
Summariser tokens (in / out)	–	1.51M / 0.94M
All tokens, summariser included	108.4M	108.7M
All tokens per resolved session	1.26M	1.10M
Prompt tokens a prefix cache could reuse	98%	96%

Table 4: Cost. The prefix-cache row is modelled from the traces: a turn reuses the previous turn’s prompt and output unless a compaction rewrote the history. Reasoning tokens are included in output tokens.

LLM Context Management

Large language model context management studies how to select, compress, retrieve, and organize information within bounded model inputs.

See all 246 papers

Benchmarks Are Surprisingly Compaction-Friendly
Oct 22, 2026Jeanne Salle, Hardik Bhatnagar, Matthias Bethge +2Context Compaction for LLM Agents LLM Context Compression
ELLIS Institute Tübingen · Max Planck Institute for Intelligent Systems · Tübingen AI Center +1
Context Compaction Theory
Aug 2, 2026Hayder Tirmazi, Sam Markelon, Allison Bishop +1LLM Context Management Context Compression
AllSpice Inc. · Boston University · Proof Trading +2
Compact-Bench: Measuring What Context Compaction Costs Long-Horizon Coding Agents
Oct 22, 2026Ruopei Sun, Jianfeng Cai, Kangwen Zhao +2Context Compaction for LLM Agents LLM Context Compression
University of Science and Technology of China

Oct 22, 2026J/K move · Enter open · S save

Benchmarks Are Surprisingly Compaction-Friendly

Jeanne Salle, Hardik Bhatnagar, Matthias Bethge, Sahar Abdelnabi+1

ELLIS Institute Tübingen · Max Planck Institute for Intelligent Systems · Tübingen AI Center+1

Context Compaction for LLM Agents LLM Context Compression LLM Agent Memory LLM Agent Evaluation Long-Horizon Agent Tasks Agent Benchmarks

Open paper AskPDF arXiv

Long horizon agentic tasks are now pushing LLM agents past their context window. To continue the task, the agent scaffold has to free up the context window, a process known as context compaction. Compaction is generally lossy by design: agents can silently lose task constraints, instructions, or their reasoning behind earlier decisions. Although compaction is a standard component of coding agent scaffolds, it has not been studied systematically with frontier models on realistic, long horizon agentic tasks. Previous work evaluate smaller open weight models on toy or synthetic trajectories. We fix the agent scaffold (OpenCode), implement five compaction strategies inside it, and evaluate three frontier models on three long horizon benchmarks (PaperBench, ProgramBench, PostTrainBench). For the two 1M context models, the compaction threshold does not measurably affect the task performance, whereas GPT-5.3-Codex, the model with the smallest context window, performs worse at lower thresholds on PaperBench and, less clearly, on PostTrainBench. No compaction method outperforms default self-summarization, and methods that discard more content cost some models up to 20 points while leaving other models the same. Compaction can, however, silently remove task constraints: on ProgramBench, the first compaction summary already drops about one third of the task's anticheating rules on average, and this rises monotonically with the number of compactions. On real user-agent sessions from SWE-chat, instructions about how the agent should work and what it should not do are dropped around 6 times more than specifications of what to build. We argue that current benchmarks are compaction-friendly: i.e., the objective is fixed and the agent's progress stays on disk, so most of what compaction discards can be recovered, except constraints that don't exist anywhere else.

Abstract

LLM Context Management

Explore similar work

Benchmarks Are Surprisingly Compaction-Friendly

Context Compaction Theory

Compact-Bench: Measuring What Context Compaction Costs Long-Horizon Coding Agents