As LLM capabilities advance, agents are tackling increasingly complex tasks over longer horizons. Their growing interaction histories make memory compaction essential for staying within context windows and reducing prefill cost. Existing methods summarize the history or compress its KV cache, often adding model computation to preserve information for future requests. A new user request can change which history matters, but reassessing that history with the model requires re-encoding it if the KV cache has expired. Past attention provides signals of historical importance and dependencies between messages, while relevance to the current task must be assessed using the new user request. We introduce ReCAP, a memory compaction method that stores attention-derived importance scores and dependency links in a lightweight, persistent context graph. For each new request, ReCAP combines stored importance with relevance cues from the request and follows dependency links to select messages and their supporting context, without additional model calls for selection. Compared with Codex's default summarization-based compaction, ReCAP reduces estimated latency for compaction and cold restoration by approximately 95% on both Qwen3-Coder and gpt-oss. It also roughly halves the historical context per call on SWE-Together at comparable task quality and improves accuracy on the code tasks of Lost-in-Conversation over full history by 19.8 and 41.2 points.
Figures & tables
Figure 1: ReCAP preserves attention-derived importance and dependency links in a persistent context graph. For each new request, it combines stored importance with the request’s relevance cues to select historical blocks and their supporting context, retaining protected contexts.
Figure 2: Attention signals for memory compaction. Left: Attention over history blocks per turn (width: tokens; dashed: user turns; blue: attention density, cube-root scale). Coral: blocks selected under a 10% token budget, concatenated below. Middle: Next-turn identifier coverage versus retained-history budget; Attn. + Query adds query-identifier overlap. Right: Attention redistribution (TV distance) over 32 fixed histories for paraphrases versus changed targets; ticks mark tasks.
Figure 3: ReCAP workflow. 1–2: Sampled attention supplies node importance and support-to-dependent edges. 3: Selection combines stored importance with the query’s identifier overlap, keeps protected records, and adds one-hop predecessors (diagnosis m5 brings in search record m2 ), without a model call. 4–5: Execution produces new messages and attention that update the graph.
Method
Qwen3-Coder-30B-A3B
gpt-oss-20b
Score ↑
Δ
Ctx. ↓
+Pre. ↓
+Dec. ↓
+Lat. ↓
+Mem. ↓
Score ↑
Δ
Ctx. ↓
+Pre. ↓
+Dec. ↓
+Lat. ↓
+Mem. ↓
SWE-Together
No compr.
Full
24.4 ± 1.87
↑ 4.1
28,504
0
0
0
0
8.2 ± 0.15
↑ 0.8
8,508
0
0
0
0
Heuristic
Recency
26.6 ± 0.40
↑ 1.9
10,154
0
0
0
0
8.4 ± 0.08
↑ 0.6
1,264
0
0
0
0
Random
23.8 ± 0.99
↑ 4.7
12,712
0
0
0
0
7.4 ± 0.35
↑ 1.6
1,011
0
0
0
0
Prompt compr.
LLMLingua-2
23.9 ± 1.17
↑ 4.6
11,062
0
0
7.39
2,236
7.8 ± 0.22
↑ 1.2
3,340
0
0
0.43
2,236
Table 1: Main results on SWE-Together (reward) and Lost-in-Conversation (LiC, accuracy), scaled by 100; ± : standard deviation across runs. Δ is ReCAP ’s score minus the row’s ( orange : ReCAP higher; teal : lower). Ctx. : mean tokens per call, excluding fixed system prompts. Compaction adds, per invocation, +Pre. prompt and +Dec. completion tokens on the backbone (means) and +Lat. seconds (one H100; ReCAP on CPU); +Mem. : memory it keeps in MB (GPU for compressors and KV cache, CPU for the context graph). Bold / underline : best/second-best scores; Appendix C gives the accounting.
Figure 4: Estimated cold-restoration latency on SWE-Together. Bars combine TTFT and compaction overhead; arrows show reductions with ReCAP . The two panels use separate scales.
Variant
Acc. ↑
Δ
Ctx. ↓
ReCAP
85.4 ± 1.95
—
94
w/o attention graph
76.6 ± 1.95
↑ 8.8
95
paragraph blocks
83.2 ± 1.92
↑ 2.2
152
line blocks
83.0 ± 2.45
↑ 2.4
168
Table 2: Component ablations on Lost-in-Conversation with Qwen3-Coder-30B-A3B ( β=0.1 ). Δ is ReCAP ’s accuracy minus the variant’s. Ctx. is the mean context size in tokens, excluding the fixed system prompt, as in Table 1 .
Figure 5: ReCAP with prefix caching (gpt-oss-20b, SWE-Together). (a) Idle boundaries prefill the compact context Sn ; warm ones reuse the cache; the graph updates in the background. (b) Prompt tokens per request, cached (hatched) and uncached (solid), with hit rates. (c) Engine TTFT at warm and idle boundaries. (d) Task reward versus warm probability; hollow point: all idle (Table 1 ).
LiC
SWE
Attention
gpt-oss
Qwen
Qwen
agent’s own
67.6 ± 3.58
85.4 ± 1.95
28.5 ± 0.40
Qwen3-0.6B proxy
68.2 ± 3.49
85.2 ± 2.17
29.0 ± 0.64
none
63.4 ± 2.61
76.6 ± 1.95
25.4 ± 0.22
Table 3: ReCAP with attention from the agent or a Qwen3-0.6B proxy ( β=0.1 ). Scores are scaled by 100; ± : standard deviation across runs.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Additional historical-context observations. Left: Aggregate attention concentration under a retained-history budget; the diagonal denotes uniform attention per unit of history. Middle: The full comparison of selection signals, including query identifiers and query embeddings. Right: User-instruction and latest-edit records omitted by attention-ranked selection, together with the fraction of transitions losing any protected record. The selector receives enough budget to fit the complete protected set.
Figure 7: Query effects across request templates. Paraphrases of the same target and requests about different targets are compared on the same 32 histories. Longer framed requests explicitly ask the agent to recall earlier facts and constraints. Both template families show greater attention redistribution under a target change. Curves share a smoothing bandwidth; bottom ticks identify the paired task-level observations, and the dashed line marks zero redistribution.
Figure 8: History retained by ReCAP on SWE-Together with Qwen3-Coder. (a) Retained blocks in each tenth of the session, relative to a uniform spread; counts include protected records. (b) Origin of the blocks retained at each user turn: kept from the previous working context, revived after being omitted from it, or created since.
Benchmark
Backbone
Blocks (median)
Median (ms)
P90 (ms)
Max (ms)
SWE-Together
Qwen3-Coder
132
20.2
45.2
112.6
SWE-Together
gpt-oss
18
1.4
4.4
20.3
LiC
Qwen3-Coder
6
0.30
0.80
1.69
LiC
gpt-oss
6
0.26
0.65
1.51
Appendix
Table 4: Per-request selection time of ReCAP on one CPU core.
Symbol
Meaning
Value
β
nominal history-character retention fraction
0.10
λ
weight of the lexical channel in Eq. 3
0.5
α
importance EMA rate (§ 4.2 )
0.6
ϵ
exploration bonus scale for unobserved blocks
0.1
κ
stale penalty (in standardized-score units)
2.0
δ
allowance for one-hop dependency expansion
0.1
Appendix
Table 5: ReCAP hyperparameters. Selection and update thresholds are shared across tasks and replicates; attention layers depend on the model architecture.