The Compaction Cliff in Long-Running AI Agent Memory

Saber Zerhoudi; Jelena Mitrovic; Michael Granitzer

cs.AIAug 24, 2026

The Compaction Cliff in Long-Running AI Agent Memory

Authors: Saber Zerhoudi, Jelena Mitrovic, Michael Granitzer

Organizations: University of Passau Passau, Germany · IT:U Linz, Austria

Abstract

A safety rule and an episodic log compete for the same tokens in an AI agent's context. When the budget overflows, both are summarized at the same rate; only the rule needs exact wording to remain enforceable. On 20 production agent configurations, Claude Code's /compact prompt on Sonnet 4.6 preserves 53% of safety rules after one compaction round and 10% after five. We name this the Compaction Cliff. We address it with Knowledge Triage, a framework that classifies each line of an agent's knowledge base by type and routes each type through its own retention policy. Three deterministic operators implement this triage across the three context-management operations: TypeCompact rewrites items in place under per-type fidelity, TypeDecompose partitions a topic too large to compact safely, replicating in-scope safety rules across partitions, and TypeRetrieve fetches items from external storage with in-scope rules pinned ahead of relevance. On five public corpora, TypeCompact preserves 2--4 $\times$ more safety rules than the strongest single-shot LLM compactor at every ratio, with 96% recall over five rounds. TypeDecompose reaches 0% locality violations against 93% under uniform partitioning. TypeRetrieve reaches 100% recall@50 against 73% for the best single-shot LLM retriever. On three downstream behavioral benchmarks, we outperform the production Sonnet compactor on medical compliance (paired McNemar $p < 10^{-8}$ on preservation, $N = 200$ ), the full-policy and hierarchical baselines on retail task pass rate ( $p < 0.01$ , $N = 115$ ), and the hierarchical compaction on the airline domain ( $p = 0.024$ ). We release AgentArtifactCorpus (396{,}934 agent configurations from 54{,}628 public GitHub repositories), the classifier, and the reference implementation.

Figures & tables

Figure 1. Knowledge Triage. Classifier $\tau$ assigns each working-set item one of five types; three deterministic operators apply per-type policies: TypeCompact compresses without dropping safety rules, TypeDecompose splits large topics and duplicates rules that span them, TypeRetrieve returns applicable rules first. A verifier checks that every constraint survived; if any are missing, the output is flagged unsafe , otherwise it becomes the next working set.

System	Typed	Op C	Op D	Op R	Safety
	model				guarantee
MemGPT ( 24 )	–	✓	–	✓	–
A-MEM ( 36 )	–	–	–	✓	–
GraphRAG ( 9 )	–	–	–	✓	–
LLMLingua-2 ( 25 )	–	✓	–	–	–
MaRS ( 2 )	✓	✓	–	–	partial

Table 1. Positioning of Knowledge Triage.

Type	Definition	Distortion tolerance	Examples
Constraint (C)	Rule that bounds agent behavior; violation causes safety failure	Zero; any loss is unsafe	“Never prescribe contraindicated drugs”
Procedural (P)	Step-by-step instruction for a task	Semantic equivalence iff execution preserved	“Run pytest -x --tb=short before merging”
Belief (B)	Factual assertion treated as true	Bounded by semantic distance	“Backend exposes a REST API over HTTPS”
Preference (F)	Soft guideline	High; may be summarized or merged	“Use 2-space indentation”
Episodic (E)	Past event or observation	Free; may be discarded	“Refactored auth module yesterday”

Table 2. The five operational knowledge types of Knowledge Triage, with their distortion tolerance under context compression and illustrative examples drawn from several domains.

Corpus	Scale	Role in this paper
AAC (ours)	396,934 items	Taxonomy, compaction, scope analysis (§§ 4.2 – 4.3 )
LongMemEval ( 32 )	500 turns	Cross-corpus taxonomy validation (§ 4.2 )
$\tau$ -bench retail ( 37 )	115 tasks	Retail-task behavioral rollouts (§ 4.5 )
$\tau$ -bench airline ( 37 )	50 tasks	Customer-service generalization test (§ 4.5 )
BEIR scifact ( 27 )	1,000 docs	Retrieval baseline corpus (§ 4.3 )
SafetyMed (ours)	200 scenarios	Medical-compliance behavioral evaluation (§ 4.5 )

Table 3. Corpora used in the empirical validation.

Type	% of items	Precision	Recall	F1
Constraint	12.3	0.91	0.87	0.89
Procedural	28.7	0.94	0.92	0.93
Belief	31.4	0.86	0.84	0.85
Preference	14.2	0.83	0.80	0.81
Episodic	13.4	0.90	0.88	0.89

Table 4. Knowledge type distribution and classifier performance on AgentArtifactCorpus (

n=2{,}000

annotations).

Strategy	C	P	B	F	E	All
Hierarchical truncation	0.50	0.82	0.75	0.63	0.77	0.69
Temporal windowing	0.58	0.80	0.64	0.78	0.91	0.74
Aggressive pruning	0.51	0.75	0.67	0.66	0.84	0.69
TypeCompact	1.00	1.00	0.50	0.51	0.84	0.77

Table 5. Preservation rate by knowledge type at 50% compaction across 50 AAC configurations.

Figure 2. Constraint preservation vs. target compression ratio on 20 AAC configurations. Structural baselines are tabulated at 50% in Table 5 ; full numerical detail in Table 6 .

Strategy	C	P	mF1	time	tok
50% compression
TypeCompact	1.00	0.92	0.75	0.0s	0
LLMLingua-2	0.21	0.72	0.63	0.5s	0
MaRS-FL (ours, no public impl)	0.99	0.89	0.69	0.5s	0
gpt-5.4-nano (single-shot)	0.52	0.80	0.69	9.9s	56K
gpt-5.4-mini (single-shot)	0.48	0.77	0.68	7.3s	57K

Table 6. TypeCompact vs SOTA prompt compactors at three compression ratios, mean across 20 AAC configurations.

Figure 3. Constraint recall after

N

sequential compaction rounds at 50% per round, 20 AAC configurations.

Strategy	Mean viol.	Configs viol.	Overhead
chunk_by_tokens	32%	93%	0%
chunk_by_topic	13%	40%	0%
chunk_by_paragraph	32%	93%	0%
TypeDecompose	0%	0%	+14.5%

Table 7. Decomposition constraint-locality on 200 AAC configurations, 25%-of-total partition budget.

Retriever	Variant	$@5$	$@10$	$@20$	$@50$	sec/q	tok/q
Non-LLM retrievers + TypeRetrieve overlay
BM25	baseline	32%	38%	43%	49%	0.0	0
Dense (octen)	baseline	34%	45%	67%	91%	0.0	0
Reranked dense	baseline	25%	36%	55%	91%	0.0	0
TypeRetrieve	dense	40%	57%	96%	100%	0.0	0
Single-shot LLM retrievers (full pipeline; given top-100 candidate pool)

Table 8. In-scope constraint recall@

k

on a 1,033-item mixed corpus, 50 queries.

Variant	C-Rec.	C-F1	mF1	Acc	Lat.
Regex only	0.60	0.63	0.35	0.53	$<$ 1 ms
Encoder only	0.27	0.32	0.33	0.41	387 ms
Regex + Encoder	0.70	0.63	0.40	0.46	266 ms
Distilled MiniLM+LR ( 25 )	0.77	0.61	0.44	0.52	$\sim$ 5 ms
gpt-5.4-nano	0.93	0.74	0.62	0.69	684 ms
gpt-5.4-mini	0.87	0.84	0.68	0.75	667 ms

Table 9. Classifier variants on the 200-item AAC test set, reference labels from gpt-5.4-mini.

Model	Full	Hier.	TypeC.	$p$ vs full	$p$ vs hier.
gpt-5.4-nano	22.5%	21.6%	36.0%	0.011	0.007
gpt-5.4-mini	27.3%	29.1%	35.5%	0.176	0.324
qwen3-next-80b	36.1%	37.0%	41.7%	0.345	0.424
Paired overall	28.6%	29.2%	37.7%	0.003	0.005

Table 10.

\tau

-bench retail pass rate,

N=115

stratified tasks per cell, 1,035 paired rollouts (1,019 completed without error).

LLM Agent Memory

Large language model agent memory lets agents retain, organize, retrieve, and update information across interactions and long-running tasks.

See all 357 papers

AI Guardrail Survival under Single-Cycle Agentic Self-Summarization
Aug 11, 2026Ted Kwartler, Alan Aqrawi, Arian AbbasiLLM Guardrails LLM Safety Evaluation
Harvard Extension School, Harvard University; Americas Advanced AI Practice Lead, Accenture. · Accenture.
CliffCompaction: Cost-Efficient Compaction for Long-Horizon Coding Agents
Sep 22, 2026Trang Nguyen, Eulrang Cho, Bingqing Chen +1Continual Learning for LLM Agents Test-Time Scaling
Carnegie Mellon University · Bosch Center for AI
Governance Decay: How Context Compaction Silently Erases Safety Constraints in Long-Horizon LLM Agents
Jun 21, 2026Shiyang ChenAdversarial Attacks on LLMs AI Agent Safety Benchmarks
Beijing Institute of Technology

Aug 11, 2026J/K move · Enter open · S save

AI Guardrail Survival under Single-Cycle Agentic Self-Summarization

Ted Kwartler, Alan Aqrawi, Arian Abbasi

Harvard Extension School, Harvard University; Americas Advanced AI Practice Lead, Accenture. · Accenture.

LLM Guardrails LLM Safety Evaluation LLM Agent Safety LLM Context Management Language Model Auditing

Open paper AskPDF arXiv

Long-running agents periodically compact their context, replacing the transcript with a model-generated summary. Recent work shows that dropping a standing safety constraint during compaction drives behavioral violations across many models (Governance Decay; Chen, 2026). We ask a finer question: under a single compaction cycle, how is a safety rule lost, and what does that imply for detection and evaluation? Our central finding is that a presence check is not a safety check: when compaction does not drop a rule outright, it often leaves something that looks like a rule but does not act like one. On behavioral replay, a degraded residue leads the model to perform the prohibited action far more often than an intact welded rule does (all-case gaps of +34 and +57 points under two replay models, both positive), category-level survival behaves like a residue, and even intact rules sometimes fail to fire, so an audit that checks only textual presence gives false assurance. Sharpening this, rule-form items are retained substantially more often than prominence-matched facts, which is exactly why presence-based checking feels adequate even though survival is not protection. Textual loss is regime-dependent (weld-or-drop with a single rule; degraded predicate-loss residues under a tighter budget), and we did not observe the hypothesized textual severing mode. Such loss is silent at runtime and detectable only by comparison with retained external ground truth (such as a constraint registry), which reveals textual absence but not whether a surviving rule still fires. We also document evaluation pitfalls where LLM-judge labels alone would have reversed a conclusion. All results concern a single compaction cycle.

Abstract

LLM Agent Memory

Explore similar work

AI Guardrail Survival under Single-Cycle Agentic Self-Summarization

CliffCompaction: Cost-Efficient Compaction for Long-Horizon Coding Agents

Governance Decay: How Context Compaction Silently Erases Safety Constraints in Long-Horizon LLM Agents