Long horizon agents accumulate growing interaction histories that increase context and inference costs. We find that geometric redundancy alone is an insufficient criterion for safe compression. Although agent histories exhibit strong low dimensional structure, similar global geometry can preserve very different amounts of task evidence. At identical retained block counts, evidence aware selection raises next action Top 3 retention from 0.31 to 0.69, while centroid similarity remains 0.98. Controlled replacement further shows that action related information can be substantially altered while global geometric measures remain nearly unchanged. Motivated by this gap between geometry and evidence, we introduce Geometry Guided Evidence Preserving Memory (GEM), a training free compressor that protects task and execution evidence before using geometric residuals to complete coverage. GEM reduces mean combined token usage from 2.69M to 2.11M per task, a 21.4% reduction, while maintaining comparable task reward. Our results show that efficient agent history compression should optimize for preserved task evidence rather than geometric coverage alone.
Figures & tables
Figure 1: Structured redundancy in agent histories. Real histories are compared with feature-shift and Gaussian controls. The two representation spaces show the same concentration pattern.
Metric
Geometry
Evidence
Δ
Action projection
0.91
0.94
↑0.03
Next-action Top-3
0.31
0.69
↑0.38
Final projection
0.90
0.94
↑0.04
Goal projection
0.91
0.94
↑0.03
Centroid similarity
0.98
0.98
−
Table 1: Matched-budget retention at 650 checkpoints.
Figure 2: Matched-budget geometry and task information. The trajectory-level visualization contrasts evidence-constrained and geometry-only selection at the same retained block count.
Figure 3: Next-action information across ordered spectral bands.
Metric
Δ
Task information
Action projection
−0.03
Next-action Top-3
−0.33
Global geometry
Centroid similarity
+0.00
Captured energy
+0.00
Table 2: Targeted block replacement effects.
Figure 4: Overview of GEM . (a) A frozen encoder represents complete interaction blocks. (b) Evidence constraints form the protected set Pt . (c) Residual geometric completion adds blocks whose directions are not covered by the current selected span. (d) Retained original blocks are reconstructed in chronological order and forwarded to the unchanged agent. The online forwarding constraints are described in Section 5.3 .
Method
Code
Office
Security
Web
Overall
S↑
T↓
S↑
T↓
S↑
T↓
S↑
T↓
S↑
T↓
Base agent
76.99
1.37
81.84
1.20
47.76
5.98
72.14
2.43
69.87
2.69
GEM
75.25
1.35
84.06
1.15
49.56
3.95
72.14
2.09
70.18
2.11
Table 3: WorkBuddyBench Full260 results. S denotes mean reward ×100 , and T denotes combined tokens per task (M). Higher S and lower T are better.
Method
Code
Office
Security
Web
Overall
S↑
T↓
S↑
T↓
S↑
T↓
S↑
T↓
S↑
T↓
Base agent
72.93
1.44
81.64
1.04
44.58
3.97
71.00
1.10
67.54
1.89
Window ( K=5 )
66.81
3.12
67.43
3.98
32.32
2.13
59.00
2.58
56.39
2.95
Window ( K=10 )
74.37
1.76
73.05
1.82
40.07
0.82
61.00
3.55
62.12
1.99
Window ( K=20 )
79.31
1.42
85.11
2.23
40.28
1.13
70.00
3.30
68.67
2.02
Periodic summary ( n=3 )
65.44
1.91
67.66
2.14
26.32
0.99
82.66
3.82
60.52
2.21
Table 4: Fixed-40 context-management reference. S is mean reward ×100 and T is combined tokens per task (M). Each domain contains ten tasks.
Variant
Code
Office
Security
Web
Overall
S↑
T↓
S↑
T↓
S↑
T↓
S↑
T↓
S↑
T↓
Base agent
59.79
2.33
74.28
3.02
62.95
10.70
46.00
4.73
60.75
5.19
GEM
70.79
2.52
69.32
3.18
53.45
4.33
48.00
3.41
60.39
3.36
Random completion
47.40
2.72
60.86
5.20
56.69
6.69
47.30
3.23
53.06
4.46
Geometry only
60.93
2.40
59.62
1.25
48.92
8.14
50.64
4.71
55.03
4.12
Table 5: Online 20-task ablation by domain. S is mean reward ×100 and T is combined tokens per task (M). Each domain contains five tasks.
Figure 5: Compression strength. (a) Next-action Top-3 retention at exact retained budgets K . (b) Top-3 retention and retained-history ratio as the coverage threshold τ increases. The exact- K sweep uses 28 trajectories and 525 common checkpoints; the coverage sweep uses 32 trajectories and 59 checkpoints.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
Value
Minimum complete blocks
16
Coverage target τ / nominal core capacity K
0.90 / 16
Recent / Goal / State / Error limits
4 / 1 / 3 / 2
Error eligibility horizon
8 blocks
Source protection activation
Source-like fraction ≥0.50
Initial-block protection
0
Appendix
Table B.1 : Shared implementation settings across Code, Office, Security, and Web.
Representation
Metric
Real
Feature-shift
Gaussian
Isolated blocks
Effective rank
8.39
16.99
19.96
Isolated blocks
r90/n
0.47
0.78
0.88
Selector space
Effective rank
14.04
23.54
30.07
Selector space
r90/n
0.48
0.78
0.85
Appendix
Table C.1 : Structured low-dimensionality of agent histories. Values are trajectory medians for centered representations; r90/n is the fraction of available blocks required to explain 90% of centered energy.
Selector
Action
Final
Goal
Top-3
Centroid
Recent- K
0.96
0.88
0.82
0.76
0.97
Random- K
0.92
0.92
0.95
0.45
0.99
Cosine Top- K
0.86
0.91
0.97
0.53
0.96
Novelty Top- K
0.93
0.93
0.96
0.33
0.99
GEM
0.95
0.95
0.96
0.78
0.99
Appendix
Table D.1 : Same-retained-count offline selector comparison on 32 trajectories and 59 checkpoints.
Variant
Retention
Action
Top-3
Centroid
Full constraints
0.46
0.95
0.62
0.99
Geometry only
0.39
0.91
0.25
0.98
Recent + Goal only
0.45
0.95
0.57
0.98
Without State
0.45
0.95
0.53
0.98
Without Error
0.45
0.95
0.56
0.98
All candidates
0.44
0.95
0.59
0.99
Appendix
Table D.2 : Offline component ablation at each variant’s natural retained fraction (32 trajectories, 59 checkpoints).
K
Geometry only
Evidence constrained
Recent- K
Recent+Goal
8
0.18
0.56
0.50
0.51
9
0.21
0.56
0.53
0.52
10
0.24
0.57
0.55
0.53
11
0.27
0.59
0.54
0.54
12
0.31
0.60
0.56
0.56
13
0.33
0.62
0.58
0.57
Appendix
Table D.3 : Exact retained-count sweep. Values are next-action Top-3 retention on 28 trajectories and 525 checkpoints common to all K .
τ
Retained fraction
Top-3
Action projection
0.85
0.38
0.58
0.94
0.86
0.39
0.58
0.94
0.87
0.41
0.58
0.95
0.88
0.43
0.61
0.95
0.89
0.44
0.62
0.95
0.90
0.48
0.64
0.96
Appendix
Table D.4 : Coverage-threshold sweep on 32 trajectories and 59 checkpoints.
Token category
Base agent
GEM
Uncached input
48.70K
201.48K
Cached input
2.60M
1.82M
Normal output
12.40K
15.68K
Reasoning output
25.38K
22.81K
Model total
2.69M
2.06M
Embedding
0
46.97K
Appendix
Table E.1 : Full260 token composition, mean per task. K and M denote thousands and millions of tokens. Values use the same reporting precision as the main text.
Long-horizon agents continuously accumulate interaction history during task execution, yet the importance of past interactions changes as the agent state evolves. Existing context management methods largely compress history based on fixed windows, periodic schedules, or current relevance, overlooking a more fundamental question: when has a past interaction become safe to replace? Premature compression may remove information still needed for future actions, while overly conservative retention leads to substantial context overhead. To address this, we propose State Conditioned Compression (StateComp), a framework that determines when historical interactions can be safely compressed according to the current agent state. StateComp constructs KEEP and READY supervision through a two-stage annotation procedure and trains an imbalance-aware router on hidden representations from a frozen language model. A bounded state representation further reduces the cost of evaluating long histories, while adjacent READY interactions are grouped into continuous spans and replaced with compact summaries during execution. Experiments on WorkBuddyBench show that StateComp reduces total agent and summarization tokens by 52.27% while maintaining task performance, and achieves a 12.67-fold speedup in representation extraction.
Mingxuan Wang, Hongyue Chen, Yinglong Guo +6
TierFlow Team · Gaoling School of Artificial Intelligence, Renmin University of China · Tsinghua University
Long horizon language model agents continuously accumulate interaction history, increasing computational cost while making relevant information harder to preserve and reuse. Existing context management methods mainly focus on how to compress or retrieve history, but largely leave open whether the model itself already represents the need for these memory operations before they occur. We study the hidden state immediately before each agent action and find that compression and recall needs are already encoded in the model's internal representations. These signals cannot be explained by simple context length or interaction progress, and they exhibit distinct formation patterns across model depth. We further show that most memory decision information is preserved in a compact recent context, while selectively restored historical evidence complements the long range dependencies that recent context misses. Based on these findings, we propose Preaction Memory with Evidence Retrieval (PaMER), which combines state guided compression with external evidence retrieval. PaMER+ further introduces step level evidence selection to recover only the historical information required by the current task. Experiments on WorkBuddyBench, across multiple context management baselines and model backbones, show that our framework substantially reduces context consumption while maintaining competitive task performance.
Mingxuan Wang, Guorun Yao, Fei Luo +6
TierFlow Team · Gaoling School of Artificial Intelligence, Renmin University of China · Tsinghua University
Long horizon language model agents continually accumulate reasoning history, increasing context length and inference cost even after earlier decisions have been executed and observed. Unlike static Chain of Thought compression, removing historical reasoning can change future actions and the resulting interaction trajectory. We study when such reasoning can be safely forgotten. We propose Interaction Aware Compression for Long Horizon Reasoning (ICLR), a training free online method that ranks reasoning blocks using frozen proxy entropy while preserving actions, tool calls, and observations. On 260 WorkBuddyBench tasks, ICLR improves average reward from 0.699 to 0.718, while reducing input, output, and cache read tokens by 25.5%, 14.4%, and 33.3%, respectively. Ablations reveal trajectory amplification, where local reasoning deletion produces nonlinear changes in total computation by altering subsequent interaction. Representation probing, activation patching, and controlled trajectory analyses further suggest that historical reasoning becomes more replaceable once task relevant derived state has been reliably externalized into code, files, tool outputs, or environmental feedback. These results characterize agent reasoning as dynamic working state rather than permanent interaction history.
Mingxuan Wang, Fei Luo, Bo Wang +6
TierFlow Team · Gaoling School of Artificial Intelligence, Renmin University of China · Tsinghua University