LLM agents can improve across tasks without updating model parameters by storing experience from their own executions in external memory and reusing it to guide later decisions. A central design question is whether to retain only observed trajectories and outcomes or also the rules, summaries, and skills inferred from them. Retaining these abstractions allows agents to reuse strategies without repeatedly deriving them from raw trajectories. However, errors in stored abstractions can propagate to subsequent decisions through retrieval and reuse. They can also shape execution trajectories, which then become the basis for new abstractions that reproduce the original errors. We term this feedback mechanism error reproduction: errors are regenerated through a cycle of abstraction, execution, and re-abstraction, extending their persistence beyond direct reuse of the original faulty knowledge. We propose Anti-Persistence, a memory design that accumulates trajectories and outcomes while reconstructing task-specific guidance on demand. An adaptive selector uses prior outcomes to decide whether and how to synthesize guidance, which is discarded after execution. This prevents its direct reuse as a stored object, although indirect feedback through retained trajectories remains possible. Across different benchmarks and backbones, Anti-Persistence achieves higher overall held-out success and the lowest average LLM-call cost among the evaluated memory designs. These results support separating experience accumulation from abstraction persistence in memorybased self-improvement.
Figures & tables
Figure 1: Error propagation and reproduction in agent memory. Left: An illustrative error in a stored trajectory is reused through retrieval. Center: A faulty inferred rule may shape later trajectories and be encoded again through subsequent synthesis. Right: Possible routes of influence. Both designs can retain feedback through stored trajectories; the examples imply neither a fixed ordering of their reach nor inevitable amplification.
Llama-4-Scout
Qwen3-Coder-Next
Gemma-4-31B-it
Method
ALF
Web
Sci
ALF
Web
Sci
ALF
Web
Sci
ReAct
29.3 ± 3.1
33.3 ± 1.2
9.4 ± 1.0
70.4 ± 1.8
36.0 ± 2.0
27.2 ± 2.6
72.7 ± 1.2
43.3 ± 2.3
44.2 ± 1.2
Reflexion
46.7 ± 6.1
45.3 ± 4.2
22.8 ± 1.0
87.3 ± 3.1
47.3 ± 1.2
33.9 ± 3.5
98.0 ± 0.8
50.0 ± 2.0
44.5 ± 2.6
ExpeL
7.2 ± 3.2
38.0 ± 5.3
5.0 ± 1.9
88.0 ± 2.0
47.3 ± 2.3
32.8 ± 4.8
96.7 ± 2.3
38.6 ± 1.9
51.3 ± 1.2
AutoRefine
36.3 ± 3.2
37.3 ± 2.3
15.3 ± 1.7
83.7 ± 6.7
40.7 ± 2.3
27.8 ± 1.0
88.0 ± 4.0
48.0 ± 4.0
43.3 ± 0.0
Traj. Mem.
41.6 ± 1.7
45.3 ± 1.2
20.0 ± 1.7
84.4 ± 2.6
50.7 ± 1.2
43.8 ± 3.7
96.0 ± 1.0
48.0 ± 0.2
51.7 ± 1.7
Table 1: Held-out task success rate (%, mean ± std over 3 seeds) on ALFWorld, WebShop, and ScienceWorld with three backbones. Anti-persistence is compared with agent baselines and persistent-memory variants.
Figure 2: Per-iteration performance of four memory configurations over ten memory-update iterations across three benchmarks and two backbones.
Figure 3: Mean LLM-call cost and held-out success across nine settings.
Figure 5
Model
Bench
Discard (Ours)
Persist
Δ
Llama-4-Scout
ALF
54.7 ± 2.9
26.7 ± 6.1
+28.0
Llama-4-Scout
Web
46.3 ± 1.8
34.0 ± 4.0
+12.3
Qwen3-Coder-Next
ALF
96.0 ± 1.4
92.7 ± 1.2
0 +3.3
Qwen3-Coder-Next
Web
54.0 ± 2.0
47.3 ± 3.1
0 +6.7
Table 2: Held-out success with abstractions persisted or discarded (%, mean ± s.d.; 3 seeds).
each mode forced at least Winit times before KNN-UCB
Appendix
Table 3: Method hyperparameters and default values. Sensitivity analyses vary the specified parameters.
Benchmark
Training tasks
Held-out tasks
Evaluation set
ALFWorld
180
134
134 unseen tasks [ 18 , 35 ]
WebShop
200
100
100 held-out instructions
ScienceWorld
150
60
2 held-out instances per task type
Appendix
Table 4: Per-benchmark training and held-out task counts used in this study.
Model
Inference hardware
Context length
Llama-4-Scout [ 10 ]
2× NVIDIA H100
64K
Qwen3-Coder-Next [ 2 ]
4× NVIDIA H100
256K
Gemma-4-31B-it [ 8 ]
4× NVIDIA L40S
256K
Appendix
Table 5: Backbone configurations.
Figure 9: Per-iteration performance on Gemma-4-31B-it across the three benchmarks.
Llama-4-Scout
Qwen3-Coder-Next
Gemma-4-31B-it
Method
ALF
Web
Sci
ALF
Web
Sci
ALF
Web
Sci
Traj. Mem.
17.87 ± 0.40
4.21 ± 0.09
5.25 ± 0.10
10.90 ± 0.15
6.12 ± 0.03
11.41 ± 0.32
11.87 ± 0.19
4.85 ± 0.06
13.00 ± 0.05
Abs. Mem.
16.74 ± 4.77
4.53 ± 0.27
1.33 ± 1.15
13.64 ± 0.03
6.91 ± 0.10
15.08 ± 1.05
13.46 ± 1.04
6.04 ± 0.17
13.80 ± 0.59
Joint Mem.
19.11 ± 0.48
3.23 ± 1.57
1.15 ± 0.86
10.95 ± 0.33
7.21 ± 0.10
13.93 ± 0.95
11.01 ± 0.50
5.96 ± 0.17
12.78 ± 0.71
Anti-pers.
15.13 ± 0.19
3.99 ± 0.05
3.56 ± 0.22
11.84 ± 0.28
6.64 ± 0.55
12.68 ± 0.40
11.20 ± 0.38
5.30 ± 0.19
13.41 ± 0.19
Appendix
Table 6: Average LLM-call cost per task after the final memory-update iteration (mean ± standard deviation over 3 seeds). Lower is better.
Figure 13
Baseline
Mean Δ
95% CI
Paired t -test
Wilcoxon signed-rank
Positive
(pp)
Raw p
Holm p
Raw p
Holm p
cells
ReAct
+19.2
[14.1,24.4]
<0.0001
0.0002
0.0039
0.0273
9/9
Reflexion
+7.0
[2.7,11.4]
0.0060
0.0179
0.0078
0.0273
8/9
ExpeL
+14.9
[3.6,26.1]
0.0157
0.0179
0.0039
0.0273
9/9
AutoRefine
+13.2
[8.7,17.6]
0.0001
0.0007
0.0039
0.0273
9/9
Trajectory memory
+6.4
[2.8,10.0]
0.0036
0.0145
0.0039
0.0273
9/9
Appendix
Table 7: Exploratory comparisons across the nine fixed settings in Table 1 . Δ denotes anti-persistence minus the named baseline, measured in percentage points. CIs are nominal, unadjusted 95% t -intervals, and all tests are two-sided. Holm correction covers the seven baseline comparisons within each test family. Shared backbones and benchmarks limit the inferential interpretation of these statistics.
Figure 12: Sensitivity to trajectory retrieval budget B and outcome-log retrieval budget B^ on Qwen3-Coder-Next. Bars show means over 3 seeds; error bars show standard deviations across seeds.
Large Language Models (LLMs) show promise as tool-using agents but remain limited in long-horizon tasks that require remembering, organizing, and reusing knowledge. Prior memory approaches aim to resolve the situation, but mainly focus on storing factual information. Recent work on procedural memory improves task reuse, yet often reduces to replaying past successes without addressing failure cases or online scalability. We introduce a unified and automatic memory framework that integrates semantic, episodic, and procedural memory in a bi-level design combining short-term and long-term stores. A multi-agent architecture with actor, memory, and critic agents enables automatic memory generation, reward annotation, and adaptive retrieval. Long-term memory is managed through reward-based evaluation, merging, and pruning, ensuring scalability and continual improvement. Experiments across various environments show that our approach improves robustness and success on long multi-turn tasks compared to existing baselines. This work highlights the importance of comprehensive, adaptive memory for advancing LLM-based agents.
Memory-augmented LLM agents offer an appealing shortcut to continual learning: rather than updating model parameters, they accumulate experience in external memory, seemingly sidestepping the stability-plasticity dilemma of parametric learning. We show that this challenge does not disappear but resurfaces at the memory level. Under a limited context window, old and new experiences compete during retrieval, relocating the continual-learning bottleneck from parameter updates to memory access. To study this phenomenon, we introduce a (k,v) framework that disentangles two fundamental design axes of external memory: how experience is represented and how it is organized for retrieval. Across sequential-task experiments in ALFWorld and BabyAI, we find that abstract procedural memories transfer more reliably than detailed trajectories, while negative transfer disproportionately harms the hard cases. Moreover, finer-grained memory organization is not universally beneficial: designs that yield strong forward transfer can simultaneously induce severe forgetting. Together, these results reveal that external memory does not resolve the continual-learning problem; it reshapes it into a problem of memory representation and retrieval design.
Current LLM agents lack principled mechanisms for managing persistent memory across long interaction horizons. We present a biologically-grounded memory architecture comprising six cognitive mechanisms: (1) sleep-phase consolidation, (2) interference-based forgetting, (3) engram maturation, (4) reconsolidation upon retrieval, (5) entity knowledge graphs, and (6) hybrid multi-cue retrieval. Each mechanism addresses a specific failure mode of naive memory accumulation. We introduce a synthetic calibration methodology that derives all pipeline thresholds without benchmark data exposure, eliminating a common source of evaluation leakage. We evaluate on two benchmarks. First, a VSCode issue-tracking dataset (13K issues, 120K events) where deduplication-based consolidation achieves 97.2% retention precision with 58% store reduction (+21.8 pp over baseline). Second, the LongMemEval personal-chat benchmark where we conduct the first streaming M-tier evaluation (475 sessions, ~540K unique turns). At a 200K-token context budget, our pipeline matches raw retrieval accuracy (70.1% vs. 71.2%, overlapping 95% CI) while exposing a tunable accuracy/store-size operating curve. At S-tier scale (50 sessions), dedup-based consolidation yields a +13.3 pp improvement in preference recall.
Doga Kerestecioglu, Alexei Robsky, Clemens Vasters +2