Organizations: The University of Hong Kong · Huazhong University of Science and Technology · The Hong Kong University of Science and Technology · National University of Singapore · Peking University · Xi’an Jiaotong University · Huawei Noah’s Ark Lab
LLM agents have made impressive progress in coding and mathematical reasoning, motivating growing interest in learning from long-horizon interaction. Memory supports this process, but plausible lessons can be redundant or reinforce inappropriate actions. Can a memory's source decision provide a practical test for whether to retain it? We localize consequential trajectory steps, extract candidate memories, and replay each source decision with and without its candidate under the same context. This reveals two properties: most candidates barely shift the probed distribution, and both successful and failed source actions can be reinforced or suppressed. These observations motivate Anchor, which tests shift magnitude and source-role alignment before retention. Admitted memories guide later trajectories, enabling experience-driven improvement under a frozen policy. On held-out ALFWorld and WebShop tasks, Anchor improves success rate and mean score by up to 19.9% and 17.2%, respectively, relative to the strongest baseline for each backbone. With Qwen3.5-27B, it retains 41% fewer ALFWorld cards than unfiltered retention and outperforms all four WebShop cost-comparison baselines while using fewer tokens.
Figures & tables
Figure 1: Memory can change the next action. In this schematic kitchen example, putting a dirty reusable plate in the sink is preferable to the cupboard or trash bin. Anchor screens for a role-aligned change.
Figure 2: Shift magnitude. Action-verb probe JS divergence for 2,785 candidate memories. Most shifts cluster near zero; a smaller tail moves the probed distribution more strongly.
Figure 4: Anchor overview. Localizer extracts a fixed candidate from a source decision. Amplituder and Aspecter test its shift magnitude and source-role alignment. BankMaster stores admitted memories and retrieves them on later tasks. Kitchen actions and probability bars are schematic.
ALFWorld
WebShop
Method
Overall ↑
Put ↑
Clean ↑
Heat ↑
Cool ↑
Examine ↑
Puttwo ↑
Score ↑
Steps ↓
Qwen2.5-14B-Instruct
ReAct
46.27
33.33
51.61
26.09
85.71
72.22
5.88
41.77
8.68
Conventional memory methods
Mem0
12.69
0.00
0.00
4.35
0.00
88.89
0.00
38.32
9.11
Reflexion
67.16
77.78
55.91
66.67
74.60
77.78
52.94
25.24
8.41
Table 1: RQ1 Held-out task performance. ALFWorld success (%) and WebShop mean score and steps. Bold and underline mark the best and second-best results per backbone; task-family ties use Overall.
Admission variant
Success (%)
ReAct (no memory)
61.19
Anchor w/o Amplituder & Aspecter
73.38
Anchor w/o Aspecter
74.88
Anchor w/o Failure Corrections
86.82
Anchor w/o Success Maintain
83.83
Anchor (ours)
96.27
Table 2: RQ2 Overall ALFWorld admission ablation. Qwen3.5-27B success (%). Underline marks the best ablation.
Figure 5: RQ3 and RQ4 Success, bank size, and token consumption with Qwen3.5-27B. ( Left ) Sample success rate (cumulative; solid) over 200 randomly sampled ALFWorld memory-construction tasks and bank size (dashed) for Anchor and Anchor w/o Amplituder & Aspecter. ( Right ) WebShop mean score versus total input and output tokens across construction and evaluation ( 10x scale). Circle area is proportional to mean interaction steps; lower, smaller circles indicate lower cost.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Variant
Overall
Put
Clean
Heat
Cool
Examine
Puttwo
ReAct
61.19
66.67
51.61
65.22
76.19
72.22
35.29
Anchor w/o Amplituder & Aspecter
73.38
77.78
70.97
68.12
82.54
85.19
54.90
Anchor w/o Aspecter
74.88
81.94
69.89
73.91
80.95
83.33
58.82
Anchor w/o Failure Corrections
86.82
91.67
86.02
79.71
90.48
92.59
80.39
Anchor w/o Success Maintain
83.83
88.89
82.80
76.81
92.06
88.89
72.55
Anchor (ours)
96.27
100.00
90.32
91.30
100.00
100.00
100.00
Appendix
Table 3: Full ALFWorld ablation. Qwen3.5-27B success (%) by task family.
Setting
Value
Setting
Value
Action decoding
Token-wise greedy, 256-token limit
Probe statistic
Token-level action scores
Replay token budget L
ALFWorld: 1; WebShop: 5
Paired context
Fixed within each pair
Failure JS threshold
0.12 bits
Success JS threshold
0.05 bits
Failure probability branch
Source decrease ≥0.10
Failure ranking branch (OR)
Injected top action = source
Success ranking
Injected top action = source
Success comparison
Probability increase
Success reinforcement
≥0.04
Localized candidates
K=5 per episode
Appendix
Table 4: Implementation settings. Branch-specific thresholds use JS in bits. The measurement interface is described in Appendix C.4 .
Figure 6: Complete prompt templates: (a) ALFWorld action selection; (b) ALFWorld memory extraction. Braced fields are filled at runtime.
Figure 11: Navigation: the same source state, opposite memory effects. The recorded rollout goes to desk 1, takes the alarm clock, and turns on the lamp to complete the task. Replay tests the initial navigation decision: M1 reinforces go to ; M2 suppresses it and favors look . Bars show the two dominant verb categories without renormalization; remaining probability is below 4×10−5 in each distribution.
Figure 12: Heating: correcting a failed source decision. The agent opens the microwave at step 4 but never executes heat , reaching the 50-step limit. Dots omit intervening logged steps; indices are zero-based. The hint suppresses open and reinforces heat , reversing their ranking.
Figure 13: Memory cards grouped by source role. Left: success-maintain memories S1–S3. Right: failure corrections F1–F3. Guidance text and source metadata are shown together; F1 preserves the tested heating hint verbatim.
When does retention matter for memory-augmented LLM agents? We study this with TraceRetain, a lightweight framework for bounded external memory in frozen LLM agents that scores entries by interpretable features (success, age, access frequency, redundancy, specificity, similarity, downstream utility) and evicts the lowest-scoring ones at capacity. On clean ALFWorld with gpt-5-mini, external memory robustly improves over no memory across two seeds, but differences among bounded retention policies fall within Wilson 95% CIs: clean ALFWorld at T=100 to T=200 does not naturally exhibit the memory pollution retention is designed to address. Under a controlled noisy-write stress (75% synthetic distractors), unbounded memory and FIFO-K50 degrade on Precision@5 (20.2% to 12.4% and 15.8% to 3.8%) while TraceRetain-CEM is essentially unchanged (16.9% to 16.6%) and preserves 97/100 task success. The mechanism: unbounded memory has the highest mean similarity (0.87) but lowest precision, indicating failed distractors close to the query in embedding space. Held-out in-distribution evaluation shows memory-augmented policies solving 47 to 49 of 50 tasks vs. 39/50 for no memory. Bounded retention buys memory and step efficiency on saturated clean benchmarks at no task-success cost, and only differentiates from cache heuristics when streams contain noise.
Memory is essential for enabling LLM-based agents to maintain coherent, personalized behavior over long-horizon interactions. However, existing memory systems share a fundamental limitation: they never proactively test their own memory, repairing it only after real queries expose weaknesses. This reactive paradigm means every retrieval failure corresponds to a real interaction in which the cost has already been paid. We propose MemDream, a framework that enables self-probing memory evolution for LLM agents. Our framework periodically enters offline dream cycles where three specialized agents (Dreamer, Analyst, Consolidator) collaboratively probe, diagnose, and repair the memory graph before failures occur. A policy trained via Group Relative Policy Optimization learns which repair operations produce durable retrieval improvements, while a soft decay mechanism provides reversible forgetting driven by the same anticipatory signal. Experiments on LoCoMo and MemoryAgentBench demonstrate that MemDream improves answer F1 by 4.5 points on LoCoMo and achieves a 9.1-point higher overall score on MAB over the strongest reactive-evolution baselines.
Mingfei Lu, Mengjia Wu, Runsong Jia +2
Australian Artificial Intelligence Institute (AAII) University of Technology Sydney
Large language model (LLM) agents have demonstrated the ability to solve multi-step tasks requiring planning, tool use, and external information access, yet growing execution histories increase inference cost and expose reasoning to outdated, irrelevant, or misleading information, potentially degrading reasoning quality. Existing memory approaches organize or compress execution histories but provide limited mechanisms for deciding which memories remain active. We introduce the, a hierarchical memory system that organizes execution into tasks, subtasks, and actions while assigning each memory a dynamic retention score. Event-based updates and selection-based decay revise these scores, allowing WMT to preserve useful information, fold completed trajectories, suppress low-utility content, and retain access to folded context. We evaluate WMT on GAIA-Text using Qwen3-8B, Gemma 4 E4B, and Llama-3.1-8B, with ablations and memory-poisoning experiments. Relative to linear memory, WMT improves accuracy by an average of 9.97 percentage points while reducing prompt-token usage by 32.8%. Memory-poisoning experiments show that WMT limits the persistence and propagation of unreliable information. Our results suggest that effective long-horizon agent memory depends less on storing more information than on deciding which information should remain active.
Quang Dao, Purvi Kathalkar, Kenneth Eaton
Rose-Hulman Institute of Technology · Georgia Institute of Technology · Georgia Tech Research Institute