Organizations: The University of Hong Kong · Huazhong University of Science and Technology · The Hong Kong University of Science and Technology · National University of Singapore · Peking University · Xi’an Jiaotong University · Huawei Noah’s Ark Lab
LLM agents have made impressive progress in coding and mathematical reasoning, motivating growing interest in learning from long-horizon interaction. Memory supports this process, but plausible lessons can be redundant or reinforce inappropriate actions. Can a memory's source decision provide a practical test for whether to retain it? We localize consequential trajectory steps, extract candidate memories, and replay each source decision with and without its candidate under the same context. This reveals two properties: most candidates barely shift the probed distribution, and both successful and failed source actions can be reinforced or suppressed. These observations motivate Anchor, which tests shift magnitude and source-role alignment before retention. Admitted memories guide later trajectories, enabling experience-driven improvement under a frozen policy. On held-out ALFWorld and WebShop tasks, Anchor improves success rate and mean score by up to 19.9% and 17.2%, respectively, relative to the strongest baseline for each backbone. With Qwen3.5-27B, it retains 41% fewer ALFWorld cards than unfiltered retention and outperforms all four WebShop cost-comparison baselines while using fewer tokens.
Figures & tables
Figure 1: Memory can change the next action. In this schematic kitchen example, putting a dirty reusable plate in the sink is preferable to the cupboard or trash bin. Anchor screens for a role-aligned change.
Figure 2: Shift magnitude. Action-verb probe JS divergence for 2,785 candidate memories. Most shifts cluster near zero; a smaller tail moves the probed distribution more strongly.
Figure 4: Anchor overview. Localizer extracts a fixed candidate from a source decision. Amplituder and Aspecter test its shift magnitude and source-role alignment. BankMaster stores admitted memories and retrieves them on later tasks. Kitchen actions and probability bars are schematic.
ALFWorld
WebShop
Method
Overall ↑
Put ↑
Clean ↑
Heat ↑
Cool ↑
Examine ↑
Puttwo ↑
Score ↑
Steps ↓
Qwen2.5-14B-Instruct
ReAct
46.27
33.33
51.61
26.09
85.71
72.22
5.88
41.77
8.68
Conventional memory methods
Mem0
12.69
0.00
0.00
4.35
0.00
88.89
0.00
38.32
9.11
Reflexion
67.16
77.78
55.91
66.67
74.60
77.78
52.94
25.24
8.41
Table 1: RQ1 Held-out task performance. ALFWorld success (%) and WebShop mean score and steps. Bold and underline mark the best and second-best results per backbone; task-family ties use Overall.
Admission variant
Success (%)
ReAct (no memory)
61.19
Anchor w/o Amplituder & Aspecter
73.38
Anchor w/o Aspecter
74.88
Anchor w/o Failure Corrections
86.82
Anchor w/o Success Maintain
83.83
Anchor (ours)
96.27
Table 2: RQ2 Overall ALFWorld admission ablation. Qwen3.5-27B success (%). Underline marks the best ablation.
Figure 5: RQ3 and RQ4 Success, bank size, and token consumption with Qwen3.5-27B. ( Left ) Sample success rate (cumulative; solid) over 200 randomly sampled ALFWorld memory-construction tasks and bank size (dashed) for Anchor and Anchor w/o Amplituder & Aspecter. ( Right ) WebShop mean score versus total input and output tokens across construction and evaluation ( 10x scale). Circle area is proportional to mean interaction steps; lower, smaller circles indicate lower cost.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Variant
Overall
Put
Clean
Heat
Cool
Examine
Puttwo
ReAct
61.19
66.67
51.61
65.22
76.19
72.22
35.29
Anchor w/o Amplituder & Aspecter
73.38
77.78
70.97
68.12
82.54
85.19
54.90
Anchor w/o Aspecter
74.88
81.94
69.89
73.91
80.95
83.33
58.82
Anchor w/o Failure Corrections
86.82
91.67
86.02
79.71
90.48
92.59
80.39
Anchor w/o Success Maintain
83.83
88.89
82.80
76.81
92.06
88.89
72.55
Anchor (ours)
96.27
100.00
90.32
91.30
100.00
100.00
100.00
Appendix
Table 3: Full ALFWorld ablation. Qwen3.5-27B success (%) by task family.
Setting
Value
Setting
Value
Action decoding
Token-wise greedy, 256-token limit
Probe statistic
Token-level action scores
Replay token budget L
ALFWorld: 1; WebShop: 5
Paired context
Fixed within each pair
Failure JS threshold
0.12 bits
Success JS threshold
0.05 bits
Failure probability branch
Source decrease ≥0.10
Failure ranking branch (OR)
Injected top action = source
Success ranking
Injected top action = source
Success comparison
Probability increase
Success reinforcement
≥0.04
Localized candidates
K=5 per episode
Appendix
Table 4: Implementation settings. Branch-specific thresholds use JS in bits. The measurement interface is described in Appendix C.4 .
Figure 6: Complete prompt templates: (a) ALFWorld action selection; (b) ALFWorld memory extraction. Braced fields are filled at runtime.
Figure 11: Navigation: the same source state, opposite memory effects. The recorded rollout goes to desk 1, takes the alarm clock, and turns on the lamp to complete the task. Replay tests the initial navigation decision: M1 reinforces go to ; M2 suppresses it and favors look . Bars show the two dominant verb categories without renormalization; remaining probability is below 4×10−5 in each distribution.
Figure 12: Heating: correcting a failed source decision. The agent opens the microwave at step 4 but never executes heat , reaching the 50-step limit. Dots omit intervening logged steps; indices are zero-based. The hint suppresses open and reinforces heat , reversing their ranking.
Figure 13: Memory cards grouped by source role. Left: success-maintain memories S1–S3. Right: failure corrections F1–F3. Guidance text and source metadata are shown together; F1 preserves the tested heating hint verbatim.