Don't Just Trust Your Memory: Evidence-Linked Memory Reduces Memory-Induced Degradation in LLM Agents

Yukyeong Lee; Yumin Choi; Jinheon Baek; Sung Ju Hwang

cs.AINov 10, 2026

Don't Just Trust Your Memory: Evidence-Linked Memory Reduces Memory-Induced Degradation in LLM Agents

Authors: Yukyeong Lee, Yumin Choi, Jinheon Baek, Sung Ju Hwang

Organizations: KAIST

Abstract

External memory enables LLM agents to improve from experience by distilling trajectories of past tasks into concise lessons for new tasks. However, memory can also lead to memory-induced degradation, where an agent fails on tasks it would solve without memory, often because it follows a lesson that is incorrect, or misapplies or misreads a valid one. This is because lessons are inferred from trajectories labeled only with task success or failure, yet are typically provided without the trajectories they came from, leaving the agent no way to check them. To address this, we introduce WikiTrace, an evidence-linked memory system designed to let agents check each lesson against its original trajectory. Specifically, motivated by the finding that a trajectory often still holds the evidence needed to catch an erroneous lesson, WikiTrace stores each lesson as a concise insight linked to a summary of why its trajectory succeeded or failed and to the raw trajectory itself. Then, on a new task, the agent starts from all insights and, whenever one lacks details needed to act, conflicts with another, or comes from a failed task, descends through its links with terminal tools to fill in missing details or correct the insight before acting. We validate WikiTrace on five agentic benchmarks, showing that it consistently outperforms existing memory approaches and a baseline given the same insights alone, while also reducing memory-induced degradation. Our code is available at https://github.com/yukyeongleee/wikitrace

Figures & tables

Figure 1: Memory-induced degradation and WikiTrace. (a) The agent solves a task without memory but fails with memory that holds a wrong insight. (b) With WikiTrace, the agent follows the wrong insight’s links to its source run, revises it, and solves the task.

Figure 2: Overview of WikiTrace. (a) A compiler turns each past run into a compilation, a diagnosis of why the run succeeded or failed, and insights, each a concise lesson. All layers stay linked. (b) Each layer’s documents follow a fixed template. (c) The agent starts from every insight and descends through the links under the conditions shown.

	ALFWorld		ScienceWorld	PlanCraft		$\tau^{3}$ -bench Retail	AppWorld		Avg.
Method	Seen	Unseen	ScienceWorld	Seen	Unseen	$\tau^{3}$ -bench Retail	Normal	Challenge	Avg.
No Memory	94.4 $\pm$ 2.0	94.3 $\pm$ 1.9	83.7 $\pm$ 0.8	72.3 $\pm$ 1.9	56.3 $\pm$ 1.2	86.8 $\pm$ 1.9	75.4 $\pm$ 3.4	72.7 $\pm$ 2.2	79.5 $\pm$ 1.6
[2pt/2pt] AWM	94.7 $\pm$ 1.2	97.3 $\pm$ 0.9	88.9 $\pm$ 4.7	76.3 $\pm$ 3.7	57.3 $\pm$ 2.1	85.5 $\pm$ 2.1	79.3 $\pm$ 0.7	73.4 $\pm$ 1.7	81.6 $\pm$ 0.3
ReasoningBank	92.1 $\pm$ 2.4	92.8 $\pm$ 0.7	85.0 $\pm$ 1.3	84.7 $\pm$ 2.6	61.3 $\pm$ 3.4	86.4 $\pm$ 1.6	80.8 $\pm$ 1.5	70.7 $\pm$ 3.0	81.7 $\pm$ 0.5
ExpeL	94.4 $\pm$ 0.9	98.0 $\pm$ 0.4	91.9 $\pm$ 1.6	74.0 $\pm$ 8.5	51.0 $\pm$ 3.6	82.5 $\pm$ 5.1	80.5 $\pm$ 5.7	70.0 $\pm$ 3.7	80.3 $\pm$ 1.6
[2pt/2pt] Insights Only	96.7 $\pm$ 0.5	97.3 $\pm$ 1.3	94.9 $\pm$ 0.6	88.7 $\pm$ 6.1	73.0 $\pm$ 10.7	85.1 $\pm$ 2.7	86.8 $\pm$ 0.4	73.7 $\pm$ 1.4	87.0 $\pm$ 2.5

Table 1: Main results. Mean over three trials,

\pm

the standard deviation across trials in small type. Avg. averages the eight suites. Best results are bolded and second-best results are underlined.

Figure 5: Gain and Regression of WikiTrace and memory-based baselines , averaged over eight suites with 95% intervals.

Figure 6: WikiTrace against Insights Only by descent in the text-world benchmarks. Score Lead

\Delta

is WikiTrace minus Insights Only, and circle radius gives the insights reported as used. Difficulty: No Memory solves the task in none (Hard), one or two (Medium) or all three (Easy) trials.

Figure 7: Score and Regression by memory subset. Memory is built from successful runs (Succ.), failed runs (Fail.) or both. Arrows mark Insights Only’s higher Regression with failed runs.

Panel of Figure 8 — Figure 8: Why WikiTrace descends and what it seeks. WikiTrace’s score lead over Insights Only by trigger and target, with descent shares in gray.

	Craft		Sci	Avg.
Method	Seen	Unseen	Sci	Avg.
NM	68.7 $\pm$ 3.3	55.3 $\pm$ 2.1	89.6 $\pm$ 1.5	71.2 $\pm$ 0.3
AWM	66.0 $\pm$ 2.8	47.0 $\pm$ 2.9	91.7 $\pm$ 1.0	68.2 $\pm$ 0.5
ReasoningBank	74.7 $\pm$ 4.2	53.3 $\pm$ 5.2	90.9 $\pm$ 1.9	73.0 $\pm$ 2.3
ExpeL	79.0 $\pm$ 12.8	67.0 $\pm$ 14.1	88.6 $\pm$ 1.4	78.2 $\pm$ 8.5
Insights Only	92.3 $\pm$ 2.1	87.7 $\pm$ 3.3	92.6 $\pm$ 0.8	90.9 $\pm$ 1.6

Table 2: Results with GLM-5.3-flash . Mean over three trials. NM is No Memory and WT is WikiTrace.

Panel of Figure 10 — Figure 10: Accuracy against build cost , averaged over the eight suites. A hollow mark is a default build and a filled mark the build run $N$ times. The dashed line shows No Memory.

Panel of Figure 11 — Figure 11: Qualitative example: revising a wrong insight (PlanCraft). Insight-0083 leads agents into the auto-craft mistake. Insights Only follows it and fails, but WikiTrace revises it through descent and succeeds.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	ReAct baseline	Tool	Max steps	Max Pi turns
ALFWorld	ReAct loop Yao et al. (2023)	act	49	150
ScienceWorld	SwiftSage’s ReAct baseline Lin et al. (2023)	act	100	400
PlanCraft	The authors’ prompt Dagan et al. (2025)	take_action	30	150
AppWorld	AppWorld’s ReAct code agent Trivedi et al. (2024)	exec	50	150
$\tau^{3}$ -bench	$\tau^{2}$ -bench’s LLM agent Barres et al. (2026)	take_action	200	150

Appendix

Table 4: Agent harness per benchmark. The instruction, exemplars, and step budget follow the listed ReAct baseline. A step is one environment tool call, except in

\tau^{3}

-bench, which counts agent, user, and tool turns as its authors do.

	Learns from
Method	Succ.	Fail.	Memory	Access
AWM	✓	–	Procedures	I
ExpeL	✓	–	Rules	I
ReasoningBank	✓	✓	Formatted Rules	R
Insights Only	✓	✓	Formatted Rules	I
WikiTrace (ours)	✓	✓	+ Linked Evidence	I + T

Appendix

Table 5: Compared methods. Formatted Rules are rules with required fields. WikiTrace links each to the compilation and raw trajectory it came from. Access is inject (I), retrieve (R) or traverse (T).

	ALFWorld		ScienceWorld		PlanCraft		$\tau^{3}$ -bench		AppWorld
Method	Gain $\uparrow$	Reg. $\downarrow$	Gain $\uparrow$	Reg. $\downarrow$	Gain $\uparrow$	Reg. $\downarrow$	Gain $\uparrow$	Reg. $\downarrow$	Gain $\uparrow$	Reg. $\downarrow$
Eligible tasks	10	225	29	71	75	125	6	70	52	158
AWM	7.0 (70%)	6.0 (3%)	13.3 (46%)	12.3 (17%)	21.7 (29%)	13.0 (10%)	2.3 (39%)	7.3 (10%)	17.0 (33%)	14.3 (9%)
ReasoningBank	4.7 (47%)	12.3 (5%)	11.0 (38%)	9.0 (13%)	30.3 (40%)	9.3 (7%)	2.0 (33%)	6.3 (9%)	18.0 (35%)	16.3 (10%)
ExpeL	8.0 (80%)	6.3 (3%)	14.7 (51%)	11.7 (16%)	16.0 (21%)	16.0 (13%)	2.0 (33%)	9.3 (13%)	21.0 (40%)	20.3 (13%)
Insights Only	8.0 (80%)	5.0 (2%)	18.7 (64%)	5.3 (8%)	43.3 (58%)	6.7 (5%)	2.7 (44%)	8.0 (11%)	25.3 (49%)	14.0 (9%)

Appendix

Table 6: Gain and Regression counts per benchmark. Tasks moved on one trial, averaged over trials and summed over suites, with the rate over the eligible tasks (first row) in small type. Bold is best and underlined second best.

Figure 13: Gain and Regression per suite , with 95% intervals. The dashed line and band denote rerunning without memory.

Panel of Figure 16 — Figure 16: Wrong item (A) in ALFWorld. The block gives the item, its source run and the same test task with and without memory. The source run is the training run the item was written from. A turn (T) shows the reasoning in italics, the action (Act) in typewriter and the observation (Obs) in grey. Shading marks the decisive turn. Item titles, summaries and the coloured notes are ours.

Panel of Figure 17 — Figure 17: Out-of-scope item (B) in AppWorld. Laid out as Figure 16 without a source run. The item card quotes the item’s description and text.

Panel of Figure 18 — Figure 18: Misused item (C) in $\tau^{3}$ -bench retail. Laid out as Figure 16 without a source run. User and Reply rows are messages between the simulated user and the agent. The highlight in turn T26 marks the sentence that stretches the highlighted item phrase.

Panel of Figure 19 — Figure 19: Missed better item (D) in AppWorld. Laid out as Figure 16 without a source run. Rules are numbered as in the memory.

Panel of Figure 20 — Figure 20: No link to memory (E) in AppWorld. Laid out as Figure 16 without a source run. The world clock of the task reads 2023-05-18, and the date the run quotes in T22 comes from its own prompt.

Panel of Figure 24 — Figure 24: A revised insight and a new fact in full. Each case gives the task, why its insight cannot be used as it stands, the run WikiTrace opened, and what WikiTrace and Insights Only then do. Reasoning is in italics, actions in typewriter and observations in grey. Shading marks the decisive reasoning. Insight summaries and the coloured notes are ours.

Agent Memory

Agent memory lets AI systems retain, update, and use facts, experiences, and skills across sessions and long-horizon tasks.

See all 206 papers

Understanding and Mitigating Inference-Time Overreliance Using Agentic Memory
Oct 5, 2026Luoxi Tang, Yuqiao Meng, Nilesh Auradkar +3Agent Memory LLM Memory
Binghamton University, State University of New York, Binghamton, NY, USA · University of Iowa, Iowa City, IA, USA · Penn Medicine, University of Pennsylvania, Philadelphia, PA, USA
Delivered, Not Used: Evaluating Agent Memory at Its Point of Use
Nov 1, 2026Changran Xu, Zeju Li, Qiang XuAgent Memory LLM Agent Evaluation
Department of Computer Science and Engineering The Chinese University of Hong Kong
MemChain: Learning Interpretable Memory Traces for Memory-Augmented LLM Agents
Jul 27, 2026Yiwen Ma, Songjun Tu, Qichao Zhang +3LLM Agent Memory Persistent Memory for Language Models
Institute of Automation, Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences +1

Oct 5, 2026J/K move · Enter open · S save

Understanding and Mitigating Inference-Time Overreliance Using Agentic Memory

Luoxi Tang, Yuqiao Meng, Nilesh Auradkar, Muchao Ye+2

Binghamton University, State University of New York, Binghamton, NY, USA · University of Iowa, Iowa City, IA, USA · Penn Medicine, University of Pennsylvania, Philadelphia, PA, USA

Agent Memory LLM Memory Memory-Augmented Agents Agent Memory Management

Open paper AskPDF arXiv

Agentic memory allows LLM agents to reuse past experience, yet retrieved memories can also distort inference even when they are benign, correctly stored, and appropriately retrieved. We study this failure mode, which we call memory over-reliance. Across benchmarks and memory architectures, we find that memory is useful when past experience transfers to the current task, but can become misleading when only part of the evidence transfers. Failures are strongest under partial query-memory overlap, a pattern further confirmed by controlled experiments thatvary the amount of overlapping evidence. Motivated by this finding, we propose MEMTRIM, a plug-and-play framework that indexes memory evidence at write time and controls its reuse at read time. MEMTRIM removes repeated or conflicting evidence while preserving useful memory-specific information, requires no retraining, and applies to both embedding-based and structured memory systems.Experiments show that MEMTRIM reduces memory overreliance while preserving the benefits of useful memory across models and memory settings.

Abstract

Agent Memory

Explore similar work

Understanding and Mitigating Inference-Time Overreliance Using Agentic Memory

Delivered, Not Used: Evaluating Agent Memory at Its Point of Use

MemChain: Learning Interpretable Memory Traces for Memory-Augmented LLM Agents