MemScope: Evolving Agent Memory Architectures through Localized Code Repair
Authors: Miao Su, Yucan Guo, Zixuan Li, Long Bai, Zhongni Hou, Xiaolong Jin, Jiafeng Guo
Organizations: State Key Laboratory of AI Safety · Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Meituan
Agent memory architectures define how LLM agents store and reuse experience across tasks, but they are typically fixed after deployment, limiting their adaptability across task types. Recent self-evolving approaches instead adapt these memory architectures using execution feedback, typically represented as evaluation scores or compressed summaries. However, such coarse-grained feedback is insufficient for precise failure localization in current agent systems, where long-horizon task execution involves multiple interacting components. Without a precise failure location, existing approaches lack a clear code-level target for modification, leaving the update scope weakly constrained and often leading to large-scale rewrites of an otherwise capable memory architecture. We propose MemScope, a framework for evidence-grounded, localized evolution of an existing memory architecture. Its tool-using Patch Agent jointly inspects raw execution traces, runtime memory state, and current architecture code to identify repair targets and apply bounded, in-place edits under provider-identity and memory-compatibility constraints. On GAIA, the trace-and-memory inspection variant achieves 86.67% accuracy, compared with 70.00% for trace-only and 63.33% for summary-only feedback. Under xBench tasks, the evolved memory architecture improves over its original version from 49 to 55 correct answers with DeepSeek V3.2 and from 50 to 53 with Kimi K2.7 Code. On disjoint ALFWorld unseen tasks, success rises from 50.75% to 71.64%. Across archived architecture-snapshot transitions, the median code-change ratio is 4.4%. These results support fine-grained execution evidence as a basis for localized, in-place memory-architecture evolution.
Figures & tables
Figure 1: Feedback and update granularity in memory-architecture evolution. Prior architecture-level approaches use outcome-level feedback to propose complete executable architectures. MemScope makes the repair target explicit by jointly inspecting execution traces, runtime memory state, and provider code, then applies a localized, in-place edit to the existing provider. In the illustrated update, only the highlighted code region is modified.
Figure 2: Overview of MemScope. (a) Evolution Target. Following the Agent Skills format, each reusable memory skill contains a required SKILL.md file and an executable provider under scripts/ ; the activated provider maintains a separate skill-specific runtime memory bank Mt . (b) MemScope Evolution Loop. Execution produces metrics and raw success and failure traces. When repair is triggered, the Patch Agent inspects these traces, the active memory bank, the current provider, and Patch History. A local edit changes pt into pt+1 without replacing the skill, while the patch and its later outcome are recorded for subsequent evolution.
Index available cases; inspect a task result and step map; read selected native steps; search matching events across traces.
Memory Inspection ( memory_store )
stats
Summarize record counts, fields, schemas, and outcome distribution.
list
List compact record summaries, optionally filtered by outcome.
read
Return one complete raw memory record.
Table 1: Evidence sources for failure localization.
System
Model Family
GAIA
xBench
Avg.
L1
L2
L3
Closed-Source Agent Frameworks
Alita (pass@1)
Claude Sonnet 4, GPT-4o
75.15
77.36
76.74
65.38
—
Langfun Agent
Claude 3.7 etc.
71.52
83.02
68.60
57.69
—
TraseAgent
Claude etc.
70.30
83.02
69.77
46.15
—
OpenAI Deep Research
o1, o3 etc.
67.36
74.29
69.06
47.60
—
Table 2: GAIA validation and xBench DeepSearch-2505 accuracy (%).
Backbone
C → C
I → C
C → I
I → I
DeepSeek V3.2
47
8
2
13
Kimi K2.7 Code
46
7
4
13
Table 3: Paired xBench outcomes on 70 questions. C/I denote correct/incorrect; transitions are from the original to the evolved provider.
System
Success
DynamicCheatsheet
34/67 (50.75)
ALMA
45/67 (67.16)
MemScope
48/67 (71.64)
Table 4: ALFWorld valid_unseen success under DeepSeek V3.2 (correct/67, %).
Experimental setting
Diagnostic inputs
Task accuracy (%)
Resources
Setting
Provider
Evidence
Memory
History
Overall
L1
L2
L3
Time (s)
Steps
No Memory
×
None
×
×
56.67
75.0
58.82
20.0
517.61
11.30
Original Provider
✓
None
×
×
56.67
62.5
64.71
20.0
771.05
32.53
Summary-only
✓
Summary
×
×
63.33
75.0
70.59
20.0
993.14
40.17
Trace-only
✓
Raw Trace
×
×
70.00
87.5
70.59
40.0
921.93
30.37
Trace + Memory Inspection
✓
Raw Trace
✓
×
86.67
75.0
94.12
80.0
804.41
25.73
Table 5: GAIA diagnostic ablation: accuracy (%), mean time, and steps.
Aspect
Summary-only
Trace-only
Evidence
Grouped failure reports
Zero-length traces; empty guidance steps
Proposed diagnosis
Insufficient cross-verification
Early-return guard under MemoryStatus.IN may suppress guidance
Edit
Add a verification rule to the storage prompt
Remove the early-return guard
Table 6: A recorded GAIA repair case under different diagnostic interfaces.
Figure 3: GAIA guidance during online memory updates. Original versus evolved provider coverage: 98.8% versus 99.4%; accuracy among guided tasks: 65.6% versus 84.1%. Code and memory state both vary.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Original field
Recorded value
patch_type
"no_patch"
needs_new_provider
false
edited_files
[]
patch_history_decision
"not_repeating_failed_direction"
Appendix
Table 7: A recorded no_patch decision in xBench.
Metric
Snapshot change
Provider-size proxy
Lines
16.5
369
Functions
1
16
Median recorded normalized fraction
4.4%
100%
Appendix
Table 8: Syntactic changes across archived provider snapshots.
Long-term memory is essential for LLM agents that operate across multiple sessions, yet existing memory systems treat retrieval infrastructure as fixed: stored content evolves while scoring functions, fusion strategies, and answer-generation policies remain frozen at deployment. We argue that truly adaptive memory requires co-evolution at two levels: the stored knowledge and the retrieval mechanism that queries it. We present EvolveMem, a self-evolving memory architecture that exposes its full retrieval configuration as a structured action space optimized by an LLM-powered diagnosis module. In each evolution round, the module reads per-question failure logs, identifies root causes, and proposes targeted configuration adjustments; a guarded meta-analyzer applies them with automatic revert-on-regression and explore-on-stagnation safeguards. This closed-loop self-evolution realizes an AutoResearch process: the system autonomously conducts iterative research cycles on its own architecture, replacing manual configuration tuning. Starting from a minimal baseline, the process converges autonomously, discovering effective retrieval strategies including entirely new configuration dimensions not present in the original action space. On LoCoMo, EvolveMem outperforms the strongest baseline by 25.7% relative and achieves a 78.0% relative improvement over the minimal baseline. On MemBench, EvolveMem exceeds the strongest baseline by 18.9% relative. Evolved configurations transfer across benchmarks with positive rather than catastrophic transfer, indicating that the self-evolution process captures universal retrieval principles rather than benchmark-specific heuristics. Code is available at https://github.com/aiming-lab/SimpleMem.
Long-horizon autonomous agents require memory systems to retain historical information, track evolving states, and reuse relevant knowledge beyond finite context windows. Existing agentic memory systems typically follow a memory construction-retrieval (MCR) pipeline, but often adapt mainly the memory bank while keeping the surrounding pipeline fixed after deployment. This fixed-pipeline design struggles to handle heterogeneous task-specific failure modes and can become misaligned with memory banks that evolve in scale and structure over time. To address these limitations, we propose MemPro, a system-level evolution framework that treats the entire MCR pipeline as an evolvable program rather than adapting only the memory bank or prompt text. MemPro maintains a version tree of runnable memory-system implementations, where an Evolving Agent iteratively selects promising versions, diagnoses recurring failures, and creates improved child versions through failure-mode-guided edit-debug refinement. Experiments on LongMemEval, LoCoMo, HotpotQA, and NarrativeQA show that MemPro consistently outperforms strong static and prompt-level evolving baselines within a few iterations, continues to improve with evolution, and achieves a favorable performance-cost trade-off. Code is available at https://github.com/wanghai673/MemPro.
Memory is essential for enabling LLM-based agents to maintain coherent, personalized behavior over long-horizon interactions. However, existing memory systems share a fundamental limitation: they never proactively test their own memory, repairing it only after real queries expose weaknesses. This reactive paradigm means every retrieval failure corresponds to a real interaction in which the cost has already been paid. We propose MemDream, a framework that enables self-probing memory evolution for LLM agents. Our framework periodically enters offline dream cycles where three specialized agents (Dreamer, Analyst, Consolidator) collaboratively probe, diagnose, and repair the memory graph before failures occur. A policy trained via Group Relative Policy Optimization learns which repair operations produce durable retrieval improvements, while a soft decay mechanism provides reversible forgetting driven by the same anticipatory signal. Experiments on LoCoMo and MemoryAgentBench demonstrate that MemDream improves answer F1 by 4.5 points on LoCoMo and achieves a 9.1-point higher overall score on MAB over the strongest reactive-evolution baselines.
Mingfei Lu, Mengjia Wu, Runsong Jia +2
Australian Artificial Intelligence Institute (AAII) University of Technology Sydney