Current agents remain largely stateless across tasks, limiting their ability to continually improve from prior interactions and making memory essential for long-horizon agentic behavior. Existing memory methods seek to reuse past experience, but most rely on a single memory representation (e.g., trajectories, reflections, skills, structured knowledge) whose effectiveness varies across task distributions. Rethinking this design space, we evaluate 13 memory methods and find that no single method generalizes across benchmarks, revealing the potential of managing heterogeneous memory providers. We formulate agent memory as a routing problem in which a memory agent decides which memory provider to retrieve from, whether to inject short-term memory, and which providers should store the resulting experience. Based on this perspective, we propose MemAgent, featuring a content-aware routing architecture and a training-data synthesis pipeline. The routing architecture combines content-aware probing before retrieval, short-term memory gating during execution, and selective multi-provider storage, while the training pipeline synthesizes phase-specific supervision for routing decisions. Across GAIA, WebWalkerQA, and xBench-DS, MemAgent improves average accuracy by 10.0% and outperforms every individual memory method across all three benchmarks. These gains come with less than 0.3% routing overhead and a 12% reduction in average task steps.
Figures & tables
Figure 1: The task agent interacts with tools to solve tasks, while experiences from past tasks are stored in heterogeneous memory providers. By routing retrieval to the appropriate provider, the memory agent helps the task agent solve the new task correctly in fewer steps.
Figure 2: Cross-benchmark gains of 13 memory methods over the no-memory baseline. No single method dominates across all three benchmarks. Stars mark the best method on each benchmark.
Figure 3: For each task, the memory agent AM makes three routing decisions. (I) Begin : It previews all providers in P , selects one, and injects the retrieved long-term memory into the task agent AT ’s prompt. (II) In : At each step, the short-term memory provider summarizes cross-step state from tool interactions (at,ot) , and AM decides whether to inject the summary or skip. (III) End : After task completion, AM selects a subset S⊆P to store the trajectory ξτ for future reuse.
Figure 4: Training data synthesis pipeline for the memory agent. Forced exploration enumerates provider-action combinations for each task, producing trajectories Ξ with binary outcomes {yτ(p,a)} . Supervision signals are derived from task correctness (for Begin ), LLM-as-Judge attribution (for Begin and In ), and downstream reuse (for End ). Finally, reward filtering retains high-quality context-decision pairs (c,d) for the supervised fine-tuning of AM .
Method
GAIA
WebWalkerQA
xBench-DS
Avg.
Overall
Level 1
Level 2
Level 3
No Memory
58.8
73.6
54.7
42.3
69.4
69.0
65.7
Individual memory methods
Agent-KB ( Tang et al., 2025 )
60.0 +1.2
73.6
54.7
50.0
70.0 +0.6
71.0 +2.0
67.0 +1.3
LightWeight ( Zhang et al., 2026a )
59.4 +0.6
67.9
61.6
34.6
73.5 +4.1
73.0 +4.0
68.6 +2.9
ExpeL ( Zhao et al., 2024 )
61.8 +3.0
73.6
61.6
38.5
77.1 +7.6
74.0 +5.0
71.0 +5.2
Table 1: Accuracy (%) on GAIA, WebWalkerQA, and xBench-DS. Gray deltas show percentage-point changes vs. the no-memory baseline. GAIA results are detailed by difficulty levels ( L1 – L3 ). Bold and underline denote the best and second-best memory methods.
Table 6
Component
Per-task cost
Notes
Memory injection (tokens inserted into context):
Begin (long-term retrieval)
∼ 370 tokens
median 268, max 2.2K
In (short-term, per step)
∼ 210 tokens
3.8 steps/task triggered
Total injected per task
∼ 1.2K tokens
< 0.5% of task tokens
Routing cost (MemAgent-specific):
Probe (13 providers, parallel)
∼ 75 ms
one-time at Begin
Table 5: Inference overhead of MemAgent on GAIA. Injection sizes count memory tokens added to each task-agent prompt. Routing latency is MemAgent’s extra wall-clock cost per task.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Provider
Memory Paradigm
Storage Strategy
Retrieval Mechanism
Voyager ( Wang et al., 2024 )
Exploration memory
Refined trajectory summaries with embeddings
Semantic search over task similarity
ExpeL ( Zhao et al., 2024 )
Experience learning
Dual-path: insights from all outcomes + refined trajectories from successes
Hybrid insight + trajectory retrieval
Generative ( Park et al., 2023 )
Scored memory pool
Task queries + trajectory summaries
Vector retrieval + LLM importance scoring
DILU ( Wen et al., 2024 )
Query-trajectory mapping
Query embeddings + trajectory summaries
Semantic search on query similarity
AWM ( Wang et al., 2025c )
Workflow templates (SOPs)
LLM-induced abstract workflow patterns
Vector search over workflow templates
Mobile-E ( Wang et al., 2025b )
Keyword-indexed tips
Tips (text) + Shortcuts (structured sequences)
Keyword-based retrieval
Appendix
Table 6: Memory providers in MemAgent. The pool comprises 13 heterogeneous long-term providers spanning diverse memory paradigms, plus a short-term memory provider for within-task state aggregation. Storage Strategy: the specific information extracted and saved from past trajectories; Retrieval Mechanism: the method used to query relevant memories for new tasks.
Provider
Emb.
k
LLM-R
Key Parameters
Voyager ( Wang et al., 2024 )
✓
1
✗
Pure semantic search; returns 6-sentence max trajectory summary with hierarchical distillation
Table 7: Implementation parameters for all 13 long-term memory providers. “Emb.”: uses a dense embedding model on the retrieval path; “LLM-R”: requires an LLM call on the full retrieval path; k : retrieval top- k , where 3→1 denotes three dense candidates reranked by an LLM to one, and 2+2 denotes two tips plus two shortcuts. All 13 providers additionally invoke an LLM during storage. The content-aware Begin probe stage is embedding/keyword-only for all providers (no LLM).
Phase
Dimension
Weight
Dir.
Description
Begin
Utilization
0.6
↑
Did the task agent reference or use information from the retrieved memory in its subsequent actions?
Relevance
0.4
↑
How relevant was the retrieved memory to the specific task?
In ( Query )
Helpfulness
0.4
↑
Did the agent use information from the injected short-term memory?
Progress
0.4
↑
Did the injection help advance the task?
Noise
0.2
↓
Did the injection distract or confuse the agent?
In ( NoOp )
Appropriateness
0.5
↑
Was skipping retrieval a reasonable choice given the current state?
Appendix
Table 8: LLM-as-Judge scoring dimensions and weights by decision phase. Dimensions marked ↓ are inverted before aggregation: s^i=1−si/smax .
Phase
Dimension
0 (lowest)
2–3 (borderline)
5 (highest)
Begin
Utilization
Memory completely ignored
Some overlap, likely coincidental
Agent directly built upon specific memory content
Begin
Relevance
Completely irrelevant domain
Somewhat related, not specifically useful
Perfectly targeted, critical information
In (Query)
Helpfulness
No reference to injected content
Minor overlap, uncertain attribution
Agent directly leveraged specific injected details
In (Query)
Noise ↓
No distraction
Some off-topic exploration triggered
Agent clearly confused, pursued wrong direction
In (NoOp)
Appropriateness
STM had highly relevant content that should have been retrieved
Borderline; STM has marginal content
STM empty or clearly irrelevant; NoOp optimal
In (NoOp)
Missed Opp. ↓
Nothing valuable missed
Missed non-critical information
Missed critical information; retrieval would have been breakthrough
Appendix
Table 9: Representative rubric anchors for selected dimensions. All dimensions follow a 0–5 integer scale. Dimensions marked ↓ are inverted before aggregation.
Student Guidance (reference plans) and Teacher Guidance (lessons learned)
Success
SkillWeaver
Analyzes trajectories to identify core algorithms, generates generalized Python functions
Matches relevant skills, dynamically wraps them as tools
Success
ExpeL
Extracts Insights (Tips or Warnings) from all outcomes; stores refined trajectories on success
Hybrid retrieval of Insights and successful trajectories
All
Voyager
Generates trajectory summaries from all outcomes, stored as JSON with embeddings
Semantic search for most similar historical tasks
All
DILU
Stores query embeddings and trajectory summaries from both successes and failures
Semantic search on query similarity
All
Appendix
Table 11: Provider descriptions injected into the memory agent prompts. The Begin prompts incorporate both storage and retrieval descriptions, whereas the End prompts include only the storage description accompanied by an outcome gate annotation.
Method
Overall
L1
L2
L3
Qwen3.5-4B w/o Training
63.6
71.7
67.4
34.6
MemAgent (Qwen3.5-4B)
69.7
83.0
67.4
50.0
Qwen3.5-9B w/o Training
63.6
75.5
61.6
46.2
MemAgent (Qwen3.5-9B)
70.9
81.1
69.8
53.8
Appendix
Table 12: Router-capacity analysis on GAIA. Accuracy (%).
Correct
GAIA
WebWalkerQA
xBench-DS
0 (unsolvable)
12.7%
10.6%
6.0%
1–2 (routing-critical)
12.1%
6.5%
8.0%
3–6
11.5%
8.2%
7.0%
7–13
28.5%
23.5%
44.0%
14 (all correct)
35.2%
51.2%
35.0%
Appendix
Table 13: Task coverage distribution: the number of methods (13 memory providers + no-memory baseline) that solve each task.
Figure 5: Failure analysis and task solvability across benchmarks. Left: among failed tasks, coverage ceiling is the dominant failure source. Right: GAIA is more routing-critical than WebWalkerQA or xBench-DS, with more tasks solved by at most two of the 14 configurations.
Method
Tokens injected
Avg. steps
Δ Steps
No Memory
N/A
9.1
N/A
InsightGraph
57
8.6
− 0.5
LightWeight
154
8.3
− 0.8
Voyager
178
8.3
− 0.8
DILU
184
8.3
− 0.8
Generative
195
8.5
− 0.6
Appendix
Table 14: Added context size in the Begin phase and average trajectory length on GAIA. SkillWeaver is reported as “tools only” because it exposes retrieved skills as callable tools rather than injecting plain text, so a token count is not directly comparable.
Family
Members
Indexable unit
Similarity-retrieval
Voyager, DILU, Generative, Cheatsheet
Trajectory or query embedding
Structural-abstraction
AWM, SkillWeaver, Memp
Workflow template, executable function, or procedural script
Memory emerges as the core module in the large language model (LLM)-based agents for long-horizon complex tasks (e.g., multi-turn dialogue, game playing, scientific discovery), where memory can enable knowledge accumulation, iterative reasoning and self-evolution. A number of memory methods have been proposed in the literature. However, these methods have not been systematically and comprehensively compared under the same experimental settings. In this paper, we first summarize a unified framework that covers existing representative agent memory methods from a high-level perspective. We then extensively compare representative agent memory methods on two long-term conversational benchmarks and an agentic memory benchmark, and examine the effectiveness of representative methods, providing a thorough analysis of those methods. As a byproduct of our experimental analysis, we also design a new memory method by exploiting modules in the existing methods, which outperforms the state-of-the-art methods. Finally, based on these findings, we offer promising future research opportunities. We believe that a deeper understanding of the behavior of existing methods can provide valuable new insights for future research.
Large Language Models (LLMs) show promise as tool-using agents but remain limited in long-horizon tasks that require remembering, organizing, and reusing knowledge. Prior memory approaches aim to resolve the situation, but mainly focus on storing factual information. Recent work on procedural memory improves task reuse, yet often reduces to replaying past successes without addressing failure cases or online scalability. We introduce a unified and automatic memory framework that integrates semantic, episodic, and procedural memory in a bi-level design combining short-term and long-term stores. A multi-agent architecture with actor, memory, and critic agents enables automatic memory generation, reward annotation, and adaptive retrieval. Long-term memory is managed through reward-based evaluation, merging, and pruning, ensuring scalability and continual improvement. Experiments across various environments show that our approach improves robustness and success on long multi-turn tasks compared to existing baselines. This work highlights the importance of comprehensive, adaptive memory for advancing LLM-based agents.
Memory data are ubiquitous in Large Language Model (LLM)-based agents (e.g., OpenClaw and Manus). A few recent works have attempted to exploit agents'memory for improving their performance on the question-answering (QA) task, but they lack a principled mechanism for effectively modeling how memory data evolves over time and retrieving memory data effectively, leading to poor performance in memory utilization. To fill this gap, we present H-Mem, a novel memory mechanism via a hybrid structure that can not only effectively model the evolution of agent memory over a long period of time, but also provide an efficient memory retrieval approach. Particularly, H-Mem builds a temporal and semantic tree structure that allows the short-term memory data to evolve progressively into long-term memory data, where the latter provides summarized information about the former, while simultaneously constructing a knowledge graph to capture the relationships between entities in memory. Moreover, it offers an effective memory retrieval approach by exploiting the hybrid structure of the tree and graph structures. Extensive experiments on three agent memory benchmarks show that H-Mem achieves state-of-the-art performance on the QA task.
Jiawei Yu, Yixiang Fang, Xilin Liu +1
The Chinese University of Hong Kong, Shenzhen · Huawei Cloud Computing Technologies CO., LTD.