MemAgent: Learning to Manage Heterogeneous Memory Providers for LLM Agents

Yongxian Wei; Yilin Zhao; Runxi Cheng; Xinrui Chen; Chun Yuan; Yaoru Wang; Jiahong Yan; Dian Li

cs.AISep 26, 2026

MemAgent: Learning to Manage Heterogeneous Memory Providers for LLM Agents

Authors: Yongxian Wei, Yilin Zhao, Runxi Cheng, Xinrui Chen, Chun Yuan, Yaoru Wang, Jiahong Yan, Dian Li

Organizations: Tsinghua University · Tencent

Abstract

Current agents remain largely stateless across tasks, limiting their ability to continually improve from prior interactions and making memory essential for long-horizon agentic behavior. Existing memory methods seek to reuse past experience, but most rely on a single memory representation (e.g., trajectories, reflections, skills, structured knowledge) whose effectiveness varies across task distributions. Rethinking this design space, we evaluate 13 memory methods and find that no single method generalizes across benchmarks, revealing the potential of managing heterogeneous memory providers. We formulate agent memory as a routing problem in which a memory agent decides which memory provider to retrieve from, whether to inject short-term memory, and which providers should store the resulting experience. Based on this perspective, we propose MemAgent, featuring a content-aware routing architecture and a training-data synthesis pipeline. The routing architecture combines content-aware probing before retrieval, short-term memory gating during execution, and selective multi-provider storage, while the training pipeline synthesizes phase-specific supervision for routing decisions. Across GAIA, WebWalkerQA, and xBench-DS, MemAgent improves average accuracy by 10.0% and outperforms every individual memory method across all three benchmarks. These gains come with less than 0.3% routing overhead and a 12% reduction in average task steps.

Figures & tables

Figure 1: The task agent interacts with tools to solve tasks, while experiences from past tasks are stored in heterogeneous memory providers. By routing retrieval to the appropriate provider, the memory agent helps the task agent solve the new task correctly in fewer steps.

Figure 2: Cross-benchmark gains of 13 memory methods over the no-memory baseline. No single method dominates across all three benchmarks. Stars mark the best method on each benchmark.

Figure 3: For each task, the memory agent $\mathcal{A}_{M}$ makes three routing decisions. (I) Begin : It previews all providers in $\mathcal{P}$ , selects one, and injects the retrieved long-term memory into the task agent $\mathcal{A}_{T}$ ’s prompt. (II) In : At each step, the short-term memory provider summarizes cross-step state from tool interactions $(a_{t},o_{t})$ , and $\mathcal{A}_{M}$ decides whether to inject the summary or skip. (III) End : After task completion, $\mathcal{A}_{M}$ selects a subset $\mathcal{S}\subseteq\mathcal{P}$ to store the trajectory $\xi_{\tau}$ for future reuse.

Figure 4: Training data synthesis pipeline for the memory agent. Forced exploration enumerates provider-action combinations for each task, producing trajectories $\Xi$ with binary outcomes $\{y_{\tau}^{(p,a)}\}$ . Supervision signals are derived from task correctness (for Begin ), LLM-as-Judge attribution (for Begin and In ), and downstream reuse (for End ). Finally, reward filtering retains high-quality context-decision pairs $(c,d)$ for the supervised fine-tuning of $\mathcal{A}_{M}$ .

Individual memory methods
Method	GAIA				WebWalkerQA	xBench-DS	Avg.
Method	Overall	Level 1	Level 2	Level 3	WebWalkerQA	xBench-DS	Avg.
No Memory	58.8	73.6	54.7	42.3	69.4	69.0	65.7
Agent-KB ( Tang et al., 2025 )	60.0 +1.2	73.6	54.7	50.0	70.0 +0.6	71.0 +2.0	67.0 +1.3
LightWeight ( Zhang et al., 2026a )	59.4 +0.6	67.9	61.6	34.6	73.5 +4.1	73.0 +4.0	68.6 +2.9
ExpeL ( Zhao et al., 2024 )	61.8 +3.0	73.6	61.6	38.5	77.1 +7.6	74.0 +5.0	71.0 +5.2

Table 1: Accuracy (%) on GAIA, WebWalkerQA, and xBench-DS. Gray deltas show percentage-point changes vs. the no-memory baseline. GAIA results are detailed by difficulty levels (

L1

–

L3

). Bold and underline denote the best and second-best memory methods.

Table 6

Memory injection (tokens inserted into context):
Component	Per-task cost	Notes
Begin (long-term retrieval)	$\sim$ 370 tokens	median 268, max 2.2K
In (short-term, per step)	$\sim$ 210 tokens	3.8 steps/task triggered
Total injected per task	$\sim$ 1.2K tokens	$<$ 0.5% of task tokens
Routing cost (MemAgent-specific):
Probe (13 providers, parallel)	$\sim$ 75 ms	one-time at Begin

Table 5: Inference overhead of MemAgent on GAIA. Injection sizes count memory tokens added to each task-agent prompt. Routing latency is MemAgent’s extra wall-clock cost per task.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Provider	Memory Paradigm	Storage Strategy	Retrieval Mechanism
Voyager ( Wang et al., 2024 )	Exploration memory	Refined trajectory summaries with embeddings	Semantic search over task similarity
ExpeL ( Zhao et al., 2024 )	Experience learning	Dual-path: insights from all outcomes + refined trajectories from successes	Hybrid insight + trajectory retrieval
Generative ( Park et al., 2023 )	Scored memory pool	Task queries + trajectory summaries	Vector retrieval + LLM importance scoring
DILU ( Wen et al., 2024 )	Query-trajectory mapping	Query embeddings + trajectory summaries	Semantic search on query similarity
AWM ( Wang et al., 2025c )	Workflow templates (SOPs)	LLM-induced abstract workflow patterns	Vector search over workflow templates
Mobile-E ( Wang et al., 2025b )	Keyword-indexed tips	Tips (text) + Shortcuts (structured sequences)	Keyword-based retrieval

Appendix

Table 6: Memory providers in MemAgent. The pool comprises 13 heterogeneous long-term providers spanning diverse memory paradigms, plus a short-term memory provider for within-task state aggregation. Storage Strategy: the specific information extracted and saved from past trajectories; Retrieval Mechanism: the method used to query relevant memories for new tasks.

Provider	Emb.	$k$	LLM-R	Key Parameters
Voyager ( Wang et al., 2024 )	✓	1	✗	Pure semantic search; returns 6-sentence max trajectory summary with hierarchical distillation
ExpeL ( Zhao et al., 2024 )	✓	3	✗	Hybrid search: TF-IDF 0.3, semantic 0.7; dual-path insight + trajectory retrieval
Generative ( Park et al., 2023 )	✓	3 $\to$ 1	✓	Semantic search returns 3 candidates; LLM importance scoring selects top-1; $<$ 150 words output
DILU ( Wen et al., 2024 )	✓	1	✗	Pure semantic search on query embeddings; ultra-concise step format ( $<$ 150 words)
AWM ( Wang et al., 2025c )	✓	1	✗	Pure embedding search; abstract workflow templates with entity generalization; $<$ 200 words
Mobile-E ( Wang et al., 2025b )	✗	2+2	✗	Jieba keyword tokenization; 2 tips + 2 shortcuts; structured action sequences

Appendix

Table 7: Implementation parameters for all 13 long-term memory providers. “Emb.”: uses a dense embedding model on the retrieval path; “LLM-R”: requires an LLM call on the full retrieval path;

k

: retrieval top-

k

, where

3{\to}1

denotes three dense candidates reranked by an LLM to one, and

2{+}2

denotes two tips plus two shortcuts. All 13 providers additionally invoke an LLM during storage. The content-aware Begin probe stage is embedding/keyword-only for all providers (no LLM).

Phase	Dimension	Weight	Dir.	Description
Begin	Utilization	0.6	$\uparrow$	Did the task agent reference or use information from the retrieved memory in its subsequent actions?
Begin	Relevance	0.4	$\uparrow$	How relevant was the retrieved memory to the specific task?
In ( Query )	Helpfulness	0.4	$\uparrow$	Did the agent use information from the injected short-term memory?
	Progress	0.4	$\uparrow$	Did the injection help advance the task?
	Noise	0.2	$\downarrow$	Did the injection distract or confuse the agent?
In ( NoOp )	Appropriateness	0.5	$\uparrow$	Was skipping retrieval a reasonable choice given the current state?

Appendix

Table 8: LLM-as-Judge scoring dimensions and weights by decision phase. Dimensions marked

\downarrow

are inverted before aggregation:

\hat{s}_{i}=1-s_{i}/s_{\max}

Phase	Dimension	0 (lowest)	2–3 (borderline)	5 (highest)
Begin	Utilization	Memory completely ignored	Some overlap, likely coincidental	Agent directly built upon specific memory content
Begin	Relevance	Completely irrelevant domain	Somewhat related, not specifically useful	Perfectly targeted, critical information
In (Query)	Helpfulness	No reference to injected content	Minor overlap, uncertain attribution	Agent directly leveraged specific injected details
In (Query)	Noise $\downarrow$	No distraction	Some off-topic exploration triggered	Agent clearly confused, pursued wrong direction
In (NoOp)	Appropriateness	STM had highly relevant content that should have been retrieved	Borderline; STM has marginal content	STM empty or clearly irrelevant; NoOp optimal
In (NoOp)	Missed Opp. $\downarrow$	Nothing valuable missed	Missed non-critical information	Missed critical information; retrieval would have been breakthrough

Appendix

Table 9: Representative rubric anchors for selected dimensions. All dimensions follow a 0–5 integer scale. Dimensions marked

\downarrow

are inverted before aggregation.

Section	Min to Show	Max Items	Priority
Collected Data	2 facts	8	Highest
Pages Visited	3 URLs	15	Medium
Recent Searches	4 queries	8	Lowest

Appendix

Table 10: Short-term memory output sections, display thresholds, and capacity limits.

Provider	Storage Description	Retrieval Description	Gate
LightWeight	Extracts strategic (planning/decisions) and operational (tools/techniques) memories after task success	Provides long-term strategic/operational experience	Success
Agent-KB	Summarizes trajectories into structured fields: Agent Planning, Search Agent Planning, Agent Experience	Student Guidance (reference plans) and Teacher Guidance (lessons learned)	Success
SkillWeaver	Analyzes trajectories to identify core algorithms, generates generalized Python functions	Matches relevant skills, dynamically wraps them as tools	Success
ExpeL	Extracts Insights (Tips or Warnings) from all outcomes; stores refined trajectories on success	Hybrid retrieval of Insights and successful trajectories	All
Voyager	Generates trajectory summaries from all outcomes, stored as JSON with embeddings	Semantic search for most similar historical tasks	All
DILU	Stores query embeddings and trajectory summaries from both successes and failures	Semantic search on query similarity	All

Appendix

Table 11: Provider descriptions injected into the memory agent prompts. The Begin prompts incorporate both storage and retrieval descriptions, whereas the End prompts include only the storage description accompanied by an outcome gate annotation.

Method	Overall	L1	L2	L3
Qwen3.5-4B w/o Training	63.6	71.7	67.4	34.6
MemAgent (Qwen3.5-4B)	69.7	83.0	67.4	50.0
Qwen3.5-9B w/o Training	63.6	75.5	61.6	46.2
MemAgent (Qwen3.5-9B)	70.9	81.1	69.8	53.8

Appendix

Table 12: Router-capacity analysis on GAIA. Accuracy (%).

Correct	GAIA	WebWalkerQA	xBench-DS
0 (unsolvable)	12.7%	10.6%	6.0%
1–2 (routing-critical)	12.1%	6.5%	8.0%
3–6	11.5%	8.2%	7.0%
7–13	28.5%	23.5%	44.0%
14 (all correct)	35.2%	51.2%	35.0%

Appendix

Table 13: Task coverage distribution: the number of methods (13 memory providers + no-memory baseline) that solve each task.

Panel of Figure 5 — Figure 5: Failure analysis and task solvability across benchmarks. Left: among failed tasks, coverage ceiling is the dominant failure source. Right: GAIA is more routing-critical than WebWalkerQA or xBench-DS, with more tasks solved by at most two of the 14 configurations.

Method	Tokens injected	Avg. steps	$\Delta$ Steps
No Memory	N/A	9.1	N/A
InsightGraph	57	8.6	$-$ 0.5
LightWeight	154	8.3	$-$ 0.8
Voyager	178	8.3	$-$ 0.8
DILU	184	8.3	$-$ 0.8
Generative	195	8.5	$-$ 0.6

Appendix

Table 14: Added context size in the Begin phase and average trajectory length on GAIA. SkillWeaver is reported as “tools only” because it exposes retrieved skills as callable tools rather than injecting plain text, so a token count is not directly comparable.

Family	Members	Indexable unit
Similarity-retrieval	Voyager, DILU, Generative, Cheatsheet	Trajectory or query embedding
Structural-abstraction	AWM, SkillWeaver, Memp	Workflow template, executable function, or procedural script
Insight-accumulation	ExpeL, Agent-KB, InsightGraph, Mobile-E, EvolveR, LightWeight	Atomic, composable insight fragments

Appendix

Table 15: Three representational families. Each family optimizes a different notion of cross-task reuse and, consequently, fails in different regimes.

LLM Agent Memory

Large language model agent memory lets agents retain, organize, retrieve, and update information across interactions and long-running tasks.

See all 373 papers

Memory in the LLM Era: Modular Architectures and Strategies in a Unified Framework
Apr 2, 2026Yanchen Wu, Tenghui Lin, Yingli Zhou +7LLM Agent Memory Agent Memory Management
CUHK-Shenzhen · CUHK · HITSZ +2
AdMem: Advanced Memory for Task-solving Agents
Jun 5, 2026Runzhe Wang, Huilin Lu, Shengjie Liu +2Continual Learning for LLM Agents LLM Agent Memory
Princeton University · Amazon · Arm
H-Mem: A Novel Memory Mechanism for Evolving and Retrieving Agent Memory via a Hybrid Structure
May 15, 2026Jiawei Yu, Yixiang Fang, Xilin Liu +1Agent Memory LLM Agent Memory
The Chinese University of Hong Kong, Shenzhen · Huawei Cloud Computing Technologies CO., LTD.

Apr 2, 2026J/K move · Enter open · S save

Memory in the LLM Era: Modular Architectures and Strategies in a Unified Framework

Yanchen Wu, Tenghui Lin, Yingli Zhou, Fangyuan Zhang+6

CUHK-Shenzhen · CUHK · HITSZ+2

LLM Agent Memory Agent Memory Management Long-Term Conversational Memory Memory Agent Benchmarks

Open paper AskPDF arXiv

Memory emerges as the core module in the large language model (LLM)-based agents for long-horizon complex tasks (e.g., multi-turn dialogue, game playing, scientific discovery), where memory can enable knowledge accumulation, iterative reasoning and self-evolution. A number of memory methods have been proposed in the literature. However, these methods have not been systematically and comprehensively compared under the same experimental settings. In this paper, we first summarize a unified framework that covers existing representative agent memory methods from a high-level perspective. We then extensively compare representative agent memory methods on two long-term conversational benchmarks and an agentic memory benchmark, and examine the effectiveness of representative methods, providing a thorough analysis of those methods. As a byproduct of our experimental analysis, we also design a new memory method by exploiting modules in the existing methods, which outperforms the state-of-the-art methods. Finally, based on these findings, we offer promising future research opportunities. We believe that a deeper understanding of the behavior of existing methods can provide valuable new insights for future research.

Abstract

LLM Agent Memory

Explore similar work

Memory in the LLM Era: Modular Architectures and Strategies in a Unified Framework

AdMem: Advanced Memory for Task-solving Agents

H-Mem: A Novel Memory Mechanism for Evolving and Retrieving Agent Memory via a Hybrid Structure