Memory in the LLM Era: Modular Architectures and Strategies in a Unified Framework

Yanchen Wu; Tenghui Lin; Yingli Zhou; Fangyuan Zhang; Qintian Guo; Xun Zhou; Sibo Wang; Xilin Liu

cs.CLApr 2, 2026

Memory in the LLM Era: Modular Architectures and Strategies in a Unified Framework

Authors: Yanchen Wu, Tenghui Lin, Yingli Zhou, Fangyuan Zhang, Qintian Guo, Xun Zhou, Sibo Wang, Xilin Liu, +2 more

Organizations: CUHK-Shenzhen · CUHK · HITSZ · BIT · Huawei Cloud

Abstract

Memory emerges as the core module in the large language model (LLM)-based agents for long-horizon complex tasks (e.g., multi-turn dialogue, game playing, scientific discovery), where memory can enable knowledge accumulation, iterative reasoning and self-evolution. A number of memory methods have been proposed in the literature. However, these methods have not been systematically and comprehensively compared under the same experimental settings. In this paper, we first summarize a unified framework that covers existing representative agent memory methods from a high-level perspective. We then extensively compare representative agent memory methods on two long-term conversational benchmarks and an agentic memory benchmark, and examine the effectiveness of representative methods, providing a thorough analysis of those methods. As a byproduct of our experimental analysis, we also design a new memory method by exploiting modules in the existing methods, which outperforms the state-of-the-art methods. Finally, based on these findings, we offer promising future research opportunities. We believe that a deeper understanding of the behavior of existing methods can provide valuable new insights for future research.

Figures & tables

Figure 1. Overview of naive long-context prompting and memory-augmented prompting.

Method	Information Extraction	Management Operations	Storage Structure	Retrieval Mechanism
A-MEM ( Wujiang et al., 2025 )	Direct archive, Summarization-based extract	Connect, Update	Flat, Vector	Vector-Based
MemoryBank ( Zhong et al., 2024 )	Direct archive	Integrate, Update, Filter	Flat, Vector	Vector-Based
MemGPT ( Packer et al., 2023 )	Direct archive	Integrate, Transform, Update	Hierarchical, Vector	Lexical-Based, Vector-Based
Mem0 ( Chhikara et al., 2025 )	Direct archive, Summarization-based extract	Integrate, Update, Filter	Flat, Vector	Vector-Based
Mem0 g ( Chhikara et al., 2025 )	Graph-based extract	Connect, Update, Filter	Flat, Graph	Vector-Based, Structure-Based
MemoChat ( Lu et al., 2023 )	Direct archive	Integrate	Flat	LLM-Assisted

Table 1. Classification of representative agent memory methods.

Figure 2. An overview of the unified framework for agent memory systems.

Figure 3. Methods of information extraction.

Panel of Figure 4 — Figure 4. A sample prompt for summarization-based extraction.

Figure 5. Workflow of the memory management process.

Method	Connecting	Integrating	Transforming	Updating	Filtering
A-MEM	Associative Links	N/A	N/A	LLM-based	N/A
MemoryBank	N/A	Summarization	N/A	Rule-based	Usage-based
MemGPT	N/A	Summarization	Stage-wise Transfer	Agent-based	N/A
Mem0	N/A	Summarization	N/A	Agent-based	Content-based
Mem0 g	Structural Edges	N/A	N/A	LLM-based	Content-based
MemoChat	N/A	Abstraction	N/A	N/A	N/A

Table 2. Comparison of implementation paradigms for memory management operations; “N/A” means that this operation is not explicitly implemented.

Method	Information Extraction						Multi-Session		Temporal		Knowledge Updates		Overall
	user		assistant		preference		Multi-Session		Temporal		Knowledge Updates		Overall
	F1	BLEU-1	F1	BLEU-1	F1	BLEU-1	F1	BLEU-1	F1	BLEU-1	F1	BLEU-1	F1	BLEU-1
Qwen3.5-9B
A-MEM	40.15	35.99	43.51	36.02	11.72	0.91	17.08	14.58	22.91	16.47	31.91	29.05	26.81	21.92
MemoryBank	56.17	46.14	66.64	55.02	20.67	9.54	17.70	10.73	22.41	14.75	37.20	27.84	33.04	24.32
MemGPT	55.54	49.79	54.67	46.00	13.63	3.82	19.19	17.19	22.30	15.86	29.54	27.07	30.36	25.37

Table 3. Comparison of methods on LONGMEMEVAL. The best and second-best results are marked in bold and underlined, respectively.

Method	Single-Hop			Multi-Hop			Temporal			Open-Domain			Overall
Method	F1	BLEU-1	Acc	F1	BLEU-1	Acc	F1	BLEU-1	Acc	F1	BLEU-1	Acc	F1	BLEU-1	Acc
Qwen3.5-9B
A-MEM	34.38	30.22	53.75	23.26	15.18	52.13	18.30	13.12	52.02	12.38	8.78	39.58	27.62	22.57	52.21
MemoryBank	45.05	39.09	71.70	36.93	28.77	78.01	27.29	22.20	54.52	13.83	10.05	38.54	37.92	31.87	67.21
MemGPT	38.97	33.32	54.34	25.11	17.45	50.00	16.57	13.47	41.74	14.99	10.99	34.38	30.27	24.88	49.68
Mem0	34.66	29.75	63.26	29.69	21.36	68.79	42.68	36.97	61.68	20.66	14.70	45.83	34.55	28.78	62.86

Table 4. Comparison of methods on LOCOMO. The best and second-best results are marked in bold and underlined, respectively.

Panel of Figure 6 — Figure 6. Performance and efficiency analysis on LOCOMO.

Panel of Figure 7 — Figure 7. Robustness analysis of memory mechanisms on LONGMEMEVAL. (a) illustrates the context scalability as the input scale varies from 50% to 200%, while (b) shows the position sensitivity regarding ground-truth information placed at different relative positions: Early (first 1/3), Middle (middle 1/3), and Late (last 1/3).

	Without Query-Time LLM Calls									With Query-Time LLM Calls
Method	A-MEM	MemoryBank	MemGPT	Mem0	Zep	MemTree	MemOS	MemGAS	LightMem	Mem0 g	MemoChat	MemoryOS
Latency (ms)	617.1	864.8	105.8	356.4	536.8	24.5	346.7	793.4	78.1	3783.1	1792.4	2329.6

Table 5. Average retrieval latency per query (ms).

Model	Metric	Zep	MemoryOS	MemOS	LightMem
Qwen3.5-9B	F1	42.89	37.36	42.21	34.70
Qwen3.5-9B	BLEU-1	36.94	29.92	36.65	28.18
Qwen3.5-27B	F1	46.62	44.36	48.88	39.67
Qwen3.5-27B	BLEU-1	39.88	36.92	41.35	32.55
DeepSeek-V4-Flash	F1	43.16	42.82	48.52	38.88
DeepSeek-V4-Flash	BLEU-1	38.71	35.31	40.11	31.69

Table 6. Comparison across LLM backbones on LOCOMO.

Figure 8. Comparison of our newly designed method in terms of average token costs.

	LOCOMO		LONGMEMEVAL
Method	F1	BLEU-1	F1	BLEU-1
Zep	42.89	36.94	—	—
MemTree	38.53	31.01	41.24	34.69
MemoryOS	37.36	29.92	41.02	35.61
MemOS	42.21	36.65	37.29	30.66
LightMem	34.70	28.18	39.31	32.49

Table 7. Comparison of our newly designed method on LOCOMO and LONGMEMEVAL.

Method Variants	SH	MH	TR	OK	Overall
Ours	49.63	36.94	39.86	19.69	43.40
w/o Tree	45.82	35.23	31.89	14.25	39.01
w/o SP	47.97	34.99	35.24	19.20	41.15
w/o HM & SP ( MemTree )	43.42	34.21	33.08	26.54	38.53
w/o Tree & SP ( MemoryOS )	43.58	35.64	28.05	18.98	37.36
w/o Tree, w/ Graph	45.52	33.68	40.06	19.59	40.60

Table 8. Component ablations and organization variants measured by F1 score on LOCOMO.

Method	Bundled Web Shopping		Group Travel Planning			Progressive Web Search		Formal Reasoning				All Task Avg SR
	Bundled Web Shopping		Group Travel Planning			Progressive Web Search		Math		Phys
	SR	PS	SR	PS	sPS	Acc	#Search	SR	PS	SR	PS
A-MEM	0.00	31.03	0.00	0.27	6.32	2.26	7.04	10.00	25.92	35.00	56.54	9.45
MemoryBank	0.00	32.08	0.00	0.21	5.97	5.43	6.09	17.50	24.49	35.00	54.04	11.59
MemGPT/Letta	0.00	36.45	0.00	0.00	4.82	3.62	7.09	20.00	28.58	20.00	49.04	8.72
Mem0	0.00	31.91	0.00	0.00	7.48	5.43	7.78	10.00	23.49	30.00	49.18	9.09

Table 9. Comparison of methods on MemoryArena. The best and second-best results are marked in bold and underlined, respectively.

Figure 9. F1 performance of different memory methods under varying retrieval top-

k

values.

Work	Primary Type	Design Taxonomy	Unified Multi-method Evaluation	Component–Performance Linkage	Evidence-guided Design Synthesis
Memory in the Age of AI Agents ( Hu et al., 2025b )	Survey	✓	✗	✗	✗
The AI Hippocampus: How Far Are We From Human Memory? ( Jia et al., 2026 )	Survey	✓	✗	✗	✗
A Survey on the Memory Mechanism of Large Language Model Based Agents ( Zhang et al., 2025 )	Survey	✓	✗	✗	✗
Rethinking Memory Mechanisms of Foundation Agents in the Second Half ( Huang et al., 2026 )	Survey	✓	✗	✗	✗
LOCOMO ( Maharana et al., 2024 )	Benchmark	✗	✗	✓	✗
LongMemEval ( Wu et al., 2025b )	Benchmark	✓	✗	✓	✓

Table 10. Positioning of our work against representative benchmarks and surveys on agent memory.

Figure 10. Task-Oriented Roadmap for Selecting Long-Term Memory Architectures under Different Scenarios.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 11 — Figure 11. A sample prompt for graph-based extraction.

Panel of Figure 12 — Figure 12. Prompt for answer simplification.

Figure 13. Context scalability of various memory methods across different task categories on LONGMEMEVAL.

Panel of Figure 14 — Figure 14. Prompt for LLM-as-a-judge.

Method	Position	user	assistant	preference
Mem0 g	Early	63.80	23.35	11.39
	Middle	57.60	20.39	12.87
	Late	74.60	20.74	12.90
	Improvement	+10.80	-2.61	+1.51
MemTree	Early	59.01	55.42	10.42
	Middle	66.91	67.61	12.07

Appendix

Table 11. Position sensitivity analysis of representative memory mechanisms across Information Extraction sub-tasks on LONGMEMEVAL (F1 Score).

Figure 15. The framework of our newly designed method.

Figure 16. An example where a correctly generated answer is incorrectly evaluated by an LLM judge.

Question:	When did Caroline go to the LGBTQ support group?
Answer:	7 May 2023
Response type	Predicted answer	F1	BLEU-1
Correct (before simplification)	Caroline went to the LGBTQ support group on May 7, 2023.	0.3750	0.2307
Correct (after simplification)	May 7, 2023	0.8571	0.7500
Incorrect	8 May 2023	0.6667	0.6667

Appendix

Table 12. Example of the effect of answer simplification on F1 and BLEU-1 scores.

	F1			BLEU-1
Method	Before	After	$\Delta$	Before	After	$\Delta$
A-MEM	26.90	27.62	+0.72	21.98	22.57	+0.59
Mem0	31.40	34.55	+3.15	25.32	28.78	+3.80
Zep	35.93	42.89	+6.96	30.78	36.94	+6.16
MemTree	35.36	38.53	+3.17	29.88	31.01	+1.13
MemoryOS	35.72	37.36	+1.64	26.15	29.92	+3.77

Appendix

Table 13. F1 and BLEU-1 scores of different methods before and after answer simplification on LOCOMO.

Organization	Vector	Graph	Tree
Flat	MemoryBank	Mem0 g	MemTree
Hierarchical	MemoryOS	Zep	MemOS

Appendix

Table 14. Classification of the selected memory baselines by storage organization and memory representation.

Figure 17. Classification and coverage of the evaluated token-level memory baselines following the taxonomy in Memory in the Age of AI Agents: A Survey.

Explore similar work

EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective
May 18, 2026Yuyao Wang, Zhongjian Zhang, Mo Chi +7LLM Agent Memory LLM Agent Evaluation
Hong Kong University of Science and Technology (Guangzhou) · Beijing University of Posts and Telecommunications · Beijing Institute of Technology +1
AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications
Feb 26, 2026Yujie Zhao, Boqin Yuan, Junbo Huang +9Agent Memory Long-Horizon Agent Evaluation
UCSD · Recursive.
AdMem: Advanced Memory for Task-solving Agents
Jun 5, 2026Runzhe Wang, Huilin Lu, Shengjie Liu +2Continual Learning for LLM Agents LLM Agent Memory
Princeton University · Amazon · Arm

May 18, 2026J/K move · Enter open · S save

EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective

Yuyao Wang, Zhongjian Zhang, Mo Chi, Kaichi Yu+6

Hong Kong University of Science and Technology (Guangzhou) · Beijing University of Posts and Telecommunications · Beijing Institute of Technology+1

LLM Agent Memory LLM Agent Evaluation AI Agent Benchmarks Memory Agent Benchmarks Agent Benchmarks

Open paper AskPDF arXiv

Recent benchmarks for Large Language Model (LLM) agents mainly evaluate reasoning, planning, and execution. However, memory is also essential for agents, as it enables them to store, update, and retrieve information over time. This ability remains under-evaluated, largely because existing benchmarks do not provide a systematic way to assess memory mechanisms. In this paper, we study agent memory from a self-evolving perspective and introduce EvoMemBench, a unified benchmark organized along two axes: memory scope (in-episode vs. cross-episode) and memory content (knowledge-oriented vs. execution-oriented). We compare 15 representative memory methods with strong long-context baselines under a standardized protocol. Results show that current memory systems are still far from a general solution: long-context baselines remain highly competitive, memory helps most when the current context is insufficient or tasks are difficult, and no single memory form works consistently across all settings. Retrieval-based methods remain strong for knowledge-intensive settings, whereas procedural and long-term memory methods are more effective for execution-oriented tasks when their stored experience matches the task structure. We hope EvoMemBench facilitates future research on more effective memory systems for LLM-based agents. Our code is available at https://github.com/DSAIL-Memory/EvoMemBench.

Abstract

Explore similar work

EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective

AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications

AdMem: Advanced Memory for Task-solving Agents