GAM: Hierarchical Graph-based Agentic Memory for LLM Agents

Zhaofen Wu; Hanrong Zhang; Fulin Lin; Wujiang Xu; Xinran Xu; Yankai Chen; Henry Peng Zou; Shaowen Chen

cs.AIApr 14, 2026

GAM: Hierarchical Graph-based Agentic Memory for LLM Agents

Authors: Zhaofen Wu, Hanrong Zhang, Fulin Lin, Wujiang Xu, Xinran Xu, Yankai Chen, Henry Peng Zou, Shaowen Chen,

Organizations: Zhejiang University · University of Illinois Chicago · Rutgers University · MBZUAI ·

Abstract

To sustain coherent long-term interactions, Large Language Model (LLM) agents must navigate the tension between acquiring new information and retaining prior knowledge. Current unified stream-based memory systems facilitate context updates but remain vulnerable to interference from transient noise. Conversely, discrete structured memory architectures provide robust knowledge retention but often struggle to adapt to evolving narratives. To address this, we propose GAM, a hierarchical Graph-based Agentic Memory framework that explicitly decouples memory encoding from consolidation to effectively resolve the conflict between rapid context perception and stable knowledge retention. By isolating ongoing dialogue in an event progression graph and integrating it into a topic associative network only upon semantic shifts, our approach minimizes interference while preserving long-term consistency. Additionally, we introduce a graph-guided, multi-factor retrieval strategy to enhance context precision. Experiments on LoCoMo and LongDialQA indicate that our method consistently outperforms state-of-the-art baselines in both reasoning accuracy and efficiency.

Figures & tables

Figure 1: Comparison between Unified Stream-based Memory Systems (upper) and GAM (lower). Unified architectures suffer from Memory Loss and Semantic Drift under direct updates, while GAM uses state switching to reduce both.

Figure 2: Architecture of GAM. The leftmost module switches between buffering and consolidation based on topic boundaries. The central Hierarchical Graph Memory decouples storage into a global Topic Associative Network and local Event Progression Graphs. The rightmost Graph-Guided Retrieval performs top-down traversal with multi-factor ranking.

Model	Method	Multi-Hop		Temporal		Open-Domain		Single-Hop		Avg
Model	Method	F1	BLEU-1	F1	BLEU-1	F1	BLEU-1	F1	BLEU-1	F1	BLEU-1
Llama 3.2-3B	ReadAgent	2.05	1.96	4.33	4.33	7.53	6.39	3.52	2.75	4.36	3.86
	Memorybank	7.35	5.34	3.90	3.41	4.91	5.42	8.76	7.04	6.23	5.30
	Memgpt	7.59	5.09	3.25	3.15	5.74	5.55	7.91	6.64	6.12	5.11
	A-Mem	17.44	11.74	26.38	19.50	12.53	11.83	28.14	23.87	21.12	16.73
	MemoryOS	21.03	13.75	19.38	13.73	11.46	8.58	28.25	20.03	20.03	14.02

Table 1: Performance comparison on the LoCoMo dataset. We report F1 and BLEU-1 scores across different LLM backbones. The best results are highlighted in bold and the second best are underlined.

Model	Method	Avg
Model	Method	F1	BLEU-1
Llama 3.2-3B	ReadAgent	3.97	4.04
	Memorybank	3.93	3.60
	Memgpt	3.90	3.36
	A-Mem	4.11	3.27
	MemoryOS	5.85	4.46

Table 2: Performance comparison on the LongDialQA dataset. We report average F1 and BLEU-1 scores. The best results are highlighted in bold and the second best are underlined.

Method	Avg
Method	F1	B-1
w/o TAN	35.07	29.00
w/o MFR	35.94	29.28
w/o SSM	32.58	26.13
w/o EPG	25.06	20.76
Ours	40.00	32.99

Table 3: Ablation study on the LoCoMo dataset. We report average F1 and BLEU-1 scores for different variants.

Method	Tokens/Q	Time (s)	F1
A-Mem	4221.12	2.21	24.20
MemoryOS	3400.41	154.22	28.86
Mem0	1533.94	0.51	35.38
Ours	1370.18	0.80	40.00

Table 4: Efficiency comparison on the LoCoMo dataset. We report token consumption, latency, and F1 score.

Figure 3: Impact of Partitioning Strategies on LoCoMo. Our semantic-boundary approach outperforms heuristic baselines across metrics.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Method	Big Bang Theory		Friends		The Office		Avg
Model	Method	F1	B-1	F1	B-1	F1	B-1	F1	B-1
Llama 3.2-3B	ReadAgent	5.19	4.70	3.49	4.24	3.24	3.18	3.97	4.04
	Memorybank	4.77	4.46	4.14	3.77	2.87	2.56	3.93	3.60
	Memgpt	3.94	2.97	3.99	3.85	3.77	3.25	3.90	3.36
	A-Mem	5.69	4.20	3.79	3.32	2.86	2.29	4.11	3.27
	MemoryOS	8.65	6.46	2.98	3.56	5.92	3.35	5.85	4.46

Appendix

Table 5: Detailed performance comparison on the LongDialQA dataset, including the newly added Mem0 baseline. We report F1 and BLEU-1 scores on three sub-datasets (Big Bang Theory, Friends, The Office) and their average. The best results for each backbone are highlighted in bold, and the second best are underlined.

Method	Multi-Hop		Temporal		Open-Domain		Single-Hop		Avg
Method	F1	B-1	F1	B-1	F1	B-1	F1	B-1	F1	B-1
w/o TAN	31.80	21.29	39.74	34.15	16.28	13.42	52.46	47.12	35.07	29.00
w/o MFR	33.73	22.91	46.12	38.06	16.30	14.20	47.61	41.96	35.94	29.28
w/o SSM	30.32	19.96	39.53	32.55	16.73	14.08	43.72	37.93	32.58	26.13
w/o EPG	25.35	18.94	33.61	27.70	14.59	13.44	26.68	22.97	25.06	20.76
Ours	35.32	25.77	48.97	39.68	21.14	17.89	54.58	48.62	40.00	32.99

Appendix

Table 6: Detailed ablation study results across different task categories on the LoCoMo dataset.

Model	Method	Multi-Hop		Temporal		Open-Domain		Single-Hop		Avg
Model	Method	F1	B-1	F1	B-1	F1	B-1	F1	B-1	F1	B-1
Llama 3.2-3B	AriGraph	13.99	7.83	3.72	2.69	20.10	17.74	14.63	11.67	12.58	9.47
Llama 3.2-3B	Ours	28.74	22.35	46.12	37.24	21.18	18.33	49.02	42.18	36.27	30.02
Qwen 2.5-7B	AriGraph	21.32	12.30	5.51	5.25	15.10	12.40	24.44	19.49	19.34	14.77
Qwen 2.5-7B	Ours	35.32	25.77	48.97	39.68	21.14	17.89	54.58	48.62	40.00	32.99
Qwen 2.5-14B	AriGraph	29.40	18.20	6.00	5.01	16.87	13.65	30.68	24.38	24.44	18.54

Appendix

Table 7: Performance comparison with AriGraph on the LoCoMo benchmark. We report F1 and BLEU-1 for each reasoning category and their average.

Method	Accuracy	F1	B-1
Baseline (Direct Input)	47.35	42.35	45.31
Ours (GAM)	55.51	48.95	51.63
Improvement	+8.16	+6.60	+6.32

Appendix

Table 8: Proxy multimodal evaluation on MovieChat-1K using dense video captions as the input stream.

Metric	Value
Avg. Sessions per Sample	27.2
Avg. Latency per Session	0.63 s
Avg. Input Tokens per Session	932.36
Avg. Output Tokens per Session	6.15

Appendix

Table 9: Cost analysis of the semantic divergence detection mechanism on LoCoMo.

Sessions	Event Nodes	Topic Nodes	Edges	Avg. Tokens	Avg. Latency (s)	Avg. F1
3	81	8	371	1604	3.49	0.65
15	363	43	1791	1864	4.16	0.57
27	657	84	3307	1836	4.71	0.48

Appendix

Table 10: Complexity evolution of the dual-layer graph memory under progressive session injection.

Figure 4: Robustness analysis of the proposed framework under varying topic segmentation noise levels

\eta

. The solid red line represents the Average F1 score, which remains remarkably stable around 40.0, significantly outperforming the Fixed Window (Gray Dotted Line) and Session-based (Black Dash-Dotted Line) baselines even at

\eta=0.4

. Dashed lines indicate performance on specific task categories, highlighting the resilience of the hierarchical graph architecture against boundary errors.

Figure 5: Parameter sensitivity analysis on the LoCoMo dataset using the Qwen 2.5-7B backbone. The three subplots illustrate the impact of varying the (a) Role Factor

\beta_{role}

, (b) Temporal Factor

\beta_{time}

, and (c) Confidence Factor

\beta_{conf}

on retrieval performance across different task categories. The solid red line represents the Average F1 Score. The relatively flat curves indicate the system’s robustness to hyperparameter variations.

Figure 6: Impact of retrieval size

k

on model performance across different task categories. We select

k=10

as the optimal setting to balance reasoning accuracy with computational efficiency.

Agent Memory Management

Agent memory management studies how AI agents store, update, retrieve, compress, and forget information across long-running tasks.

See all 446 papers

AMA: Adaptive Memory via Multi-Agent Collaboration
Jan 28, 2026Weiquan Huang, Zixuan Wang, Hehai Lin +6Long-Term Conversational Memory Multi-Agent LLMs
The Hong Kong University of Science and Technology (Guangzhou) · Shandong University · Nanyang Technological University +1
Retrieval-Driven Memory Reconsolidation for Long-Term LLM Agents
Sep 13, 2026Yuanyi Song, Yukai Wang, Xinbei Ma +7LLM Agent Memory Long-Term Conversational Memory
Shanghai Jiao Tong University · OPPO · National University of Singapore
Memory is Reconstructed, Not Retrieved: Graph Memory for LLM Agents
Jun 4, 2026Shuo Ji, Yibo Li, Bryan HooiAgent Memory LLM Agent Memory
National University of Singapore, Singapore.

Jan 28, 2026J/K move · Enter open · S save

AMA: Adaptive Memory via Multi-Agent Collaboration

Weiquan Huang, Zixuan Wang, Hehai Lin, Sudong Wang+5

The Hong Kong University of Science and Technology (Guangzhou) · Shandong University · Nanyang Technological University+1

Long-Term Conversational Memory Multi-Agent LLMs Agent Memory Management

Open paper AskPDF arXiv

The rapid evolution of Large Language Model (LLM) agents has necessitated robust memory systems to support cohesive long-term interaction and complex reasoning. Benefiting from the strong capabilities of LLMs, recent research focus has shifted from simple context extension to the development of dedicated agentic memory systems. However, existing approaches typically rely on rigid retrieval granularity, accumulation-heavy maintenance strategies, and coarse-grained update mechanisms. These design choices create a persistent mismatch between stored information and task-specific reasoning demands, while leading to the unchecked accumulation of logical inconsistencies over time. To address these challenges, we propose Adaptive Memory via Multi-Agent Collaboration (AMA), a novel framework that leverages coordinated agents to manage memory across multiple granularities. AMA employs a hierarchical memory design that dynamically aligns retrieval granularity with task complexity. Specifically, the Constructor and Retriever jointly enable multi-granularity memory construction and adaptive query routing. The Judge verifies the relevance and consistency of retrieved content, triggering iterative retrieval when evidence is insufficient or invoking the Refresher upon detecting logical conflicts. The Refresher then enforces memory consistency by performing targeted updates or removing outdated entries. Extensive experiments on challenging long-context benchmarks show that AMA significantly outperforms state-of-the-art baselines while reducing token consumption by approximately 80% compared to full-context methods, demonstrating its effectiveness in maintaining retrieval precision and long-term memory consistency.

Abstract

Agent Memory Management

Explore similar work

AMA: Adaptive Memory via Multi-Agent Collaboration

Retrieval-Driven Memory Reconsolidation for Long-Term LLM Agents

Memory is Reconstructed, Not Retrieved: Graph Memory for LLM Agents