Selective Forgetting: A Graph-Based Memory Framework for Long-Term LLM Agents

Theo Rusu; Sourena Khanzadeh; Manar Alalfi

cs.AIAug 29, 2026

Selective Forgetting: A Graph-Based Memory Framework for Long-Term LLM Agents

Authors: Theo Rusu, Sourena Khanzadeh, Manar Alalfi

Organizations: Department of Computer Science Toronto Metropolitan University Toronto, Ontario, Canada · The Creative School Toronto Metropolitan University Toronto, Ontario, Canada · Flybits, Creative AI Hub Toronto, Ontario, Canada

Abstract

Knowledge graphs have been proposed as a structured alternative to flat retrieval-augmented generation for long-term agent memory, on the assumption that representing conversations as entities and relations improves recall. We evaluate that assumption directly. Our framework extracts each conversational turn into typed nodes and attributed edges, answers questions from a two-hop subgraph, and periodically prunes nodes that score low on a weighted combination of recency, access frequency, degree centrality, and age. On LongMemEval, the graph does not outperform a flat vector baseline at a matched candidate-generation budget of five retrieval roots: token F1 is $0.417$ against $0.468$ , and a paired bootstrap over 500 questions gives $Δ= -0.050$ (95% CI $[-0.085, -0.016]$ ). The gap is widest on questions that require recalling a specific prior assistant turn, where judged correctness falls from $0.911$ to $0.607$ , suggesting that decomposing a turn into entities discards the surface form these questions depend on. The forgetting module is more successful. Applied once to a persistent 27{,}021-node graph, it removes 9.8% of nodes and 9.5% of stored bytes; token F1 is unchanged ( $+0.001$ , 95% CI $[-0.015, +0.016]$ ) and judged correctness falls by $1.6$ points, with the 95% interval bounding any loss at $3.8$ points ( $[-0.038, +0.006]$ ). Because our extractor is a single small model evaluated on one benchmark, these results characterise this extraction-based pipeline rather than graph-structured memory in general. Code: https://github.com/skhanzad/Selective-Amnesia

Figures & tables

Panel of Figure 1 — Figure 1: Overview of the proposed memory framework. Each conversational turn is processed by the update stage, which extracts, embeds, de-duplicates, and writes nodes and edges into the persistent knowledge graph. At question time, the retrieval stage embeds the referenced entities, selects the top-5 matching nodes ranked by cosine similarity, and expands them via a 2-hop subgraph traversal to build the answer-generation context. Every 400 turns, the retention stage scores each node by recency, access frequency, centrality, and turn age, pruning nodes whose importance falls below the threshold together with their incident edges.

Type	n	P	F1	J	n	P	F1	J
	Graph RAG (Ours)				Baseline RAG
Single (user)	70	0.743	0.737	0.771	70	0.823	0.819	0.929
Single (asst.)	56	0.680	0.575	0.607	56	0.916	0.774	0.911
Single (pref.)	30	0.312	0.083	0.233	30	0.274	0.114	0.367
Know.-update	78	0.507	0.456	0.513	78	0.553	0.511	0.590
Multi-session	133	0.384	0.326	0.398	133	0.381	0.342	0.436

Table 1: Experiment 1 results: Graph RAG vs. Baseline RAG on LongMemEval. J = LLM-judge; Single = single-session.

Variant	Nodes	Edges	Size	F1	Judge
No-forgetting	27,021	46,538	440.6 MB	0.292	0.300
Forgetting	24,368	43,978	398.6 MB	0.293	0.284
Change	$-$ 9.8%	$-$ 5.5%	$-$ 9.5%	$+$ 0.001	$-$ 0.016

Table 2: Experiment 2 results: storage and retrieval quality for the persistent graph with and without forgetting.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Value	Basis
De-duplication cosine threshold $\tau_{\text{dedup}}$	0.92	E
Retrieval cosine floor $\tau_{\text{ret}}$	0.75	E
Retrieval roots (top- $k$ )	5	C
Subgraph expansion depth	2 hops	B
Subgraph node cap	15	B
Forgetting interval	400 turns	B

Appendix

Table 3: Parameters, settings, and basis for selection. E = empirical probe, B = budget or cost constraint, C = convention or symmetry with the baseline.

Stage	Cost	Note
Embed extracted entities	$O(kd)$
Title de-duplication	$O(1)$ per entity	hash index
Embedding de-duplication	$O(kNd)$	linear scan
Graph write	$O(k+\ell)$	indexed updates
Candidate retrieval	$O(qNd)$	linear scan
Subgraph traversal	$O(N+M+\|V_{S}\|+\|E_{S}\|)$	adjacency build

Appendix

Table 4: Per-stage complexity under exhaustive vector search.

T

is the forgetting interval (400 turns), and

V_{S},E_{S}

are the nodes and edges of the retrieved subgraph.

Condition	F1	Precision	Recall	Judge acc.
Baseline RAG (per-question, Experiment 1 control)	$0.468\pm 0.019$	$0.532\pm 0.019$	$0.486\pm 0.020$	$0.536\pm 0.022$
Graph RAG (per-question, Experiment 1 treatment)	$0.417\pm 0.019$	$0.505\pm 0.020$	$0.412\pm 0.019$	$0.454\pm 0.022$
Persistent graph, no forgetting (Experiment 2 control)	$0.292\pm 0.017$	$0.363\pm 0.018$	$0.295\pm 0.018$	$0.300\pm 0.021$
Persistent graph, with forgetting (Experiment 2 treatment)	$0.293\pm 0.017$	$0.359\pm 0.018$	$0.294\pm 0.018$	$0.284\pm 0.020$

Appendix

Table 5: Mean

\pm

SEM (1

\sigma

) and 95% bootstrap CI for each condition,

n=500

questions per condition.

Condition	F1	Precision	Recall	Judge acc.
Baseline RAG (per-question, Experiment 1 control)	$[0.432,0.505]$	$[0.496,0.571]$	$[0.447,0.525]$	$[0.492,0.580]$ (Wilson: $[0.492,0.579]$ )
Graph RAG (per-question, Experiment 1 treatment)	$[0.381,0.454]$	$[0.467,0.544]$	$[0.375,0.451]$	$[0.410,0.496]$ (Wilson: $[0.411,0.498]$ )
Persistent graph, no forgetting (Experiment 2 control)	$[0.260,0.325]$	$[0.328,0.399]$	$[0.261,0.330]$	$[0.260,0.342]$ (Wilson: $[0.262,0.342]$ )
Persistent graph, with forgetting (Experiment 2 treatment)	$[0.260,0.326]$	$[0.323,0.395]$	$[0.259,0.329]$	$[0.244,0.324]$ (Wilson: $[0.246,0.325]$ )

Appendix

Table 6: 95% bootstrap confidence intervals corresponding to Table 5 . For judge accuracy, the 95% Wilson score interval is shown alongside the bootstrap CI.

Comparison	F1	Precision	Recall	Judge acc.
Experiment 1: Graph RAG vs Baseline RAG	$-0.050$ ∗ $[-0.085,-0.016]$	$-0.028$ $[-0.067,+0.011]$	$-0.073$ ∗ $[-0.107,-0.039]$	$-0.082$ ∗ $[-0.128,-0.034]$
Experiment 2: Forgetting vs No forgetting	$+0.001$ $[-0.015,+0.016]$	$-0.004$ $[-0.023,+0.014]$	$-0.001$ $[-0.017,+0.016]$	$-0.016$ $[-0.038,+0.006]$

Appendix

Table 7: Paired bootstrap comparisons between conditions. A positive mean difference favors the first-named condition; ∗ denotes significance at

\alpha=0.05

Graph-Based Agent Memory

Graph-based agent memory organizes an agent’s experiences, facts, and procedures as linked structures for persistent, evidence-aware retrieval and reuse.

See all 133 papers

TAGGRAPH: Tag-Augmented Graphs for Graph Retrieval of Agent Persistent Histories
Sep 29, 2026Yu-Shu Chen, Yu-Jung Liang, Pengtao XieGraph-Based Retrieval Memory Agent Benchmarks
Department of Electrical and Computer Engineering University of California, San Diego La Jolla, CA 92093, USA
LazyMem: Retrieve Broadly, Construct Selectively for Efficient Long-Term Agent Memory
Jul 17, 2026Jing Yu, Yibo Zhao, Jiaming Zhang +1Persistent Memory for Language Models Long-Term Conversational Memory
School of Data Science and Engineering, East China Normal University
A Graph-Native Bitemporal Memory Store for Conversational AI Agents
Jul 29, 2026Alp Niksarli, Gopesh BahetiConversational Memory LLM Agent Memory
Davidson College

Sep 29, 2026J/K move · Enter open · S save

TAGGRAPH: Tag-Augmented Graphs for Graph Retrieval of Agent Persistent Histories

Yu-Shu Chen, Yu-Jung Liang, Pengtao Xie

Department of Electrical and Computer Engineering University of California, San Diego La Jolla, CA 92093, USA

Graph-Based Retrieval Memory Agent Benchmarks Graph-Based Agent Memory Long-Term Conversational Memory

Open paper AskPDF arXiv

Long-term memory lets LLM agents recall past interactions and remain consistent across sessions, but memory systems are hard to compare because they often vary in representation, indexing, retrieval, and evaluation. We present a controlled evaluation framework based on shared 5W-style conversational memories. Localized graph configurations traverse a common base graph; AdaptiveGraph adds chronological edges and Personalized PageRank diffusion. We also evaluate BM25 over the same extracted notes and OpenClaw as a raw-input external reference. Retrieval rankings vary across memory settings. On LongMemEval-S, AdaptiveGraph is the strongest graph configuration at 0.844 MRR, but BM25 reaches 0.867 and OpenClaw 0.880. On ATANT Core, localized graph traversal outperforms diffusion and BM25, whereas BM25 leads the stress rounds. Reducing LongMemEval-S within the tested range does not reproduce the ATANT diffusion penalty, but the smallest tested store remains larger than ATANT Core, so store size cannot be ruled out. The penalty also persists under a permissive content-match criterion. Vocabulary normalization and extraction quality substantially affect graph retrieval, and missing extraction tags are common among top-five misses. Retrieval strategies should therefore be evaluated jointly with the memory setting and against strong lexical baselines.

Abstract

Graph-Based Agent Memory

Explore similar work

TAGGRAPH: Tag-Augmented Graphs for Graph Retrieval of Agent Persistent Histories

LazyMem: Retrieve Broadly, Construct Selectively for Efficient Long-Term Agent Memory

A Graph-Native Bitemporal Memory Store for Conversational AI Agents