Selective Forgetting: A Graph-Based Memory Framework for Long-Term LLM Agents

Theo Rusu; Sourena Khanzadeh; Manar Alalfi

cs.AIAug 29, 2026

Selective Forgetting: A Graph-Based Memory Framework for Long-Term LLM Agents

Authors: Theo Rusu, Sourena Khanzadeh, Manar Alalfi

Organizations: Department of Computer Science Toronto Metropolitan University Toronto, Ontario, Canada · The Creative School Toronto Metropolitan University Toronto, Ontario, Canada · Flybits, Creative AI Hub Toronto, Ontario, Canada

Abstract

Knowledge graphs have been proposed as a structured alternative to flat retrieval-augmented generation for long-term agent memory, on the assumption that representing conversations as entities and relations improves recall. We evaluate that assumption directly. Our framework extracts each conversational turn into typed nodes and attributed edges, answers questions from a two-hop subgraph, and periodically prunes nodes that score low on a weighted combination of recency, access frequency, degree centrality, and age. On LongMemEval, the graph does not outperform a flat vector baseline at a matched candidate-generation budget of five retrieval roots: token F1 is $0.417$ against $0.468$ , and a paired bootstrap over 500 questions gives $Δ= -0.050$ (95% CI $[-0.085, -0.016]$ ). The gap is widest on questions that require recalling a specific prior assistant turn, where judged correctness falls from $0.911$ to $0.607$ , suggesting that decomposing a turn into entities discards the surface form these questions depend on. The forgetting module is more successful. Applied once to a persistent 27{,}021-node graph, it removes 9.8% of nodes and 9.5% of stored bytes; token F1 is unchanged ( $+0.001$ , 95% CI $[-0.015, +0.016]$ ) and judged correctness falls by $1.6$ points, with the 95% interval bounding any loss at $3.8$ points ( $[-0.038, +0.006]$ ). Because our extractor is a single small model evaluated on one benchmark, these results characterise this extraction-based pipeline rather than graph-structured memory in general. Code: https://github.com/skhanzad/Selective-Amnesia

Figures & tables

Panel of Figure 1 — Figure 1: Overview of the proposed memory framework. Each conversational turn is processed by the update stage, which extracts, embeds, de-duplicates, and writes nodes and edges into the persistent knowledge graph. At question time, the retrieval stage embeds the referenced entities, selects the top-5 matching nodes ranked by cosine similarity, and expands them via a 2-hop subgraph traversal to build the answer-generation context. Every 400 turns, the retention stage scores each node by recency, access frequency, centrality, and turn age, pruning nodes whose importance falls below the threshold together with their incident edges.

Type	n	P	F1	J	n	P	F1	J
	Graph RAG (Ours)				Baseline RAG
Single (user)	70	0.743	0.737	0.771	70	0.823	0.819	0.929
Single (asst.)	56	0.680	0.575	0.607	56	0.916	0.774	0.911
Single (pref.)	30	0.312	0.083	0.233	30	0.274	0.114	0.367
Know.-update	78	0.507	0.456	0.513	78	0.553	0.511	0.590
Multi-session	133	0.384	0.326	0.398	133	0.381	0.342	0.436

Table 1: Experiment 1 results: Graph RAG vs. Baseline RAG on LongMemEval. J = LLM-judge; Single = single-session.

Variant	Nodes	Edges	Size	F1	Judge
No-forgetting	27,021	46,538	440.6 MB	0.292	0.300
Forgetting	24,368	43,978	398.6 MB	0.293	0.284
Change	$-$ 9.8%	$-$ 5.5%	$-$ 9.5%	$+$ 0.001	$-$ 0.016

Table 2: Experiment 2 results: storage and retrieval quality for the persistent graph with and without forgetting.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Value	Basis
De-duplication cosine threshold $\tau_{\text{dedup}}$	0.92	E
Retrieval cosine floor $\tau_{\text{ret}}$	0.75	E
Retrieval roots (top- $k$ )	5	C
Subgraph expansion depth	2 hops	B
Subgraph node cap	15	B
Forgetting interval	400 turns	B

Appendix

Table 3: Parameters, settings, and basis for selection. E = empirical probe, B = budget or cost constraint, C = convention or symmetry with the baseline.

Stage	Cost	Note
Embed extracted entities	$O(kd)$
Title de-duplication	$O(1)$ per entity	hash index
Embedding de-duplication	$O(kNd)$	linear scan
Graph write	$O(k+\ell)$	indexed updates
Candidate retrieval	$O(qNd)$	linear scan
Subgraph traversal	$O(N+M+\|V_{S}\|+\|E_{S}\|)$	adjacency build

Appendix

Table 4: Per-stage complexity under exhaustive vector search.

T

is the forgetting interval (400 turns), and

V_{S},E_{S}

are the nodes and edges of the retrieved subgraph.

Condition	F1	Precision	Recall	Judge acc.
Baseline RAG (per-question, Experiment 1 control)	$0.468\pm 0.019$	$0.532\pm 0.019$	$0.486\pm 0.020$	$0.536\pm 0.022$
Graph RAG (per-question, Experiment 1 treatment)	$0.417\pm 0.019$	$0.505\pm 0.020$	$0.412\pm 0.019$	$0.454\pm 0.022$
Persistent graph, no forgetting (Experiment 2 control)	$0.292\pm 0.017$	$0.363\pm 0.018$	$0.295\pm 0.018$	$0.300\pm 0.021$
Persistent graph, with forgetting (Experiment 2 treatment)	$0.293\pm 0.017$	$0.359\pm 0.018$	$0.294\pm 0.018$	$0.284\pm 0.020$

Appendix

Table 5: Mean

\pm

SEM (1

\sigma

) and 95% bootstrap CI for each condition,

n=500

questions per condition.

Condition	F1	Precision	Recall	Judge acc.
Baseline RAG (per-question, Experiment 1 control)	$[0.432,0.505]$	$[0.496,0.571]$	$[0.447,0.525]$	$[0.492,0.580]$ (Wilson: $[0.492,0.579]$ )
Graph RAG (per-question, Experiment 1 treatment)	$[0.381,0.454]$	$[0.467,0.544]$	$[0.375,0.451]$	$[0.410,0.496]$ (Wilson: $[0.411,0.498]$ )
Persistent graph, no forgetting (Experiment 2 control)	$[0.260,0.325]$	$[0.328,0.399]$	$[0.261,0.330]$	$[0.260,0.342]$ (Wilson: $[0.262,0.342]$ )
Persistent graph, with forgetting (Experiment 2 treatment)	$[0.260,0.326]$	$[0.323,0.395]$	$[0.259,0.329]$	$[0.244,0.324]$ (Wilson: $[0.246,0.325]$ )

Appendix

Table 6: 95% bootstrap confidence intervals corresponding to Table 5 . For judge accuracy, the 95% Wilson score interval is shown alongside the bootstrap CI.

Comparison	F1	Precision	Recall	Judge acc.
Experiment 1: Graph RAG vs Baseline RAG	$-0.050$ ∗ $[-0.085,-0.016]$	$-0.028$ $[-0.067,+0.011]$	$-0.073$ ∗ $[-0.107,-0.039]$	$-0.082$ ∗ $[-0.128,-0.034]$
Experiment 2: Forgetting vs No forgetting	$+0.001$ $[-0.015,+0.016]$	$-0.004$ $[-0.023,+0.014]$	$-0.001$ $[-0.017,+0.016]$	$-0.016$ $[-0.038,+0.006]$

Appendix

Table 7: Paired bootstrap comparisons between conditions. A positive mean difference favors the first-named condition; ∗ denotes significance at

\alpha=0.05

Graph-Based Agent Memory

Graph-based agent memory organizes an agent’s experiences, facts, and procedures as linked structures for persistent, evidence-aware retrieval and reuse.

See all 133 papers

Sep 29, 2026

TAGGRAPH: Tag-Augmented Graphs for Graph Retrieval of Agent Persistent Histories

Long-term memory lets LLM agents recall past interactions and remain consistent across sessions, but memory systems are hard to compare because they often vary in representation, indexing, retrieval, and evaluation. We present a controlled evaluation framework based on shared 5W-style conversational memories. Localized graph configurations traverse a common base graph; AdaptiveGraph adds chronological edges and Personalized PageRank diffusion. We also evaluate BM25 over the same extracted notes and OpenClaw as a raw-input external reference. Retrieval rankings vary across memory settings. On LongMemEval-S, AdaptiveGraph is the strongest graph configuration at 0.844 MRR, but BM25 reaches 0.867 and OpenClaw 0.880. On ATANT Core, localized graph traversal outperforms diffusion and BM25, whereas BM25 leads the stress rounds. Reducing LongMemEval-S within the tested range does not reproduce the ATANT diffusion penalty, but the smallest tested store remains larger than ATANT Core, so store size cannot be ruled out. The penalty also persists under a permissive content-match criterion. Vocabulary normalization and extraction quality substantially affect graph retrieval, and missing extraction tags are common among top-five misses. Retrieval strategies should therefore be evaluated jointly with the memory setting and against strong lexical baselines.

Yu-Shu Chen, Yu-Jung Liang, Pengtao Xie

Department of Electrical and Computer Engineering University of California, San Diego La Jolla, CA 92093, USA

Graph-Based Retrieval Memory Agent Benchmarks Graph-Based Agent Memory Long-Term Conversational Memory

Ask PDF

Jul 17, 2026

LazyMem: Retrieve Broadly, Construct Selectively for Efficient Long-Term Agent Memory

Long-term memory enables LLM agents to leverage past interactions, but dialogue histories quickly exceed the context window, forcing agents to retrieve relevant subsets at query time. Because useful evidence is sparse and scattered across verbose conversations, retrieval faces a fundamental tension: broadening recall improves coverage but floods downstream reasoning with noise, while compressing memories at write time eases retrieval but irreversibly discards details that future queries may need. We introduce LazyMem, which resolves this tension by deferring all memory construction to query time. Given a retrieved candidate pool, a lightweight model processes it in overlapping parallel windows, selectively retaining and compressing only query-relevant content. The model is trained with supervised fine-tuning followed by reinforcement learning, using a reward that jointly encourages the identification of relevant messages and the generation of compressions that are faithful to the source and useful for answering the query. On LongMemEval, LazyMem-4B achieves an LLM-judge accuracy of 0.85, outperforming the strongest non-oracle baseline while using only 213 answer-context memory tokens, 21.0 times fewer than the baseline. It further generalizes to LoCoMo without target-domain training and reduces mean latency relative to the prior query-time baseline. Code is available at https://github.com/allacnobug/LazyMem.

Jing Yu, Yibo Zhao, Jiaming Zhang +1

School of Data Science and Engineering, East China Normal University

Persistent Memory for Language Models Long-Term Conversational Memory Agent Memory Management Memory Management for LLM Agents

Ask PDF

Jul 29, 2026

A Graph-Native Bitemporal Memory Store for Conversational AI Agents

Conversational AI agents commonly lack persistent memory across sessions. The obvious fixes like injecting full chat histories into the context window, or delegating to a third-party memory service, either exhaust the model's context budget or send personal data through infrastructure the user does not control. We describe a memory store that avoids both problems: an agent-local Neo4j property graph augmented with HNSW vector indexes and a full bitemporal data model. Each memory is stored as an immutable identity node linked to versioned content nodes carrying two closed-open time intervals: valid time (when the fact was true in the world) and transaction time (when the database recorded it). This design supports point-in-time semantic retrieval without physically overwriting history. Semantic edges between related memories are maintained automatically at write time using cosine similarity over 1024-dimensional embeddings. We evaluate the system on LongMemEval, a 500-question benchmark spanning six question types designed to stress long-term memory. Across 60 sampled questions, the current-state semantic search path achieves 46.7% R@10 overall, rising to 80% on knowledge-update questions. The time-travel path yields 80% R@10 on knowledge-update but decreases recall on temporal-reasoning questions (50% to 37.5%), a consequence of post-filter dilution that points directly to a concrete design improvement. We discuss what these results reveal about the limits of pure retrieval for different question types and what each failure mode suggests for future work.

Alp Niksarli, Gopesh Baheti

Davidson College

Conversational Memory LLM Agent Memory Persistent Memory for Language Models Temporal IR Graph-Based Agent Memory Temporal KGs

Ask PDF

Abstract

Graph-Based Agent Memory

Explore similar work

TAGGRAPH: Tag-Augmented Graphs for Graph Retrieval of Agent Persistent Histories

LazyMem: Retrieve Broadly, Construct Selectively for Efficient Long-Term Agent Memory

A Graph-Native Bitemporal Memory Store for Conversational AI Agents