Agent Zero Memory: Provenance-Aware Long-Term Memory for LLM Agents

Ming Wu; Pengyuan Zhu

cs.CLAug 30, 2026

Agent Zero Memory: Provenance-Aware Long-Term Memory for LLM Agents

Authors: Ming Wu, Pengyuan Zhu

Organizations: Zero Labs

Abstract

Large language model (LLM) agents need durable, faithful memory of everything a user or organization has said and stored, yet most memory systems commit to a single organizing structure (a fact store, a vector index, or a knowledge graph) and inherit its blind spots. We present Agent Zero Memory, a provenance-aware long-term memory system that distils a user's conversations, files, and connected sources into three parallel memory systems, each capturing a different facet of the same history: an episodic Memory Events timeline that makes when and what changed first-class, an associative entity-event knowledge graph that links people and projects across sessions, and a semantic, curated, citation-locked Hierarchical Documentary Memory (HDM) of durable facts. A retrieval turn runs an intent gate (so self-contained turns add no latency), a source router, and three concurrent agentic searches, one per system, each a tool-using loop over hybrid (embedding + lexical) search under agent-controlled filters; their grounded, cited answers are integrated into one answer with a single confidence. We formalize the reading discipline: every learned item is a provenanced item carrying its origin, timestamp, and evidence pointer, and every answer is read under a citation lock, so it may cite only evidence its reader actually opened; fabrication is structurally excluded and the system abstains rather than guesses. On two public benchmarks the system sets a new state of the art: 95.60% on LongMemEval and 93.60% on LoCoMo, improving over the strongest prior systems by +0.73 and +1.10 points. A controlled study across eight backbone LLMs characterizes the accuracy-cost-latency frontier: accuracy varies by only 3.4 points while per-query cost varies by ~30x, with near-state-of-the-art quality at up to 20x lower cost per query, the signature of memory-driven, rather than model-driven, quality.

Figures & tables

System	Memory organization	Update semantics	Provenance	Read cost vs. history	Scope
Agent Zero (ours)	timeline $+$ graph $+$ HDM	append, non-destructive	citation-locked	gated, routed, top- $k$	org-shared
MemGPT [ 3 ]	paged raw context	append (raw log)	raw log only	paging grows w/ history	per agent
Mem0 [ 4 ]	flat extracted facts	overwrite / delete	lost on update	bounded (top- $k$ )	per user
EmergenceMem [ 15 ]	raw turns, vector index	append (raw)	raw turns	grows w/ session length	per user
Supermemory [ 16 ]	atomic memories $+$ links	LLM-inferred links	source chunk kept	bounded (top- $k$ )	per user
Memobase [ 18 ]	profile slots $+$ events	slot overwrite	sources deleted	bounded (SQL)	per user

Table 1: The memory systems compared against in § 5 , along the design dimensions that decide long-term-memory benchmarks. “Read cost” describes how per-query cost scales as the stored history grows.

Panel of Figure 1 — Figure 1: The Agent Zero Memory architecture. (1) Ingest & Build: raw sources are preprocessed and classified, then in parallel indexed (embedding $+$ lexical) and distilled by an LLM into events and documentary-memory entries, which are connected into an entity–event graph. (2) Three parallel memories: an episodic Memory Events timeline (temporal / knowledge-update queries), an associative Entity Graph (multi-hop across sessions), and a semantic , citation-locked Hierarchical Documentary Memory (profile / preferences), each with its own retrieval mode. (3) Retrieve: an intent gate decides whether a turn needs memory, a router selects buckets, then three concurrent agentic searches (one per memory, each hybrid dense $+$ lexical under agent-controlled filters) read the stores on demand, and their grounded results are integrated into one cited answer with a confidence.

	Source	Lifecycle
Factual	Read directly from chats / files / sources	Persistent; re-synced on change
Experiential	Inferred, learned from how the user works	Persistent; continuously refined
Working	Current chat, instructions, task context	Session-bound; useful parts promoted

Table 2: The three kinds of memory the brain draws on, by source and lifecycle.

#	System	Note	Acc. (%)
1	Agent Zero	New 2026 algorithm	95.60
2	Mastra	Observational memory, 2026	94.87
3	Hindsight	Vectorize, 2025	91.40
4	EmergenceMem	Emergence AI, 2026	86.00
5	Supermemory	Supermemory, 2026	85.20
6	Zep [ 5 ]	Open-source memory layer	71.20

Table 3: LongMemEval (500 questions, 6 categories). Judged answer accuracy.

#	System	Note	Acc. (%)
1	Agent Zero	New 2026 algorithm	93.60
2	Mem0 [ 4 ]	Mem0 research, 2026	92.50
3	ByteRover 2.0	Agent memory, 2026	92.20
4	Hindsight	Vectorize, 2025	89.60
5	Memobase	Open-source memory	75.80
6	Zep [ 5 ]	Open-source memory layer	75.10

Table 4: LoCoMo (1,540 questions, 4 categories). Judged answer accuracy.

Panel of Figure 2 — Figure 2: Accuracy–cost frontier on LongMemEval across eight backbones (data from Table 5 ). Accuracy varies by only $3.4$ points while per-query cost varies by $\sim\!30\times$ ; several backbones are Pareto-efficient. Each backbone keeps the same color here and in Figure 3 .

Backbone	Accuracy (%)	Median latency (s)	Avg. cost (USD)	Avg. prompt tok.	Avg. cached-in tok.	Avg. compl. tok.
gpt-5.5	95.60	16.311	0.034788	12,224.8	5,324.8	446.1
gpt-5.6-sol	95.20	15.780	0.030663	11,234.8	5,990.4	406.5
gpt-5.6-terra	94.00	21.183	0.018913	11,881.0	6,412.8	390.6
opus4.8	93.80	12.623	0.052221	23,779.1	14,897.8	591.0
sonnet5	93.20	10.754	0.028520	24,520.4	17,807.1	592.1
glm5.2fast	93.00	6.158	0.014139	10,986.4	5,118.0	339.1

Table 5: Backbone study on LongMemEval . Only the pipeline’s main model is varied; memory and retrieval configuration are held fixed. Cost is average end-to-end USD per query from provider list prices; latency is per-query median.

Backbone	Accuracy (%)	Median latency (s)	Avg. cost (USD)	Avg. prompt tok.	Avg. cached-in tok.	Avg. compl. tok.
sonnet5	93.57	22.855	0.047083	45,818.7	35,987.8	1,279.8
opus4.8	91.49	23.530	0.056355	33,816.2	26,466.6	1,102.2
gpt-5.6-sol	91.49	18.259	0.039919	14,791.3	7,288.3	685.0
deepseek-v4-pro	90.45	34.741	0.010733	49,057.8	38,247.7	2,533.1
glm5.2fast	90.45	17.042	0.022719	31,340.9	22,650.3	1,021.6
gpt-5.6-terra	88.38	18.353	0.023848	16,778.7	9,162.2	751.4

Table 6: Backbone study on LoCoMo . Conventions as in Table 5 .

Panel of Figure 3 — Figure 3: Accuracy–latency frontier on LongMemEval (data from Table 5 ), complementary to the accuracy–cost view of Figure 2 ; point colors identify the same backbones in both figures. The fastest backbone ( glm5.2fast , $6.16$ s) is within $2.6$ points of the most accurate one.

Retrieval exposed to agent	Acc. (%)	$\Delta$
embedding $+$ lexical	95.20	—
Embedding	94.00	$-1.20$
grep	93.60	$-1.60$
Lexical	93.40	$-1.80$

Table 7: Retrieval-channel ablation on LongMemEval (backbone gpt-5.6-sol ; all other components fixed). The agentic searches are restricted to a single retrieval channel.

LLM Agent Memory

Large language model agent memory lets agents retain, organize, retrieve, and update information across interactions and long-running tasks.

See all 368 papers

Apr 20, 2026

HiGMem: A Hierarchical and LLM-Guided Memory System for Long-Term Conversational Agents

Long-term conversational large language model (LLM) agents require memory systems that can recover relevant evidence from historical interactions without overwhelming the answer stage with irrelevant context. However, existing memory systems, including hierarchical ones, still often rely solely on vector similarity for retrieval. It tends to produce bloated evidence sets: adding many superficially similar dialogue turns yields little additional recall, but lowers retrieval precision, increases answer-stage context cost, and makes retrieved memories harder to inspect and manage. To address this, we propose HiGMem (Hierarchical and LLM-Guided Memory System), a two-level event-turn memory system that allows LLMs to use event summaries as semantic anchors to predict which related turns are worth reading. This allows the model to inspect high-level event summaries first and then focus on a smaller set of potentially useful turns, providing a concise and reliable evidence set through reasoning, while avoiding the retrieval overhead that would be excessively high compared to vector retrieval. On the LoCoMo10 benchmark, HiGMem achieves the best F1 on four of five question categories and improves adversarial F1 from 0.54 to 0.78 over A-Mem, while retrieving an order of magnitude fewer turns. Code is publicly available at https://github.com/ZeroLoss-Lab/HiGMem.

Shuqi Cao, Jingyi He, Fei Tan

East China Normal University, Shanghai, China · Shanghai Jiao Tong University, Shanghai, China

Persistent Memory for Language Models Hierarchical Memory Long-Term Conversational Memory Agent Memory Management

Ask PDF

Jul 31, 2026

Zero-Mem: Zero-Token Memory Operations for LLM Agents

LLM agents need memory to act consistently over long interactions, yet many systems use additional LLM calls to operate that memory. Generating intermediate records and mediating their retrieval adds recurring token and time costs, while omitted or merged details can obscure the original evidence. We ask whether structured memory access requires generation at all. Zero-Mem introduces zero-token memory operations: no step outside final question answering invokes an LLM or consumes LLM input or output tokens; encoder computation is accounted for separately. Zero-Mem preserves original interaction traces as its source of record. It organizes the traces in two complementary ways. An entity--context graph exposes connections across interactions, while a temporal hierarchy preserves conversational locality and session state. For each query, Zero-Mem weighs the two views, retrieves from both, and follows their structure to recover supporting relations or surrounding context. Deterministic calibration first discards conflicting evidence and then keeps the reader's answer grounded in the retrieved traces. Only the final-QA reader invokes an LLM. Across long-memory and long-context question-answering benchmarks, Zero-Mem achieves competitive performance while eliminating LLM calls and LLM-token consumption from memory operations. With the same final-QA reader and context budget, it reduces memory-operation time cost by 57.6% relative to the fastest compared baseline. Ablations support the contribution of the two views and their query-dependent coordination. Overall, the results show that structured agent memory need not generate an intermediate representation of the past. After peer review, the code and implementation details will be available at \textcolor{blue}{https://github.com/TheMoon0815/Zero-mem}.

Yilin Xiao, Zhehan Zhu, Yujing Zhang +8

The Hong Kong Polytechnic University, Hong Kong SAR · School of Computing and Artificial Intelligence, Southwestern University of Finance and Economics · School of Artificial Intelligence, Jilin University

Persistent Memory for Language Models Long-Context QA LLM Agent Memory Conversational Memory Agent Memory Management Graph-Based Agent Memory+1

Ask PDF

Jun 9, 2026

Infini Memory: Maintainable Topic Documents for Long-Term LLM Agent Memory

Long-term LLM agents need persistent memory that can track changing facts and provide relevant evidence across sessions. Existing memory systems often store observations as isolated records, summaries, or indexed fragments, which makes evidence aggregation, fact revision, and memory maintenance difficult. We propose Infini Memory, a maintainable text-based persistent memory architecture that treats agent memory as topic-structured documents. Each topic document serves as a semantic unit for collecting related evidence, preserving metadata, and revising facts over time. New observations are first staged in a buffer and periodically consolidated into coherent textual contexts. At inference time, an agentic retrieval procedure lets the LLM read memory through iterative tool calls rather than a single retrieval step. On MemoryAgentBench, Infini Memory achieves 64.7% overall score. Ablations show that topic-structured maintenance and iterative evidence inspection improve complementary aspects of long-term memory use.

Suozhao Ji, Baodong Wu, Zehao Wang +8

1Infinigence AI · 2Tsinghua University · 3Shanghai Jiaotong University

Agentic RAG Persistent Memory for Language Models Memory Consolidation Agent Memory Management LLM Agent Memory Memory Management for LLM Agents+1

Ask PDF

Abstract

LLM Agent Memory

Explore similar work

HiGMem: A Hierarchical and LLM-Guided Memory System for Long-Term Conversational Agents

Zero-Mem: Zero-Token Memory Operations for LLM Agents

Infini Memory: Maintainable Topic Documents for Long-Term LLM Agent Memory