MemMachine: A Ground-Truth-Preserving Memory System for Personalized AI Agents

Shu Wang; Edwin Yu; Oscar Love; Tom Zhang; Tom Wong; Steve Scargall; Charles Fan

cs.AIApr 6, 2026

MemMachine: A Ground-Truth-Preserving Memory System for Personalized AI Agents

Authors: Shu Wang, Edwin Yu, Oscar Love, Tom Zhang, Tom Wong, Steve Scargall, Charles Fan

Organizations: MemVerge, Inc.

Abstract

Large Language Model (LLM) agents require persistent memory to maintain personalization, factual continuity, and long-horizon reasoning, yet standard context-window and retrieval-augmented generation (RAG) pipelines degrade over multi-session interactions. We present MemMachine, an open-source memory system that integrates short-term, long-term episodic, and profile memory within a ground-truth-preserving architecture that stores entire conversational episodes and reduces lossy LLM-based extraction. MemMachine uses contextualized retrieval that expands nucleus matches with surrounding context, improving recall when relevant evidence spans multiple dialogue turns. Across benchmarks, MemMachine achieves strong accuracy-efficiency tradeoffs: on LoCoMo it reaches 0.9169 using gpt4.1-mini; on LongMemEvalS (ICLR 2025), a six-dimension ablation yields 93.0 percent accuracy, with retrieval-stage optimizations -- retrieval depth tuning (+4.2 percent), context formatting (+2.0 percent), search prompt design (+1.8 percent), and query bias correction (+1.4 percent) -- outperforming ingestion-stage gains such as sentence chunking (+0.8 percent). GPT-5-mini exceeds GPT-5 by 2.6 percent when paired with optimized prompts, making it the most cost-efficient setup. Compared to Mem0, MemMachine uses roughly 80 percent fewer input tokens under matched conditions. A companion Retrieval Agent adaptively routes queries among direct retrieval, parallel decomposition, or iterative chain-of-query strategies, achieving 93.2 percent on HotpotQA-hard and 92.6 percent on WikiMultiHop under randomized-noise conditions. These results show that preserving episodic ground truth while layering adaptive retrieval yields robust, efficient long-term memory for personalized LLM agents.

Figures & tables

† Randomized-noise setting.
Benchmark	Metric	Result
LoCoMo	Overall score (gpt-4.1-mini)	91.69%
LongMemEval S	Ablation best (gpt-5-mini)	93.0%
HotpotQA hard	Retrieval Agent accuracy	93.2%
WikiMultiHop †	Retrieval Agent accuracy	92.6%
Mem0 comparison	Input token reduction	~80% less

Table 1: Benchmark snapshot (quick overview). Detailed setup and full breakdowns are provided in later sections.

Criterion	Episodic	Semantic
Query type	Specific past events	General preferences
Accuracy need	Ground truth	Approximate
Temporal scope	Point-in-time	Cross-session
Data form	Raw conversation	Extracted facts
Example	“What did I say about X?”	“What foods do I like?”

Table 2: Episodic vs. Semantic Memory Usage

Panel of Figure 1 — Figure 1: MemMachine system architecture. Clients (AI agents, Python SDK, MCP servers) access MemMachine through REST API and SDK interfaces. Internally, episodic memory comprises working memory (short-term) and persistent memory (long-term), while semantic memory stores user profiles. All memory types are backed by SQL, vector, and graph databases.

Panel of Figure 2 — Figure 2: Memory recall workflow. The query passes through STM, LTM vector search, contextualization, deduplication, reranking, and chronological sorting before returning results.

Panel of Figure 3 — Figure 3: Retrieval Agent tool tree. The ToolSelectAgent classifies each query and routes it to one of three strategies. All strategies ultimately delegate to the same declarative memory search, ensuring that index and reranker improvements propagate automatically.

Benchmark	Questions	Answer LLM	MemMachine		Retrieval Agent		Baseline
			Acc.	Recall	Acc.	Recall	Acc.
LoCoMo	1,540	gpt-5-mini	90.5%	82.2%	90.2%	82.5%	91.7%
WikiMultiHop	500	gpt-4.1-mini	88.8%	90.8%	90.0%	92.5%	96.7%
WikiMultiHop †	500	gpt-5-mini	87.4%	91.7%	92.6%	83.3%	96.7%
MRCR	300	gpt-5.2	79.6%	99.1%	81.4%	99.4%	32.3%
EpBench	546	gpt-5-mini	73.4%	66.1%	71.8%	65.3%	77.5%

Table 3: Retrieval Agent benchmark results across five datasets. Accuracy is LLM-judge score; Recall measures gold supporting fact retrieval. Best result per benchmark in bold . All MemMachine and Agent results use surrounding episodes enabled unless noted.

Tool	Questions	Accuracy	Recall
MemMachine	201	93.53%	89.31%
SplitQuery	118	94.07%	92.83%
ChainOfQuery	181	92.27%	95.31%
Overall (Agent)	500	93.20%	92.31%

Table 4: HotpotQA hard set: per-tool performance breakdown for the Retrieval Agent (n=500, gpt-5-mini).

Component	Input Tokens	Output Tokens
ToolSelect (router)	1,049	195
ChainOfQuery	2,874	1,614
SplitQuery	1,229	435
MemMachine	0	0

Table 5: Average token cost per question by Retrieval Agent strategy (HotpotQA hard set).

Use Agent Mode	When…
Best fit	Queries involve multi-hop reasoning, co-reference resolution, or multi-entity fan-out. Accuracy is prioritized over latency. Use cases include research assistants, compliance review, and complex QA.
Not needed	Queries are predominantly single-hop factual lookups. Latency requirements are strict ( $<$ 200ms). Token budget is constrained. Use cases include simple chatbot recall, preference lookup, and session context.

Table 6: Decision guidance for enabling agent mode.

Configuration	LLM Score	Mode
gpt-4.1-mini (agent)	0.9169	Agent
gpt-4.1-mini (memory)	0.9123	Memory
gpt-4o-mini (agent)	0.8812	Agent
gpt-4o-mini (memory)	0.8747	Memory

Table 7: Impact of eval-LLM on MemMachine v0.2 LoCoMo scores.

System	Input Tokens	Output Tokens
MemMachine v0.2 (memory)	4.20M	43,169
MemMachine v0.2 (agent)	8.57M	93,210
Mem0 main/HEAD (memory)	19.21M	14,840

Table 8: Token usage comparison on LoCoMo (gpt-4.1-mini).

Component	Specification
Operating System	Ubuntu 24.04 LTS
CPU	8 vCPUs
RAM	16 GiB
GPU	Not required (CPU-only runs)
Python Version	3.11
MemMachine Version	v0.3.x

Table 9: Benchmark test environment.

Eval-LLM	Mode	Multi-hop	Temporal	Open-domain	Single-hop	Overall
gpt-4.1-mini	Agent	0.8830	0.9159	0.7188	0.9512	0.9169
gpt-4.1-mini	Memory	0.8972	0.8910	0.7500	0.9441	0.9123
gpt-4o-mini	Agent	0.8404	0.8069	0.7396	0.9394	0.8812
gpt-4o-mini	Memory	0.8759	0.7352	0.7083	0.9465	0.8747

Table 10: MemMachine v0.2 LoCoMo results by category (LLM Judge Score).

System	Single-hop	Temporal	Multi-hop	Open-domain	Overall
MemMachine v0.2	0.9465	0.7352	0.8759	0.7083	0.8747
Memobase (v0.0.37)	0.7092	0.8505	0.4688	0.7717	0.7578
Zep	0.7411	0.7979	0.6604	0.6771	0.7514
Mem0	0.6713	0.5551	0.5115	0.7293	0.6688
LangMem	0.6223	0.2343	0.4792	0.7112	0.5810
OpenAI	0.6379	0.2171	0.4292	0.6229	0.5290

Table 11: LoCoMo benchmark comparison across AI agent memory systems (LLM Judge Score). MemMachine results are with gpt-4o-mini for fair comparison with published baselines; gpt-4.1-mini results shown separately in Table 10 .

ID	Answer LLM	Chunk	User-Q	JSON-str	Prompt	top_ $k$	LLM Score
C5	GPT-5	—	—	✓	Edwin1	20	0.855
C6	GPT-5	—	✓	✓	Edwin1	20	0.870
C7	GPT-5	—	✓	✓	Edwin1	30	0.912
C8	GPT-5	—	✓	✓	Edwin1	50	0.890
C9	GPT-5	✓	✓	✓	Edwin1	20	0.878
C11	GPT-5	✓	✓	✓	Edwin3	20	0.896

Table 12: LongMemEval S configuration matrix (

n=500

questions per run). “Chunk” = sentence-level chunking enabled; “User-Q” = user-role query prefix; “JSON-str” = structured message formatting; “Edwin{1,3}” = search prompt variants of increasing refinement.

Retrieval-stage optimizations
Optimization	Comparison	$\Delta$ Score
Retrieval depth ( $k$ : 20 $\to$ 30)	C6 vs. C7	+4.2%
Context formatting (JSON-str)	C4 vs. C5	+2.0%
Search prompt (Edwin1 $\to$ 3)	C9 vs. C11	+1.8%
COT $\to$ simple prompt (GPT-5)	C1 vs. C4	+1.6%
User-query bias correction	C5 vs. C6	+1.4%

Table 13: LongMemEval S ablation: contribution of each optimization to overall LLM score, measured by comparing configuration pairs that differ in one variable.

Config	LLM	SSU	SSP	SSA	TR	KU	MS	Total
C5 (baseline)	GPT-5	0.986	0.700	1.000	0.798	0.925	0.797	0.855
C6 (user-q)	GPT-5	0.957	0.667	0.982	0.850	0.885	0.835	0.870
C7 ( $k$ =30)	GPT-5	0.986	0.800	1.000	0.917	0.923	0.850	0.912
C12 (mini, $k$ =20)	GPT-5-mini	0.986	0.933	1.000	0.902	0.962	0.850	0.922
C14 (mini, $k$ =50)	GPT-5-mini	1.000	0.933	1.000	0.932	0.949	0.842	0.928
C15 (mini, $k$ =100)	GPT-5-mini	1.000	0.933	0.982	0.917	0.949	0.872	0.930

Table 14: LongMemEval S per-category LLM scores for selected configurations (

n=500

). SSU = single-session-user, SSP = single-session-preference, SSA = single-session-assistant, TR = temporal reasoning, KU = knowledge update, MS = multi-session.

Config (LLM)	$k$	Input (M)	Score
C6 (GPT-5)	20	2.94	0.870
C7 (GPT-5)	30	4.03	0.912
C8 (GPT-5)	50	6.47	0.890
C12 (GPT-5-mini)	20	2.58	0.922
C14 (GPT-5-mini)	50	5.97	0.928
C15 (GPT-5-mini)	100	9.79	0.930

Table 15: LongMemEval S token consumption per 500-question run. Input tokens scale with

k

; output tokens remain approximately constant.

Property	MemMachine	Mem0	Zep	Mastra OM	MemOS	Full Context
Memory approach	Retrieval	Retrieval	Retrieval	In-context	Hybrid	In-context
Ground truth preserved	✓	Partial	Partial	$\times$	Partial	✓
Prompt cacheable	Partial	$\times$	$\times$	✓	$\times$	✓
Scales beyond context window	✓	✓	✓	$\times$	✓	$\times$
LLM calls per message	Low	High	Moderate	Moderate	High	None
Specialized DB required	Yes	Yes	Yes	No	Yes	No

Table 16: Architectural design space comparison across agent memory systems.

LLM Agent Memory

Large language model agent memory lets agents retain, organize, retrieve, and update information across interactions and long-running tasks.

See all 357 papers

MemORAI: Memory Organization and Retrieval via Adaptive Graph Intelligence for LLM Conversational Agents
May 2, 2026Hung Pham Van, Nguyen Manh Hieu, Khang Pham Tran Tuan +4Conversational Memory Graph Retrieval-Augmented Generation
1Independent Researcher · 2Hanoi University of Science and Technology · 3VNU University of Engineering and Technology +1
Memori: A Persistent Memory Layer for Efficient, Context-Aware LLM Agents
Mar 20, 2026Luiz C. Borro, Luiz A. B. Macarini, Gordon Tindall +2Persistent Memory for Language Models LLM Memory
Memori Labs Inc.
Human-Inspired Memory Architecture for LLM Agents
May 8, 2026Doga Kerestecioglu, Alexei Robsky, Clemens Vasters +2Agent Memory Management LLM Agent Memory
Microsoft

May 2, 2026J/K move · Enter open · S save

MemORAI: Memory Organization and Retrieval via Adaptive Graph Intelligence for LLM Conversational Agents

Hung Pham Van, Nguyen Manh Hieu, Khang Pham Tran Tuan, Nam Le Hai+3

1Independent Researcher · 2Hanoi University of Science and Technology · 3VNU University of Engineering and Technology+1

Conversational Memory Graph Retrieval-Augmented Generation Persistent Memory for Language Models Graph-Based Agent Memory Long-Term Conversational Memory Agent Memory Management

Open paper AskPDF arXiv

Large Language Models (LLMs) lack persistent memory for long-term personalized conversations. Existing graph-based memory systems suffer from information dilution, absent provenance tracking, and uniform retrieval that ignores query context. We introduce MemORAI (Memory Organization and Retrieval via Adaptive Graph Intelligence), a framework that integrates three innovations: selective memory filtering with dual-layer compression to retain user-persona-relevant content, a provenance-enriched multi-relational graph tracking factual origins at the turn level, and query-adaptive subgraph retrieval with Dynamic Weighted PageRank that applies query-conditioned edge weighting. Evaluated on LOCOMO and LongMemEval benchmarks, MemORAI achieves state-of-the-art performance in memory retrieval and personalized response generation, demonstrating that selective storage, enriched representation, and adaptive retrieval are essential for coherent, personalized LLM agents.

Abstract

LLM Agent Memory

Explore similar work

MemORAI: Memory Organization and Retrieval via Adaptive Graph Intelligence for LLM Conversational Agents

Memori: A Persistent Memory Layer for Efficient, Context-Aware LLM Agents

Human-Inspired Memory Architecture for LLM Agents