Persistent conversational AI systems face a choice between passing full conversation histories to a long-context large language model (LLM) and maintaining a dedicated memory system that extracts and retrieves structured facts. We compare a fact-based memory system built on the Mem0 framework against long-context LLM inference on three memory-centric benchmarks - LongMemEval, LoCoMo, and PersonaMemv2 - and evaluate both architectures on accuracy and cumulative API cost. Long-context GPT-5-mini achieves higher factual recall on LongMemEval and LoCoMo, while the memory system is competitive on PersonaMemv2, where persona consistency depends on stable, factual attributes suited to flat-typed extraction. We construct a cost model that incorporates prompt caching and show that the two architectures have structurally different cost profiles: long-context inference incurs a per-turn charge that grows with context length even under caching, while the memory system's per-turn read cost remains roughly fixed after a one-time write phase. At a context length of 100k tokens, the memory system becomes cheaper after approximately ten interaction turns, with the break-even point decreasing as context length grows. These results characterize the accuracy-cost trade-off between the two approaches and provide a concrete criterion for selecting between them in production deployments.
Figures & tables
Dataset
N
Min
Max
Med.
Mean
LongMemEval
500
95,271
103,935
101,747
101,601
LoCoMo
10
9,690
19,242
17,788
15,966
PersonaMem v2
222
14,828
28,479
23,546
22,997
Table 1: Conversation token statistics per dataset.
Dataset
N
Min
Max
Med.
Mean
LongMemEval
500
5
65
15
17.8
LoCoMo
1,986
4
28
11
12.0
PersonaMem v2
589
7
163
28
58.4
Table 2: Question token statistics per dataset.
Dataset
Memory System
LC GPT-5-mini
LC GPT-OSS-120B
LoCoMo
57.68
92.85
81.69
PersonaMem v2
62.48
69.75
60.50
LongMemEval
49.00
82.40
48.20
Table 3: Accuracy (%) on three datasets Maharana et al. (2024) ; Jiang et al. (2025) ; Wu et al. (2025) . “Memory System” denotes the Mem0 pipeline (GPT-5-nano extraction + GPT-5-mini reader). LC = long-context, no memory extraction.
N (turns)
Memory System
LC GPT-5-mini
Result
1
$0.0450
$0.0265
LC cheaper
5
$0.0502
$0.0408
LC cheaper
10
$0.0568
$0.0588
Memory cheaper
15
$0.0634
$0.0768
Memory saves ≈ 17%
20
$0.0700
$0.0947
Memory saves ≈ 26%
Table 4: Cumulative cost per user at context length 101,601 tokens as a function of N interaction turns. LC prices assume prompt caching from turn 2 onward (90% discount on input tokens).
Context L (tokens)
Write Cost
LC Turn 1
LC Turn N
Break-even N
30,000
$0.0129
$0.0086
$0.0018
13
100,000
$0.0430
$0.0261
$0.0036
10
200,000
$0.0859
$0.0511
$0.0061
9
500,000
$0.2148
$0.1261
$0.0136
9
Table 5: Break-even analysis across context sizes. Write Cost is the one-time memory extraction cost. LC costs are per-turn: Turn 1 reflects the uncached rate, Turn N>1 reflects the cached rate. Break-even N is the first turn at which the memory system becomes cheaper.
Figure 1: Break-even heatmap: cumulative cost difference (LC minus Memory) as a function of context length L and number of turns N . Red regions indicate where the long-context approach is cheaper; blue regions indicate where the memory system is cheaper. The black curve marks the break-even boundary.
Loss type
Question
Retrieved memory (Mem0)
Mem0 answer
LC answer
Temporal marker
How many days passed between my visit to MoMA and the ‘Ancient Civilizations’ exhibit at the Met?
“Interested in art-themed day trips” (visit dates not stored)
0 days
7 days
Implicit coreference
What was my previous occupation?
“Previously managed a team of interns at a startup” (activity stored, not the role title)
Managed interns at a startup
Marketing specialist at a startup
Ephemeral update
How often do I attend yoga classes to help with my anxiety?
“Yoga twice a week” (earlier session; later update to three times not propagated)
Twice a week
Three times a week
Table 6: Concrete examples of information categories irretrievably lost under flat-typed fact extraction (LongMemEval). “LC answer” refers to LC GPT-5-mini, which reads the full conversation and answers correctly in all three cases.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 2: Custom instructions passed to the Mem0 add stage for memory extraction.
Figure 3: System and user prompt templates used for LLM-as-a-judge evaluation.
Role
Model
Snapshot
Reasoning effort
Temp.
max_tokens
Fact extractor
GPT-5-nano
2025-08-07
low
—
—
Memory reader
GPT-5-mini
2025-08-07
medium
—
—
Long-context LLM
GPT-5-mini
2025-08-07
low
—
—
Long-context LLM
GPT-OSS-120B
—
—
0
2,000
Judge
GPT-5-mini
2025-08-07
high
—
—
Appendix
Table 7: Intended model inference parameters. GPT-5-nano and GPT-5-mini use reasoning_effort and are pinned to the 2025-08-07 snapshot; GPT-OSS-120B has no versioned snapshot and uses temperature and max_tokens.