Is Complex Agentic Memory Always Needed? Naive RAG Is Stronger Than We Thought

Zhengye Bao; Shengtao Zhang; Haokai Zhang; Shiqi Lou; Muning Wen; Minnan Luo

cs.AIOct 22, 2026

Is Complex Agentic Memory Always Needed? Naive RAG Is Stronger Than We Thought

Authors: Zhengye Bao, Shengtao Zhang, Haokai Zhang, Shiqi Lou, Muning Wen, Minnan Luo

Organizations: Luo Lab Undergraduate Division (LUD), Xi’an Jiaotong University · Shanghai Jiao Tong University · Zhejiang University · The Chinese University of Hong Kong

Abstract

LLM memory is widely used in long-term conversation understanding. Agentic memory systems often use mutation, structural organization, learned retrieval strategies, or other complex operations. Is this complexity always necessary? We study Raw-RAG, a naive RAG baseline using dense retrieval and reranking over raw history to supply evidence to a reader LLM. Across LoCoMo, LongMemEval, MEME, and MemoryAgentBench, Raw-RAG ranks first among the compared memory systems in six of seven benchmark-model settings, outperforming the strongest compared agentic memory system by 0.13-5.53 points. We investigate why raw-history retrieval remains competitive. Evidence analysis shows that its higher coverage of necessary evidence can offset lower utilization. Useful evidence includes both answer values and the relations and dependency paths needed to derive them. Even complete access to transformed memory stores leaves an accuracy gap relative to raw evidence, highlighting the importance of preserving raw information. These findings challenge the necessity of complex agentic memory for the evaluated conversation tasks and establish naive RAG as an essential baseline.

Figures & tables

Figure 2: Mem0 proposes few mutations, and fewer take effect. Top : write-action distributions. Bottom : per-conversation proposed and effective mutation rates on LoCoMo (left) and LongMemEval (right). Dashed lines: decision-weighted means. Rates are relative to all emitted decisions.

Table 2

Figure 3: Pipelines of agentic memory systems (EverMemOS and Memora) and naive RAG ( Raw-RAG ) across build, store, and recall stages.

Method	LoCoMo										LongMemEval
	GPT-4o-mini					GPT-4.1-mini					GPT-4.1-mini
	Single	Multi	Temp.	Open	Overall	Single	Multi	Temp.	Open	Overall	S-U.	S-A.	S-P.	M-S.	KU.	TR.	Overall
Full-context	83.00	66.80	56.20	48.60	72.30	86.90	77.20	74.20	56.60	80.60	85.70	98.20	16.70	51.10	76.90	60.20	65.60
MemU [ 31 ]	72.77	62.41	33.96	46.88	61.15	74.91	72.34	43.61	54.17	66.67	67.14	19.64	76.67	42.10	41.02	17.29	38.40
Mem0 [ 4 ]	67.13	51.15	55.51	72.93	66.88	68.97	61.70	58.26	50.00	64.20	82.86	26.78	90.00	63.15	66.67	72.18	66.40
A-MEM [ 54 ]	66.83	52.13	61.99	30.21	60.84	73.25	59.93	72.90	42.71	68.83	95.71	100.00	63.33	61.65	82.05	52.63	71.60

Table 3: LoCoMo and LongMemEval accuracy by question type (%). For LongMemEval, S-U., S-A., and S-P. denote single-session user, assistant, and preference; M-S. denotes multi-session; KU. knowledge update; and TR. temporal reasoning. Bold: column maxima excluding Full-context.

Method	MEME							MemoryAgentBench
Method	ER.	Agg.	Tr.	Del.	Cas.	Abs.	Overall	AR.	TTL.	LRU.	CR.	Overall
GPT-4o-mini
Full-context	93.00	10.00	44.00	16.00	3.66	9.23	29.31	–	–	–	–	–
MemU [ 31 ]	52.00	6.00	31.00	19.00	1.22	10.00	19.87	52.30	9.12	22.54	20.50	26.12
EverMemOS [ 10 ]	44.00	3.00	50.00	20.00	2.44	10.00	21.57	43.38	48.30	24.77	29.50	36.49
MemOS [ 21 ]	78.00	19.00	49.00	21.00	3.66	10.77	30.24	52.03	45.73	33.13	12.00	35.72

Table 4: MEME and MemoryAgentBench scores by subset (%). Bold: reader-wise column maxima excluding Full-context. Overall: six-task macro-average (MEME) or mean of four capability macros (MemoryAgentBench). Dashes: Full-context results unavailable due to context-length limits.

Table 7

Appendix figures & tables25 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Single	Multi	Temporal	Open	Overall
GPT-4o-mini
MemoryOS [ 16 ]	74.55	58.87	44.24	46.88	63.64
Mem0 [ 4 ]	65.52	52.84	52.02	37.50	58.64
A-MEM [ 54 ]	66.83	52.13	61.99	30.21	60.84
RecMem [ 5 ]	79.79	72.70	57.32	58.33	72.47
SimpleMem [ 23 ]	86.01	74.94	63.76	68.40	78.25

Appendix

Table 10: LoCoMo accuracy (%) under the text-embedding-3-small evaluation setting, scored by a large language model (LLM) judge. Single, Multi, and Open denote single-hop, multi-hop, and open-domain questions. Overall is question-weighted; adversarial questions are excluded. Bold marks the highest score in each column within each reader block.

Method	Single	Multi	Temporal	Open	Overall
GPT-4o-mini
MemoryOS [ 16 ]	62.43	56.50	37.18	40.28	54.70
Mem0 [ 4 ]	66.71	58.16	55.45	40.62	61.00
MemU [ 31 ]	72.77	62.41	33.96	46.88	61.15
MemOS [ 21 ]	81.45	69.15	72.27	60.42	75.87
Zep [ 36 ]	88.11	71.99	74.45	66.67	81.06

Appendix

Table 11: LoCoMo LLM-judge accuracy (%) with Qwen3-Embedding-4B and Qwen3-Reranker-4B. Single, Multi, and Open denote single-hop, multi-hop, and open-domain questions. Overall is question-weighted; adversarial questions are excluded. Bold marks column maxima within each reader block, including ties at the displayed precision.

GPT-4o-mini
Method	Source	$J$	F1	$B_{1}$
Full-context	M	72.90	–	–
A-MEM [ 54 ]	M	48.38	–	–
LangMem	M	58.10	–	–
Zep [ 36 ]	M	65.99	–	–
OpenAI	M	52.90	–	–

Appendix

Table 12: Overall LoCoMo scores under the short-answer evaluation setting (0–100). Metrics are LLM-judge accuracy (

J

), token F1, and unigram BLEU (

B_{1}

). Sources M, N, and L denote the Mem0 [ 4 ] paper, the Nemori [ 30 ] paper, and this study. RAG (256/2) uses 256-token chunks and top-2 retrieval; Mem0 g [ 4 ] is the graph variant. “Minimal” requests only the shortest sufficient answer. Dashes indicate unavailable scores; bold marks column maxima per reader, excluding Full-context.

GPT-4o-mini
Method	Src.	Single			Multi			Temporal			Open
Method	Src.	$J$	F1	$B_{1}$	$J$	F1	$B_{1}$	$J$	F1	$B_{1}$	$J$	F1	$B_{1}$
A-MEM [ 54 ] (rerun)	M	39.79	20.76	14.90	18.85	9.22	8.81	49.91	35.40	31.08	54.05	33.34	27.58
LangMem	M	62.23	35.51	26.86	47.92	26.04	22.32	23.43	30.75	25.84	71.12	40.91	33.63
Zep [ 36 ]	M	61.70	35.74	23.30	41.35	19.37	14.82	49.31	42.00	34.53	76.60	49.56	38.92
OpenAI	M	63.79	34.30	23.72	42.92	20.09	15.42	21.71	14.04	11.25	62.29	39.31	31.16

Appendix

Table 13: LoCoMo short-answer scores by question type with GPT-4o-mini (0–100). Metrics are LLM-judge accuracy (

J

), token F1, and unigram BLEU (

B_{1}

). Single, Multi, and Open mean single-hop, multi-hop, and open-domain. Src. identifies the source: M, Mem0 [ 4 ] paper; N, Nemori [ 30 ] paper; L, this study. Mem0 g [ 4 ] is the graph variant; “minimal” requests the shortest sufficient answer. Bold marks column maxima excluding Full-context.

GPT-4.1-mini
Method	Single			Multi			Temporal			Open
Method	$J$	F1	$B_{1}$	$J$	F1	$B_{1}$	$J$	F1	$B_{1}$	$J$	F1	$B_{1}$
Full-context	86.90	61.40	53.40	77.20	44.20	33.70	74.20	47.50	40.00	56.60	28.40	22.20
LangMem	84.50	51.00	43.60	71.00	41.50	32.50	50.80	48.50	40.90	59.00	32.80	26.40
Zep [ 36 ]	66.90	45.50	40.00	53.70	30.50	20.40	60.20	23.90	20.00	43.80	24.20	19.30
Mem0 [ 4 ]	71.40	48.60	42.00	68.20	40.10	30.30	56.90	39.20	33.20	47.90	23.70	17.70

Appendix

Table 14: LoCoMo short-answer scores by question type with GPT-4.1-mini (0–100). Metrics are LLM-judge accuracy (

J

), token F1, and unigram BLEU (

B_{1}

). Single, Multi, and Open mean single-hop, multi-hop, and open-domain. Baseline scores are from the Nemori [ 30 ] paper; Raw-RAG uses the shortest-sufficient-answer instruction. Bold marks column maxima excluding Full-context.

Method	S-U.	S-A.	S-P.	M-S.	KU.	TR.	Overall
GPT-4.1-mini
MemoryOS [ 16 ]	94.29	89.29	100.00	67.67	80.77	54.89	74.40
A-MEM [ 54 ]	95.71	100.00	63.33	61.65	82.05	52.63	71.60
RecMem [ 5 ]	95.71	96.43	86.67	65.41	75.64	68.42	76.80
Nemori [ 30 ]	90.00	92.90	86.70	55.60	79.50	72.20	74.60
MemMachine [ 49 ]	92.86	100.00	55.56	66.67	86.32	58.65	74.33

Appendix

Table 15: LongMemEval-S accuracy (%) with GPT-4.1-mini under the text-embedding-3-small evaluation setting, using all 500 questions. S-U., S-A., and S-P. denote single-session user, assistant, and preference questions; M-S., multi-session; KU., knowledge update; TR., temporal reasoning. Overall is question-weighted. Bold marks column maxima.

Method	S-U.	S-A.	S-P.	M-S.	KU.	TR.	Overall
GPT-4.1-mini
MemU [ 31 ]	67.14	19.64	76.67	42.10	41.02	17.29	38.40
Zep [ 36 ]	92.90	75.00	53.30	47.40	74.40	54.10	63.80
Mem0 [ 4 ]	82.86	26.78	90.00	63.15	66.67	72.18	66.40
MemOS [ 21 ]	95.71	67.86	96.67	70.67	74.26	77.44	77.80
EverMemOS [ 10 ]	97.14	85.71	93.33	73.68	89.74	77.44	83.00

Appendix

Table 16: LongMemEval-S LLM-judge accuracy (%) with GPT-4.1-mini, Qwen3-Embedding-4B, and Qwen3-Reranker-4B, using all 500 questions. S-U., S-A., and S-P. denote single-session user, assistant, and preference questions; M-S., multi-session; KU., knowledge update; TR., temporal reasoning. Overall is question-weighted. Bold marks column maxima.

Method	S-U.	S-A.	S-P.	M-S.	KU.	TR.	Overall
GPT-4.1-mini
EMem [ 59 ]	93.80	82.10	46.70	82.10	95.40	80.60	83.00
EMem-G [ 59 ]	94.80	87.50	50.00	82.60	94.40	83.70	84.90
Raw-RAG (textemb3)	100.00	98.21	100.00	76.31	94.44	81.36	87.80
Raw-RAG (Qwen3)	100.00	96.43	95.56	78.79	94.44	91.08	90.57

Appendix

Table 17: LongMemEval-S accuracy (%) with GPT-4.1-mini on the 470 non-abstention questions. The textemb3 setting uses text-embedding-3-small; Qwen3 uses Qwen3-Embedding-4B. EMem-G [ 59 ] is EMem [ 59 ] ’s graph variant. S-U., S-A., and S-P. denote single-session user, assistant, and preference questions; M-S., multi-session; KU., knowledge update; TR., temporal reasoning. Overall is question-weighted. Bold marks column maxima.

Method	Profile	CON	EP	TR	EPR	CW	WP	KQ	SM	CH	Overall
GPT-4.1-mini
Zep [ 36 ]	No	39.51	42.51	36.92	37.59	41.22	40.54	63.43	32.35	44.87	43.40
Mem0 [ 4 ]	No	43.21	41.30	43.08	42.41	42.86	34.75	59.20	38.66	40.30	43.85
MemU [ 31 ]	Yes	37.86	33.20	38.46	32.76	35.51	35.14	56.97	34.03	34.22	38.70
MemoryOS [ 16 ]	Yes	35.80	36.84	40.00	35.86	35.51	35.91	57.96	35.29	36.88	40.05
MemOS [ 21 ]	Yes	48.15	49.80	51.92	50.00	48.16	48.26	61.94	46.64	44.87	50.72

Appendix

Table 18: PersonaMem-v2 multiple-choice accuracy (%) with GPT-4.1-mini and 32k-token histories. Raw-RAG ’s Overall score is question-weighted over 5,000 questions. Subset abbreviations: CON = Consultation; EP = Email (Personal); TR = Translation; EPR = Email (Professional); CW = Creative Writing; WP = Writing (Professional); KQ = Knowledge Query; SM = Social Media; CH = Chat. Profile indicates use of persona profiles. Bold marks the highest accuracy in each scenario and overall.

Final $k$	Total	$\Delta$	95% CI
GPT-4o-mini
1	36.59	-11.73	[-14.05,-9.43]
5	47.10	-1.21	[-2.81,+0.40]
10	47.57	-0.75	[-2.19,+0.66]
15	48.32	–	–
30	48.10	-0.22	[-1.40,+0.93]

Appendix

Table 19: Effect of retrieval depth on Raw-RAG with GPT-4o-mini on MemoryAgentBench. Final

k

is the number of reranked chunks selected for the reader; Total (%) averages the four capability scores. Gray marks the default

k=15

\Delta

and its paired-bootstrap 95% confidence interval (CI) compare each setting with the default, in percentage points.

Benchmark	$N$	Mean	Median	P95	Maximum	$>5{,}000$
LoCoMo	1,540	4,778.8	4,778.0	4,966.1	5,177	41
LongMemEval	500	11,355.6	11,248.5	13,915.3	27,375	500
MEME	1,188	11,986.1	10,186.0	21,329.6	42,490	1,188
MemoryAgentBench	2,071	3,915.8	4,173.0	4,209.5	4,262	0

Appendix

Table 20: Length of the evidence supplied to Raw-RAG readers, in tokens, including formatting but excluding the question and fixed prompt.

N

is the number of questions per reader; P95 is the 95th percentile;

>5{,}000

counts questions exceeding 5,000 evidence tokens.

System	Native mean	Median	P95	Maximum	5k mean	Cut
EMem [ 59 ]	2,637.7	171	7,081.0	125,646	1,417.7	511
Mem0 [ 4 ]	3,572.8	1,033	6,319.0	39,485	1,961.3	500
EverMemOS [ 10 ]	15,394.8	9,443	50,750.0	70,861	4,160.6	2,011
MemMachine [ 49 ]	26,296.9	21,806	82,868.5	88,967	4,479.2	2,030
Raw-RAG	3,915.8	4,173	4,209.5	4,262	3,915.8	0

Appendix

Table 21: Reader-evidence lengths before and after applying a 5,000-token cap on MemoryAgentBench (2,071 questions per system). Native mean, Median, P95 (95th percentile), and Maximum describe evidence before capping; 5k mean describes capped evidence. Cut counts questions whose evidence changes. Mem0 [ 4 ] uses the earlier hosted-service configuration.

System	AR	TTL	LRU	CR	Overall (5k)	Native	$\Delta$
GPT-4o-mini
EMem [ 59 ]	57.60	45.87	16.94	30.00	37.60	38.44	$-0.84$
Mem0 [ 4 ]	65.10	46.25	21.13	46.50	44.74	45.27	$-0.53$
EverMemOS [ 10 ]	42.42	41.38	22.62	28.50	33.73	36.49	$-2.76$
MemMachine [ 49 ]	53.55	23.39	26.13	23.50	31.64	35.53	$-3.89$
MemOS [ 21 ]	52.72	46.06	21.83	15.00	33.90	34.18	$-0.28$

Appendix

Table 22: MemoryAgentBench scores (%) with GPT-4o-mini and a 5,000-token evidence cap. AR denotes Accurate Retrieval; TTL, Test-Time Learning; LRU, Long-Range Understanding; CR, Conflict Resolution. Overall averages the four capabilities; Native is the corresponding score before capping, and

\Delta

is 5k minus Native in percentage points. Mem0 [ 4 ] and MemOS [ 21 ] use earlier hosted-service configurations. Bold marks the highest capped score in each column.

Benchmark	$N$	Changed	Mean	Maximum	Native	5k	$\Delta$
			5k evidence tokens		Score
LoCoMo	1,540	41	4,753.5	4,999	89.07	89.42	$+0.35$
LongMemEval	500	500	4,144.4	4,271	87.53	89.20	$+1.67$
MEME	1,188	1,188	4,173.1	4,273	39.30	38.21	$-1.09$

Appendix

Table 23: Raw-RAG results with a 5,000-token evidence cap, using GPT-4o-mini on LoCoMo and MEME and GPT-4.1-mini on LongMemEval.

N

counts questions; Changed counts contexts shortened by disabling turn expansion, with all selected chunks retained. Scores are percentages;

\Delta

is 5k minus Native in percentage points. Bold marks the higher score for each benchmark.

LoCoMo: GPT-4o-mini
Condition	Single	Multi	Temp.	Open			Overall
Native	94.57	85.93	82.04	73.61			89.07
fixed	95.05	88.06	80.17	75.00			89.42
LongMemEval: GPT-4.1-mini
Condition	S-U.	S-A.	S-P.	M-S.	KU.	TR.	Overall
Native	100.00	98.21	100.00	77.69	91.03	81.45	87.53

Appendix

Table 24: Raw-RAG accuracy (%) by task with native evidence and a 5,000-token cap (fixed). For LoCoMo, Single, Multi, Temp., and Open denote single-hop, multi-hop, temporal, and open-domain questions. For LongMemEval, S-U., S-A., and S-P. denote single-session user, assistant, and preference questions; M-S., multi-session; KU., knowledge update; TR., temporal reasoning. For MEME, ER denotes exact recall; Agg, aggregation; Tr, tracking; Del, deletion; Cas, cascade; Abs, invalidation requiring abstention. Bold marks the higher score in each column, including ties.

Benchmark	Metric	Statistic/range	$p$	Holm sig.
MemoryAgentBench	Coverage	$Q(6)=897.802$	$1.12\!\times\!10^{-190}$	20/21
MemoryAgentBench	Utilization	9.71 pp	$6.00\!\times\!10^{-5}$	7/21
MEME	Coverage	40.66 pp	$2.00\!\times\!10^{-5}$	19/21
MEME	Utilization	21.04 pp	$2.00\!\times\!10^{-5}$	17/21

Appendix

Table 25: Tests of cross-system differences in evidence coverage and utilization across seven systems. MemoryAgentBench coverage uses Cochran’s

Q

with six degrees of freedom; the remaining tests use a centered bootstrap of the maximum-minus-minimum range, measured in percentage points (pp). Holm sig. counts significant contrasts among the 21 system pairs after Holm correction at

\alpha=0.05

	Coverage			Utilization
System	%	$n/N$	95% CI	%	$n/N$	95% CI
MemoryAgentBench
Mem0 [ 4 ]	40.03	829/2071	[38.29, 41.77]	82.39	683/829	[79.84, 84.86]
EMem [ 59 ]	34.72	719/2071	[33.17, 36.26]	89.71	645/719	[87.46, 91.87]
MemOS [ 21 ]	34.48	714/2071	[32.83, 36.17]	85.85	613/714	[83.33, 88.30]
EverMemOS [ 10 ]	25.11	520/2071	[23.56, 26.65]	80.00	416/520	[76.60, 83.30]

Appendix

Table 26: Evidence coverage and conditional reader accuracy (utilization), with paired-bootstrap 95% confidence intervals (CIs). Separate columns give each percentage, its supporting count

n/N

, and its CI. For coverage,

n/N

counts sufficient contexts among all questions; for utilization, correct answers among sufficient contexts. Bold marks the highest estimate per benchmark and metric.

System	Mem0 [ 4 ]	EMem [ 59 ]	MemOS [ 21 ]	EverMemOS [ 10 ]	MemMach. [ 49 ]	MemU [ 31 ]	Raw-RAG
Mem0 [ 4 ]	—	+5.31 * [+3.43,+7.19] $4.96\!\times\!10^{-7}$	+5.55 * [+3.77,+7.34] $2.91\!\times\!10^{-7}$	+14.92 * [+12.99,+16.85] $5.02\!\times\!10^{-46}$	+18.49 * [+16.37,+20.57] $3.12\!\times\!10^{-54}$	+20.96 * [+18.98,+22.89] $3.47\!\times\!10^{-67}$	-5.26 * [-7.39,-3.14] $9.84\!\times\!10^{-6}$
EMem [ 59 ]	+7.32 * [+4.43,+10.22] $0.00084$	—	+0.24 [-1.64,+2.12] $0.853$	+9.61 * [+7.63,+11.59] $3.27\!\times\!10^{-18}$	+13.18 * [+11.25,+15.11] $1.74\!\times\!10^{-33}$	+15.64 * [+13.71,+17.58] $1.40\!\times\!10^{-37}$	-10.57 * [-12.51,-8.64] $4.84\!\times\!10^{-23}$
MemOS [ 21 ]	+3.47 [+0.54,+6.41] $0.225$	-3.85 [-6.76,-0.94] $0.124$	—	+9.37 * [+7.44,+11.35] $6.64\!\times\!10^{-18}$	+12.94 * [+10.91,+14.97] $6.02\!\times\!10^{-27}$	+15.40 * [+13.42,+17.38] $7.51\!\times\!10^{-37}$	-10.82 * [-12.80,-8.84] $1.26\!\times\!10^{-19}$
EverMemOS [ 10 ]	-2.39 [-6.01,+1.20] $1.000$	-9.71 * [-13.23,-6.22] $0.00084$	-5.85 * [-9.63,-2.10] $0.0352$	—	+3.57 * [+1.69,+5.46] $0.00235$	+6.04 * [+4.01,+8.06] $1.76\!\times\!10^{-6}$	-20.18 * [-22.11,-18.25] $3.86\!\times\!10^{-73}$
MemMachine [ 49 ]	+1.92 [-1.89,+5.71] $1.000$	-5.40 * [-8.94,-1.88] $0.0408$	-1.55 [-5.28,+2.09] $1.000$	+4.30 [+0.22,+8.42] $0.355$	—	+2.46 * [+0.48,+4.44] $0.0445$	-23.76 * [-25.64,-21.87] $3.18\!\times\!10^{-107}$
MemU [ 31 ]	+6.98 * [+3.36,+10.46] $0.00408$	-0.34 [-3.88,+3.11] $1.000$	+3.51 [-0.16,+7.12] $0.492$	+9.37 * [+5.07,+13.61] $0.00084$	+5.06 [+0.77,+9.30] $0.225$	—	-26.22 * [-28.25,-24.19] $1.22\!\times\!10^{-95}$

Appendix

Table 27: Pairwise differences in MemoryAgentBench evidence coverage (upper triangle, blue) and utilization (lower triangle, gray). Each cell lists the row-minus-column difference in percentage points (pp), its paired-bootstrap 95% confidence interval (CI), and the Holm-adjusted

p

-value

p_{\mathrm{H}}

. Bold and ∗ mark

p_{\mathrm{H}}<0.05

, with correction over 21 pairs per metric. MemMach. [ 49 ] denotes MemMachine [ 49 ] .

System	Mem0 [ 4 ]	EMem [ 59 ]	MemOS [ 21 ]	EverMemOS [ 10 ]	MemMach. [ 49 ]	MemU [ 31 ]	Raw-RAG
Mem0 [ 4 ]	—	+13.13 * [+9.98,+16.22] $0.00084$	+1.09 [-1.51,+3.66] $0.432$	+15.57 * [+12.23,+18.92] $0.00084$	-7.15 * [-10.03,-4.28] $0.00084$	+27.53 * [+24.06,+30.96] $0.00084$	-13.13 * [-15.95,-10.35] $0.00084$
EMem [ 59 ]	+21.04 * [+16.94,+25.08] $0.00084$	—	-12.04 * [-14.80,-9.26] $0.00084$	+2.44 [-0.82,+5.68] $0.294$	-20.29 * [-22.85,-17.63] $0.00084$	+14.39 * [+11.66,+17.06] $0.00084$	-26.26 * [-28.78,-23.69] $0.00084$
MemOS [ 21 ]	+1.15 [-2.93,+5.25] $0.890$	-19.89 * [-23.18,-16.55] $0.00084$	—	+14.48 * [+11.39,+17.64] $0.00084$	-8.25 * [-11.34,-5.16] $0.00084$	+26.43 * [+23.40,+29.43] $0.00084$	-14.23 * [-17.15,-11.24] $0.00084$
EverMemOS [ 10 ]	+11.34 * [+6.41,+16.11] $0.00084$	-9.70 * [-13.57,-5.91] $0.00084$	+10.19 * [+6.14,+14.14] $0.00084$	—	-22.73 * [-25.54,-19.93] $0.00084$	+11.95 * [+8.53,+15.38] $0.00084$	-28.70 * [-31.71,-25.67] $0.00084$
MemMachine [ 49 ]	+17.14 * [+13.10,+21.11] $0.00084$	-3.90 * [-6.86,-0.95] $0.0474$	+15.99 * [+12.57,+19.34] $0.00084$	+5.80 * [+2.37,+9.36] $0.00512$	—	+34.68 * [+31.42,+37.94] $0.00084$	-5.98 * [-7.65,-4.34] $0.00084$
MemU [ 31 ]	+18.69 * [+13.87,+23.40] $0.00084$	-2.36 [-6.42,+1.80] $0.789$	+17.53 * [+13.71,+21.42] $0.00084$	+7.34 * [+2.92,+11.90] $0.00744$	+1.54 [-2.48,+5.53] $0.890$	—	-40.66 * [-43.70,-37.60] $0.00084$

Appendix

Table 28: Pairwise differences in MEME evidence coverage (upper triangle, blue) and utilization (lower triangle, gray). Each cell lists the row-minus-column difference in percentage points (pp), its paired episode-bootstrap 95% confidence interval (CI), and the Holm-adjusted

p

-value

p_{\mathrm{H}}

. Bold and ∗ mark

p_{\mathrm{H}}<0.05

, with correction over 21 pairs per metric. MemMach. [ 49 ] denotes MemMachine [ 49 ] .

System	$N$	$C$	$R$	$C-N$ [95% CI]	$R-C$ [95% CI]
GPT-4o-mini
MemMachine [ 49 ]	37.06	57.46	61.60	+20.40 [17.30, 23.40]	+4.14 [0.23, 8.07]
EverMemOS [ 10 ]	21.57	51.59	61.60	+30.02 [26.53, 33.47]	+10.01 [6.17, 13.87]
EMem [ 59 ]	33.84	48.36	61.60	+14.52 [10.58, 18.42]	+13.24 [9.27, 17.19]
MemOS [ 21 ]	30.24	45.12	61.60	+14.88 [11.04, 18.57]	+16.48 [12.76, 20.19]
Mem0 [ 4 ]	27.76	32.26	61.60	+4.50 [1.75, 7.27]	+29.34 [25.47, 33.10]

Appendix

Table 29: MEME complete-store diagnostic with GPT-4o-mini, reported as six-task macro-accuracy (%).

N

uses native retrieval;

C

uses gold-informed selection from the complete store;

R

is the shared raw-evidence control. Both

C

and

R

use a 5,000-token budget and task-aware reader instructions. Differences and paired episode-bootstrap 95% confidence intervals (CIs) are in percentage points. Bold marks the highest score in

N

and

C

GPT-4o-mini
Condition	Affected	Dropped	$C_{5k}$	$C_{\infty}$	$C_{\infty}-C_{5k}$ [95% CI]	$R-C_{\infty}$ [95% CI]
Raw control	0	0.00%	61.60	61.60	+0.00 [+0.00, +0.00]	+0.00 [+0.00, +0.00]
MemMachine [ 49 ]	1,172	83.39%	57.46	54.02	$-3.44$ [ $-6.49$ , $-0.37$ ]	+7.58 [+3.03, +12.15]
Mem0 [ 4 ]	62	1.45%	32.26	32.22	$-0.04$ [ $-1.04$ , +0.96]	+29.38 [+25.58, +33.18]
MemOS [ 21 ]	408	12.79%	45.12	44.58	$-0.53$ [ $-1.60$ , +0.44]	+17.02 [+13.22, +20.80]
MemU [ 31 ]	604	63.07%	31.07	32.55	+1.49 [ $-0.33$ , +3.28]	+29.05 [+25.77, +32.36]

Appendix

Table 30: Effect of removing the 5,000-token evidence cap in the MEME complete-store diagnostic with GPT-4o-mini.

C_{5k}

and

C_{\infty}

are capped and uncapped six-task macro-accuracies (%);

R

is the shared raw control. Affected counts questions whose evidence changes; Dropped is the percentage of evidence tokens omitted by the cap. Differences and paired episode-bootstrap 95% confidence intervals (CIs) are in percentage points. Bold marks score-column maxima excluding the raw control.

Raw-source setting	ER	Agg	Tr	Del	Cas	Abs	Overall
GPT-4o-mini
Answer-location oracle	98	47	81	39	1.83	36.15	50.50
Gold-informed raw control	94	52	69	92	34.15	28.46	61.60

Appendix

Table 31: MEME accuracy (%) with GPT-4o-mini under two raw-source oracle configurations that differ in evidence selection, context assembly, and reader instructions. ER denotes exact recall; Agg, aggregation; Tr, tracking; Del, deletion; Cas, cascade; Abs, invalidation requiring abstention. Overall is the six-task macro-average. Bold marks the higher score in each column.

Configuration	Calls	Prompt (M)	Compl. (M)	Tokens (M)	Add (s)	Search (s)	Total (s)
Atomic facts	40,248	24.0	22.3	46.3	28,837.0	2,836.8	31,673.8
Raw summary	32,144	14.8	6.8	21.6	8,804.4	1,523.1	10,327.5
Graph-expanded	33,748	105.7	3.7	109.3	11,506.3	4,093.3	15,599.6
Raw-RAG	0	0	0	0	272.6	445.8	718.4

Appendix

Table 32: LLM usage and elapsed time for Raw-RAG and its memory-management variants on the 11 MemoryAgentBench sources outside Conflict Resolution. Calls counts LLM requests; Prompt, Compl. (completion), and Tokens report input, output, and total tokens in millions (M). Add and Search measure ingestion and retrieval time; Total is their sum. All times are in seconds (s). Measurements exclude answer generation and evaluation. Bold marks column minima.

System	Exported reader evidence	Provenance and scope
Mem0 [ 4 ]	All stored textual memories, excluding the auxiliary entity graph.	Match each memory to its conversation and map its creation time (or recorded timestamp if unavailable) to the source session. This is the final local mutable store.
MemOS [ 21 ]	All memories in the fine-grained stores used by the Native condition, including preference, tool, and skill memories.	Map source-message timestamps to source sessions. Source-message content is excluded from reader text.
MemU [ 31 ]	All item summaries and any nonempty linked category summaries in the local stores; no original resource text.	Resource timestamps map items to sessions, including segmented resources. Category-item links supply the union of source sessions; a category is visible only after all mapped sources. Empty category summaries are checked but excluded.
EMem [ 59 ]	All elementary discourse units (EDUs) in the 200 stored before/after snapshots.	Use the largest completed snapshot when multiple copies exist. Snapshot boundaries determine visibility; graph attributes map EDU IDs to source sessions for display order. No source transcript is appended. Task-specific packing is described below.
EverMemOS [ 10 ]	Every episode text and atomic-fact text in the per-conversation persisted vector-index files.	Map original-message timestamps to source sessions; original-message content is excluded. Historical versions are unavailable in this export.
MemMachine [ 49 ]	All episodic and semantic memories, plus persisted working-memory summaries. Render episodic text with timestamp/speaker and semantic text as category/name/value.	Use episodic source metadata and database citation episodes. Uncited semantic features and current summaries conservatively depend on all conversation sessions. Citations serve only to locate source episodes.

Appendix

Table 33: Stored evidence and temporal eligibility rules for the seven systems in the MEME complete-store diagnostic. The complete store comprises the listed reader-visible representations; source provenance or before/after snapshots determine eligibility at the question’s temporal cutoff. EDUs are elementary discourse units.

Work and locator	Reported RAG configuration and reference score	Missing or under-tuned design choice
Mem0 [ 4 ] Sec. 3.3; Tab. 2	LoCoMo RAG (256/2) reaches 60.97 with GPT-4o-mini. The paper searches seven chunk sizes, 128–8192 tokens, but always uses $k\in\{1,2\}$ and embedding similarity only.	Top- $k$ is too low ( $k\leq 2$ ): even the small-chunk configurations retrieve at most two evidence locations across the chunk-size search.
Memobase [ 28 ] LoCoMo evaluation	The public raw-RAG configuration uses 500-token chunks and $k=1$ , ranked by embedding cosine similarity. No independent RAG score is reported; the comparison scores for other methods are imported from Mem0.	Top- $k$ is too low ( $k=1$ ): only one 500-token chunk reaches the reader, so evidence distributed across multiple chunks cannot be combined.
Nemori [ 30 ] Sec. 4	RAG-4096 uses 4096-token chunks; its LoCoMo RAG reference is 30.20 with GPT-4o-mini and 32.90 with GPT-4.1-mini.	Chunk size is too large (4096 tokens): each retrieval unit bundles many dialogue details, reducing the specificity of dense retrieval and spending context on unrelated material.
RecMem [ 5 ] App. E.1, E.5	Message-preserving $\sim$ 2048-token chunks; $k=3/5$ on LoCoMo/LongMemEval-S. Scores: LoCoMo 49.68/54.42; LongMemEval-S 59.40/67.00 (GPT-4o-mini/GPT-4.1-mini).	Chunk size is too large ( $\sim$ 2048 tokens), and top- $k$ is too low ( $k=3/5$ ). The matched budgets still use coarse retrieval units and restrict evidence to a few locations.
Memora [ 53 ] Sec. 5.1	500-token chunks, $k=3$ ; LoCoMo 63.30 with GPT-4.1-mini. The reported 65.60 LongMemEval result belongs to Full-context and is excluded here.	Top- $k$ is too low ( $k=3$ ): only three chunks, totaling at most 1500 source tokens, can supply evidence from the conversation.
xMemory [ 12 ] Secs. 4.1, 5.3	Top-20 original-message chunks by vector similarity; LoCoMo 27.95 BLEU-1 / 36.48 F1 with Qwen3-8B.	Cross-encoder reranking is absent from the reported raw baseline. LLMLingua-2 is evaluated separately as a pruning control.

Appendix

Table 34: Review of raw-history retrieval-augmented generation (RAG) baselines in 14 prior evaluations.

k

is the number of retrieved units; listed scores are on a 0–100 scale. Retrieval limitations are our assessments of the reported configurations, not controlled estimates of their effects. Scores are not directly comparable across studies. For Memobase, only the public configuration is reviewed.

Conversational QA

See all 17 papers

Do Conversational Agents Need Explicit Memory? A Controlled Comparison with Raw-History Retrieval
Oct 17, 2026Yongjie Zhou, Shuai Wang, Bevan Koopman +1LLM Agent Memory Long-Term Conversational Memory
The University of Queensland, Brisbane, Australia · CSIRO · Google
HiGMem: A Hierarchical and LLM-Guided Memory System for Long-Term Conversational Agents
Apr 20, 2026Shuqi Cao, Jingyi He, Fei TanPersistent Memory for Language Models Hierarchical Memory
East China Normal University, Shanghai, China · Shanghai Jiao Tong University, Shanghai, China
RaMem: Contextual Reinstatement for Long-term Agentic Memory
Jun 22, 2026Wei Yang, Bryce Kan, Shixuan Li +5Agent Memory LLM Agent Memory
University of Southern California

Oct 17, 2026J/K move · Enter open · S save

Do Conversational Agents Need Explicit Memory? A Controlled Comparison with Raw-History Retrieval

Yongjie Zhou, Shuai Wang, Bevan Koopman, Guido Zuccon

The University of Queensland, Brisbane, Australia · CSIRO · Google

LLM Agent Memory Long-Term Conversational Memory Retrieval-Augmented Generation LLM Agent Evaluation Conversational QA Memory Agent Benchmarks

Open paper AskPDF arXiv

Explicit memory systems transform conversation history into persistent representations intended to improve future access, often at substantial construction cost. Yet it remains unclear whether this added complexity yields better answer accuracy than retrieving directly from raw history. This study compares six state-of-the-art explicit memory systems with raw-history RAG on LongMemEval-S and BEAM-100K under original and controlled retrieval settings. We find only localized gains for particular question types and tight run-time context budgets, with no consistent advantage over strong raw-history RAG baselines. Oracle retrieval shows that some systems fail to preserve answer-supporting information during explicit memory construction, while others preserve it but fail to retrieve it effectively, despite construction costs reaching hours per conversation. Our findings suggest that while many existing memory systems for AI agents emphasize increasingly elaborate explicit memory construction, they are often evaluated without strong controlled comparisons to raw-history RAG and may underemphasize retrieval. Retrieval remains a major bottleneck when useful information is preserved, suggesting that improving access to existing information may be at least as important as constructing richer explicit memory.

Abstract

Conversational QA

Explore similar work

Do Conversational Agents Need Explicit Memory? A Controlled Comparison with Raw-History Retrieval

HiGMem: A Hierarchical and LLM-Guided Memory System for Long-Term Conversational Agents

RaMem: Contextual Reinstatement for Long-term Agentic Memory