Explicit memory systems transform conversation history into persistent representations intended to improve future access, often at substantial construction cost. Yet it remains unclear whether this added complexity yields better answer accuracy than retrieving directly from raw history. This study compares six state-of-the-art explicit memory systems with raw-history RAG on LongMemEval-S and BEAM-100K under original and controlled retrieval settings. We find only localized gains for particular question types and tight run-time context budgets, with no consistent advantage over strong raw-history RAG baselines. Oracle retrieval shows that some systems fail to preserve answer-supporting information during explicit memory construction, while others preserve it but fail to retrieve it effectively, despite construction costs reaching hours per conversation. Our findings suggest that while many existing memory systems for AI agents emphasize increasingly elaborate explicit memory construction, they are often evaluated without strong controlled comparisons to raw-history RAG and may underemphasize retrieval. Retrieval remains a major bottleneck when useful information is preserved, suggesting that improving access to existing information may be at least as important as constructing richer explicit memory.
Figures & tables
Figure 1: Experimental framework. Explicit memory systems and raw-history RAG share a three-stage evaluation pipeline of construction, retrieval, and answering (Section 3.1 ).
Method
Retrieval
LME-S
BEAM
Full context
—
22.2
24.4
Raw-history RAG (unified prompt)
all-turns
dense (MiniLM)
63.2
56.2
dense (Qwen3-Emb-0.6B)
78.0
56.0
user-turns
dense (MiniLM)
65.6
54.8
dense (Qwen3-Emb-0.6B)
72.0
55.9
Table 1: RQ1: Accuracy under original system configurations. Results are reported on LongMemEval-S (LME-S) and BEAM-100K (BEAM). † denotes systems using the unified answer prompt; bold indicates the best result per benchmark.
Figure 2: RQ2: Accuracy across retrievers and context budgets. Each box summarizes one method over 64 configurations (8 retrievers × 8 context budgets). Boxes show the interquartile range and median, diamonds the mean, and whiskers the minimum and maximum.
Figure 3: RQ2: Accuracy across context budgets. Solid lines show the median accuracy of each explicit memory system across eight retrievers, with bands indicating the minimum and maximum. Dashed lines show the corresponding median for user-turns RAG. The x-axis reports the number of tokens actually supplied to the answer generator.
Best score
Question type
n
RAG
Memory
Δ
LongMemEval-S
Knowledge update
72
88.9
79.2
−9.7
Single-session user
64
98.4
90.6
−7.8
Multi-session
121
74.4
69.4
−5.0
Single-session assistant
56
96.4
92.9
−3.5
Table 2: RQ2: Best accuracy by question type. For each type, RAG and Memory report the best result among the three raw-history RAG baselines and six explicit memory systems, respectively, using Qwen3-Embedding-0.6B, a 1,000-token budget, and the unified answer prompt. Δ=Memory−RAG ; positive differences are bold.
Figure 4: RQ2: Accuracy across answer generators and token budgets. Thick lines highlight the raw-history RAG baseline and explicit memory system with the highest mean accuracy across these budgets.
Figure 5: RQ3: Oracle versus retrieved accuracy on BEAM-100K. ⋆ denotes oracle accuracy and ∙ denotes accuracy under the best retrieval setting from RQ2. The dashed line marks the all-turns oracle. Pink bars show the oracle gap to all-turns , and grey bars show the gap between oracle accuracy and the best retrieval setting.
Construction
Retrieval
Answer
Overall
System
Calls
Tokens (k)
Min
Tok/q (k)
s/q
Tok/q (k)
s/q
Tok/q (k)
s/q
Acc. (%)
LongMemEval-S
Full context
0
0
0
0
0
95.7
8.7
95.7
8.7
22.2
all-turns
0
0
0
0
0.014
1.8
1.1
1.8
1.1
78.0
user-turns
0
0
0
0
0.010
1.1
1.0
1.1
1.0
72.0
pair-turns
0
0
0
0
0.010
4.7
1.4
4.7
1.4
63.8
Table 3: RQ4: Cost by stage. Construction reports per-conversation LLM calls, tokens, and wall time; Retrieval and Answer report per-question cost. Overall is the sum of Retrieval and Answer. Tokens include LLM input and output; retrieval tokens are zero when no LLM is used at query time. Bold indicates the best score per dataset.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
MemoryBank
A-Mem
Mem0
LightMem
SimpleMem
HyperMem
Construction
Input
Session
Turn
User–assistant pair
User turns, topic-segmented
20-turn window, 5-turn overlap
Session
Retrieval unit
Session summary; turn pair
Note with keywords, tags, links
Fact
Fact sentence
Atomic fact with metadata
Topic, episode, fact
Key step
Summarize session
Link to k=5 neighbors; memory evolution a
Extract, then add / update / delete b
LLMLingua-2 compression, rate 0.8; offline update, θ=0.8 c
Extract per window
Extract episodes and facts; group by topic
Retrieval
Query-time LLM
–
Keyword query
–
–
Query plan; ≤2 reflections
–
Appendix
Table 4: Configurations of explicit memory systems. Construction components are used in all experiments; retrieval and answering components are those of the original configurations used in RQ1 (Section 4 ) and RQ4 (Section 7 ); – denotes none. Qwen3-Emb abbreviates Qwen3-Embedding. a An LLM may update the new note’s links and tags and its neighbors’ context. b An LLM compares each new fact with similar stored facts. c Similarity threshold for rewriting an earlier fact. d all-MiniLM-L6-v2. e Semantic / keyword / structured legs. f Candidates, then topics / episodes / facts. g Hits from the same session are concatenated.
Construction model
Memory system
Qwen3-30B-A3B
GPT-4o mini
Δ
MemoryBank
53.9
54.3
+0.4
A-Mem
33.5
28.8
−4.7
Mem0
42.7
54.2
+11.5
LightMem
51.5
51.1
−0.4
SimpleMem
29.0
31.8
+2.8
Appendix
Table 5: RQ1: Accuracy of explicit memory systems across memory construction models. Results are reported on BEAM-100K. Δ=GPT-4o mini−Qwen3-30B-A3B ; bold indicates the best result under both construction models. The bottom row is the strongest raw-history RAG baseline from Table 1
Figure 6: RQ2: Accuracy across four additional answer generators. Qwen3-4B, Qwen3-32B, Gemma-4-26B-A4B ( Team et al., 2026 ) , and GPT-4o mini, with Qwen3-Embedding-0.6B as the retriever. Layout as in Figure 4 .
Raw-history RAG
Explicit memory systems
Question type
all-turns
user-turns
pair-turns
MemoryBank
A-Mem
Mem0
LightMem
SimpleMem
HyperMem
Contradiction resolution
61.9
60.0
51.6
48.4
56.6
54.1
60.9
48.8
47.2
Event ordering
71.3
74.8
68.4
70.0
76.6
33.9
57.6
28.1
30.6
Information extraction
88.9
79.9
88.5
88.5
88.2
71.2
75.6
71.5
47.9
Instruction following
77.5
81.2
86.2
97.5
90.0
78.1
73.1
76.9
81.2
Knowledge update
72.5
63.8
68.8
71.2
72.5
32.5
53.8
41.2
37.5
Appendix
Table 6: RQ3: Oracle accuracy by question type on BEAM-100K. Each question type contains 40 questions; Overall reports accuracy across all 400 questions, including those with empty oracle contexts. Bold indicates the best result per row, except for abstention, where all methods tie.