Long-term conversational memory poses a fundamental system challenge for LLM agents: reasoning over all past interactions improves coverage but incurs prohibitive token costs and noise, while retrieval-based methods are efficient yet often fail on multi-hop and temporal questions. Existing multi-step retrievers partially address this gap, but typically operate over an ever-growing textual history, causing context expansion and noise accumulation across iterations. We propose MemR3, a backend-agnostic closed-loop controller that reformulates conversational memory retrieval as explicit stateful decision-making. At each iteration, MemR3 maintains an evidence-gap state consisting of grounded information (evidence) and unresolved information requirements (gaps), and uses this state to route among three actions: retrieve, reflect, and answer. Newly retrieved snippets are incorporated into the state and then masked from subsequent prompts, so the model conditions on a compact summary of progress rather than the full retrieval history. This design turns multi-step memory retrieval into a bounded, inspectable control process whose query-time context scales with the state instead of raw accumulated text. Experiments on LongMemEvals show that MemR3 surpasses implicit multi-step retrieval baselines and often approaches or exceeds full-context prompting while retrieving only 1%-5% of the full conversation tokens on long conversations. On LoCoMo, MemR3 consistently improves over its underlying RAG- and Zep-based memory backbones. These results suggest that explicit evidence-gap tracking is an effective abstraction for token-efficient conversational memory.
Figures & tables
Figure 1: Illustration of three memory-usage paradigms. In this example, Full-Context exposes the LLM to many distractors, while Retrieve-then-Answer retrieves relevant snippets but still miscalculates. In contrast, MemR 3 iteratively retrieves and reflects using an explicit evidence-gap tracker (Acts 0-3). By absorbing information and discarding raw texts, it refines the query and produces the correct answer (3 months) without unbounded context expansion.
Figure 2: Pipeline of MemR 3 . MemR 3 transforms retrieval into a closed-loop process: a router dynamically switches between Retrieve, Reflect, and Answer nodes while a global evidence-gap tracker maintains what is known and what is still missing. This enables iterative query refinement, targeted retrieval, and early stopping, making MemR 3 an autonomous, backend-agnostic controller.
Figure 3: Results on LongMemEval s . (a) Overall accuracy across six LLM backends. (b) Retrieved-context tokens versus accuracy across six categories. (c) LLM-response latency. Repeated-run statistics and full query-time costs are reported in Appendices E.1 and E.4 .
Backend
RAG
Self-RAG
RCR
MemR 3
Full-Context
GPT-4o-mini
1.53
7.06
2.81
3.87
11.04
GPT-4.1-mini
1.72
10.71
3.10
5.59
7.55
GPT-5-mini
11.51
41.38
15.80
15.26
36.88
Table 2: Average end-to-end latency (seconds) on LongMemEval s .
Method
1. Multi-Hop
2. Temporal
3. Open-Domain
4. Single-Hop
Overall
4o
4.1
4o
4.1
4o
4.1
4o
4.1
4o
4.1
A-Mem ( Xu et al., 2025 )
61.70
71.99
64.49
74.77
40.62
58.33
76.63
79.88
69.06
76.00
LangMem ( LangChain Team, 2025 )
62.23
74.47
23.43
61.06
47.92
67.71
71.12
86.92
58.10
78.05
Mem0 ( Chhikara et al., 2025 )
67.13
62.41
55.51
57.32
51.15
44.79
72.93
66.47
66.88
62.47
Self-RAG ( Asai et al., 2024 )
69.15
75.89
64.80
75.08
34.38
54.17
88.31
90.12
76.46
82.08
RCR
71.28
80.85
71.03
81.62
42.71
62.50
86.99
90.12
77.96
84.89
Table 3: LLM-as-a-Judge Score (%) on LoCoMo. Categories are grouped, with GPT-4o-mini (4o) and GPT-4.1-mini (4.1) shown side-by-side. Δ denotes absolute improvements over backbones. MemR 3 rows are three-run means (Appendix E.1 ); baselines are single runs .
Figure 4: Granularity bias (failed case). The agent recovers the coarse fact “in 2022” and produces the correct answer, but the gap requires the exact date. Because of the granularity mismatch, the controller cannot close the loop and stops only at the iteration limit nmax .
Method
MH *
Temporal
OD *
SH *
Overall
RAG
68.79
65.11
58.33
83.86
75.54
MemR 3
71.39
76.22
61.11
89.44
81.55
w/o mask
62.41
68.54
55.21
72.17
68.54
w/o Δqk
66.67
75.08
60.42
83.37
77.11
w/o reflect
65.25
73.83
61.46
83.37
76.65
Table 4: Ablation studies on LoCoMo.
R
E
Interpretation
Questions
Human accuracy (%)
0
0
Retrieval failure
23
0.0
1
0
State/compression failure
24
4.2
1
1
Evidence preserved
105
90.5
Table 5: Human audit of retrieval and evidence preservation on 152 LoCoMo questions (GPT-4o-mini). R=1 : retrieved snippets contain answer-critical evidence; E=1 : the final state faithfully retains it.
Backend
Subset
n
Cumulative recall
J
Initial
Iter. 2
Iter. 3
Iter. 4
Iter. 5
GPT-4o-mini
R0=0
192
0.00
16.11
24.33
29.77
33.90
45.83
0<R0<1
206
48.00
52.49
56.92
58.66
59.99
70.87
R0=1
1,124
100.00
100.00
100.00
100.00
100.00
90.04
GPT-4.1-mini
R0=0
192
0.00
21.84
34.16
41.50
47.58
64.58
0<R0<1
206
48.00
54.72
59.54
63.32
65.23
78.64
Table 6: Cumulative evidence recall and final J score (%) on LoCoMo, grouped by initial retrieval recall .
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: LLM-as-a-Judge score (%) with different a) number of chunks per iteration, b) number of facts/entities, and c) max iterations.
RAG
Self-RAG
RCR
Full-Context
MemR 3
Mean ± SD
56.60±0.72
41.53±0.61
56.07±0.31
58.53±1.10
71.67±0.76
Run 1
57.40
42.20
56.00
59.80
72.00
Run 2
56.00
41.00
55.80
57.80
70.80
Run 3
56.40
41.40
56.40
58.00
72.20
Appendix
Table 7: Three-run J scores (%) on LongMemEval s with GPT-4o-mini.
Backend
Method
J (%)
Completion tokens
Individual runs (J)
GPT-4o-mini
FAIR-RAG
54.93±0.46
996±4
55.20 / 55.20 / 54.40
MemR 3
71.67±0.76
357±74
72.00 / 70.80 / 72.20
GPT-4.1-mini
FAIR-RAG
60.00±0.92
1,060±29
60.80 / 60.20 / 59.00
MemR 3
76.40±1.06
491±5
77.20 / 76.80 / 75.20
GPT-5-mini
FAIR-RAG
73.20±1.40
10,144±185
74.60 / 73.20 / 71.80
MemR 3
81.80±0.72
5,029±78
82.60 / 81.20 / 81.60
Appendix
Table 8: Three-run comparison with FAIR-RAG on LongMemEval s across six backends. The last column lists individual runs .
Method
GPT-4o-mini
GPT-4.1-mini
J (%)
Completion tokens
J (%)
Completion tokens
FAIR-RAG
68.96±0.23
891±5
78.46±0.20
786±2
MemR 3 (RAG)
81.55±0.34
300±6
86.75±0.46
431±1
Zep
74.69±0.24
45±2
78.94±0.15
64±2
MemR 3 (Zep)
76.26±0.34
390±1
80.88±0.46
496±4
Appendix
Table 9: Three-run results on LoCoMo. FAIR-RAG and MemR 3 (RAG) share the same RAG retriever; MemR 3 (Zep) is compared with its Zep backbone .
LLM
Retriever
Run
MH
T
OD
SH
Overall
GPT-4o-mini
Zep
1
68.09
73.52
68.75
80.72
76.13
2
69.86
72.59
67.71
80.36
76.00
3
70.21
75.39
64.58
80.72
76.65
Mean ± SD
69.39 ± 1.14
73.83 ± 1.43
67.01 ± 2.17
80.60 ± 0.21
76.26 ± 0.34
RAG
1
71.63
77.26
61.46
89.28
81.75
2
70.21
76.01
59.38
89.40
81.16
Appendix
Table 10: Individual MemR 3 runs on LoCoMo, with category means and sample standard deviations. MH = multi-hop; T = temporal; OD = open-domain; SH = single-hop.
Backend
Method
P
A
TR
MS
KU
U
Overall
GPT-4o-mini
RAG
56.67
94.64
42.11
33.83
70.51
87.14
57.40
Self-RAG
26.67
83.93
18.05
18.80
66.67
78.57
42.20
RCR
53.33
92.86
34.59
38.35
71.79
84.29
56.00
Full-Context
40.00
94.64
39.10
45.11
76.92
88.57
59.80
MemR 3
60.00
92.86
55.64
67.67
75.64
95.71
72.00
Improvement
20.00
-1.78
16.54
22.56
-1.28
7.14
12.20
Appendix
Table 11: J scores (%) on LongMemEval s . P/A/U = single-session preference/assistant/user; TR = temporal reasoning; MS = multi-session; KU = knowledge update. Improvements are absolute differences from Full-Context. Three-run results are in Tables 7 and 8 .
Backend
Decomp-RAG
Rewrite-RAG
MemR 3
GPT-4o-mini
62.00
63.40
72.00
GPT-4.1-mini
68.40
68.40
77.20
GPT-5-mini
79.00
76.40
82.60
Appendix
Table 12: J scores (%) for query decomposition and iterative rewriting on LongMemEval s .
Backend
Method
Input
Completion
Total
J (%)
GPT-4o-mini
RAG
1,403
8
1,411
57.40
Self-RAG
3,461
159
3,619
42.20
RCR
4,044
106
4,150
56.00
MemR 3
6,195
314
6,509
72.00
Full-Context
109,624
12
109,636
59.80
GPT-4.1-mini
MemR 3
6,596
486
7,082
77.20
Appendix
Table 13: Average query-time tokens per question on LongMemEval s , including all controller, state-generation, and answer-generation calls. GPT-5-mini completion counts include internal reasoning.
Figure 6: Average token usage of the retrieved context (y-axis) and LLM-as-a-Judge (J) Score (x-axis) of RAG, Self-RAG, RCR, MemR 3 , and Full-Context across four categories on LoCoMo.
Metric
RAG
Self-RAG
RCR
Full-Context
MemR 3 (default)
MemR 3 (concise)
EM
23.60
0.20
0.00
24.20
0.00
19.00
F1
43.21
14.28
13.90
46.48
17.72
43.49
J
57.40
42.20
56.00
59.80
72.00
68.40
Appendix
Table 14: Auxiliary metrics (%) on LongMemEval s with GPT-4o-mini.
Figure 7: Number of questions requiring different numbers of iterations before final answers, across four categories on LoCoMo (left) and six categories on LongMemEval s (right).
Method
RAG
Full-Context
MemR 3 (stall-2)
MemR 3 (stall-3)
MemR 3
J (%)
57.40
59.80
66.00
67.40
72.00
Appendix
Table 15: J scores (%) on LongMemEval s with GPT-4o-mini. Stall- m stops after m consecutive iterations without evidence-state updates.
Figure 8: A successful case executed by MemR 3 on LoCoMo.
Figure 9: A failed case executed by MemR 3 with coverage bias (false negatives). Although the retrieved snippets contain evidence supporting the requirement, the LLM-based tracker fails to extract and record it in Ek .
Figure 10: A failed case executed by MemR 3 with hallucination bias (false positives).
Figure 11: A case executed by MemR 3 with granularity bias. Although the final answer is correct, it cannot close the loop until the max iteration is reached.
Category
Category 1
Category 2
Category 3
Category 4
Category 5
Name
Multi-Hop
Temporal
Open-Domain
Single-Hop
Adversarial
# Questions
282
321
96
830
445
Appendix
Table 16: The alignment of the category orders and names in the LoCoMo dataset.