Current evaluations of long-term memory in LLMs are fundamentally static. By fixating on simple retrieval and short-context inference, they neglect the multifaceted nature of complex memory systems, such as dynamic state tracking and hierarchical reasoning in continuous interactions. To overcome these limitations, we propose MemGround, a rigorous long-term memory benchmark natively grounded in rich, gamified interactive scenarios. To systematically assess these capabilities, MemGround introduces a three-tier hierarchical framework that evaluates Surface State Memory, Temporal Associative Memory, and Reasoning-Based Memory through specialized interactive tasks. Furthermore, to comprehensively quantify both memory utilization and behavioral trajectories, we propose a multi-dimensional metric suite comprising Question-Answer Score (QA Overall), Memory Fragments Unlocked (MFU), Memory Fragments with Correct Order (MFCO), and Exploration Trajectory Diagrams (ETD). Extensive experiments reveal that state-of-the-art LLMs and memory agents still struggle with sustained dynamic tracking, temporal event association, and complex reasoning derived from long-term accumulated evidence in interactive environments.
Figures & tables
Figure 1: Overview of MemGround. MemGround includes three-tier hierachical memory evaluation framework, consisting of Surface State Memory, Temporal Associative Memory, and Reasoning-Based Memory. Each is evaluated in a particular gamified scenario.
Figure 2: Overview of Game Ground. Game Ground consists of two components, data collection pipeline as well as model evaluation pipeline.
Figure 3: The progress chart of GPT-5.2 and its memory agents in No Case Should Remain Unsolved. The pink, purple, yellow, and green points stand for the model unlocking a pink-lock file, unlocking a purple-lock file, unlocking a yellow-lock file, and sorting 2 memory fragments in correct timeline, respectively.
Model
Method
TRPG
No Case Should Remain Unsolved
Type Help
QA Overall ↑ (%)
QA Overall ↑ (%)
MFCO ↑ (%)
MFU ↑ (%)
QA Overall ↑ (%)
MFU ↑ (%)
Closed-Source
Gemini-3-Pro-Preview
In-context
26.28
41.61
45.46
83.33
28.62
50.91
DeepSeek-V3.2
In-context
31.41
37.58
27.27
81.48
27.98
17.27
Claude-Opus-4.6
In-context
38.07
47.76
38.64
81.48
23.81
37.27
GPT-5.2
In-context
51.51
28.56
15.91
81.48
23.61
14.55
Table 1: Main results of multiple state-of-the-art LLMs and memory agents in MemGround. ↑ indicates higher is better. The highest and second highest scores are marked in blue and green , respectively.
Figure 5: Example ETD of both human and evaluated models in a selected subsection of Type Help.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Method
Cam
Overall ↑ (%)
Acc. ↑ (%)
Inst. ↑ (%)
Cit. ↑ (%)
Read. ↑ (%)
Comp. ↑ (%)
Dep. ↑ (%)
Closed-Source
Gemini-3-Pro-Preview
In-context
T
23.66
21.39
97.86
42.78
15.78
16.88
19.66
S
30.81
32.06
96.18
70.23
18.32
20.00
29.17
C
24.38
21.98
90.11
57.14
15.38
17.50
21.24
DeepSeek-V3.2
In-context
T
27.48
26.74
96.79
60.43
12.30
25.97
26.75
S
33.14
32.82
97.71
70.23
19.08
29.09
32.08
Appendix
Table 2: QA results in TRPG. The column Cam stands for Campaign of TRPG. In the Cam column, T , S , and C stands for Terror on the Orient Express, Spring Snow Incident, and Cold Wind Howling, respectively. The columns Acc. , Inst. , Cit. , Read. , Comp. , and Dep. stand for Acc Judge score, Inst Judge score, Cit Judge score, Read Judge score, Comprehension score, and Depth score, respectively. ↑ indicates higher is better.
Model
Method
Overall ↑ (%)
Acc. ↑ (%)
Inst. ↑ (%)
Cit. ↑ (%)
Read. ↑ (%)
Comp. ↑ (%)
Dep. ↑ (%)
Closed-Source
Gemini-3-Pro-Preview
In-context
41.61
33.33
96.30
96.30
40.74
42.86
22.22
DeepSeek-V3.2
In-context
37.58
29.63
81.48
85.19
44.44
35.71
11.11
Claude-Opus-4.6
In-context
47.76
44.44
81.48
94.44
59.26
35.71
16.67
GPT-5.2
In-context
28.56
18.52
92.59
92.59
35.19
14.29
6.94
Mem0
31.87
22.22
100.00
81.48
25.93
28.57
27.08
Appendix
Table 3: QA results in No Case Should Remain Unsolved. The columns Acc. , Inst. , Cit. , Read. , Comp. , and Dep. stand for Acc Judge score, Inst Judge score, Cit Judge score, Read Judge score, Comprehension score, and Depth score, respectively. ↑ indicates higher is better. The highest scores are marked in bold .
Model
Method
Overall ↑ (%)
Acc. ↑ (%)
Inst. ↑ (%)
Cit. ↑ (%)
Read. ↑ (%)
Comp. ↑ (%)
Dep. ↑ (%)
Closed-Source
Gemini-3-Pro-Preview
In-context
28.62
26.09
71.74
47.83
25.00
22.22
26.52
DeepSeek-V3.2
In-context
27.98
28.26
80.43
50.00
21.74
16.67
26.41
Claude-Opus-4.6
In-context
23.81
23.91
78.26
50.00
17.39
11.11
22.3
GPT-5.2
In-context
23.61
23.91
100.00
36.96
14.13
16.67
19.00
Mem0
28.65
21.74
100.00
76.09
20.65
22.22
25.30
Appendix
Table 4: QA results in Type Help. The columns Acc. , Inst. , Cit. , Read. , Comp. , and Dep. stand for Acc Judge score, Inst Judge score, Cit Judge score, Read Judge score, Comprehension score, and Depth score, respectively. ↑ indicates higher is better. The highest scores are marked in bold .
Model
Common Nodes
Human Edges
Model Edges
TP
FP
FN
Precision ↑
Recall ↑
F1 ↑
Jaccard ↑
Gemini-3-Pro-Preview
8
7
4
3
1
4
0.7500
0.4286
0.5455
0.3750
DeepSeek-V3.2
8
7
5
3
2
4
0.6000
0.4286
0.5000
0.3333
Claude-Opus-4.6
8
7
11
6
5
1
0.5455
0.8571
0.6667
0.5000
GPT-5.2
8
7
3
2
1
5
0.6667
0.2857
0.4000
0.2500
Appendix
Table 5: Example ETD comparisons between human and evaluated models in a selected subsection of Type Help. ↑ indicates higher is better. The highest scores are marked in bold .
Figure 6: The progress chart of all evaluated models and memory agents in No Case Should Remain Unsolved. The pink, purple, yellow, and green points stand for the model unlocking a pink-lock file, unlocking a purple-lock file, unlocking a yellow-lock file, and sorting 2 memory fragments in correct timeline, respectively.
Figure 7: The progress chart of all evaluated models and memory agents in Type Help. The blue and yellow points stand for the model unlocking a file by itself and unlocking a file by receiving a hint, respectively.