Organizations: School of Information, Renmin University of China, Beijing, China · China Telecom Research Institute, Beijing, China · MemTensor (Shanghai) Technology, ShangHai, China
Memory systems address the challenge of context loss in Large Language Models during prolonged interactions. However, as an essential tool for warm companionship and personalized services, the effectiveness of memory systems in processing affective information remains underexamined. To bridge this gap, we propose an emotion-enhanced memory evaluation benchmark to assess the performance of mainstream and state-of-the-art memory systems in integrating and harnessing emotional information. First, we design a benchmark called MemEmo to assess the affective reasoning and tracking capabilities of memory systems across four dimensions: emotional information extraction, emotional memory updating, emotional memory question-answering, and emotional conflict detection. Second, we construct the Human-Like Memory Emotion (HLME) dataset to provide an empirical foundation for scrutinizing memory systems. Finally, We evaluate five representative memory systems on the HLME dataset, reporting 11 performance metrics across four dimensions. Experimental results reveal that aggregating scores for all tested memory systems is up to 45%, far below 100% scale, exposing a profound disparity between memory system and affective intelligence.
Figures & tables
Figure 1. Example of memory system emotional processing. The right panel presents a dialogue focused on affective content. Throughout the interaction, the memory system consistently fails to capture affective nuances. For comparative fairness, all evaluated systems are represented by identical icons. The example shown in the figure demonstrates a progression of escalating affective errors and hallucinations.
Category
Benchmark
Year
TC
ASP
ATP
LTMS
MSI
EA
EQA
EIF
ERC
EO
Personalization
Emotional Interaction Frameworks
DAM-LLM ( Lu and Li, 2025 )
2025
100
1.0
10
✓
✓
✓
×
✓
×
LLM
Dynamic memory adaptation
SO-AI ( Park, 2025 )
2025
-
-
-
✓
×
✓
×
×
×
w/o
Relational identity support
EmoHarbor ( Ye et al., 2026 )
2026
100*
-
-
×
✓
✓
✓
✓
×
LLM
Inner-state simulation
EC2ER ( Sreedar et al., 2026 )
2026
-
-
-
×
×
✓
×
✓
×
LLM
Synthetic reasoning logic
CoEM ( Liu et al., 2025b )
2025
15K*
-
-
✓
✓
✓
✓
✓
×
LLM
Long-text collaborative modeling
EvoEmo ( Long et al., 2025 )
2025
-
-
-
×
×
✓
×
✓
×
LLM
Adversarial emotional strategy
Table 1. Comparison of MemEmo with related frameworks. Metrics include long-term memory support ( LTMS ), multi-session interaction ( MSI ), emotion analysis ( EA ), emotion QA ( EQA ), emotional information inference ( EIF ), and resolution conflict ( ERC ). We report evaluation object ( EO ), total conversations ( TC ), average sessions per ( ASP ), and turns per ( ATP ). ‘-’ indicates unavailable data; MS denotes Memory System. MemEmo(Medium) statistics are per-instance averages ( N denotes sample count).
Figure 2. MemEmo benchmark evaluation dimension.
Figure 3. The HLME dataset construction process pipeline.
Model
Setting
EIE
EMU
EQA
ECD
Overall
Acc ↑
MAE ↓
F1 slot↑
Acc upd↑
MAE Δ↓
MSS ↑
Acc QA↑
F1 evid↑
ρ↑
F1 conf↑
RC ↑
Score ↑
Mem0
Med.
0.1438
2.5521
0.0115
0.0132
1.9737
0.7813
0.4957
0.6140
0.7359
0.0759
0.0395
0.2791
Letta
Med.
0.1917
2.4229
0.1104
0.0461
1.9079
0.6711
0.6168
0.7058
0.7922
0.1118
0.0592
0.3337
MemOS
Med.
0.1127
2.6539
0.0382
0.0184
1.9632
0.6626
0.5846
0.7013
0.7890
0.2473
0.1411
0.3345
Mirix
Med.
0.5838
1.2453
0.2950
0.2108
1.5250
0.9474
0.6298
0.7185
0.8017
0.1237
0.0663
0.4331
MemoBase
Med.
0.6229
1.1063
0.1774
0.2632
1.4737
0.9671
0.5769
0.6910
0.7600
0.0637
0.0329
0.3849
Table 2. Overall emotion-related performance of different memory systems. ↑ / ↓ indicate higher / lower is better. MAE,MAEΔ∈[0,+∞) (lower is better); all other metrics are normalized to 1.0 ( 100% ). Bold and underline denote best and worst results .
Figure 4. Comparison of scores on subtasks for the HLME.
LLM
Model
EIE ↑
EMU ↑
EQA ↑
ECD ↑
Overall ↑
5-mini
Mem0
0.1259
0.2174
0.5829
0.1932
0.2799±0.0412
5-mini
Letta
0.3246
0.2576
0.6690
0.1735
0.3562±0.0045
5-mini
MemOS
0.2635
0.2360
0.6077
0.2438
0.3628±0.0238
5-mini
Mirix
0.4053
0.3441
0.6052
0.1301
0.3712±0.0264
5-mini
MemoBase
0.3801
0.3359
0.6133
0.1527
0.3705±0.0715
deepsk3
Mem0
0.1429
0.1138
0.5217
0.2713
0.2624±0.0389
Table 3. Memory systems were evaluated on different LLMS. Bold and underline denote best and worst results .
Memory
4o-mini vs Human
4o-mini vs 5-mini
4o-mini vs deepsk3
κ↑
r↑
ρ↑
κ↑
r↑
ρ↑
κ↑
r↑
ρ↑
Mem0
0.602
0.768
0.754
0.768
0.894
0.872
0.412
0.564
0.538
Letta
0.624
0.785
0.771
0.785
0.912
0.891
0.435
0.591
0.562
MemOS
0.615
0.772
0.760
0.752
0.881
0.859
0.398
0.542
0.515
Mirix
0.668
0.824
0.815
0.814
0.935
0.918
0.428
0.588
0.565
MemoBase
0.655
0.811
0.798
0.826
0.941
0.927
0.452
0.612
0.585
Table 4. Evaluation comparison between gpt-4o-mini and alternative LLM judges or human judgment.
Ablation
Model
EIE ↑
EMU ↑
EQA ↑
ECD ↑
Overall ↑
W/O Mem.Sys.
No-Memory
0.0000
0.1730
0.5428
0.2965
0.2531±0.0067
Full-Context
0.5372
0.6889
0.5761
0.9077
0.6752±0.0247
Oracle
0.9489
0.9950
0.7949
0.2728
0.7529±0.0040
Mem.Sys.
Mem0
0.1298
0.2215
0.6021
0.1080
0.2654±0.0441
Letta
0.1813
0.2458
0.6974
0.1601
0.3231±0.0042
MemOS
0.1060
0.2201
0.7018
0.2766
0.3314±0.0251
Table 5. Evaluation results of diagnostic baselines and various memory systems (Mem.Sys.) on the HLME dataset.
Dataset
Model
EQA (Top-20)
EQA (Top-50)
Acc QA↑
ρ↑
F1 evid.↑
Acc QA↑
ρ↑
F1 evid.↑
Medium
Mem0
0.5085
0.7420
0.6280
0.4915
0.7380
0.6350
Letta
0.6280
0.7980
0.7180
0.6215
0.7940
0.7260
MemOS
0.5980
0.7950
0.7120
0.5910
0.7910
0.7210
Mirix
0.6410
0.8080
0.7310
0.6345
0.8050
0.7420
MemoBase
0.5890
0.7680
0.7020
0.5820
0.7630
0.7110
Table 6. EQA performance in different dataset. Best results are in bold , worst results are underlined .
Dataset
System
Add(s)
Search(s)
Total(m)
In(T)
Out(T)
Cost($)
Medium
Mem0
82.93
1837.85
32.01
8139951
620555
1.59
Letta
419.22
1741.16
36.01
18284460
596802
3.10
MemOS
190.13
813.94
16.73
7735666
632969
1.54
Mirix
57.33
2741.33
46.64
18499552
662535
3.17
Memobase
59.19
1602.40
27.69
11673332
644132
2.14
Large
Mem0
230.94
2427.92
44.31
15449430
651040
2.71
Table 7. Efficiency analysis of memory systems. s, m, and t represent second, minute, and token, respectively.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5. Workflow for batch emotion evaluation in memory systems.
Metric
Medium
Large
Context Length (tokens)
170,304
1,000,229
Session Num
89
156
Dialogue Turns
2017
3169
Memories Points Num
769
769
Questions Num
234
234
Question Type Distribution
Appendix
Table 8. A representative dataset instance and the statistical summary of its key components.
Figure 6. Distribution of Mean Absolute Error (MAE) for emotional intensity estimation in the EIE task across different retrieval depths. The boxplots compare the performance stability and error variance of memory systems on both Medium (blue) and Large (orange) datasets.
Figure 7. MAEΔ metrics in different top-k for EMU task.
Figure 8. Statistical correlation analysis of metrics.
Figure 9. Comparative analysis of memory systems across EIE, EMU, EQA, and ECD tasks on the Medium (left) and Large (right) datasets under the Top-10 retrieval setting.
Memory-augmented language models combine persistent or temporary memory with neural computation to retain, retrieve, and use information beyond the active context.