ETAMem: Decoupling and Aligning Heterogeneous Memory for Long-Term Conversational Agents
Organizations: Nankai University, Tianjin, China · Tiangong University, Tianjin, China · The University of Edinburgh, Edinburgh, United Kingdom
Abstract
Long-term memory enables Large Language Models (LLMs) to maintain coherent and personalized interactions across extended conversations. However, existing memory systems face two key challenges: (1) Information Entanglement, where heterogeneous information is mixed within coarse-grained memory units; and (2) Organizational Mismatch, where memory structures based primarily on semantic similarity may overlook other associations. To address these challenges, we propose ETAMem, a multi-view memory framework that decouples heterogeneous conversational information and aligns memory representations and organization with its distinct characteristics. Specifically, heterogeneous information parsing extracts atomic memory units and identifies affective information. View-specific memory structuring then organizes these units into three complementary hierarchical views: Event, Affect, and Time. ETAMem incrementally maintains memory structures with adaptively calibrated thresholds. For retrieval, entropy-based retrieval allocates query-specific budgets across views and performs hierarchical search within each view. Experiments on LoCoMo and LongMemEval-S demonstrate that ETAMem outperforms state-of-the-art baselines, achieving relative accuracy improvements of 12.46% and 7.96%, respectively, while maintaining competitive token efficiency. Further analyses demonstrate the complementary benefits of different memory views.
Figures & tables
| Dataset | View | Structural Parameters | Maintenance Thresholds | |||||
| LoCoMo | Event | 29 | 6 | 4 | 0.402 | 0.460 | 0.515 | 0.572 |
| Affect | 33 | 6 | 4 | 0.446 | 0.504 | 0.562 | 0.619 | |
| LongMemEval | Event | 22 | 5 | 4 | 0.426 | 0.481 | 0.534 | 0.582 |
| Affect | 11 | 4 | 4 | 0.435 | 0.486 | 0.537 | 0.582 | |
| Method | Single-Hop | Temporal | Open-Domain | Multi-Hop | Overall | Token Cost | |
| (841Q) | (321Q) | (96Q) | (282Q) | (1540Q) | Build (k) | Inference | |
| MemoryBank | – | 8063.77 | |||||
| A-MEM | 1149.43 | 2431.40 | |||||
| Mem0 | 1693.39 | 1070.10 | |||||
| MemoryOS | 287.00 | 4659.09 | |||||
| MemOS | – | 1371.42 | |||||
| Method | KU | MS | SSA | SSP | SSU | TR | Overall | Token Cost | |
| (78Q) | (133Q) | (56Q) | (30Q) | (70Q) | (133Q) | (500Q) | Build (k) | Inference | |
| MemoryBank | – | 13906.81 | |||||||
| A-MEM | 1605.81 | 3971.60 | |||||||
| Mem0 | 1152.62 | 1647.56 | |||||||
| MemoryOS | 2938.41 | 7574.30 | |||||||
| MemOS | – | 1091.51 | |||||||
| Method | LoCoMo | LongMemEval-S |
| ETAMem | 84.68 | 83.00 |
| w/o AMU | 82.14 | 80.60 |
| w/o Memory Tree | 82.86 | 80.60 |
| w/o Fallback | 84.48 | 74.20 |
| Method | LoCoMo | LongMemEval-S | ||||||||||
| Overall | Single | Temp. | Open | Multi | Overall | KU | MS | SSA | SSP | SSU | TR | |
| ETAMem | 84.68 | 88.47 | 80.06 | 80.21 | 80.14 | 83.00 | 87.18 | 78.95 | 92.86 | 76.67 | 95.71 | 75.19 |
| Unified Semantic | 83.31 | 87.75 | 76.64 | 77.08 | 79.79 | 80.49 | 84.61 | 76.69 | 85.71 | 73.33 | 95.71 | 73.28 |
| Unified Affective | 77.20 | 80.85 | 71.34 | 66.67 | 76.60 | 80.60 | 83.33 | 74.44 | 92.86 | 76.67 | 92.86 | 74.44 |
| Unified Temporal | 82.92 | 86.92 | 77.88 | 75.00 | 79.43 | 79.20 | 80.76 | 73.68 | 82.14 | 76.67 | 94.29 | 75.19 |
| Method | Overall | KU | MS | SSA | SSP | SSU | TR |
| Shared Text | 72.80 | 78.21 | 69.92 | 89.29 | 73.33 | 92.85 | 54.89 |
| Shared Affect | 74.40 | 76.92 | 69.92 | 89.29 | 76.67 | 94.28 | 60.15 |
| ETAMem | 80.40 | 87.18 | 70.68 | 91.07 | 93.33 | 95.71 | 70.68 |
| Configuration | LoCoMo | LongMemEval-S |
| ETAMem | 80.84 | 80.40 |
| Route Configuration | ||
| Event-only | 73.44 | 74.00 |
| Time-only | 77.73 | 74.40 |
| Affect-only | 56.23 | 37.40 |
| Event + Time | 77.92 | 74.80 |
| Method | Choice Acc. | Memory Helpful | Memory Recall |
| A-MEM | 87.48 | 0.6275 | 0.5490 |
| Mem0 | 85.28 | 0.6182 | 0.4860 |
| Flat | 87.74 | 0.7767 | 0.6492 |
| ETAMem | 92.59 | 0.9200 | 0.8031 |
| Question | What brand of BBQ sauce am I currently obsessed with? |
| View Routing | Event: weight 0.284; Affect: weight 0.452; Time: weight 0.264 |
| Event Tree | BBQ cooking activities brisket, ribs, pulled pork, and sauce recommendations general BBQ sauce alternatives, including Sweet Baby Ray’s BBQ sauce |
| Affect Tree | food preferences and enthusiasm favorite and currently preferred BBQ sauces the user is currently obsessed with Kansas City Masterpiece BBQ sauce |
| Time Tree | WEEK_2023-26 DAY_2023-06-30 BBQ cooking and sauce-preference session |
| Answer | The user is obsessed with Kansas City Masterpiece BBQ sauce . |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| Failure Mode | Question and Outcome | View Routing | Retrieved Path and Evidence Coverage |
| Partial Retrieval Failure | Dataset: LongMemEval Question: How many different types of citrus fruits have I used in my cocktail recipes? Gold: 3 Prediction: 2 (orange and lemon) | Event: 0.333 Affect: 0.147 Time: 0.519 | Event view: culinary experiences cocktail recipes Time view: WEEK_2023-W21 DAY_2023-05-21 orange-bitters session Coverage: the orange- and lemon-related evidence is retained, but the May 25 Daiquiri memory containing fresh lime juice is omitted; recall . |
| Retrieval Miss | Dataset: LoCoMo Question: What book did Caroline recommend to Melanie? Gold: Becoming Nicole Prediction: The specific book is not mentioned | Event: 0.613 Affect: 0.147 Time: 0.239 | Event view: personal growth and art pottery and creative activities Affect view: support and encouragement self-acceptance Time view: July 2023 sessions Coverage: the recommendation turn containing Becoming Nicole is absent after both dialogue fallback; recall . |
| Given a memory unit, determine whether it contains affective information based solely on its explicitly expressed meaning. |
| Affective (1): The memory unit expresses emotions, psychological feelings, personal preferences, likes or dislikes, or sentiment-bearing subjective evaluations and attitudes. |
| Non-Affective (0): The memory unit describes neutral events, factual information, objective attributes, general plans, task-oriented intentions, or information requests without explicit affective content. |
| Guidelines: Judge only the information expressed in the memory unit. Do not infer emotions from surrounding dialogue or the potential implications of an event. Distinguish emotional expectations from general plans, and subjective evaluations from objective descriptions. |
| Output: Assign exactly one binary label (0 or 1) to each memory unit. |
| Determine whether the atomic fact expresses a person’s subjective affect, such as emotion, attitude, preference, like/dislike, desire, or evaluation. |
| Answer "yes" if it contains subjective affective information. Answer "no" if it is only an objective or factual statement. |
| Do not infer emotions that are not explicitly stated. |
| Return only: yes or no |
| Method | Precision | Recall | F1 |
| GPT-4o-mini | 73.60 | 86.18 | 79.39 |
| RoBERTa-Sentiment | 80.79 | 80.26 | 80.53 |
| Dataset | Question Type | Retrieved | Unretrieved | Partially Retrieved | |||
| Correct | Incorrect | Correct | Incorrect | Correct | Incorrect | ||
| LongMemEval | KU | 87.2 | 12.8 | 0.0 | 0.0 | 0.0 | 0.0 |
| MS | 78.9 | 17.3 | 0.0 | 0.0 | 0.0 | 3.8 | |
| SSA | 91.1 | 7.1 | 1.8 | 0.0 | 0.0 | 0.0 | |
| SSP | 76.7 | 20.0 | 0.0 | 3.3 | 0.0 | 0.0 | |
| SSU | 95.7 | 4.3 | 0.0 | 0.0 | 0.0 | 0.0 | |