ETAMem: Decoupling and Aligning Heterogeneous Memory for Long-Term Conversational Agents

Xinying Qian; Ying Zhang; Yu Zhao; Xubo Liu; Baohang Zhou; Jeff Z. Pan

cs.CLOct 27, 2026

ETAMem: Decoupling and Aligning Heterogeneous Memory for Long-Term Conversational Agents

Authors: Xinying Qian, Ying Zhang, Yu Zhao, Xubo Liu, Baohang Zhou, Jeff Z. Pan

Organizations: Nankai University, Tianjin, China · Tiangong University, Tianjin, China · The University of Edinburgh, Edinburgh, United Kingdom

Abstract

Long-term memory enables Large Language Models (LLMs) to maintain coherent and personalized interactions across extended conversations. However, existing memory systems face two key challenges: (1) Information Entanglement, where heterogeneous information is mixed within coarse-grained memory units; and (2) Organizational Mismatch, where memory structures based primarily on semantic similarity may overlook other associations. To address these challenges, we propose ETAMem, a multi-view memory framework that decouples heterogeneous conversational information and aligns memory representations and organization with its distinct characteristics. Specifically, heterogeneous information parsing extracts atomic memory units and identifies affective information. View-specific memory structuring then organizes these units into three complementary hierarchical views: Event, Affect, and Time. ETAMem incrementally maintains memory structures with adaptively calibrated thresholds. For retrieval, entropy-based retrieval allocates query-specific budgets across views and performs hierarchical search within each view. Experiments on LoCoMo and LongMemEval-S demonstrate that ETAMem outperforms state-of-the-art baselines, achieving relative accuracy improvements of 12.46% and 7.96%, respectively, while maintaining competitive token efficiency. Further analyses demonstrate the complementary benefits of different memory views.

Figures & tables

Figure 1. Illustration of challenges in existing memory systems and our proposed multi-view memory framework.

Figure 2. Overview of ETAMem. Given dialogue histories, the framework extracts AMUs and organizes them into three memory views: Event, Affect, and Time. During retrieval, entropy-based routing and beam search are used for response generation.

Dataset	View	Structural Parameters			Maintenance Thresholds
Dataset	View	$M_{\max}^{(v)}$	$b_{\max}^{(v)}$	$H^{(v)}$	$\theta_{\mathrm{new}}$	$\theta_{\mathrm{attach}}$	$\theta_{\mathrm{coh}}$	$\theta_{\mathrm{merge}}$
LoCoMo	Event	29	6	4	0.402	0.460	0.515	0.572
LoCoMo	Affect	33	6	4	0.446	0.504	0.562	0.619
LongMemEval	Event	22	5	4	0.426	0.481	0.534	0.582
LongMemEval	Affect	11	4	4	0.435	0.486	0.537	0.582

Table 1. Final parameter configurations of ETAMem.

Method	Single-Hop	Temporal	Open-Domain	Multi-Hop	Overall	Token Cost
Method	$\uparrow$ (841Q)	$\uparrow$ (321Q)	$\uparrow$ (96Q)	$\uparrow$ (282Q)	$\uparrow$ (1540Q)	Build (k)	Inference
MemoryBank	$46.18\pm 0.32$	$29.34\pm 0.45$	$36.67\pm 0.47$	$33.36\pm 0.54$	$39.77\pm 0.27$	–	8063.77
A-MEM	$52.82\pm 0.28$	$60.87\pm 0.37$	$43.75\pm 0.00$	$38.37\pm 0.27$	$51.29\pm 0.06$	1149.43	2431.40
Mem0	$62.09\pm 0.42$	$59.25\pm 0.41$	$37.70\pm 0.47$	$50.14\pm 0.32$	$57.79\pm 0.34$	1693.39	1070.10
MemoryOS	$68.37\pm 0.46$	$52.46\pm 0.49$	$46.67\pm 1.79$	$52.76\pm 0.49$	$60.79\pm 0.48$	287.00	4659.09
MemOS	$76.07\pm 0.10$	$69.47\pm 0.25$	$45.14\pm 0.49$	$56.85\pm 0.69$	$69.24\pm 0.11$	–	1371.42

Table 2. Performance and token costs on LoCoMo. Build and Inference denote token costs for memory construction and inference, averaged per session (k) and per question, respectively.

Method	KU	MS	SSA	SSP	SSU	TR	Overall	Token Cost
Method	$\uparrow$ (78Q)	$\uparrow$ (133Q)	$\uparrow$ (56Q)	$\uparrow$ (30Q)	$\uparrow$ (70Q)	$\uparrow$ (133Q)	$\uparrow$ (500Q)	Build (k)	Inference
MemoryBank	$21.79\pm 0.00$	$9.77\pm 0.00$	$50.00\pm 0.00$	$12.00\pm 1.83$	$29.71\pm 0.64$	$17.14\pm 0.34$	$21.04\pm 0.09$	–	13906.81
A-MEM	$72.82\pm 0.51$	$40.30\pm 0.37$	$87.50\pm 0.00$	$39.33\pm 2.49$	$82.86\pm 0.00$	$36.09\pm 0.48$	$55.44\pm 0.15$	1605.81	3971.60
Mem0	$78.72\pm 0.70$	$66.17\pm 0.92$	$51.79\pm 0.00$	$50.00\pm 2.36$	$94.29\pm 0.00$	$49.17\pm 0.67$	$64.96\pm 0.41$	1152.62	1647.56
MemoryOS	$56.15\pm 0.57$	$44.81\pm 0.41$	$78.18\pm 0.00$	$51.33\pm 1.83$	$81.14\pm 0.64$	$53.38\pm 0.00$	$58.04\pm 0.18$	2938.41	7574.30
MemOS	$76.67\pm 0.51$	$58.80\pm 0.30$	$67.86\pm 0.00$	$50.67\pm 1.33$	$93.71\pm 0.70$	$65.11\pm 0.37$	$68.68\pm 0.16$	–	1091.51

Table 3. Performance and token costs on LongMemEval-S. KU: Knowledge Update; MS: Multi-Session; SSA/P/U: Single-Session Assistant/Preference/User; TR: Temporal Reasoning. Build and Inference denote token costs for memory construction and inference, averaged per session (k) and per question, respectively.

Method	LoCoMo	LongMemEval-S
ETAMem	84.68	83.00
w/o AMU	82.14	80.60
w/o Memory Tree	82.86	80.60
w/o Fallback	84.48	74.20

Table 4. Ablation study of key components.

Method	LoCoMo					LongMemEval-S
Method	Overall	Single	Temp.	Open	Multi	Overall	KU	MS	SSA	SSP	SSU	TR
ETAMem	84.68	88.47	80.06	80.21	80.14	83.00	87.18	78.95	92.86	76.67	95.71	75.19
Unified Semantic	83.31	87.75	76.64	77.08	79.79	80.49	84.61	76.69	85.71	73.33	95.71	73.28
Unified Affective	77.20	80.85	71.34	66.67	76.60	80.60	83.33	74.44	92.86	76.67	92.86	74.44
Unified Temporal	82.92	86.92	77.88	75.00	79.43	79.20	80.76	73.68	82.14	76.67	94.29	75.19

Table 5. Effect of view-specific memory organization on LoCoMo and LongMemEval-S.

Method	Overall	KU	MS	SSA	SSP	SSU	TR
Shared Text	72.80	78.21	69.92	89.29	73.33	92.85	54.89
Shared Affect	74.40	76.92	69.92	89.29	76.67	94.28	60.15
ETAMem	80.40	87.18	70.68	91.07	93.33	95.71	70.68

Table 6. Effect of view-specific memory representations on LongMemEval-S using Qwen3-30B.

Configuration	LoCoMo	LongMemEval-S
ETAMem	80.84	80.40
Route Configuration
Event-only	73.44	74.00
Time-only	77.73	74.40
Affect-only	56.23	37.40
Event + Time	77.92	74.80

Table 7. Comparison of memory view configurations and retrieval routing strategies using Qwen3-30B.

Method	Choice Acc.	Memory Helpful	Memory Recall
A-MEM	87.48	0.6275	0.5490
Mem0	85.28	0.6182	0.4860
Flat	87.74	0.7767	0.6492
ETAMem	92.59	0.9200	0.8031

Table 8. Performance comparison on CloneMem. Choice accuracy is reported on a 0–100 scale, while memory helpful score and memory recall are reported on a 0–1 scale.

Figure 3. Distribution of routing weights across Experience, Emotion, and Opinion questions in CloneMem.

Figure 4. Hyperparameter sensitivity on LoCoMo.

Question	What brand of BBQ sauce am I currently obsessed with?
View Routing	Event: weight 0.284; Affect: weight 0.452; Time: weight 0.264
Event Tree	BBQ cooking activities $\rightarrow$ brisket, ribs, pulled pork, and sauce recommendations $\rightarrow$ general BBQ sauce alternatives, including Sweet Baby Ray’s BBQ sauce
Affect Tree	food preferences and enthusiasm $\rightarrow$ favorite and currently preferred BBQ sauces $\rightarrow$ the user is currently obsessed with Kansas City Masterpiece BBQ sauce
Time Tree	WEEK_2023-26 $\rightarrow$ DAY_2023-06-30 $\rightarrow$ BBQ cooking and sauce-preference session
Answer	The user is obsessed with Kansas City Masterpiece BBQ sauce .

Table 9. A case study of a knowledge-updating question in LongMemEval-S.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Failure Mode	Question and Outcome	View Routing	Retrieved Path and Evidence Coverage
Partial Retrieval Failure	Dataset: LongMemEval Question: How many different types of citrus fruits have I used in my cocktail recipes? Gold: 3 Prediction: 2 (orange and lemon)	Event: 0.333 Affect: 0.147 Time: 0.519	Event view: culinary experiences $\rightarrow$ cocktail recipes Time view: WEEK_2023-W21 $\rightarrow$ DAY_2023-05-21 $\rightarrow$ orange-bitters session Coverage: the orange- and lemon-related evidence is retained, but the May 25 Daiquiri memory containing fresh lime juice is omitted; recall $=0.75$ .
Retrieval Miss	Dataset: LoCoMo Question: What book did Caroline recommend to Melanie? Gold: Becoming Nicole Prediction: The specific book is not mentioned	Event: 0.613 Affect: 0.147 Time: 0.239	Event view: personal growth and art $\rightarrow$ pottery and creative activities Affect view: support and encouragement $\rightarrow$ self-acceptance Time view: July 2023 sessions Coverage: the recommendation turn containing Becoming Nicole is absent after both dialogue fallback; recall $=0$ .

Appendix

Table 10. Representative partial and complete retrieval failures.

Given a memory unit, determine whether it contains affective information based solely on its explicitly expressed meaning.

Affective (1): The memory unit expresses emotions, psychological feelings, personal preferences, likes or dislikes, or sentiment-bearing subjective evaluations and attitudes.

Non-Affective (0): The memory unit describes neutral events, factual information, objective attributes, general plans, task-oriented intentions, or information requests without explicit affective content.

Guidelines: Judge only the information expressed in the memory unit. Do not infer emotions from surrounding dialogue or the potential implications of an event. Distinguish emotional expectations from general plans, and subjective evaluations from objective descriptions.

Output: Assign exactly one binary label (0 or 1) to each memory unit.

Appendix

Table 11. Annotation guidelines for affect classification.

Determine whether the atomic fact expresses a person’s subjective affect, such as emotion, attitude, preference, like/dislike, desire, or evaluation.

Answer "yes" if it contains subjective affective information. Answer "no" if it is only an objective or factual statement.

Do not infer emotions that are not explicitly stated.

Return only: yes or no

Appendix

Table 12. Prompt for GPT-4o-mini-based affect classification.

Method	Precision	Recall	F1
GPT-4o-mini	73.60	86.18	79.39
RoBERTa-Sentiment	80.79	80.26	80.53

Appendix

Table 13. Affect classification performance.

Figure 5. Visualization of the memory structure on LoCoMo.

Figure 6. T-SNE visualization of LongMemEval memory representations under raw conversation, event, and affect view.

Dataset	Question Type	Retrieved		Unretrieved		Partially Retrieved
Dataset	Question Type	Correct	Incorrect	Correct	Incorrect	Correct	Incorrect
LongMemEval	KU	87.2	12.8	0.0	0.0	0.0	0.0
	MS	78.9	17.3	0.0	0.0	0.0	3.8
	SSA	91.1	7.1	1.8	0.0	0.0	0.0
	SSP	76.7	20.0	0.0	3.3	0.0	0.0
	SSU	95.7	4.3	0.0	0.0	0.0	0.0

Appendix

Table 14. Error distribution (%) by evidence retrieval status.

Memory Management for LLM Agents

See all 88 papers

AutoViewMem: Self-Configuring Orthogonal Views for Conversational Long-Term Memory
Sep 18, 2026Zijie Cao, Xijun Qu, Zhicheng Gu +7LLM Agent Memory Long-Term Conversational Memory
National University of Defense Technology, Changsha, China
HyperMem: Hypergraph Memory for Long-Term Conversations
Apr 9, 2026Juwei Yue, Chuanrui Hu, Jiawei Sheng +5Long-Term Conversational Memory Graph-Based Agent Memory
Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences · EverMind AI
LeanMem: Simple and Efficient Long-Term Memory for LLM Agents
Aug 4, 2026Yuxin Liao, Le Wu, Min Hou +3LLM Agent Memory Persistent Memory for Language Models
Hefei University of Technology

Sep 18, 2026J/K move · Enter open · S save

AutoViewMem: Self-Configuring Orthogonal Views for Conversational Long-Term Memory

Zijie Cao, Xijun Qu, Zhicheng Gu, Xiaoshu Chen+6

National University of Defense Technology, Changsha, China

LLM Agent Memory Long-Term Conversational Memory Agent Memory Management Memory-Augmented Language Model Agents

Open paper AskPDF arXiv

Long-term memory is essential for large language model (LLM) agents to maintain consistency and personalization over extended interactions. Existing memory systems typically rely on fixed granularities or static schemas, but these designs struggle when heterogeneous information, such as preferences, events, constraints, and temporal updates, is embedded in a single mixed representation. The resulting semantic interference makes top-K retrieval sensitive to noise and often leaves relevant evidence poorly ranked. We present AutoViewMem, a data-driven framework that organizes long-term conversational memory into self-configuring, low-overlap semantic views before indexing. AutoViewMem discovers candidate views from interaction traces, selects a compact complementary view set, and uses these views to guide write-time structured extraction of provenance-grounded memories. This representation-first design moves semantic disentanglement from retrieval time to write time, allowing standard top-K similarity search to retrieve focused evidence without explicit routing or iterative retrieval. We further apply offline consolidation to improve memory compactness and consistency. Experiments on the LoCoMo and PersonaMem benchmarks, under both Qwen3-8B and Qwen3-14B backbones, show that AutoViewMem improves long-horizon question answering and personalization over strong memory baselines while preserving a simple inference pipeline.

Abstract

Memory Management for LLM Agents

Explore similar work

AutoViewMem: Self-Configuring Orthogonal Views for Conversational Long-Term Memory

HyperMem: Hypergraph Memory for Long-Term Conversations

LeanMem: Simple and Efficient Long-Term Memory for LLM Agents