cs.CVMar 31, 2026
SaveMulti-Layered Memory Architectures for LLM Agents: An Experimental Evaluation of Long-Term Context Retention
Organizations: Fulloop
Abstract
Long-horizon dialogue systems suffer from semanticdrift and unstable memory retention across extended sessions. This paper presents a Multi-Layer Memory Framework that decomposes dialogue history into working, episodic, and semantic layers with adaptive retrieval gating and retention regularization. The architecture controls cross-session drift while maintaining bounded context growth and computational efficiency. Experiments on LOCOMO, LOCCO, and LoCoMo show improved performance, achieving 46.85 Success Rate, 0.618 overall F1 with 0.594 multi-hop F1, and 56.90% six-period retention while reducing false memory rate to 5.1% and context usage to 58.40%. Results confirm enhanced long-term retention and reasoning stability under constrained context budgets.
Figures & tables
| Ref | Dataset Used | Methodology | Limitation | Evaluation Results (Reported in Paper) |
| Hu et al. [ 10 ] | Blocksworld, Gripper, Tyreworld, Barman, Jericho | Hierarchical working memory using subgoal-based memory chunks; summarized observation replacement for selective retention | Focus limited to in-trial working memory; no cross-session long-term evaluation | Overall Results: SR 21.00 42.00; PR 38.61 62.55; Steps 26.41 22.61; Context usage 100% 64.98%; Time 100% 80.58% |
| Ming et al. [ 11 ] | Chat-based long-context dialogue tasks | Integrated Long-Short Term Memory Architecture (ILSTMA) combining systematic memory space planning with most-relevant dialogue retrieval strategy | Additional memory planning overhead; evaluated primarily in chat scenarios | Execution efficiency improved by 21.45%; answer accuracy increased to 88.4%; outperforms baseline long-term retrieval architectures |
| Kang et al. [ 12 ] | GVD; LoCoMo | Three-tier Memory Operating System (short-, mid-, long-term) with dialogue-chain FIFO updating, segmented paging, and tier-aware retrieval-generation integration | Primarily dialogue-focused; evaluation limited to conversational benchmarks | GVD (GPT-4o-mini): Acc 93.3, Corr. 91.2, Coher. 92.3 (+5.4% over best baseline). LoCoMo (GPT-4o-mini): Single-hop F1 23.2, Multi-hop F1 30.1, Temporal F1 8.18, Open-domain F1 22.39; Avg Rank (F1) 1.0; improvements up to +32.35% |
| Phadke et al. [ 13 ] | Long-context dialogue benchmarks | Four-tier hierarchical memory (Working, Summarized, Archival, Flagged) with proactive write-time truth verification, token gating, and credibility-aware retrieval | Additional verification overhead; multi-agent control stack increases computational complexity | Dialogue Benchmark: Accuracy 78.2%, Understanding 94.5%, False Memory Rate 6.8%, DAR 82.4%, CDR 41.2%. SGD Benchmark: BLEU 5.75, F1 0.654, Intent Accuracy 55.5%, FMR 10.5%, DAR 73.3%, CDR 34.2% |
| Jia et al. [ 14 ] | LOCCO (3080 dialogues, 100 users); LOCCO-L | Parameter-based memory via supervised fine-tuning; dialogue QA evaluation with trained consistency model | Generated dialogues; mainly evaluates explicit factual memory | Openchat-3.5: ( ); ChatGLM3-6B retained 48.25% after 6 periods; 20-user vs 100-user ; at : 0.15 vs 0.033; Consistency model accuracy 98% |
| Maharana et al. [ 15 ] | LOCOMO | Very long-term conversational benchmark with persona and temporal event graphs | Dataset contribution only | Average 588.2 turns per conversation; 27.2 sessions; 16,618.1 tokens per conversation |
TABLE I: Comparison of multi-layer memory and long-context management approaches
Fig. 1: Descriptive overview of the multi-layer memory framework showing dialogue processing, layered memory consolidation, adaptive retrieval, retention control, and response generation for stable long-horizon interactions.
| Dataset | Ref | SR / Accu | F1 | Retention / Context | Effici |
|---|---|---|---|---|---|
| LOCOMO | [ 10 ] | 42.00 | – | 64.98% Context | 9 |
| LOCOMO | MLMF | 46.85 | 0.618 | 58.40% Context | 10.4 |
| LOCCO | [ 14 ] | 98% | – | 48.25% Retention | – |
| LOCCO | MLMF | 99.1% | – | 56.90% Retention | – |
| LoCoMo | [ 18 ] | – | 0.583 | – | – |
| LoCoMo | MLMF | – | 0.618 (0.594 Multi-hop) | – | – |
TABLE II: Performance Comparison Across LOCOMO, LOCCO, and LoCoMo
Fig. 2: Retention after six temporal periods on LOCCO. MLMF achieves higher long-term retention compared to Jia et al. [ 14 ] .
| Ref | SR / Accu | F1 | BLEU-1 | Retention / Context | Efficiency |
| [ 10 ] | SR: 42.00 | – | – | Context: 64.98% usage | 9 speedup |
| [ 12 ] | Acc: 93.3 | 0.301 (Multi-hop) | – | – | +32.35% improvement |
| [ 13 ] | 78.2% | 0.654 | 5.75 | FMR: 6.8% | – |
| [ 18 ] | – | 0.583 | 0.599 | – | +78.3% over A-MEM |
| [ 14 ] | 98% (consistency model) | – | – | 48.25% retention after 6 periods | – |
| [ 20 ] | – | – | – | LongBench Avg: 41.45 (25% KV) | 9 faster decoding |
TABLE III: Comparison of Performance Metrics with Prior Work
Fig. 3: F1 score comparison across representative memory architectures. The dotted line indicates the strongest baseline performance.
| Method | Retention (6 periods) | False Memory Rate | Context Usage |
|---|---|---|---|
| [ 14 ] | 48.25% | – | – |
| [ 13 ] | – | 6.8% | – |
| [ 10 ] | – | – | 64.98% |
| MLMF | 56.90% | 5.1% | 58.40% |
TABLE IV: Long-Term Retention and Stability Comparison on LOCCO
| Variant | F1 | Retention (6 periods) | FMR |
|---|---|---|---|
| (No Semantic Layer) | 0.591 | 50.84% | 6.4% |
| (No Episodic Consolidation) | 0.602 | 52.13% | 6.1% |
| (No Retention Loss) | 0.608 | 53.27% | 6.9% |
| No Adaptive Gating | 0.604 | 52.98% | 6.5% |
| Full MLMF | 0.618 | 56.90% | 5.1% |
TABLE V: Ablation Study on Memory Components
Fig. 4: Ablation analysis of MLMF components across F1, retention, and false memory rate. The full model consistently outperforms its reduced variants, confirming the contribution of semantic consolidation, episodic accumulation, adaptive gating, and retention regularization.
More in this topic
Persistent Memory for Language Models
Persistent memory for language models studies how models retain, update, and use information across interactions and beyond their context windows.
See all 271 papers