cs.AIDec 23, 2025

Knowing What is Missing: Efficient Conversational Memory via Explicit Evidence-Gap Tracking

Authors: Xingbo Du, Loka Li, Duzhen Zhang, Leonard Song, Le Song

Organizations: Mohamed bin Zayed University of Artificial Intelligence.

Abstract

Long-term conversational memory poses a fundamental system challenge for LLM agents: reasoning over all past interactions improves coverage but incurs prohibitive token costs and noise, while retrieval-based methods are efficient yet often fail on multi-hop and temporal questions. Existing multi-step retrievers partially address this gap, but typically operate over an ever-growing textual history, causing context expansion and noise accumulation across iterations. We propose MemR3^3, a backend-agnostic closed-loop controller that reformulates conversational memory retrieval as explicit stateful decision-making. At each iteration, MemR3^3 maintains an evidence-gap state consisting of grounded information (evidence) and unresolved information requirements (gaps), and uses this state to route among three actions: retrieve, reflect, and answer. Newly retrieved snippets are incorporated into the state and then masked from subsequent prompts, so the model conditions on a compact summary of progress rather than the full retrieval history. This design turns multi-step memory retrieval into a bounded, inspectable control process whose query-time context scales with the state instead of raw accumulated text. Experiments on LongMemEvals_s show that MemR3^3 surpasses implicit multi-step retrieval baselines and often approaches or exceeds full-context prompting while retrieving only 1%-5% of the full conversation tokens on long conversations. On LoCoMo, MemR3^3 consistently improves over its underlying RAG- and Zep-based memory backbones. These results suggest that explicit evidence-gap tracking is an effective abstraction for token-efficient conversational memory.

Figures & tables

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. JustMem: Just-Enough Memory Access for Long-Term Conversations

    Sep 17, 2026Guanhua Chen, Yanting Wang, Wenjing Zhi +1Conversational MemoryConversational QA

  2. EviMem: Evidence-Gap-Driven Iterative Retrieval for Long-Term Conversational Memory

    Apr 30, 2026Yuyang Li, Yime He, Zeyu Zhang +1Temporal IREvidence Retrieval

  3. HiGMem: A Hierarchical and LLM-Guided Memory System for Long-Term Conversational Agents

    Apr 20, 2026Shuqi Cao, Jingyi He, Fei TanPersistent Memory for Language ModelsHierarchical Memory