Learning to Retrieve Missing Evidence for Long-Term Memory QA

Yi-Xuan Deng; Yi Zhang; Wei Liu; Chao Xue; Shuojin Yang

cs.CLSep 29, 2026

Learning to Retrieve Missing Evidence for Long-Term Memory QA

Authors: Yi-Xuan Deng, Yi Zhang, Wei Liu, Chao Xue, Shuojin Yang

Organizations: Tsinghua University

Abstract

Long-term memory enables language models to use past interactions in future conversations. However, evidence needed to answer a question may be scattered across distant turns, while the question itself omits clues needed to locate it. Retrieved facts can reveal these clues, motivating retrieval decisions conditioned on evidence already found. We introduce MERA (Missing-Evidence Retrieval Augmentation), which separates globally searchable memory from a question-specific evidence state. Verified evidence guides subsequent retrieval without restricting access to the global memory. We train a lightweight planner through reinforcement learning, rewarding queries that recover previously missing evidence. MERA achieves strong answer accuracy across Qwen3-30B and GPT-4o-mini backbones. With Qwen3-30B for evidence processing and answer generation, the trained 0.6B planner achieves 77.40% accuracy on LoCoMo and 71.29% on LongMemEval-S, exceeding a 30B planner without retrieval-grounded training by 4.10% and 3.96%, respectively. On LoCoMo, later retrieval rounds increase cumulative evidence recall from 55.5% to 80.5%.

Figures & tables

Figure 1: A question-specific evidence state should guide retrieval without defining the search boundary. (a) A preconstructed graph’s explored region may exclude relevant evidence. (b) MERA searches a separate global memory index; verified evidence updates the state and guides subsequent retrieval. Orange and green diamonds mark required and verified evidence; panels are schematic.

Figure 2: Overview of MERA. (a) Conversation turns are decomposed into atomic facts and indexed in global memory. (b) Planning, retrieval, aggregation, and verification update the evidence state for continued retrieval or answering. (c) Newly recovered gold evidence provides the reward for GRPO; the dashed arrow denotes the training-only planner update.

LongMemEval-S
Backbone	Method	ACC (%)	Sum. In (k)	Sum. Out (k)	Upd. In (k)	Upd. Out (k)	Total (k)
Qwen3-30B	FullText	54.80	–	–	–	–	105.07
	NaiveRAG	60.80	–	–	–	–	–
	IRCoT ∗	55.45	–	–	–	–	–
	GraphReader ∗	46.53	155.52	99.61	–	–	255.13
	LangMem	50.80	–	–	1,311.96	118.06	1,430.02

Table 1: Main results on LongMemEval-S and LoCoMo. ACC is answer accuracy. Offline generative-model costs are reported in thousands per question (k/Q) for LongMemEval-S and thousands per conversation (k/Conv.) for LoCoMo. Superscript ∗ marks results obtained from our own runs. † † footnotemark: “–” denotes not applicable or not reported; IRCoT makes no offline generative calls.

Planner	LoCoMo	LongMemEval-S
MERA-Small	68.40	59.80
MERA-Large	73.30	67.33
MERA-RL	77.40	71.29

Table 2: Planner-scale and policy-training ablation: answer accuracy (%).

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Histories	Generated QAs	Collected states	Rollout input	Retained
LoCoMo	170	24,787	74,742	74,742	2,996 (4.0%)
LongMemEval	277	11,017	17,752	14,475	2,312 (16.0%)

Appendix

Table 3: Construction and filtering of planner-training data. A state is the input saved before one planner call in a retrieval trajectory. Retention is measured against the states entering rollout filtering.

Method	In (k)	Out (k)	Total (k)
IRCoT	4.29	0.13	4.43
GraphReader	24.50	0.54	25.04
MERA	13.47	1.80	15.27

Appendix

Table 4: Online inference cost on LoCoMo, in thousands of tokens per question (k/Q).

Method	In (k)	Out (k)	Total (k)
IRCoT	19.02	0.24	19.26
GraphReader	32.82	0.52	33.34
MERA	14.97	2.43	17.40

Appendix

Table 5: Online inference cost on LongMemEval-S, in thousands of tokens per question (k/Q).

Configuration	LLM calls/Q	Runtime/Q (s)	Relative runtime
MERA-Small	6.8	43.2	1.00 $\times$
MERA-RL	6.6	48.3	1.12 $\times$
MERA-Large	10.9	155.5	3.60 $\times$

Appendix

Table 6: Online runtime on LongMemEval-S. Calls and wall-clock time are averaged per question and then across three MERA-Small runs, two MERA-RL runs, and three MERA-Large runs. Relative runtime is normalized by MERA-Small.

Metric	MERA-Small	MERA-RL	Change
Average query word count	3.30	3.50	+0.20
Intents per planner call	2.75	2.98	+0.23
Normalized query repetition	17.1%	16.0%	$-1.1$ pp
JSON parsing failures	0.0%	0.0%	0.0 pp
First-round evidence recall	0.623	0.631	+0.8 pp
Final evidence recall	0.761	0.776	+1.5 pp

Appendix

Table 7: LoCoMo query behavior and retrieval outcomes before and after retrieval-grounded training. The comparison between MERA-Small and MERA-RL examines how policy training changes the use of the retrieval interface: query word count and intents per call characterize action construction; repetition and parsing failures characterize action quality; and evidence recall and premature stopping measure downstream retrieval effectiveness. Change is computed as MERA-RL minus MERA-Small.

Explore similar work

EviMem: Evidence-Gap-Driven Iterative Retrieval for Long-Term Conversational Memory
Apr 30, 2026Yuyang Li, Yime He, Zeyu Zhang +1Conversational Memory Retrievers
The Australian National University · UNSW Sydney
Where to Look and What to Use: Retrieve-Localize-Generate for Long-Term Conversational Memory Question Answering
Sep 7, 2026Yifan Wang, Xinkui Lin, Yongxiu Xu +9Conversational Memory Peak Memory
University of Electronic Science and Technology of China, Chengdu, China 4State Key Laboratory of Internet Architecture, Tsinghua University, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China 3School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China · 5Independent Researcher
DeferMem: Query-Time Evidence Distillation via Reinforcement Learning for Long-Term Agent Memory
May 21, 2026Jianing Yin, Tan Tang, Yingcai WuLarge Language Model Memory Nativemem
State Key Lab of CAD&CG, Zhejiang University

Apr 30, 2026, cs.CVJ/K move · Enter open · S save

EviMem: Evidence-Gap-Driven Iterative Retrieval for Long-Term Conversational Memory

Yuyang Li, Yime He, Zeyu Zhang, Dong Gong

The Australian National University · UNSW Sydney

Conversational Memory Retrievers Long-Term

Open paper AskPDF arXiv

Long-term conversational memory requires retrieving evidence scattered across multiple sessions, yet single-pass retrieval fails on temporal and multi-hop questions. Existing iterative methods refine queries via generated content or document-level signals, but none explicitly diagnoses the evidence gap, namely what is missing from the accumulated retrieval set, leaving query refinement untargeted. We present EviMem, combining IRIS (Iterative Retrieval via Insufficiency Signals), a closed-loop framework that detects evidence gaps through sufficiency evaluation, diagnoses what is missing, and drives targeted query refinement, with LaceMem (Layered Architecture for Conversational Evidence Memory), a coarse-to-fine memory hierarchy supporting fine-grained gap diagnosis. On LoCoMo, EviMem improves Judge Accuracy over MIRIX on temporal (73.3% to 81.6%) and multi-hop (65.9% to 85.2%) questions at 4.5x lower latency. Code: https://github.com/AIGeeksGroup/EviMem.

Abstract

Explore similar work

EviMem: Evidence-Gap-Driven Iterative Retrieval for Long-Term Conversational Memory

Where to Look and What to Use: Retrieve-Localize-Generate for Long-Term Conversational Memory Question Answering

DeferMem: Query-Time Evidence Distillation via Reinforcement Learning for Long-Term Agent Memory