From Relevant Memories to Sufficient Evidence: Rethinking Memory Utilization in LLM Agents

Shixuan Li; Wei Yang; Peiyu Zhang; Anzhe Cheng; Heng Ping; Xiaole Zhang; Jesse Thomason; Xuezhe Ma

cs.AIOct 15, 2026

From Relevant Memories to Sufficient Evidence: Rethinking Memory Utilization in LLM Agents

Authors: Shixuan Li, Wei Yang, Peiyu Zhang, Anzhe Cheng, Heng Ping, Xiaole Zhang, Jesse Thomason, Xuezhe Ma,

Organizations: University of Southern California · Georgia Institute of Technology

Abstract

Long-term memory enables LLM agents to reuse past experiences, yet retrieving relevant memories does not necessarily provide sufficient evidence for answering a query. Essential information may remain fragmented across memory records, obscured by abstraction, or disconnected from the relations required for reasoning. We identify this relevance-sufficiency gap and propose QEMem, a framework that reformulates memory utilization as the construction of query-specific evidence sets, shifting the focus from individual retrieval relevance to collective evidential support. QEMem integrates three complementary mechanisms: query-guided discovery retrieves evidence across multiple memory granularities; source-grounded reification organizes it into structured ViewCards with explicit evidential roles and source references; and sufficiency-guided refinement selectively recovers missing information. Operating on existing memory stores without modifying memory formation, QEMem achieves 91.43% LLM-judge accuracy on LoCoMo with Claude Opus 4.5, outperforming the strongest evaluated baseline, MemR $^3$ , by 7.6 percentage points and NEMORI by 10.7 points under a shared memory store. Further analyses demonstrate improved evidence coverage and strong performance on questions requiring cross-session evidence integration.

Figures & tables

Figure 1: Overview of QEMem. The framework addresses the relevance–sufficiency gap by moving from individually relevant memories to collectively sufficient evidence. Given a query, QEMem discovers candidate evidence across multiple memory granularities, constructs source-grounded evidence representations, and iteratively refines the evidence set based on identified information gaps. The resulting query-specific evidence set supports grounded answer generation.

	Claude Haiku 4.5					Claude Opus 4.5
Method	Multi-Hop	Temporal	Open	Single-Hop	Overall	Multi-Hop	Temporal	Open	Single-Hop	Overall
NEMORI	69.50	71.65	50.00	83.71	76.49	70.92	77.88	61.46	87.28	80.71
Mem0	65.25	61.68	52.08	74.67	68.83	62.77	68.85	62.50	80.26	73.57
A-Mem	50.35	32.09	44.79	68.73	56.23	51.77	29.28	48.96	73.60	58.83
LightMem	63.48	81.62	57.29	78.24	74.94	62.41	84.42	59.38	80.14	76.49
SimpleMem	78.72	79.13	53.12	80.62	78.25	78.01	81.62	59.38	84.54	81.17

Table 1: Main results on LoCoMo across two Claude backbones, measured by LLM-judge accuracy (%). All memory stores are constructed using Claude Opus 4.5, while query-time processing uses the indicated backbone. Bold and underlined values indicate the best and second-best results within each backbone, respectively.

Figure 2: Evidence coverage and answer accuracy on LoCoMo. (a) Complete coverage of annotated supporting turns. Retrieval-source coverage (

R

) includes turns belonging to retrieved episodes or present in the final raw-turn context. Source visibility (

V

) counts original turns included verbatim in the answer prompt. Card-source coverage (

C

) includes directly cited turns and all turns belonging to cited episodes. Coverage is complete when all annotated supporting turns are covered. (b) Answer accuracy grouped by NEMORI’s retrieval-source coverage. Results use Opus 4.5 on 1,531 questions with fully mappable evidence annotations.

Variant	Multi-Hop	Temporal	Open-Domain	Single-Hop	Overall	$\boldsymbol{\Delta}$	Visibility
Matched-Budget Retrieval	79.1	80.4	64.6	88.7	83.7	$-5.9$	77.0
Query-Guided Retrieval	84.4	85.0	66.7	93.5	88.4	$-1.2$	82.6
w/o Evidence Refinement	84.0	88.2	67.7	93.6	89.1	$-0.5$	89.3
Full QEMem	86.9	89.1	68.8	93.1	89.6	–	91.6

Table 2: Ablation analysis of query-specific evidence construction on LoCoMo. Accuracy (%) is evaluated on 1,540 questions;

\Delta

is the difference from Full QEMem in percentage points. Visibility is the percentage of questions with all annotated supporting turns exposed verbatim in the final answer context, evaluated on 1,531 questions with fully mappable annotations. Matched-Budget Retrieval matches Full QEMem’s final answer-context token budget. Bold indicates the best result in each accuracy and visibility column.

Grounding Metric	Haiku 4.5	Opus 4.5
Provenance Resolution	98.4	100.0
Exact Quotation Fidelity	82.7	95.9
Normalized Quotation Fidelity	85.5	97.7
Multi-Excerpt Quotation Fidelity	90.0	99.1
Original-Turn Traceability	100.0	100.0

Table 3: Source traceability and quotation fidelity of QEMem ViewCards on LoCoMo. Percentages are computed over all generated cards (10,121 for Haiku and 10,213 for Opus), except original-turn traceability, which uses cards citing original dialogue turns. Multi-excerpt fidelity permits segment-wise quotation matching.

Evidence Complexity	$\boldsymbol{n}$	NEMORI	SimpleMem	MemR 3	QEMem	$\boldsymbol{\Delta}$ vs. Best
Single evidence turn	1122	79.9	80.8	86.8	92.1	+5.3
$\geq$ 3 evidence turns	184	63.6	75.0	53.8	81.0	+6.0
Cross-session evidence	332	63.3	72.0	59.0	81.3	+9.3
Cross-session list/count	235	59.6	74.0	54.5	80.4	+6.4

Table 4: LLM-judge accuracy (%) across evidence-complexity groups on LoCoMo. Overlapping groups are defined by supporting turns and sessions among 1,535 questions with valid evidence annotations.

\Delta

is QEMem’s gain over the strongest included baseline per group (percentage points). Bold/underlined values mark the best/second-best accuracy.

Figure 3: Complete evidence coverage under source-turn budgets. (a) Haiku 4.5. (b) Opus 4.5. Curves measure coverage of all annotated supporting turns in offline retrieval prefixes, preserving retrieval order and candidate boundaries. Budgets count unique original turns exposed verbatim. QEMem uses its initial retrieval pool. NEMORI (actual) exposes source turns from its top-two episodes; NEMORI top-10 hypothetically expands all ten retrieved episodes into their associated source turns. Results cover 1,531 questions with fully mappable evidence annotations.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Method / Metric	Multi-Hop	Temporal	Open-Domain	Single-Hop	Overall
NEMORI – R	31.5	75.6	38.0	82.0	68.8
NEMORI – V	1.8	19.7	8.7	20.5	16.2
QEMem – R	78.8	97.8	70.7	99.3	93.5
QEMem – V	72.8	95.9	69.6	98.7	91.6
QEMem – C	49.5	90.9	59.8	94.2	83.3

Appendix

Table 5: Additional RQ2 analysis with Claude Haiku 4.5. (a) Complete coverage of annotated supporting evidence across question categories. R denotes retrieval-record coverage, V denotes original source-turn visibility in the answer prompt, and C denotes ViewCard citation coverage. (b) Answer accuracy conditioned on NEMORI’s retrieval coverage. All values are percentages unless otherwise stated. Results are computed on 1,531 questions with fully mappable evidence annotations.

Backbone	Samples	Fully Supported	Partially Supported	Unsupported	Contradicted	Full + Partial
Haiku 4.5	144	57.6	39.6	0.7	2.1	97.2
Opus 4.5	144	66.0	32.6	0.7	0.7	98.6

Appendix

Table 6: Claim–source semantic consistency of sampled ViewCards. Results are based on 288 stratified samples evaluated by Claude Opus 4.5. The percentages below are unweighted sample proportions. Fully or partially supported claims include both strict entailment and claims containing additional details not fully established by the cited sources. Population-weighted estimates are reported in the text.

Backbone	Time Fields	Mechanically Verified	LLM-Written Time Consistency	Yesterday / Last Night	Last Year
Haiku 4.5	9,541	5,398/5,746 (93.9%)	4,857/5,205 (93.3%)	121/194 (62.4%)	39/64 (60.9%)
Opus 4.5	9,598	5,822/6,060 (96.1%)	5,287/5,525 (95.7%)	176/218 (80.7%)	49/65 (75.4%)

Appendix

Table 7: Temporal-grounding consistency of QEMem ViewCards. The audit covers cards with explicit time fields and reports agreement with deterministic source-timestamp-based rules. LLM-written time fields exclude values rewritten by the pipeline’s deterministic normalizer. Relative-time results are reported over applicable cases only.

Figure 4: Accuracy–cost tradeoffs on LoCoMo. Query-time inference cost versus LLM-judge accuracy for (a) Claude Haiku 4.5 and (b) Claude Opus 4.5. Dashed lines indicate the Pareto frontier within each backbone. Query-guided retrieval with direct answering improves both accuracy and cost relative to matched-context retrieval. Full QEMem achieves the highest observed accuracy while providing structured evidence representations and selective refinement. Costs include all query-time LLM calls but exclude shared memory construction and offline evaluation.

Method	Accuracy	Calls	Input Tokens	Output Tokens	Cost (cents)	Visibility
(a) Claude Haiku 4.5
NEMORI	76.5	1.00	5,186	205	0.68	16.2
Matched-Budget Retrieval	83.7	1.04	12,119	224	1.46	77.0
Query-Guided Retrieval (Direct Answer)	88.4	2.02	9,784	344	1.27	82.6
QEMem w/o Refinement	89.1	3.01	19,396	971	2.67	89.3
Full QEMem	89.6	3.34	23,003	1,199	3.19	91.6

Appendix

Table 8: Query-time efficiency comparison on LoCoMo. Accuracy and complete evidence visibility are percentages. Input and output tokens are average counts per question. Cost is reported in US cents per question. Visibility measures complete coverage of annotated supporting turns in the final answer context, evaluated on 1,531 questions with fully mappable evidence annotations.

Figure 5: Answer accuracy across evidence cardinality on LoCoMo. Results are grouped by the number of annotated supporting dialogue turns: one turn (

n=1{,}122

), two turns (

n=229

), and three or more turns (

n=184

). (a) Claude Haiku 4.5. (b) Claude Opus 4.5. QEMem maintains the highest absolute accuracy among the evaluated methods across all complexity groups, while NEMORI and MemR 3 exhibit substantially greater performance degradation as evidence cardinality increases.

(a) Claude Haiku 4.5
Temporal Span	$\boldsymbol{n}$	NEMORI	SimpleMem	MemR 3	QEMem
Same day	81	81.5	69.1	79.0	88.9
1–30 days	83	62.7	72.3	66.3	80.7
31–180 days	183	67.8	73.2	59.6	85.2
$>180$ days	66	51.5	68.2	48.5	71.2
(b) Claude Opus 4.5

Appendix

Table 9: Accuracy across temporal dispersion of supporting evidence. Temporal span is measured between the earliest and latest annotated supporting sessions for questions with multiple supporting turns. All results are LLM-judge accuracy percentages.

1 supporting turn $\rightarrow$ 3+ supporting turns
Complexity Contrast	Baseline	Haiku 4.5	Opus 4.5
	NEMORI	$+5.3\;[-4.4,\,+16.8]$	$+8.7\;[+3.0,\,+15.5]$
	SimpleMem	$-5.3\;[-14.7,\,+4.2]$	$-2.1\;[-7.8,\,+4.6]$
	MemR 3	$\mathbf{+21.9}\;[+11.1,\,+36.4]$	$\mathbf{+18.4}\;[+11.9,\,+27.9]$
Single-session $\rightarrow$ cross-session evidence
	NEMORI	$+6.3\;[-0.4,\,+13.4]$	$+8.2\;[+4.5,\,+11.6]$

Appendix

Table 10: Change in QEMem’s accuracy advantage under increasing evidence complexity. Values represent

\Delta_{\mathrm{gap}}

in percentage points with 95% conversation-cluster bootstrap confidence intervals. Positive values indicate a larger advantage for QEMem in high-complexity groups.

(a) Claude Haiku 4.5
Retrieval Strategy	$\boldsymbol{B=20}$	$\boldsymbol{B=40}$	$\boldsymbol{B=80}$	$\boldsymbol{B=120}$	AUC
NEMORI (actual)	9.3	15.0	16.2	16.2	14.5
NEMORI top-10 (hypothetical)	12.3	32.3	57.5	65.6	44.4
Matched Retrieval-Only	34.5	52.5	67.2	73.7	58.6
QEMem (Initial Retrieval)	56.6	67.9	80.4	82.6	73.1
(b) Claude Opus 4.5

Appendix

Table 11: Complete evidence coverage under different source-turn budgets. Coverage and normalized AUC are reported as percentages. AUC is computed over budgets of 10–120 source turns. NEMORI top-10 denotes a hypothetical offline expansion of all retrieved episodes, not the original NEMORI answer context.

Figure 6: Complete evidence recovery under source-turn token budgets. (a) Claude Haiku 4.5 and (b) Claude Opus 4.5. Budgets are measured using the cl100k_base tokenizer over original dialogue turns exposed verbatim. These counts exclude episodic summaries and semantic-memory text and are not equivalent to total answer-prompt tokens. The relative advantage of QEMem’s initial retrieval pool remains consistent with the source-turn-budget analysis in Figure 3 .

Figure 7: Answer-level groundedness across memory systems on LoCoMo. (a) Grounding classifications considering all factual claims, including auxiliary explanations. (b) Grounding classifications restricted to core claims constituting the final answer. Results are population-weighted estimates from 240 stratified questions using Claude Opus 4.5. The comparison distinguishes full grounding, partial grounding, unverifiable claims, unsupported claims, contradictions, and abstentions. Differences between the two panels highlight the influence of auxiliary explanatory content on answer-level faithfulness.

Metric	NEMORI	SimpleMem	MemR 3	QEMem
Fully Grounded $\uparrow$	62.2	82.3	66.4	77.7
Fully or Partially Grounded $\uparrow$	75.1	89.7	75.5	90.2
Correct & Fully Grounded $\uparrow$	55.4	73.6	58.9	73.2
Contradicted Answers $\downarrow$	14.3	2.4	17.7	4.6
Abstention Rate $\downarrow$	7.1	5.8	4.3	0.7
Core-Claim Groundedness $\uparrow$	76.7	82.3	75.8	79.3

Appendix

Table 12: Answer-level groundedness and faithfulness on LoCoMo. All metrics except claims per answer are percentages. Results are population-weighted estimates from 240 stratified questions, evaluated using Claude Opus 4.5 predictions. Fully grounded requires all checkable factual claims in an answer to be supported by source evidence. Core-only grounding excludes auxiliary explanations. Grounding is evaluated over complete model outputs, including auxiliary reasoning and explanations; core-answer grounding considers only claims constituting the final answer.

LLM Agent Memory

Large language model agent memory lets agents retain, organize, retrieve, and update information across interactions and long-running tasks.

See all 373 papers

Relevance Is Not Sufficiency: What Actually Closes the Evidence Gap in Long-Term Memory QA
Oct 7, 2026Yufeng Li, Shuxin Li, Zhenhua Xu +7Long-Term Conversational Memory Agent Memory Management
East China Normal University, Shanghai, China · Nanyang Technological University, Singapore · Zhejiang University, Hangzhou, China +3
Remember by Asking: Retrieval-Induced Memory Evolution for LLM Agents
Sep 28, 2026Wanqi Zhou, Jiawei Lu, Yang Wang +4LLM Agent Memory Persistent Memory for Language Models
Chang’an University
RaMem: Contextual Reinstatement for Long-term Agentic Memory
Jun 22, 2026Wei Yang, Bryce Kan, Shixuan Li +5Agent Memory LLM Agent Memory
University of Southern California

Oct 7, 2026J/K move · Enter open · S save

Relevance Is Not Sufficiency: What Actually Closes the Evidence Gap in Long-Term Memory QA

Yufeng Li, Shuxin Li, Zhenhua Xu, Junxian Li+6

East China Normal University, Shanghai, China · Nanyang Technological University, Singapore · Zhejiang University, Hangzhou, China+3

Long-Term Conversational Memory Agent Memory Management Evidence Retrieval

Open paper AskPDF arXiv

LLM agents that interact with a user across many sessions accumulate histories that exceed their context window, so they store past interactions in an external memory and answer each question from a small set of retrieved records. Existing memory systems rank records by lexical or embedding relevance, yet the top-ranked memories can each be relevant while jointly omitting a complementary fact that the answer requires, especially for multi-session and temporal questions. Drawing on the distinction between relevance and sufficiency in legal evidence scholarship, we recast memory retrieval as constructing a sufficient memory set. To operationalize this view, we introduce a blinded LLM judgment over the retrieved set, together with Gold Hit and Turn Hit as evidence-coverage proxies. We then propose Budgeted Flat Reconstruction (BFR), which builds sufficient sets over a fixed flat memory store in two stages. Specifically, we first apply Formal Concept Analysis for Memory Selection (FCA-MS) to decompose the question into information requirements and select a compact candidate subset that jointly covers them. Then, we repeatedly acquire unseen records through deeper text search or complementary entity and session views, stopping when the budget is exhausted. Experiments on LoCoMo and LongMemEval-S show that BFR outperforms same-store adaptations of recent agent-memory systems in both answer quality and evidence coverage. Specifically, on LongMemEval-S it raises judged accuracy from 72.4% to 82.2% and Turn Hit to 91.4%.

Abstract

LLM Agent Memory

Explore similar work

Relevance Is Not Sufficiency: What Actually Closes the Evidence Gap in Long-Term Memory QA

Remember by Asking: Retrieval-Induced Memory Evolution for LLM Agents

RaMem: Contextual Reinstatement for Long-term Agentic Memory