Authors: Zhi Song, XiMing Xing, Chunhan Li, Weian Mao, Zhenchao Tang, Hanbo Huang, Fan Xu, Jiale Zhou, +4 more
Organizations: Department of Computer Science, City University of Hong Kong, Hong Kong SAR · Tencent, China · Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, USA
External memory lets language-model agents answer questions about histories too long for the answer model's context window. Updates create a harder problem than retrieving a recent fact: changing one relation can redirect a multi-hop question to records about an entity absent from the question. We study this update-dependent evidence selection problem and introduce the Contract Memory Compiler (CMC). Before seeing a question, CMC uses a language model to identify relations in the history and record where each one was stated. It applies later updates to determine the current relations, follows them from entities named in the question, and passes the corresponding original records to the answer model in one call. Thus the current state determines which evidence is read, rather than merely refreshing values in a previously selected context. To the best of our knowledge, CMC achieves state-of-the-art multi-hop accuracy on FactConsolidation, reaching 78.25% overall and 61.0% at 262K. With the extracted relations and answer model held fixed, selecting evidence before resolving updates reduces multi-hop accuracy to 21.50%. We also introduce MQuAKE-MemStream, a derived dataset of ordered memory streams built from MQuAKE-Remastered counterfactual cases.
Figures & tables
Figure 1: The Contract Memory Compiler between the assertion memory and the reader. Typed-edge parsing and version resolution build one current object per subject–relation key. Anchoring, closure, and source-order rendering run once per query: explicit subjects start a five-hop traversal in which each object becomes the next subject, and the selected source records are rendered for one reader call.
View
SH
MH
Tokens
Trunc.
Untruncated views
CMC
97.25
78.25
185
0
Prior read + refresh
97.25
21.50
195
0
Stale closure
32.00
11.75
182
0
Fact BM25
97.50
20.75
1,528
0
Budgeted (600/800 truncated)
Table 1: FactConsolidation with shared extraction and one HY4-preview reader. SH and MH are accuracy (%); Tokens is mean reader-input length; Trunc. counts clipped contexts.
Figure 2: FactConsolidation accuracy by history length. Each point covers 100 questions with the same HY4-preview reader. Latest-state and raw-history scores include truncation on 600 of 800 questions.
Figure 3: FactConsolidation at matched evidence length. BM25 packs whole current facts within each question’s CMC budget. Panel (c) shows mean HY4-preview reader-input tokens over all 800 questions.
Reader
Frontend
CMC SH
CMC MH
Ctrl. SH
Ctrl. MH
Grammar adapter; control = full current state
Qwen3.5-4B
Grammar
97.25
79.00
95.25
22.50
Qwen3.5-9B
Grammar
98.00
80.25
94.25
22.50
Shared LLM extraction; control = fact BM25
Qwen3.5-4B
Shared LLM
96.50
76.50
96.50
20.50
Llama-3.1-8B
Shared LLM
90.75
55.00
92.00
11.50
Table 2: FactConsolidation reader/frontend settings, 800 questions each. The grammar rows share one extraction and use untruncated current state as control. The LLM rows share the extracted facts and original reader messages; the control is fact BM25. Each reader uses its own chat interface.
Context given to the reader
Single hop
Multi hop
CMC
99.63
96.62
Latest state
98.75
53.38
Stale closure
30.88
13.41
Raw history (windowed)
89.63
60.53
CMC − latest state
+0.88
+43.23
CMC − stale closure
+68.75
+83.21
Table 3: MQuAKE-MemStream, constructed from the two-hop MQuAKE-Remastered cases. Frozen Qwen3.5-4B reader; 800 single-hop and 798 multi-hop questions. Raw history uses its newest suffix; the last two rows are percentage-point differences from CMC.
AR: original windows
MCC
Rec: windows
LRU
SF: current graph
System
SH-QA
MH-QA
LME(S*)
EventQA
MCC
Rec.
Sum
DetQA
FC-SH
FC-MH
Published long-context (reference)
GPT-5-mini (400K)
85.0
71.0
63.3
78.2
84.0
13.2
56.3
76.1
78.0
28.0
Published retrieval (reference)
BM25
66.0
56.0
45.3
74.6
75.4
13.6
19.0
52.1
48.0
3.0
Text-Embed-3-Small
60.0
44.0
48.3
63.0
70.0
15.3
17.7
54.9
28.0
3.0
Table 4: MemoryAgentBench. Upper blocks copy published Table 3 numbers as reference; their readers differ and they are not bolded against this work. The last block holds one HY4-preview reader. AR and Rec are original-window retrieval. SF is current-graph traversal on overwritten assertions; the two cells are the official 262K split (200 questions). MCC, Summary, and DetectiveQA are the original same-reader campaign, not Algorithm 1 : card/raw lookup on MCC and DetectiveQA, coverage notes on Summary, and the registered JSON-field DetQA decoder. Bold is the highest value inside the last block of each column.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Accuracy (%)
Context
FC-SH
FC-MH
Mean tokens
CMC
97.25
79.00
183
Latest state
95.25
22.50
71,316
Stale closure
31.50
15.50
—
Raw history
42.25
3.75
97,139
Dense+reranker top-10
41.25
3.75
5,559
Appendix
Table 5: Complete grammar-adapter FactConsolidation results under one frozen Qwen3.5-4B reader, pooled across 800 questions. Mean tokens count the full reader input; dashes mark arms without a recorded cost total. The disjoint random arm is a negative control: its current-state keys exclude CMC’s selected keys. G-Walk and MeLLo use a resolved one-value-per-key bank.
FactCon-SH
FactCon-MH
Context
6K
32K
64K
6K
32K
64K
CMC
98
97
98
91
81
85
Latest state
100
96
98
43
23
15
Stale closure
26
38
36
19
18
10
Raw history
31
40
45
2
0
8
Dense+reranker top-10
37
44
46
3
2
8
Appendix
Table 6: Grammar-adapter arms on the non-overflow 6K–64K histories under Qwen3.5-4B, with 100 questions per cell. The 262K raw-history prompts are truncated and are therefore omitted from this comparison.
SF
Frontend
FC-SH
FC-MH
All
Grammar adapter
97.50
79.75
88.625
Direct LLM
97.25
78.25
87.750
Source-induced hybrid
97.00
76.50
86.750
Appendix
Table 7: Source-extraction frontends with the same CMC read, HY4-preview reader, and 800 FactConsolidation questions. The grammar adapter uses manually specified patterns; direct LLM extraction names relations from source records; the hybrid induces literal patterns from source examples. Column names follow MemoryAgentBench Table 3. Each hop class contains 400 questions. Scores use the official substring scorer, including failed requests as incorrect.
FactCon-SH
FactCon-MH
View
6K
32K
64K
262K
6K
32K
64K
262K
CMC
98
98
98
95
90
77
85
61
Latest state
94
67
44
7
39
23
6
0
Stale closure
26
39
36
27
13
9
12
13
Raw history
26
43
33
8
2
11
6
1
Fact BM25
98
98
98
96
33
19
20
11
Appendix
Table 8: Shared direct-LLM controls by source-history length, following MemoryAgentBench Table 5 (FactCon-SH / FactCon-MH × length). Each cell is accuracy over 100 questions with the same HY4 reader. The 262K columns are the original overall-table SF population; Table 1 pools all four lengths.
View
SH
MH
Tokens
Trunc.
CMC
97.25
78.25
185
0
Fact BM25
97.50
20.75
1,528
0
Fact BM25, size matched
97.00
7.25
182
0
Fact dense (BGE-M3)
96.50
33.25
1,535
0
Appendix
Table 9: Shared LLM extraction and HY4-preview on all 800 FactConsolidation questions. Each view selects from the same current facts and unresolved records. Dense retrieval uses BGE-M3 top-100. Tokens are mean reader-input tokens; Trunc. counts reads clipped by the 60,000-character rendering limit.
Method
SH
MH
SH-Conflict / CAR (adapted)
98.00
60.75
CMC
97.25
78.25
Appendix
Table 10: Same-backbone FactConsolidation comparison: 400 SH and 400 MH questions. Both systems receive the original histories and use HY4-preview. The adapted released SH-Conflict/CAR methods retain their query-dependent extraction and stage prompts; CMC uses source-only extraction and one reader call. Frontends and output caps differ.
Single-hop
Multi-hop
Evidence
6K
32K
64K
262K
6K
32K
64K
262K
CMC
98
98
98
95
90
77
85
61
Fact BM25
98
98
98
96
33
19
20
11
BM25, matched cap
97
97
98
96
9
6
9
5
Appendix
Table 11: All length groups in the budget comparison. Each cell contains 100 questions scored with the released substring exact-match metric.
Population
n
Prior + refresh
CMC
Difference
Single-hop
400
97.25
97.25
0.00
Multi-hop
400
21.50
78.25
56.75
MH: keys unchanged
31
100.00
100.00
0.00
MH: keys changed
369
14.91
76.42
61.52
Appendix
Table 12: Complete order control with frozen extraction and HY4-preview. Accuracy is substring exact match (%). Key-change strata are computed from source graphs before scoring.
History length
CMC replay (s)
CMC read (ms)
Fact BM25 read (ms)
6K
0.004
0.35 / 0.42
0.47 / 0.59
32K
0.023
1.61 / 1.96
1.79 / 2.15
64K
0.047
3.32 / 3.90
3.35 / 4.24
262K
0.204
13.78 / 144.34
14.33 / 17.96
Appendix
Table 13: CPU time for state replay and evidence selection on frozen extraction. Replay has 40 runs, and each read arm has 1,000 runs per history length. Read columns give median / 95th percentile in milliseconds.
Evidence arm
Median (s)
p95 (s)
Cached prompt fraction
CMC
1.29
1.95
0.00
Fact BM25
1.42
3.15
0.003
Latest state
2.46
4.88
0.557
Appendix
Table 14: Observed answer-request duration in recorded HY4-preview calls, 800 completed requests per arm. Prompt-cache fraction is the share of prompt tokens recorded as cached.
Conversational memory retrieval optimizes relevance, yet a retrieved memory can be relevant and simultaneously outdated: a later turn updates, corrects, or supersedes it. ConvMemory v3 adds a validity context layer that detects and surfaces this update evidence through target-conditioned relation verification, sitting after the v1/v2 retrieval path. The core mechanism is a dual-evidence gate that conditions a relation judgment on the specific target proposition, scoring a (target, source) pair through the product of a MiniLM slot head and a DeBERTa-v3 slot head and gating it by conservative event/operation evidence. On a synthetic multi-hop validity benchmark the gate reaches 90.12% +/- 1.73 accuracy; through a real-data feedback loop that mines failure patterns but trains on synthetic pairs only, the verifier transfers to Memora role binding with zero target-side labels, reaching 98.8% +/- 0.9 group-all-correct. The deployed layer preserves retrieval by default: a context mode attaches structured validity metadata while keeping the candidate set and rank order fixed, and a query-conditioned demote mode is an explicit opt-in for dense current-state workloads, where it raises current-active H@1 from a never-demote baseline of 45.1% to 95.7% +/- 1.2 while protecting non-superseded memories at 99.4% recall. Six machine-verifiable safety contracts pin the layer's behavior. Multi-hop graph propagation is validated as a mechanism; fully automatic construction of strict prerequisite edges is characterized as a boundary, since strict necessity requires counterfactual world knowledge. This report extends ConvMemory v1 (arXiv:2605.28062) and v2 (arXiv:2606.10842).
Taiheng Pan
School of Computing and Information Systems University of Melbourne
Long-term memory enables language models to use past interactions in future conversations. However, evidence needed to answer a question may be scattered across distant turns, while the question itself omits clues needed to locate it. Retrieved facts can reveal these clues, motivating retrieval decisions conditioned on evidence already found. We introduce MERA (Missing-Evidence Retrieval Augmentation), which separates globally searchable memory from a question-specific evidence state. Verified evidence guides subsequent retrieval without restricting access to the global memory. We train a lightweight planner through reinforcement learning, rewarding queries that recover previously missing evidence. MERA achieves strong answer accuracy across Qwen3-30B and GPT-4o-mini backbones. With Qwen3-30B for evidence processing and answer generation, the trained 0.6B planner achieves 77.40% accuracy on LoCoMo and 71.29% on LongMemEval-S, exceeding a 30B planner without retrieval-grounded training by 4.10% and 3.96%, respectively. On LoCoMo, later retrieval rounds increase cumulative evidence recall from 55.5% to 80.5%.
Memory enables language model agents to use information beyond their context windows, but maintaining and accessing this information reliably remains difficult. Long-horizon conversational memory is especially challenging in multi-actor settings, where relevant evidence is distributed across participants and contexts, and previously established information may later be revised. We introduce EGMEMORY, a training-free agentic memory framework centered on evidence chains that preserve message-level evidence and provenance while tracking active states and their revision histories. At write time, adaptive memory inspection and an evidence-grounded propose-verify-commit protocol govern state updates. At read time, adaptive evidence navigation iteratively identifies the state and supporting evidence required for a query, using conversational structure to narrow the search space and lexical-semantic relevance to rank candidates. The system operates through prompting and tool use without memory-specific policy training. EGMEMORY achieves 68.2% on GroupMemBench and 77.9% on EverMemBench, outperforming the strongest evaluated baselines by 22.7 and 21.4 percentage points, respectively. It further reaches 73.6% on the dyadic LoCoMo benchmark, supporting its applicability beyond multi-actor conversations.