Reusable key-value (KV) memory avoids repeatedly prefilling retrieved text, but stored and injected memories do not necessarily become usable evidence. We present EASE-KV, a controlled framework for understanding this gap under fixed candidates and a frozen language model. EASE-KV separates three interfaces: whether required relations are preserved during encoding, whether preserved memories receive appropriate influence during allocation, and whether outcome supervision can identify useful allocations. Matched experiments show that causal encoding improves compositional accuracy by 25.0 percentage points over block-diagonal encoding, while the tested inference-time bridging methods fail to recover the lost contextual information. With unchanged stored states, diagnostic evidence-guided allocation improves LongMemEval accuracy from 31.2% to 38.4%, demonstrating that memory influence can be controlled without re-encoding. However, controllability does not guarantee learnability: binary correctness provides informative reward contrast for 53.0% of sampled query groups, compared with 73.6% under the evaluated shaped-reward setting. These findings show that effective KV memory requires more than retrieval and injection: the required relations must be encoded, preserved memories must receive appropriate influence, and supervision must distinguish better allocations from worse ones.
Figures & tables
Figure 1: EASE-KV asks three sequential questions under matched candidates and a frozen model: what is encoded, what can be allocated, and what can be learned from outcome supervision.
Figure 2: Headline evidence for encoding, allocation, and supervision. Panels report separate controlled experiments and are compared only within panel.
Backbone
Block-diag.
Causal
Gain
Qwen 7B
54.5
79.5
+25.0
Llama 8B
64.0
74.0
+10.0
Gemma 12B
69.0
79.0
+10.0
Table 1: Cross-backbone encoding accuracy (%). Block-diagonal and causal encoding share the token layout and positions; their difference measures the effect of cross-fact visibility.
Setting
Text
Latent
Diag.
Learned
LongMemEval
51.6
31.2
38.4
–
Full-haystack
55.5
38.6
41.5
–
Temporal
77.8
56.9
76.4
70.8
Table 2: End-to-end accuracy (%). Diagnostic allocation uses oracle evidence or temporal information and is not a deployable policy result.
Regime
Method
Test (%)
Gain (pp)
Weak base
Uniform reference
48.0
0.0
Outcome RL
52.5
+4.5
Strong base
Uniform ±2
75.0
0.0
Warm-neutral RL
75.5
+0.5
From-scratch RL
73.5
-1.5
Search–distill
Uniform ±2
75.0
0.0
Table 3: Descriptive accuracy of learned and distilled allocations against their matched references. Gains are percentage-point differences; no statistical significance is inferred from these comparisons.
Condition
Acc.
Δ (pp)
Visibility
Independent
46.5
-33.0
Block-diagonal
54.5
-25.0
Causal
79.5
0.0
Inference bridge
Text reference
31.2
-0.8
Table 4: Encoding ablations. Visibility differences use causal encoding as reference; bridge differences use session-level KV.
Allocation
Overall
1-hop
Multi-hop
Text ref.
74.5
77.9
64.7
Unsteered
44.5
51.0
25.5
Sim. top-1
30.0
33.6
19.6
Uniform ±2
75.0
83.9
49.0
Table 5: Allocation accuracy (%). Uniform ±2 uses oracle evidence labels and serves as a diagnostic reference.
Initialization
Dev (%)
Test (%)
Dominant action
Warm-neutral
75.0
75.5
85% at 0 ; 14% at +2
From scratch
73.2
73.5
81% at −4
Warm-legacy
64.3
–
46% at −4 ; 32% at −2
Uniform ±2
–
75.0
Fixed reference
Table 6: Initialization under sparse within-query reward contrast.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
Candidate condition
Primary role
Controlled composition
Required facts are present; single- and multi-hop questions are separated
Isolate cross-memory encoding visibility and post-retrieval allocation.
LongMemEval
Retrieved memories are fixed within each text–latent comparison
Measure representation loss and recoverable allocation headroom under long conversational histories.
Full-haystack
The candidate set contains additional irrelevant memories
Stress competition between evidence and distractors.
Temporal conflict
Earlier and updated facts remain simultaneously available
Test whether allocation can express a recency preference without deleting history.
Test whether memory relevance can be predicted without conflating selection with answer accuracy.
Appendix
Table 7: Evaluation settings and the interface each one is designed to expose. Candidate control describes the comparison used in this paper, not a property of the original benchmark.
Allocation
All
1-hop
Multi-hop
Text reference
149/200
116/149
33/51
Unsteered KV
89/200
76/149
13/51
Similarity top-1
60/200
50/149
10/51
Uniform ±2
150/200
125/149
25/51
Appendix
Table 8: Verified correct/total counts for the four retained allocation-ablation rows in Table 5 . These are correctness counts, not counts of evidence memories.
Condition
What changes
Interpretation
Text reference
Memories are serialized into the normal prompt
Delivery reference; not compute-matched to reusable KV.
Independent KV
Each fact is encoded in a separate forward pass
Maximum modularity with no cross-fact encoding context.
Block-diagonal KV
Facts share positions and sequence length, but cross-fact attention is masked
Matched layout control for causal visibility.
Causal KV
Later facts can attend to earlier facts during cache construction
Measures the effect of cross-fact contextualization.
Turn/session KV
Progressively more source context is bound into one stored unit
Measures the fidelity–reuse trade-off.
Query-level KV
Current candidates are encoded jointly for the evaluated query
Contextualization reference; not a query-independent cache.
Appendix
Table 9: Delivery and encoding baselines. Conditions in one comparison share memory contents, prompt, frozen backbone, decoding, and evaluator.
Condition
Information used
Role
Unsteered KV
b=0
Fixed-representation reference.
Similarity top-1
Highest retrieval-similarity candidate
Tests whether a retrieval rank is also a useful allocation rule.
Memory-augmented language models combine persistent or temporary memory with neural computation to retain, retrieve, and use information beyond the active context.
When a fact changes, how should a language model update the history stored in its key-value (KV) cache? Hiding the old record is cheap, but it may still contain needed details or answer questions about the past. We compare hiding whole records, hiding only replaced values, and deleting old text and recomputing the cache. In a controlled quantity task, masking makes all eight models prefer the new value more strongly, yet six lose complete answers through unit errors or failure to stop; keeping the unit preserves all current answers. Later states also retain useful information from earlier records: on multi-hop updates, rebuilding these states at unchanged positions lowers historical accuracy by 20-41 percentage points, whereas moving the existing states has little effect. Keeping object dependencies and unchanged revision passages prevents many losses. Recognition is a separate challenge. Learned readouts recover distinctions missed by fixed cache similarities on synthetic record pairs. On natural text, text-detector-selected masks show no clear advantage over random masks at the same rate in 14 same-model detector-generator comparisons. Query-dependent access can avoid some losses, with additional storage or access costs. These findings identify what must be preserved beyond the replaced value when using a KV cache as updatable memory.
Long-term memory enables language models to use past interactions in future conversations. However, evidence needed to answer a question may be scattered across distant turns, while the question itself omits clues needed to locate it. Retrieved facts can reveal these clues, motivating retrieval decisions conditioned on evidence already found. We introduce MERA (Missing-Evidence Retrieval Augmentation), which separates globally searchable memory from a question-specific evidence state. Verified evidence guides subsequent retrieval without restricting access to the global memory. We train a lightweight planner through reinforcement learning, rewarding queries that recover previously missing evidence. MERA achieves strong answer accuracy across Qwen3-30B and GPT-4o-mini backbones. With Qwen3-30B for evidence processing and answer generation, the trained 0.6B planner achieves 77.40% accuracy on LoCoMo and 71.29% on LongMemEval-S, exceeding a 30B planner without retrieval-grounded training by 4.10% and 3.96%, respectively. On LoCoMo, later retrieval rounds increase cumulative evidence recall from 55.5% to 80.5%.
Under a limited context budget, a long-term agent must recall not merely memories that are individually relevant, but evidence that is collectively sufficient for the current query. Conventional memory access retrieves, scores, and truncates memories, often spending multiple context slots on redundant facts while leaving complementary evidence behind. We cast the read phase of long-term memory as query-conditioned evidence composition: each memory is valued by what it adds beyond the evidence already selected. We instantiate this operation with MemPursuit, a training-free mechanism that uses the residual of a sparse convex query reconstruction as a representation-space proxy for what the current memory set is missing. By reusing the query and candidate embeddings already available from retrieval, MemPursuit progressively composes nested evidence contexts without invoking a language model or cross-encoder to score each query--memory pair. The effectiveness of our method is verified on 3 popular benchmarks in comparison to strong baselines. In fixed-budget LoCoMo experiments, MemPursuit improves end-to-end accuracy by 7.2 percentage points at Top-10 and 5.5 percentage points at Top-20, with consistent gains across multiple memory frameworks, while achieving a 32.9× speedup in median reranking latency over fully batched BGE-v2-m3. Our implementation is available at https://github.com/FengQi-HITSZ/MemPursuit.
Qi Feng, Jicong Fan
School of Data Science, The Chinese University of Hong Kong, Shenzhen