Causal Memory Policy: Making Memory Utility Identifiable by Intervening on Retrieval

Arman Behnam; Binghui Wang

cs.AIOct 1, 2026

Causal Memory Policy: Making Memory Utility Identifiable by Intervening on Retrieval

Authors: Arman Behnam, Binghui Wang

Organizations: Department of Computer Science Illinois Institute of Technology Chicago, IL

Abstract

Memory-augmented large language models must decide which memories to retain, and recent systems do so by estimating each memory's effect on task performance. However, these estimates rely entirely on retrieved memories. When a memory is never retrieved, store-level interventions produce identical outcomes, leaving its utility unidentified. This is a retrieval-level positivity violation, invisible to diagnostics that examine only memory operations. We introduce Causal Memory Policy (CMP), a causal framework that restores identification by intervening on retrieval itself, reserving a fixed number of context slots for memories sampled with known propensities. CMP estimates memory utility by self-normalized inverse propensity weighting under a balanced assignment design. We prove the causal factorization of memory utility through retrieval, the unbiasedness and exact variance of the estimator, and the optimal decision rule under irreversible operations. Empirically, identification fails for 54% of required memories on LongMemEval and 67% on LoCoMo, and the failure persists in a deployed memory system. CMP improves discrimination between required and non-required memories from 0.54 to 0.66 AUC. Finally, we show that identified memory utility alone is insufficient for retention decisions: per-query utility reaches 0.78 AUC on the query for which it is estimated, yet no aggregation available to a retention policy predicts a memory's value on unseen queries. Code is available at: https://anonymous.4open.science/r/cmp-release-D0C3/.

Figures & tables

Panel of Figure 1 — Figure 1: Causal DAG of memory-augmented LLMs across two timesteps.

Panel of Figure 2 — Figure 2: Overview of CMP. (a) Balanced exposure assigns the $Tk$ exposure slots evenly across the exposure pool, fixing each memory’s propensity by design. (b) At each timestep, the context consists of the top- $(B-k)$ retrieved memories together with $k$ randomly exposed memories. (c) CMP compares three designs that differ only in the randomized variable while using the same estimator.

Design	AUC	Lift	Gold loss
Store-level rand.	$0.542$	$0.97$	$10.9$
Exposure (CMP)	$\mathbf{0.664}$	$\mathbf{0.68}$	$\mathbf{5.2}$

Table 2: Estimation and decision quality on two designs (only differ in randomized variable) and LongMemEval independent subset.

Support	$n$	AUC gap	Median $\|t\|$
$P=0$	$61$	exposure only	$0.83$
$0<P<0.8$	$15$	$+0.09$	$1.7$
$P\geq 0.8$	$37$	$+0.003$	$3.66$

Table 3: Discrimination by retrieval support, LongMemEval independent subset. Median

|t|

is the store-level estimate’s significance.

Variant	AUC	Destroyed	Gold loss
CMP (full)	$\mathbf{0.664}$	$\mathbf{6}$	$\mathbf{5.2}$
– one-sided abstention	$0.664$	$27$	$9.0$
– exposure design	$0.542$	$15$	$10.9$
– both	$0.542$	$35$	$11.7$
two-sided abstention	$0.664$	$22$	$14.1$
two-sided exposure	$0.671$	$10$	$6.6$

Table 4: Component ablation on LongMemEval. Gold loss is reported per

100

reclaimed slots.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Description
$\mathcal{M}=(\mathcal{U},\mathcal{V},\mathcal{F})$	Structural causal model (Def. 1 )
$M_{t}$	Memory state at time $t$
$M^{-m}_{t}$	Memory state excluding $m$ , $M_{t}\setminus\{m\}$
$\mathbb{M}$	Universe of possible memories
$R_{t}$	Retrieved subset at time $t$
$O_{t}$	Memory operation at time $t$

Appendix

Table 5: Summary of notation used throughout the paper.

Analysis	Benchmark	$B$	$k$	$T$	$\|\mathcal{P}\|$	Draws/mem.
Identification, single query	LongMemEval	$6$	$2$	$90$	$\approx 32$	$5.6$
Two-sided exposure	LongMemEval	$6$	$2$	$90$	$\approx 32$	$5.6$
Identification, pooled	LoCoMo	$6$	$2$	$90$	$36$	$5.0$
Identification, per query	LoCoMo	$6$	$6$	$30$	$36$	$5.0$

Appendix

Table 6: Exposure design parameters by experiment. Draws per memory is

Tk/|\mathcal{P}|

; the identification bar requires at least

n_{\min}=5

in each arm. On LoCoMo the horizon is spread across roughly

30

questions per conversation, so the pooled row supplies

5.0

draws per memory but only

3

per

(\text{memory},\text{question})

cell, which is why the per-query estimand requires a separate design.

Nomination rule	Signal used	Lift over chance
Lowest access frequency	Access counts	$0.00$ – $0.11$
Lowest cosine similarity	Query similarity	$0.00$
Recency-weighted blend	Both	$0.00$ – $0.01$
Newest-first	Insertion order	$0.34$ – $0.86$
Utility, store-level estimates	Estimated $U(m)$	$\mathbf{0.97}$
Utility, exposure estimates	Estimated $U_{R}(m)$	$\mathbf{0.68}$

Appendix

Table 7: Nomination of required memories as removal candidates, LongMemEval independent subset. Lift is the observed rate divided by the rate expected under uniform nomination;

1.00

is chance.

Panel of Figure 4 — Figure 4: Arm structure for three required memories. (a) Store-level randomization does not surface the memory, so both arms contain identical contexts and no estimate exists; exposure recovers one. (b) The mirror case: the memory always ranks high enough that exposure does not exclude it, so exposure has no estimate and store-level does. (c) Where both designs have both arms, they agree to three decimal places, but exposure’s standard error is $4.5$ times larger: randomization spent on a memory the ranker already varies is wasted.

Benchmark	Pairs	$P=0$	Store	AUC store	AUC exposure
LongMemEval (independent)	$113$	$54.0\%$	$487$	$0.542$	$0.664$
LongMemEval (all $-$ suspect)	$170$	$61.2\%$	$487$	$0.529$	—
LoCoMo (pooled estimand)	$359$	$66.9\%$	$588$	$0.513$	$0.551$
LoCoMo (per-query estimand)	$359$	$66.9\%$	$588$	—	$\mathbf{0.784}$
Mem0 $2.0.18$	$109$	$0.284$ [ $0.208$ , $0.375$ ]	$291$	—	—

Appendix

Table 8: Retrieval support failure and discrimination by benchmark. Support failure is the fraction of required

(\text{question},\text{memory})

pairs with

P(m\in R\mid q,c)=0

. AUC separates required from non-required memories by estimated utility.

Population	$n$	Exposure	Cosine	Diff.	$t$	$p$
Independent	$30$	$0.197$	$0.223$	$-0.026$	$-0.76$	$0.45$
All $-$ suspect	$43$	$0.155$	$0.168$	$-0.013$	$-0.54$	$0.59$
All	$51$	$0.165$	$0.175$	$-0.010$	$-0.50$	$0.62$

Appendix

Table 9: Answer-quality cost of one-sided exposure (

k=2

B=6

), paired per item.

Variant	AUC	Evicted	Destroyed	Gold loss
Exposure $+$ one-sided gate	$0.664$	$116$	$\mathbf{6}$	$\mathbf{5.2}$
Exposure, ungated	$0.664$	$300$	$27$	$9.0$
Store-level $+$ one-sided gate	$0.542$	$138$	$15$	$10.9$
Store-level, ungated	$0.542$	$300$	$35$	$11.7$
Exposure $+$ two-sided gate	$0.664$	$156$	$22$	$14.1$
Store-level $+$ two-sided gate	$0.542$	$132$	$28$	$21.2$

Appendix

Table 10: Component ablation, LongMemEval independent subset, eviction budget

10

per item. Destroyed counts required memories evicted; gold loss normalizes by slots reclaimed.

Selector	Type	Headroom recovered
Five bi-encoders (MiniLM to gemini-embedding-2)	Pointwise	at chance on contradiction pairs
Cross-encoder reranker (BGE-reranker-v2-m3)	Pointwise	$10\%$ LongMemEval, $30$ – $33\%$ HotpotQA, $17$ – $24\%$ MuSiQue
Maximal marginal relevance	Set-aware	$-2.7\%$ to $+4.8\%$
Coverage-greedy (ceiling)	Set-aware	$95\%$

Appendix

Table 11: Recall of required evidence at matched context budget. Bi-encoders span two orders of magnitude in parameter count. Coverage-greedy uses gold knowledge and serves as a ceiling.

Predictor	Deletion	Insertion
Pooled estimate	$-0.060$	$+0.023$
Nearest estimation query	$<0.11$	$<0.11$
Similarity-weighted mean	$<0.11$	$<0.11$
Maximum over estimation queries	$<0.11$	$<0.11$
Ground-truth evidence labels (control)	$\mathbf{+0.286}$	—

Appendix

Table 12: Correlation between estimated utility and realized contribution on held-out queries in LoCoMo. Ground-truth per-query evidence labels provide a positive control, confirming that the signal is recoverable from the same data.

Explore similar work

Causal Intervention-Based Memory Selection for Long-Horizon LLM Agents
May 17, 2026Saksham Sahai SrivastavaLong-Term Memory Causal Intervention
School of Computing, University of Georgia, Athens, Georgia, USA.
Learning What to Remember: Long-horizon Counterfactual Memory Optimization
Sep 29, 2026Jiaming Tang, Mingyan Liu, Armin Sarabi
Department of Electrical Engineering and Computer Science, University of Michigan
Selective Memory Retention for Long-Horizon LLM Agents
Jun 28, 2026Pranath ReddyRetention Traces
Independent Researcher, Huntsville, Alabama, USA.

May 17, 2026, cs.AIJ/K move · Enter open · S save

Causal Intervention-Based Memory Selection for Long-Horizon LLM Agents

Saksham Sahai Srivastava

School of Computing, University of Georgia, Athens, Georgia, USA.

Long-Term Memory Causal Intervention Causal

Open paper AskPDF arXiv

Long-horizon LLM agents rely on persistent memory to support interactions across sessions, yet existing memory systems often retrieve context using semantic similarity or broad history inclusion, treating retrieved memories as uniformly useful. This assumption is fragile because memories may be topically related while remaining irrelevant, stale, or misleading. We propose Causal Memory Intervention (CMI), a causal memory-selection technique that estimates how candidate memories affect the model's answer under controlled interventions, selecting memories that improve task performance while suppressing unstable, irrelevant, or harmful ones. To evaluate this setting, we introduce Causal-LoCoMo, a causally annotated benchmark derived from long conversational data, where each example contains a user request, a structured memory bank, useful memories, irrelevant distractors, and synthetic harmful memories. We compare CMI against vector, graph, reflection, summary, full-history, and no-memory baselines. Results show that CMI achieves a stronger balance between answer quality and robustness to misleading memory, suggesting that reliable long-term memory requires selecting context based on causal usefulness rather than relevance alone. The full framework, benchmark construction code, and experimental pipeline are available at https://github.com/Saksham4796/causal-memory-intervention.

Abstract

Explore similar work

Causal Intervention-Based Memory Selection for Long-Horizon LLM Agents

Learning What to Remember: Long-horizon Counterfactual Memory Optimization

Selective Memory Retention for Long-Horizon LLM Agents