The Right Memory in the Wrong Context: Verifying Retrieval Admissibility in Long-Term Agent Memory

Zi Wang; Xingqiao Wang; Emmanuel Addai; Devika Ambekar; Xiaowei Xu

cs.AIOct 5, 2026

The Right Memory in the Wrong Context: Verifying Retrieval Admissibility in Long-Term Agent Memory

Authors: Zi Wang, Xingqiao Wang, Emmanuel Addai, Devika Ambekar, Xiaowei Xu

Organizations: University of Arkansas at Little Rock

Abstract

Long-term-memory agents can retrieve relevant information that is inadmissible for the current request because it belongs to another principal, violates policy, or reflects an incompatible lifecycle state. Recall and final-answer accuracy do not reveal this: a route can appear safe by missing required evidence, while a correct answer may follow inadmissible prompt exposure. We introduce a retrieval-admissibility verification framework that assigns each memory-query pair one of three statuses (admissible, inadmissible, or unresolved), compares routes at matched required-evidence recall with bounds for unresolved cases, and tracks memory IDs through prompt exposure while linking exposure to target-level disclosure. We evaluate its stages on separate, non-pooled populations. A post-hoc top-20 reanalysis of frozen rankings from two public long-term-memory benchmarks, RHELM and MemOps, covers 3,767 queries. All released anchors lie within trusted query namespaces; with within-namespace scores unchanged, off-namespace filtering cannot lower their ranks. Top-20 anchor recall increases from 0.432 to 0.533, 80% recall feasibility from 0.237 to 0.311, and exact similarity evaluations decrease by 98.3%. In a frozen 72-case development diagnostic, a released-metadata reference preserves required evidence, whereas neither text-only verifier detects violations under the 1% required-anchor false-denial limit. Across 1,523 paired benchmark-native cases, namespace routing is associated with judged-accuracy gains of 0.053-0.068 across three readers; recall also changes, so this comparison is observational. In 16 controlled exposure scenarios, only one of four reader-specific 95% confidence intervals excludes zero for relevant-inadmissible literal disclosure (+0.156, 95% CI [0.031, 0.312]). Results motivate separate verification of candidate support, admissibility, prompt exposure, and answer disclosure.

Figures & tables

Figure 1: Audited retrieval-admissibility cases. Wrong-scope, superseded, and explicitly forgotten records are excluded, whereas superseded medication states are retained for a history query. The first row is from RHELM and the remainder from MemOps; text is abridged, but verdicts are not.

Figure 2: Agent-memory retrieval-admissibility verification framework. Query metadata constrain candidate support, semantic retrieval ranks IDs, and policy and lifecycle verification precede agent-prompt assembly. The lower trace follows one memory ID from storage through retrieval and exposure, followed by target-level disclosure scoring; disclosure is observable output, not internal causal use. Each study reports its population and reader-specific estimates separately.

Question	Population and comparison	Primary outcomes
RQ1: Support	$3{,}767$ RHELM/MemOps queries; global, namespace, random, pre/post-filter, routed, and released-field arms; metadata stress	Recall, feasibility, bounds, coverage, constrained loss, typed exposure, work
RQ2: Verification	96 public-development queries with fixed top-20 pools plus $16$ controlled scenarios; fields versus text decisions	ROC-AUC, violation precision/recall, anchor false denial, consistency, overflip
RQ3: Enforcement	$1{,}523$ paired benchmark-native reader cases plus $192$ controlled pairs per reader; no reader pooling	Answer correctness/quality, non-answer action, protected/stale disclosure, cell effects, selectivity

Table 1: Experimental map. Each row identifies the population, compared systems, and outcomes; rows are analyzed separately and never pooled.

Figure 3: Three separate evaluations yield different stagewise verdicts. (a) Namespace beats random same-size support; late filtering retains global work. (b) The released-field reference preserves evidence, but the text-only gates do not. (c) Only DeepSeek V4 Pro’s relevant-inadmissible contrast excludes zero; ∗ marks the Claude Opus 5 replication. Populations (

3{,}767

queries, 72 verifier cases, and 192 pairs per reader) are not pooled.

Method	Visible information	Recall $\uparrow$	Feasible $\uparrow$	$[C^{L,A},C^{U,A}]$ $\downarrow$	$\mathcal{L}^{\mathrm{RC},A}$ $\downarrow$	Similarity evals. $\downarrow$
Global dense	text	.432	.237	[.187,.195]	.798	90,122
Global recency+dense	text + released order	.452	.256	[.198,.206]	.784	90,122
Namespace dense	text + trusted namespace	.533	.311	[.118,.126]	.713	1,551
$\Delta$ Namespace $-$ global		+.101	+.074	—	$-$ .085	58.1 $\times$ fewer

Table 2: Frozen-route admissibility audit and downstream reader association. (a) Post-hoc admissibility, top-20 results; rankings and settings are unchanged, coverage is .992; similarity evaluations count query–candidate score computations, not wall-clock latency. (b) Equal-source reader averages and paired namespace-minus-global contrasts. (c) Typed matched-prefix exposure; axes can overlap. Readers remain separate; † marks a gated replication.

Support intervention	Recall $\uparrow$	Feasible $\uparrow$	$\mathcal{L}^{\mathrm{RC},A}$ $\downarrow$	Similarity evals. $\downarrow$
Global dense	.432	.237	.798	90,122
Random same-size	.061	.024	.978	1,540
Namespace pre-filter	.533	.311	.713	1,551
Global post-filter ( $d=500$ )	.531	.311	.713	90,122
Anchor-preserving same-size oracle	.840	.747	.390	1,551

Table 3: Support ablations and attribution controls. Frozen top-20 estimates at

\tau=.8

. Random and anchor-preserving rows are ten-seed means; the latter uses released anchors. Similarity evaluations count query–candidate score computations, not wall-clock latency.

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Support	Known non-usable	Coverage	Lower	Upper
Global dense	0.575	0.538	0.308	0.770
Namespace dense	0.348	0.355	0.134	0.778

Appendix

Table 4: Historical non-usable diagnostics within feasible matched-recall prefixes. These source-macro

B=U

quantities do not use the infeasibility penalty and are not pure wrong-scope measures. Namespace support lowers the resolved rate and lower bound but also lowers coverage; the upper bound remains high.

Axis family	Route	Recall	Feasible	$[C^{L},C^{U}]$	$\mathcal{L}^{\mathrm{RC}}$	Any violation
Scope + policy + lifecycle	Global	.432	.237	[.187,.195]	.798	.528
Scope + policy + lifecycle	Namespace	.533	.311	[.118,.126]	.713	.396
Scope + lifecycle	Global	.432	.237	[.102,.111]	.787	.356
Scope + lifecycle	Namespace	.533	.311	[.017,.025]	.693	.134

Appendix

Table 5: The namespace result does not depend on the low-agreement policy axis. Equal-source macro estimates at target recall .8. For the policy-omitted loss difference, the paired 10,000-sample namespace-group bootstrap gives

-.0933

[

-.115,-.071

Route	Known risk	Coverage	$[C^{L},C^{U}]$	Any violation
Global dense	.1893	.9918	[.1873,.1955]	.5285
Namespace dense	.1122	.9913	[.1105,.1191]	.3721

Appendix

Table 6: The matched-prefix risk reduction persists on an identical feasible-query set. Both routes reach target recall on the same 609 queries. Equal-source macro estimates use the full scope+policy+lifecycle target; the paired namespace-minus-global upper-bound difference is

-.0763

[95% CI

-.0929,-.0600

] from 10,000 namespace-group bootstrap replicates.

Axis	Exact agreement	Uncertain	Evaluable	$\alpha$	Reviewer–source
Relevance	0.8937 (185/207)	0	207	0.8135	.797–.802
Scope	0.9952 (206/207)	0	207	0.9808	.995–1.000
Lifecycle state	0.8599 (178/207)	23	184	0.6703	.841–.947
Prohibited evidence	0.8841 (183/207)	0	207	0.0829	.729–.826
Legacy usable composite (derived)	0.9710 (201/207)	0	207	0.9405	.836–.855

Appendix

Table 7: Scope and relevance labels are reproducible; lifecycle and prohibited status remain the harder axes. Raw independent, unadjudicated agreement on the blinded human-audit sample. Uncertainty counts labels across both reviewers; nominal Krippendorff’s

\alpha

excludes pairs containing an uncertain label. The legacy usable composite is derived rather than independently elicited and excludes lifecycle. Exact agreement with the deterministic released-field construction is reported as a consistency check, not external gold validation.

Figure 4: Relevant records require query-conditioned eligibility checks. Four abridged expert-audited examples underlying the admissibility taxonomy: wrong principal, superseded current state, explicit forgetting, and a historical query that requires superseded states. Packet IDs, memory IDs, released labels, and verdicts are preserved in the example manifest; wording is shortened for legibility. These cases illustrate coverage, not population prevalence.

Work	Invalidity target	Primary unit / stages	Matched-recall bounds	Paired exposure estimand
GateMem [ Ren et al., 2026 ]	Principal access, updates, forgetting	Episode/checkpoint utility and leakage	Not primary	Not primary
MemOps [ Hao et al., 2026 ]	Operation target, scope, state transition	Structured operation trace and probes	Not primary	Not primary
STALE [ Chao et al., 2026 ]	Implicit invalidation of earlier state	State-resolution and answer probes	Not primary	Not primary
A-TMA [ Shi et al., 2026 ]	Current, historical, and transition state	Bank, retrieval, and answer diagnostics	Not primary	Not primary
MemConflict [ Tao et al., 2026 ]	Temporal, factual, and contextual conflicts	Retrieval, ranking, and final answers	Not primary	Not primary
MemGate [ Zhang et al., 2026b ]	Query-conditioned contextual appropriateness	Learned admission gate, threat, and utility	Not primary	Not primary

Appendix

Table 8: This work audits eligibility and downstream stages at the record level. Relationship to nearby evaluation frameworks. “Not primary” describes each paper’s reported estimands, not the capabilities of its system. Our emphasis is one record-level audit across eligibility axes and stages.

Figure 5: More filtering is not monotonically better. Changes from namespace dense in percentage points; right is better. Circles show utility and open squares recall-constrained loss reduction for (a) released metadata on

3{,}767

queries and (b) fixed-candidate gates on 72 development cases. Whiskers are pointwise 95% bootstrap intervals; verifiers remain separate. Full corruption brackets appear in Table 14 .

Figure 6: Semantic verifiers recognize eligibility changes but do not apply them selectively. (a) Feasibility and sign-reversed recall-constrained-loss differences versus namespace dense; labels give recall differences. Whiskers are pointwise 95% intervals from 10,000 namespace-group bootstrap replicates stratified by source and query intent over 72 analysis cases. (b) Controlled focal consistency versus stable-control overflip; labels report stable-admissible false-denial rate. The benchmark-native and controlled populations are separate and never pooled.

Figure 7: No tested threshold satisfies the declared anchor-denial constraint while removing violations. Thresholds span 0 to 1 in .05 steps; panels (a)–(b) show the practical

\geq.5

region. Open markers are the calibration-selected .95 thresholds. Candidate precision, violation recall, and anchor denial are analysis-candidate micro averages; route outcomes in (c) use the prespecified equal-stratum macro. GPT-5.6 Sol meets the 1% false-denial constraint only by retaining every candidate and detecting no violations; Gemini 3.6 Flash has no tested point below the line. The gray cross is the retain-all namespace route.

Model	ROC-AUC	Violation precision	Violation recall	Anchor false deny
GPT-5.6 Sol	$0.626$	$0.350$	$0.232$	$0.059$
Gemini 3.6 Flash	$0.522$	$0.679$	$0.049$	$0.020$

Appendix

Table 9: Text-only verification does not yield a low-denial operating point. Candidate-level diagnostics for the benchmark-native text-only verification experiment. Values use the fixed development-selected operating points. Higher ROC-AUC, violation precision, and violation recall are better; lower required-anchor false-denial rate is better.

Figure 8: Trusted namespace support improves retrieval and downstream utility. Across

3{,}767

queries, namespace raises (a) recall and (b) feasibility at every budget. (c) All three readers gain accuracy on the frozen

1{,}523

cases; whiskers are paired 95% intervals. GPT-5.6 Luna is sequential. Claude Opus 5 was not executed as a reader on this population; Claude Haiku 4.5 served only as the shared blinded answer judge.

Figure 9: Namespace support outperforms arbitrary candidate-count matching. (a) The ten-seed random partition preserves source-level namespace-label counts but breaks provenance assignment; points show recall and feasible rate. (b) Global top-

d

is namespace-filtered and truncated to 20; dashed lines are namespace pre-filtering. (c) Namespace-minus-global penalized risk as the explicit infeasibility cost varies; negative is better. Scope-excluded risk retains only released policy/lifecycle violations and unresolved labels. Across all 87 groups, leave-one-group-out recall deltas are +.097 to +.108; the exact seven-group RHELM sign-flip

p

-value is .0156.

Figure 10: Zero-call diagnostics expose uncertainty and retrieval headroom. (a) Established composite evaluator judgments are hidden after retrieval, so routes, anchors, recall, and feasibility remain fixed. Solid lines report known-label coverage and dashed lines the resulting partial-identification interval width; the callout gives lower and upper risk at 20% hiding. (b) The gold-preserving arm retains every released required anchor and samples non-anchors to exactly the namespace support size. Circles are recall, squares are feasible-query rate, and whiskers show one seed standard deviation where applicable. * The oracle reads released gold and is an attribution diagnostic, not a deployable router.

Reader	$\Delta$ answer accuracy	$\Delta$ answer quality	$\Delta$ non-answer rate
DeepSeek V4 Pro	+0.053 [+0.020, +0.084]	+0.043 [+0.023, +0.062]	-0.048 [-0.068, -0.031]
Gemini 3.6 Flash	+0.068 [+0.047, +0.090]	+0.050 [+0.036, +0.065]	-0.041 [-0.057, -0.025]
GPT-5.6 Luna †	+0.066 [+0.039, +0.096]	+0.048 [+0.031, +0.066]	-0.038 [-0.059, -0.015]

Appendix

Table 10: Namespace support improves answer utility for every frozen reader. Benchmark-native same-population route-to-reader effects. Entries are namespace dense minus global dense with pointwise 95% paired namespace-group bootstrap intervals. Readers are reported separately and share one blinded Claude Haiku 4.5 judge. † GPT-5.6 Luna is a sequential robustness replication.

Reader	Source	$\Delta$ answer accuracy	$\Delta$ non-answer rate
DeepSeek V4 Pro	RHELM	+.021 [ $-$ .037,+.075]	$-$ .044 [ $-$ .075, $-$ .016]
	MemOps	+.084 [+.052,+.116]	$-$ .053 [ $-$ .077, $-$ .030]
Gemini 3.6 Flash	RHELM	+.059 [+.032,+.094]	$-$ .042 [ $-$ .068, $-$ .022]
	MemOps	+.076 [+.045,+.106]	$-$ .039 [ $-$ .061, $-$ .018]
GPT-5.6 Luna †	RHELM	+.046 [+.004,+.094]	$-$ .059 [ $-$ .096, $-$ .016]
	MemOps	+.086 [+.053,+.121]	$-$ .017 [ $-$ .034, $-$ .001]

Appendix

Table 11: The direction is stable across sources, with wider RHELM uncertainty. Source-specific namespace-dense minus global-dense effects in the benchmark-native reader analysis. Intervals use 10,000 paired namespace-group bootstrap replicates. The equal-source macro result in the main table is prespecified, but these rows expose source heterogeneity: DeepSeek V4 Pro accuracy on RHELM is the only accuracy interval that crosses zero. Non-answer means

\mathbf{1}\{\mathrm{action}\neq\mathrm{answer}\}

, not gold-action-aware over-refusal. RHELM intervals resample only seven evaluation namespace groups and should be read with that cluster count in view.

Reader	$\Delta$ answer accuracy	$\Delta$ answer quality	$\Delta$ non-answer rate
DeepSeek V4 Pro	+.062 [+.033,+.092]	+.050 [+.033,+.068]	$-$ .050 [ $-$ .069, $-$ .032]
Gemini 3.6 Flash	+.070 [+.048,+.094]	+.052 [+.037,+.067]	$-$ .040 [ $-$ .057, $-$ .024]
GPT-5.6 Luna †	+.072 [+.045,+.101]	+.053 [+.036,+.071]	$-$ .032 [ $-$ .049, $-$ .014]

Appendix

Table 12: Case weighting preserves the namespace benefit. Post-hoc case-weighted sensitivity for namespace dense minus global dense. Each evaluable case receives equal weight across RHELM and MemOps; 95% intervals resample namespace groups within source. The prespecified equal-source macro remains primary, and readers remain separate.

Arm	Selected setting
Global BM25	$k_{1}=1.5,\ b=0.75$
Global dense	Parameter-free
Global BM25+dense RRF	$k_{1}=0.9,\ b=0.75,\ k_{\mathrm{RRF}}=60$
Global recency dense	$\gamma=0.1$
Namespace dense	Parameter-free
Query-agnostic current-only	Remove stale and superseded records for every query

Appendix

Table 13: All route settings were selected on development data only. Frozen development-selected settings; all arms use

k=100

Channel	Reference	Last dominant	First failure
Namespace false deny	Global dense	.10	.20
Namespace missing	Global dense	.10	.20
Namespace source swap	Global dense	.10	.20
Namespace false allow	Global dense	.50	Not observed
Policy false deny	Clean namespace	.00	.02
Policy false allow	Clean namespace	.30	.40

Appendix

Table 14: False-deny corruption is the earliest failure mode. Observed metadata-corruption weak-dominance brackets over ten nested seeds. Namespace channels use global dense as the reference; policy, lifecycle, and intent channels use clean namespace dense. Rates are tested grid points, not interpolated thresholds. “Not observed” means no failure through the maximum tested rate .50.

Arm	Recall	Feasible	Penalized non-usable upper risk	Candidates
Global BM25	0.448	0.284	0.913	90122
Global dense	0.717	0.539	0.873	90122
Global BM25+dense RRF	0.691	0.503	0.883	90122
Global recency dense	0.728	0.550	0.864	90122
Namespace dense	0.866	0.762	0.837	1551
Query-agnostic current-only	0.824	0.672	0.845	1549

Appendix

Table 15: Namespace dense gives the strongest retrieval trade-off among the frozen routes. All nine frozen v1 arms at

k=100

on the source-macro public evaluation; main-text Table 2 a truncates the same frozen rankings to top-20. The legacy penalized non-usable upper-risk field assigns 1.0 to infeasible or unresolved query outcomes. Candidates scored is a local diagnostic, not a production latency estimate.

Figure 11: Namespace support, rather than the tested routing refinements, separates the frozen retrieval regimes. (a) Recall versus mean candidates scored for all nine arms. (b) Namespace-dense minus global-dense effects under the historical non-usable loss. (c) Conditional non-usable bounds and label coverage. (d) Threshold, cluster, and historical released-field v1 differences versus namespace dense. This figure preserves the original route semantics and cannot be read as the corrected v2 policy/lifecycle attribution.

Figure 12: Observed exposure predicts downstream disclosure but does not identify a causal effect. (a) Exposed-minus-unexposed answer disclosure over

721

discordant checkpoints. (b) Predictive value of exposure on

4{,}470

held-out rows. (c) GPT-4o mini G1-minus-G0 leakage, bounded utility, and GateMem bounded over-refusal. Readers are not pooled; route contrasts are non-causal and leakage and utility use different populations. Exact reader snapshots are gpt-4o-mini-2024-07-18 and gpt-4o-2024-08-06 .

Figure 13: Controlled exposure can become disclosure. (a) Paired effects by evidence type; (b) selectivity gaps and relevant-inadmissible effects. Whiskers are pointwise 95% scenario-bootstrap intervals. Readers are separate; Claude Opus 5 is an unpooled replication, and DeepSeek V4 Pro’s result is constructed, not a safety guarantee.

Support	Evidence recall	Wrong-scope leakage	Non-usable fraction
Global dense	1.0	0.025	0.1189
Namespace dense	1.0	0.000	0.0824

Appendix

Table 16: The scope intervention behaves as intended in a controlled smoke. Candidate-pool results on 16 curated evaluation packets.

Layer	Status	Relation to earlier evidence
Retrieval setting selection	Prespecified development procedure	Selected before evaluation with historical $B=U$ loss; no test retuning.
Frozen v1 evaluation	Held-out evaluation of the frozen v1 route family	Preserves the original non-usable status semantics and all nine arms.
Admissibility v2 score	Post-hoc correction; frozen routes	Replaces $B=U$ by $B=A$ in scoring only; no reranking or retuning.
Released-governance v2	Post-hoc untuned attribution; frozen routes	Applies policy independently of intent and reports lifecycle separately.
Two-reader analysis	Prespecified paired execution	Cases, routes, prompts, and two readers frozen before outcome scoring.
GPT-5.6 Luna reader	Gated sequential replication	Repeats the three primary routes after the two-reader continuation criterion.

Appendix

Table 17: The paper separates prespecified, corrected, replicated, and post-hoc analyses. Provenance of the main analysis layers. “Frozen route” means that no ranking, threshold, or candidate order was changed for the analysis.

Explore similar work

Oct 7, 2026cs.AI

Relevance Is Not Sufficiency: What Actually Closes the Evidence Gap in Long-Term Memory QA

LLM agents that interact with a user across many sessions accumulate histories that exceed their context window, so they store past interactions in an external memory and answer each question from a small set of retrieved records. Existing memory systems rank records by lexical or embedding relevance, yet the top-ranked memories can each be relevant while jointly omitting a complementary fact that the answer requires, especially for multi-session and temporal questions. Drawing on the distinction between relevance and sufficiency in legal evidence scholarship, we recast memory retrieval as constructing a sufficient memory set. To operationalize this view, we introduce a blinded LLM judgment over the retrieved set, together with Gold Hit and Turn Hit as evidence-coverage proxies. We then propose Budgeted Flat Reconstruction (BFR), which builds sufficient sets over a fixed flat memory store in two stages. Specifically, we first apply Formal Concept Analysis for Memory Selection (FCA-MS) to decompose the question into information requirements and select a compact candidate subset that jointly covers them. Then, we repeatedly acquire unseen records through deeper text search or complementary entity and session views, stopping when the budget is exhausted. Experiments on LoCoMo and LongMemEval-S show that BFR outperforms same-store adaptations of recent agent-memory systems in both answer quality and evidence coverage. Specifically, on LongMemEval-S it raises judged accuracy from 72.4% to 82.2% and Turn Hit to 91.4%.

Yufeng Li, Shuxin Li, Zhenhua Xu +7

East China Normal University, Shanghai, China · Nanyang Technological University, Singapore · Zhejiang University, Hangzhou, China +3

Long-Term Conversational Memory Agent Memory Management Evidence Retrieval

Ask PDF

Jun 15, 2026cs.AI

MemTrace: Probing What Final Accuracy Misses in Long-Term Memory

LLM agents increasingly maintain long-term memory of user facts across sessions. Yet such memory is usually evaluated by aggregating accuracy over question rows or episodes. Because this approach scores question rows independently, even when several questions probe the same fact, it cannot show how that fact behaves as conditions change. We introduce MemTrace, a benchmark whose unit of measurement is the knowledge point: a single typed fact about the user, rather than an individual question. MemTrace probes each fact along three controlled dimensions: memory age, defined by how many sessions ago the fact appeared in the history; question type, covering current state, earlier state, and trajectory of change; and evidence condition, covering present, missing, and contradicted-by-false-premise settings. Evaluating 13 memory-system configurations across four paradigms, we find that similar pooled accuracy hides different failures: recovering a fact's current and earlier states does not imply tracking how it changed, and safe abstention does not imply correcting a false premise. The dominant bottleneck is evidence use, not retrieval: when systems fail, the evidence was retrievable 10 times more often than it was missing. These results suggest that improving long-term memory requires better use of reachable evidence, not simply more storage or retrieval.

Xianxuan Long, Zhikai Chen, Shenglai Zeng +3

Michigan State University · Case Western Reserve University

Temporal Reasoning in Language Models LLM Agent Memory Persistent Memory for Language Models AI Agent Benchmarks Memory Agent Benchmarks Language Modeling+2

Ask PDF

Jul 27, 2026cs.CL

Keep It InMind: Benchmarking the Implicit-Association Blind Spot in Agent Memory

Long-term memory systems store what a user says in an external store and retrieve it when a related query arrives. This interface rests on an assumption so natural that it is rarely stated: a memory that is needed will resemble the query that needs it. World knowledge breaks the assumption. A tree-nut allergy should change the answer to a macaron request through their almond-flour ingredient, yet the two texts share no cue a retriever can see. We call this failure mode the implicit-association blind spot and introduce InMind, a 125-task, expert-verified benchmark spanning ten life domains, with 113 tasks grounded in citable public sources. Its paired controls separate three explanations that existing evaluations conflate: the fact was never stored, the model lacks the bridging knowledge, or the fact was stored and never surfaced. The verdict is clean. With the decisive memory placed in context, the backbone answers 84.0 percent of indirect queries; when the same memory must be retrieved, six vector, graph, and agentic memory systems reach at most 14.4 percent, even though they recall the same facts on demand at up to 100 percent. An embedding with eight times the dimensionality raises answer-blind target recall for every system yet leaves the gap essentially intact. A minimal diagnostic probe that keeps memory visible before the query arrives recovers most of the gap, locating the failure in the query-conditioned interface itself and pointing to routing, deciding which facts must stay visible, as the open problem InMind is built to score.

Ruizhe Li, Mingxuan Du, Benfeng Xu +1

University of Science and Technology of China · Metastone Technology

Agent Memory In-Context Retrieval Memory Agent Benchmarks

Ask PDF