When Stale Constraints Go Unchecked: Budgeted Verification Failures in Inherited Agent Memory

Kazuki Nakayashiki

cs.IRAug 26, 2026

When Stale Constraints Go Unchecked: Budgeted Verification Failures in Inherited Agent Memory

Authors: Kazuki Nakayashiki

Organizations: Glasp

Abstract

Provenance links keep the evidence behind an inherited belief reachable; an agent with a verification budget must still choose which links to inspect. We study a consolidated memory that states a decision constraint and whose source record has since been superseded by a record that withdraws it: provenance is immutable, the current record has changed, and the memory is stale. In a controlled six-memory scenario with a budget of two records, sixteen language models rarely re-verified a constraint that read as settled: they inspected its provenance path in about one episode in five and, once the constraint had been superseded, produced stale-consistent decisions in 77.3%, 74.7% and 74.7% of episodes across a primary run, a replication and a held-out domain. Re-assigning one of the same two slots to the critical path removed most of them: +74.0, +72.7 and +61.3 points (positive in every model), +80.7 in a prospectively frozen interleaved replication with a repaired non-critical control, and +62.0 on a panel of 10 models from 9 organisations; a corrected re-run of the held-out scenario gave +73.3. The forced-critical policy uses experimenter knowledge of the critical path: it quantifies how much stale-decision risk the same budget can recover and is not a scheduler. Two further deposited experiments locate the failure and a remedy: in this store the constraint's path is selected in 17.0% of episodes at two slots and 88.7% at four of six (above uniform allocation), and at two slots a one-sentence, target-blind rule (prefer memories that state a limit on a candidate direction) moved the agent's own allocation onto the constraint's path and recovered the oracle contrast on decisions (+89.3 points) where that constraint limits the tempting action, while a content-free freshness cue did not materially redirect allocation and a content-matched control rule changed neither selection nor decisions.

Figures & tables

Panel of Figure 1 — Figure 1: The supersession setting: provenance $M\!\to\!S_{0}$ never changes; what changes at $t_{1}$ is which record is current; $S_{1}$ enters the agent’s context only if a verification slot reaches $M$ ’s path, which the assigned policy manipulates; the decision is scored against the current record.

run	total run $N$	native $Y$	forced-critical $Y$	RD [95%]	positive models
primary	1,800	34/150	145/150	+74.0 [+68.0, +80.0]	6/6
fresh-wording replication	1,800	38/150	147/150	+72.7 [+66.7, +78.7]	6/6
original held-out †	900	38/150	130/150	+61.3 [+54.0, +68.0]	6/6
corrected held-out robustness	900	36/150	146/150	+73.3 [+68.7, +77.3]	5/6
interleaved replication (Exp. A) ‡	900	24/150	145/150	+80.7 [+74.0, +86.7]	6/6
cross-organisation panel (Exp. X) ‡	1,498	92/250	247/250	+62.0 [+57.2, +66.8]	10/10

Table 1: The same-budget contrast across the four original runs and the post-review, prospectively frozen replications (stated form, superseded world; Exp. C: the withdraws world of its store, its oracle positive control; its non-oracle rule is in § 4.6 ).

Y

= decision consistent with the record the archive marks current. Headline comparison

n=150

per policy arm (

n=250

in Exp. X); total run

N

counts every cell. Intervals: model-stratified bootstrap 95%; two-record budget in every arm.

Panel of Figure 2 — Figure 2: Share of decisions consistent with the stale memory rather than the current record ( $1-Y$ ; stated form, superseded world; Exp. C rows: the withdraws world of its store) under native allocation (filled) and the same budget with one slot re-assigned to the critical path (open); Wilson 95% intervals. † Reported as run, with the inconsistency of § 4.4 . Lower rows: Experiments A, X and C; ∗ the open point is the agent’s own allocation under the target-blind rule P1, not forced-critical.

Panel of Figure 3 — Figure 3: The same-budget effect tracks the native missed-path rate: one point per model and run ( $n=25$ per arm; 40 points, coincident values overlapping); 37 lie within 12 points of the dashed line, the departures being GPT-5.6 Sol (original held-out, $-$ 32); GPT-5.6 Terra (original held-out, $-$ 16); Llama 4 Maverick (Exp. X, $-$ 44). † Original held-out run.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

form	world	policy	$Y$	%	path named	record returned
stated	valid	native	149/150	99.3	27/150	27/150
stated	valid	forced-critical	150/150	100.0	22/150	150/150
stated	valid	forced-noncritical	146/150	97.3	32/150	13/150
stated	superseded	native	34/150	22.7	32/150	32/150
stated	superseded	forced-critical	145/150	96.7	36/150	150/150
stated	superseded	forced-noncritical	17/150	11.3	32/150	13/150

Appendix

Table 2: Primary run (1,800 episodes): authoritative-consistent decisions

Y

, target provenance path named at turn 1, and target record returned, by cell (

n=150

per cell, 25 per model).

form	world	policy	$Y$	%	path named	record returned
stated	valid	native	147/150	98.0	43/150	43/150
stated	valid	forced-critical	150/150	100.0	33/150	150/150
stated	valid	forced-noncritical	148/150	98.7	32/150	21/150
stated	superseded	native	38/150	25.3	37/150	37/150
stated	superseded	forced-critical	147/150	98.0	32/150	150/150
stated	superseded	forced-noncritical	16/150	10.7	31/150	13/150

Appendix

Table 3: Fresh-wording replication (1,800 episodes, families r1–r6, fresh seeds), same layout.

form	world	policy	$Y$	%	path named	record returned
stated	valid	native	149/150	99.3	32/150	32/150
stated	valid	forced-critical	150/150	100.0	35/150	150/150
stated	valid	forced-noncritical	150/150	100.0	33/150	22/150
stated	superseded	native	38/150	25.3	40/150	40/150
stated	superseded	forced-critical	130/150	86.7	40/150	150/150
stated	superseded	forced-noncritical	18/150	12.0	37/150	18/150

Appendix

Table 4: Original held-out run, procurement world (900 episodes; stated form only). Reported exactly as run; see § 4.4 .

form	world	policy	$Y$	%	path named	record returned
stated	valid	native	150/150	100.0	43/150	43/150
stated	valid	forced-critical	150/150	100.0	41/150	150/150
stated	valid	forced-noncritical	150/150	100.0	48/150	24/150
stated	superseded	native	36/150	24.0	36/150	36/150
stated	superseded	forced-critical	146/150	97.3	46/150	150/150
stated	superseded	forced-noncritical	21/150	14.0	42/150	21/150

Appendix

Table 5: Corrected held-out robustness replication (900 episodes; one sentence changed, fresh seeds, externally deposited before execution). Supplements, and does not replace, the original run.

	primary		replication		original held-out		corrected held-out
model	RD	$U$	RD	$U$	RD	$U$	RD	$U$
Opus 5	+88.0	100	+96.0	100	+88.0	96	+96.0	100
Sonnet 5	+16.0	24	+16.0	24	+60.0	60	+56.0	68
Haiku 4.5	+96.0	100	+84.0	88	+16.0	20	+0.0	0
GPT-5.6 Sol	+88.0	88	+76.0	76	+56.0	88	+96.0	96
GPT-5.6 Terra	+68.0	68	+76.0	76	+72.0	88	+92.0	92

Appendix

Table 6: Per-model same-budget effect (forced-critical minus native on

Y

, stated form, superseded world;

n=25

per arm per model) with each model’s native missed-path rate

U=100\times(1-\text{native target verification})

. Direction generalises; magnitude tracks

U

family	forced-critical	native	RD
f1	22/22	6/22	+72.7
f2	21/23	7/23	+60.9
f3	20/22	5/22	+68.2
f4	26/27	9/27	+63.0
f5	28/28	3/28	+89.3
f6	28/28	4/28	+85.7

Appendix

Table 7: Per-wording-family same-budget effect (stated form, superseded world): forced-critical vs native

Y

. Families f1–f6 (primary) are the prior work’s slot-grammar families; r1–r6 (replication) are new.

world	policy	V0Y0	V0Y1	V1Y0	V1Y1	$Y$
valid	native	3	116	0	31	147/150
valid	forced-critical	0	117	0	33	150/150
valid	repaired-noncritical	1	117	0	32	149/150
superseded	native	126	1	0	23	24/150
superseded	forced-critical	3	117	2	28	145/150
superseded	repaired-noncritical	115	3	0	32	35/150

Appendix

Table 8: Experiment A (interleaved; stated form): cell counts of

V

(target path named at turn 1)

\times

Y

(current-record-consistent decision),

n=150

per cell.

model	RD	$U$
Opus 5	+92.0	100
Sonnet 5	+52.0	64
Haiku 4.5	+96.0	96
GPT-5.6 Sol	+84.0	88
GPT-5.6 Terra	+68.0	68
GPT-5.6 Luna	+92.0	92

Appendix

Table 9: Experiment A per model (

n=25

per arm): forced-critical minus native on

Y

(superseded world) and the native missed-path rate

U

world	arm	$V$	$Y$	$Y\mid V{=}1$	$Y\mid V{=}0$	m31 ∗	m57	m73 ∗†	m86	m91 ∗
withdraws	hidden	19.3	19.3	100.0	0.0	61.3	0.0	19.3	94.0	25.3
withdraws	visible	15.3	15.3	100.0	0.0	77.3	0.0	15.3	81.3	26.0
agrees	hidden	20.7	98.7	100.0	98.3	71.3	0.7	20.7	88.0	19.3
agrees	visible	19.3	98.7	100.0	98.3	74.7	0.0	19.3	79.3	26.7

Appendix

Table 10: Experiment B decomposition (

n=150

per cell): target-path selection

V

, current-record-consistent decision

Y

Y

given

V

, and the share of episodes selecting each provenance path (%).

model	$\Delta V$	$\Delta Y$
Opus 5	+0.0	+0.0
Sonnet 5	+4.0	+4.0
Haiku 4.5	$-$ 4.0	$-$ 4.0
GPT-5.6 Sol	$-$ 12.0	$-$ 12.0
GPT-5.6 Terra	$-$ 16.0	$-$ 16.0
GPT-5.6 Luna	+4.0	+4.0

Appendix

Table 11: Experiment B per model (withdraws world,

n=25

per arm):

\Delta V

and

\Delta Y

, metadata-visible minus metadata-hidden.

world	policy	V0Y0	V0Y1	V1Y0	V1Y1	$Y$
valid	native	10	155	1	83	238/249
valid	forced-critical	1	162	1	85	247/249
valid	repaired-noncritical	13	151	0	86	237/250
superseded	native	155	14	3	78	92/250
superseded	forced-critical	2	161	1	86	247/250
superseded	repaired-noncritical	162	15	1	72	87/250

Appendix

Table 12: Experiment X (cross-organisation panel; stated form): cell counts of

V

(target path named at turn 1)

\times

Y

(current-record-consistent decision),

n=250

per cell (249 in the cells with an error file).

model	episodes	errors	cap hits	schema attempts (parse)	transport (429)	reasoning	provider constant	served = requested
gpt-oss-120b	149	1	0	1 (0)	161 (122)	yes	yes	yes
DeepSeek V4 Pro	150	0	0	0 (0)	44 (21)	yes	yes	yes
Kimi K3	150	0	0	5 (0)	170 (167)	yes	yes	yes
MiniMax M3	150	0	3	5 (4)	29 (11)	no	yes	yes
Llama 4 Maverick	149	1	0	7 (0)	114 (0)	no	yes	yes
Hunyuan HY3	150	0	0	0 (0)	0 (0)	yes	yes	yes

Appendix

Table 14: Experiment X serving record per model: episodes, error files, episodes with an output-cap hit, extra schema attempts in completed episodes (of which parse failures; the rest are transport exhaustions re-booked as attempts by the frozen rule), transport retry lines (HTTP 429 in parentheses; error files included), reasoning tokens returned on any call, whether the routed provider was constant across all calls, and whether the served model string returned by the routing service equalled the requested identifier on every call (an echo, not a version identifier).

$k$	$V(k)$ [95%]	$\rho(k)$ [95%]	ids named	RD $(k)$ [95%]	$Y$ (fc)	$U(k)$	valid fc $-$ native
1	5.3 [3.0, 8.0]	0.32 [0.18, 0.48]	1.00	—	—	—	—
2	17.0 [13.7, 20.7]	0.51 [0.41, 0.62]	2.00	+76.7 [+70.0, +82.7]	95.3	83.3	+2.0
3	41.7 [37.3, 46.0]	0.83 [0.75, 0.92]	2.99	+54.0 [+46.7, +61.3]	96.7	58.7	+1.3
4	88.7 [84.7, 92.3]	1.33 [1.27, 1.39]	3.90	+10.7 [+4.0, +17.3]	95.3	12.0	+0.0
4 (removed form)	97.0 [94.7, 99.0]	—	—	—	—	—	—

Appendix

Table 15: Experiment C, Block K (Study A’s store; six models;

n=150

per cell, 300 pooled over worlds for

V

). Native target-path selection

V(k)

, the selection ratio

\rho(k)=V(k)/(k/6)

against the uniform benchmark

\rho=1

(bootstrap over (model, run) blocks, 95%), mean ids named, the removed-form control at

k=4

, and the forced-critical bundle:

\mathrm{RD}(k)

Y

under forced-critical and the native missed-path rate

U(k)

(superseded world).

$k$	arm	$V$	$\Delta V$ [95%]	$\Delta Y$ withdraws [95%]	$\Delta Y$ agrees [95%]	decoy %	reading
1	P0 (no rule)	0.3	—	—	—	76.0	reference
1	P1 rule	43.7	+43.3 [+38.3, +48.7]	+42.7 [+36.0, +48.7]	+0.7 [ $-$ 2.7, +4.0]	56.3	material
1	P2 dates	2.7	+2.3 [+0.3, +4.3]	+4.7 [+1.3, +8.7]	+2.0 [ $-$ 0.7, +4.7]	63.3	no material redirection
1	P3 rule+dates (ceiling)	98.7	+98.3 [+97.0, +99.7]	+97.3 [+94.7, +99.3]	+2.7 [+0.7, +5.3]	1.3	material
1	P4 control rule	0.3	+0.0 [ $-$ 1.0, +1.0]	+0.0 [ $-$ 2.7, +2.7]	+2.0 [ $-$ 0.7, +5.3]	1.0	no material redirection
2	P0 (no rule)	10.0	—	—	—	89.0	reference

Appendix

Table 16: Experiment C, Block P (three-constraint store; Experiment B’s archive semantics;

n=150

per arm and world). Target-path selection

V

(worlds pooled),

\Delta V

and

\Delta Y

against P0 (withdraws world; agrees world as the cost diagnostic), the decoy-constraint selection rate (memory_86 or memory_57 named), and the registered reading. P3 is the instruction-following ceiling; P4 the content-matched control; FC-first

-

P0 the outcome-contingency positive control; FC-second

-

FC-first the order contrast on FC-first’s own turn 1.

model	$V_{1}$	$V_{2}$	$V_{3}$	$V_{4}$	$V_{4}^{\mathrm{rem}}$	RD 2	RD 3	RD 4
Opus 5	0.0	0.0	14.0	92.0	100.0	+88.0	+72.0	$-$ 12.0
Sonnet 5	18.0	50.0	94.0	100.0	98.0	+28.0	+4.0	+12.0
Haiku 4.5	10.0	2.0	4.0	78.0	94.0	+96.0	+80.0	+24.0
GPT-5.6 Sol	0.0	6.0	46.0	94.0	100.0	+92.0	+52.0	+8.0
GPT-5.6 Terra	2.0	44.0	60.0	98.0	100.0	+56.0	+44.0	+4.0
GPT-5.6 Luna	2.0	0.0	32.0	70.0	90.0	+100.0	+72.0	+28.0

Appendix

Table 17: Experiment C per model, Block K (

n=25

per cell and world): native target-path selection

V(k)

, the removed-form

V

k=4

, and forced-critical minus native on

Y

(superseded world) at

k=2,3,4

model	P0 1	P1 1	P3 1	P4 1	P0 2	P1 2	$Y$ P0	$Y$ P1	$Y$ FC1	$Y$ FC2
Opus 5	0.0	34.0	100.0	0.0	0.0	100.0	0.0	100.0	96.0	100.0
Sonnet 5	2.0	54.0	100.0	0.0	34.0	100.0	28.0	100.0	100.0	100.0
Haiku 4.5	0.0	96.0	100.0	0.0	2.0	98.0	0.0	100.0	100.0	100.0
GPT-5.6 Sol	0.0	12.0	100.0	0.0	4.0	100.0	4.0	100.0	100.0	100.0
GPT-5.6 Terra	0.0	20.0	100.0	2.0	16.0	100.0	24.0	100.0	100.0	100.0
GPT-5.6 Luna	0.0	46.0	92.0	0.0	4.0	100.0	8.0	100.0	100.0	100.0

Appendix

Table 18: Experiment C per model, Block P (

n=25

per cell and world): target-path selection under P0/P1/P3/P4 at

k=1

and P0/P1 at

k=2

(worlds pooled), and

Y

in the withdraws world for P0, P1, FC-first and FC-second at

k=2

Explore similar work

Sep 2, 2026cs.AI

ERRAND: Budgeted Maintenance of Agent Memory

Deployed agents run on handed-over knowledge: a frozen policy consults a briefing of consolidated items written before the stream begins. The world then moves while the store stands still: paths close, flags change, price bands move; every item was true at handover, and the failure is staleness, not ignorance. We introduce ERRAND, which treats revalidation as a priced errand: a recheck competes with the task it protects for the same scarce actions, funded only when the value per action of resolving a doubt clears a running wage. The errand index is single-peaked, vanishing at both ends of belief, so certainty in either direction costs nothing; free en-route receipts maintain on-path knowledge, and repair writes a version, never a deletion. Under equal action budgets in two drifting tool-use worlds, ERRAND clears every non-oracle policy on the preregistered calibers, primary in every setting and conditional at every binding budget, leading eager revalidation by 10.0pp at the base cap. Restraint wins: given no cap, ERRAND stops on its own, spending 11.0% of steps, while uncapped eager revalidation spends 70.7% and still finishes 4.5pp behind capped ERRAND. The margin sits where the briefing's coverage is thinnest, the shadow price of long-tail knowledge: a small budget, well priced, beats a bigger store that never rechecks.

Beining Wu, Zihao Ding, Jun Huang

Department of Electrical Engineering and Computer Science South Dakota State University, Brookings, SD 57007, USA

Agent Memory Management Memory-Augmented Agents

Ask PDF

Sep 3, 2026cs.IR

Plan Pointers and Record-Directive Form in Budgeted Verification of Inherited Agent Memory

A model that inherits one-line memories may pull one archived source record before acting; a directive in the store can steer that pull: a pointer, a criterion or both. Across sixteen registered studies (179,352 attempts) we measured where the request goes under each form; every result is descriptive, with registered intervals, no mechanism claim. A length-matched criterion exceeded a bare id on six direct-provider models (D) and failed its registered superiority rule on a nine-model OpenRouter panel (E). On generated worlds (K2-K5): the two registered signatures held on Opus 5 and Fable 5.1, Fable 5 followed the same sign, Haiku 4.5 reversed, and Sonnet 5, the GPT-5.6 endpoints and GPT-6 Astra lay near zero (K2). With a defensive adapter at five gains, the 70B rule for a gain-dependent change of the composite - criterion contrast was not met (K3 and K4); under the 8B attenuation rule (0.95 intervals: slope below zero; change beyond the margin), the 8B change of -17.5 [-26.7, -8.1] did not meet it on 36 families (K4) and at registered power on 337 families -16.6 [-19.4, -13.8] did (realised one-sided error at the margin 1.8 to 3.2% per corner of a finite grid, nominal 2.5%, not a uniform-error guarantee; K4's status stands; K5, first ladder), while a second SecAlign++ adapter under the imposed Meta-SecAlign template did not (-11.8 [-14.3, -9.3]; K5, second ladder); no NOT-MET is a statement that the contrast was unchanged; their difference (+4.7 [+2.3, +7.2]) describes two fixed execution paths, licenses no superiority, equivalence or 'significant difference' claim; nothing follows from the statuses differing (K5). Intervals describe family-reweighting stability conditional on the execution, not reproducibility across engine executions; audit replays were neither substituted for nor averaged into outcomes; no missingness gate fired and directional completions changed no status.

Kazuki Nakayashiki

Glasp

LLM Agent Memory LLM Agent Verification Agentic Retrieval Agent Memory Management

Ask PDF

Aug 3, 2026cs.AI

When Memory Updates but Behavior Does Not: Repairing Implicit Stale Dependencies in Personalized Agent Responses

Memory-augmented agents can know that a user's stored state is outdated and still plan around the old value. The STALE benchmark calls this the implicit policy adaptation (IPA) gap. We identify one structural contributor: draft-anchored verification checks what a response says, and in an open-ended response the stale dependency is usually unsaid. StateAuditor therefore audits in the opposite direction, from stored state to draft. An LLM proposes candidate old-to-new transitions from timestamped evidence; deterministic code pins each quotation to a single entry, checks that the new evidence really is newer, and lets only these verified transitions trigger repair. What is verified is provenance and chronology - not semantic supersession. On STALE's full protocol (400 scenarios, 50-session histories, one independent response per query), strict single-query VTA scores .736 against .686 for our locked predecessor under the same judge: a +5.0-point paired gain (95% CI [+2.9, +7.2]) coming almost entirely from IPA and premise resistance (PR). The benchmark's own judge, from a third model family, reproduces the gain (.738 vs. .680). On an independent cross-family preference-evolution benchmark (HorizonBench), the full draft-audit-repair pipeline over a gold-derived structured store raises current-preference accuracy (user-clustered p<.01), though a matched control shows most of this external gain is the draft-side audit itself; a harder authored lifecycle set gives no gain, bounding the claim while false invalidation stays controlled. On STALE, by contrast, a matched control (same evidence, adapter, and call budget) scores only .692 (+0.6 over the predecessor, n.s.), attributing the STALE gain to the transition machinery rather than added context or calls. We make no claim about general-purpose agent memory.

Haofei Sun, Lin He

The University of Texas at Arlington · The University of Tennessee, Knoxville

LLM Agent Memory Memory Agent Benchmarks Agent Memory Management Memory-Augmented Language Model Agents

Ask PDF