Externalise the Memory: A Pre-Registered Study of Where an Embodied Agent's Experience Should Live

Cheng-Yin Tu

cs.ROOct 21, 2026

Externalise the Memory: A Pre-Registered Study of Where an Embodied Agent's Experience Should Live

Authors: Cheng-Yin Tu

Organizations: Nikon Research Corporation of America

Abstract

Should an embodied agent's experience be baked into its policy weights, or kept external and retrieved at inference? A memory-augmented teacher (a VLM that retrieves lessons, with their episode images, from its own past attempts) produces trajectories on a vision-conditioned reach task; from the same trajectories we distil LoRA student policies and evaluate them under matched protocols, including one that re-attaches the teacher's frozen retrieval buffer to a fixed set of student weights. We pre-registered the hypotheses, arms, tests, and analysis rules before training and before any p-value. Distilling the memory-augmented behaviour into weights beat an action-only control by +1 pp (n.s.); the registered test proves underpowered for its own minimum effect, so we withdraw - rather than refute - the memory-in-weights claim. Attaching retrieval to a fixed set of student weights added +34 pp (z=5.15), reaching the pooled memory-augmented teacher reference (49.3%); the pre-registered protocol contrast agrees (+23 pp, z=3.36); under a pre-registered auto-reset sensitivity count the two are +36 and +29 pp. An exploratory behavioural probe with pre-registered per-arm predictions is consistent with behavioural distillation transferring the marginal correction but not the conditional, situation-specific correction that retrieval supplies. The recipe that follows is to externalise the memory: in follow-up controls registered before they ran, the same frozen buffer lifts the base model with no adapter to 48% (sensitivity 48%) and the other three adapters from 19-26% to 53-55% (sensitivity 17-21% to 50-55%), and the adapter adds nothing detectable once the buffer is attached. On a second, harder task, the retrieval advantage points the same direction but falls short of significance (+6 pp, 95% CI [-4.5, +16.4]); exploratory diagnostics tie the attenuation to memory coverage.

Figures & tables

	no gist	with gist
no native thought	A_action_only	D_gist
with native thought	A_ctrl_rat	B_main

Table 1

Protocol	Adapter	Inference-time retrieval
A_action_only	own	none
A_ctrl_rat	own	none
B_main	own	none
D_gist	own	none
C_retrieval	A_ctrl_rat’s	frozen teacher buffer snapshot

Table 2

#	Date	Change	Anchor
1	07-06	Sample-count control arm (B_matched) added	pre-reg §12
2	07-06	Filter + stratified blinded audit; escalation rule (agreement $<$ 85% escalates to LLM judge)	—
3	07-06	Asymmetric KTO drop policy (desirable vs undesirable) + auto-balance invariant	2ba693c
4	07-07	Rule-1 sentence classifier (past/intent) fix	—
5	07-07	Rule-2 demoted to flag; composite-agreement rejected	6a29103
6	07-07	Coherence filter demoted to advisory flags (v-final 992+1810)	9b0a9da

Table 3

rung	student	teacher (same rung)
no-memory	B_main / A_action_only $\approx$ 25–26% (sens. 19–21)	memoryless teacher 22% $\dagger$ (sens. 19)
with-memory	C_retrieval 49% (sens. 48)	memory-teacher 49.3% $\ddagger$ (sens. 47.5)

Table 4

Weights + frozen buffer	SR	without buffer (D11)	$\Delta$ vs own, $z$	$\Delta$ vs C_retrieval, $p$	R1 ′ $r$ (above band)
base, no adapter (a)	48 (48)	—	—	$-$ 1 pp, 0.887	+0.29 (yes)
A_action_only (b)	53 (53)	25 (21)	+28 pp, 4.06	+4 pp, 0.572	+0.49 (yes)
B_main (b)	53 (50)	26 (19)	+27 pp, 3.91	+4 pp, 0.572	+0.36 (yes)
D_gist (b)	55 (55)	19 (17)	+36 pp, 5.27	+6 pp, 0.396	+0.44 (yes)

Table 1: Amendment 15 adapter controls (

n

=100 per arm; auto-reset sensitivity counts in parentheses).

\Delta

against the same weights’ D11 run without retrieval, and against C_retrieval ; two-proportion

z

, two-sided. R1 ′ slope: Spearman

r

of round-1

\Delta

X against the required correction under the 15 mm recovery gate (A15.1; pre-registered reading for the four with-buffer rows).

Arm	R1 $\Delta$ X $\mu$	$\sigma$	closer to
A_action_only	$-$ 16.62	8.61	mem-teacher ( $-$ 15.8)
A_ctrl_rat	$-$ 15.75	10.11	mem-teacher
B_main	$-$ 16.67	8.53	mem-teacher
D_gist	$-$ 18.15	7.89	mem-teacher
C_retrieval	$-$ 7.10	14.39	(bias nearly erased)

Table 6

distilled arm	r (round 1)	r (round 2)	r (round 3)
A_action_only	$-$ 0.12 (0.23)	+0.05 (0.70)	+0.18 (0.088)
B_main	$-$ 0.12 (0.22)	+0.05 (0.65)	+0.40 (0.001)
A_ctrl_rat	+0.03 (0.78)	+0.11 (0.30)	+0.27 (0.013)
D_gist	$-$ 0.12 (0.23)	+0.10 (0.34)	+0.12 (0.27)
n (paired episodes)	100	89–90	84–85

Table 2: Correlation of each distilled arm’s per-episode residual with C_retrieval ’s, by round (two-sided permutation p, 2000 draws). All rounds are a 2026-10 post-hoc recomputation from the persisted replays with the round-1 definition; round 1 reproduces the original exploratory output exactly.

Test	Contrast	$\Delta$ SR	$\Delta$ 95% CI (Newcombe)	z	p_z	McNemar p	agree	verdict @ pre-reg $\alpha$
T1	B_main $-$ A_action_only	+1 pp	[ $-$ 11.0, +13.0]	0.16	0.87	1.0	✓	FAIL @ $\alpha$ =0.020 $\rightarrow$ null
T1a	B_main $-$ A_ctrl_rat	+11 pp	[ $-$ 0.2, +22.0]	1.93	0.054	0.046	✓	n.s. @ $\alpha$ =0.020
T4	C_retrieval $-$ B_main	+23 pp	[+9.6, +35.3]	3.36	7.8e-4	1.0e-3	✓	PASS @ $\alpha$ =0.010
T3	B_main SR vs 0.7 $\times$ teacher floor	26% vs 36.2%	—	$-$ 2.12	0.98 1	—	—	BELOW floor
Auto-reset sensitivity (Amendment 15 pin 8; reported beside, never replacing, the rows above)
T1 sens.	19 vs 21	$-$ 2 pp	[ $-$ 13.1, +9.1]	$-$ 0.35	0.72	0.80	✓	null

Table 3: Confirmatory contrasts (

n

=100/arm, two-sided

z

primary).

Protocol	Adapter	Training target	Inference retrieval
A_action_only	own	canonical action only	none
A_ctrl_rat	own	native thought + action	none
B_main	own	gist + native thought + action	none
D_gist	own	gist + action	none
C_retrieval	A_ctrl_rat’s	(reuses A_ctrl_rat)	frozen D10-ext buffer

Table 4: Arm / protocol definitions (from pre-reg §10.3).

protocol	successes / n	SR
A_ctrl_rat (bare)	14 / 100	14%
C_retrieval (same weights + frozen re-tagged buffer)	20 / 100	20%

Table 10

#	Date	Change	Anchor
L2-1	07-15	Per-arm scoring (post-hoc, disclosed); SFT single-arm target; task rename	l2_amendment_1.md
L2-1a	07-15	Buffer re-tag (dual label) + arm-aligned success-floor; pins 83337879 $\rightarrow$ a79581cd	l2_amendment_1.md §1a
L2-2	07-15	Scope to answerable question (T4-class only); Stage-1 A_ctrl_rat; T1 not run (MDE design choice)	l2_amendment_2.md
L2-2 add	07-16	Per-arm SFT pool selected by label not directory (verify-layer 2/2)	l2_amendment_2.md
L2-2 add	07-21	GGUF conversion version-drift caught at load gate; rollback + arch byte-patch (verify-layer 3/3)	l2_amendment_2.md
L2-3	07-22 decided / 07-28 re-filed	Per-arm EVAL: single-arm inference matches single-arm training; non-scored arm frozen; left-scored $\times$ 2 protocols n=100, seed 4600; position-only success; format-contract gate (verify-layer 4/4)	l2_amendment_3.md ( 96e1388 ; spec-implementing code 0398568 , 07-22, before the eval)

Table 11

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Collect	Teacher	Reasoning
D10 / L0a	gemma-4-31B Q4_K_M, ctx 16384	budget 512 (on)
L2 (from 2026-07-14)	same	off (budget 0)

Appendix

Table 12

Memory-Augmented Agents

Memory-augmented agents use stored experience and knowledge to act coherently across long tasks, changing environments, and repeated interactions.

See all 346 papers

Aug 30, 2026

Hindsight Memory-PRM: Supervising Memory Management with Auditable Hindsight Credit

Memory operations of long-horizon LLM agents are hard to supervise: an operation's value is unobservable when it is taken. But they are special -- they leave machine-readable evidence in the trajectory: retrieval hits and answer-time citations. Hindsight Memory-PRM exploits this audit trail twice: offline to train an operation-conditioned memory-utility critic, and online, where retrievals, citations, and one controlled deletion-and-reanswer per probe settle an intervention-calibrated entry-level presence credit, propagated along version chains as an action-level proxy reward -- no per-operation human labels, no Monte-Carlo replay of continuations. On held-out LoCoMo a local 8B policy reaches 77.5% under a fixed shared reader, surpassing its API teacher (65.1%) and all reproduced external systems, at one eighth the context of Mem0's official operating point; on LongMemEval, 79.0%. Ablations attribute the gain to causal calibration rather than signal density, and the policy converges to a multi-version memory organization whose gains no tested open-loop baseline reproduces.

Haoxuan Jia, Yang Liu, Yingguang Yang +14

Fullive-AI · Nanyang Technological University · Supply Chain Tech Team Y, JD.com +7

Agent Memory Management Long-Horizon LLM Agents Process Reward Models Credit Assignment

Ask PDF

Jun 28, 2026

Selective Memory Retention for Long-Horizon LLM Agents

When does retention matter for memory-augmented LLM agents? We study this with TraceRetain, a lightweight framework for bounded external memory in frozen LLM agents that scores entries by interpretable features (success, age, access frequency, redundancy, specificity, similarity, downstream utility) and evicts the lowest-scoring ones at capacity. On clean ALFWorld with gpt-5-mini, external memory robustly improves over no memory across two seeds, but differences among bounded retention policies fall within Wilson 95% CIs: clean ALFWorld at T=100 to T=200 does not naturally exhibit the memory pollution retention is designed to address. Under a controlled noisy-write stress (75% synthetic distractors), unbounded memory and FIFO-K50 degrade on Precision@5 (20.2% to 12.4% and 15.8% to 3.8%) while TraceRetain-CEM is essentially unchanged (16.9% to 16.6%) and preserves 97/100 task success. The mechanism: unbounded memory has the highest mean similarity (0.87) but lowest precision, indicating failed distractors close to the query in embedding space. Held-out in-distribution evaluation shows memory-augmented policies solving 47 to 49 of 50 tasks vs. 39/50 for no memory. Bounded retention buys memory and step efficiency on saturated clean benchmarks at no task-success cost, and only differentiates from cache heuristics when streams contain noise.

Pranath Reddy

Independent Researcher, Huntsville, Alabama, USA.

LLM Agent Memory Persistent Memory for Language Models Long-Horizon LLM Agents Agent Memory Management Memory-Augmented Language Model Agents Memory Management for LLM Agents

Ask PDF

Jun 29, 2026

DuoMem: Towards Capable On-Device Memory Agents via Dual-Space Distillation

Large Language Model (LLM)-based agents can solve complex procedural tasks by interacting with environments over multiple turns, but this ability typically depends on large models, long contexts, and repeated inference calls. This makes advanced memory-augmented agents difficult to deploy on resource-constrained devices. We introduce DuoMem, a dual-space distillation framework that transfers procedural problem-solving ability from a large teacher model to compact student models. DuoMem distils in two complementary spaces: (1)context-space distillation, which replaces student-generated memories with higher-quality teacher-generated procedural memories prepended to the student's input, and (2)parameter-space distillation, which fine-tunes lightweight LoRA adapters on successful teacher trajectories. Evaluated on ALFWorld, a challenging embodied decision-making benchmark, DuoMem boosts a 4B-parameter model from 4.3% to 77.9% task success rate, closing most of the gap to a 72B teacher model (87.1%), while adding fewer than 10M trainable parameters and only a few megabytes of pre-computed teacher memories. Moreover, the DuoMem-enhanced 4B model completes tasks over 3x faster than the 72B teacher in wall-clock time, making it viable for real-time edge deployment, which would be challenging for the teacher.Extensive ablations across eight models spanning 2B-72B parameters reveal that both distillation axes contribute complementary

Peyman Hosseini, Ondrej Bohdal, Ahmed Alajrami +6

Samsung R&D Institute UK · Queen Mary University of London

LLM Agents LLM Agent Training Context Distillation Knowledge Distillation Memory-Augmented Agents

Ask PDF

Abstract

Memory-Augmented Agents

Explore similar work

Hindsight Memory-PRM: Supervising Memory Management with Auditable Hindsight Credit

Selective Memory Retention for Long-Horizon LLM Agents

DuoMem: Towards Capable On-Device Memory Agents via Dual-Space Distillation