REMM: Revisable Embodied Memory for Manipulation

Revanth Krishna Senthilkumaran; Ruwan Wickramarachchi; Alessandro Oltramari; Jonathan Francis

cs.ROOct 8, 2026

REMM: Revisable Embodied Memory for Manipulation

Authors: Revanth Krishna Senthilkumaran, Ruwan Wickramarachchi, Alessandro Oltramari, Jonathan Francis

Organizations: Carnegie Mellon University · Bosch Center for AI

Abstract

Long-horizon manipulation requires robots to retain task-relevant history while revising conclusions no longer supported by physical evidence. Retaining observations alone does not specify which completed conditions should be withdrawn after a disturbance or which obligations require repair. We introduce REMM, Revisable Embodied Memory for Manipulation, a supervisory memory layer for a frozen G0.5 vision-language-action executor. REMM separates an append-only evidence history from revisable task state and explicitly links derived relations and completed conditions to their supporting facts. When accepted evidence changes a supporting fact, it withdraws dependent conclusions and schedules targeted recovery at verified skill boundaries without updating the executor. Across 51 physical trials, including operator-assisted runs, REMM improves mean recorded task progress over the strongest baseline from 57% to 91% in intervention recovery and from 43% to 70% in sequence recall. Complementary video diagnostics show improved grounding of retained history into the next instruction, including comparisons with $π_{0.5}$ -compatible high-level planner adaptations. These results support revisable memory as a practical interface for continued manipulation without policy retraining. Project: https://revanthsenthil.github.io/remm/.

Figures & tables

Fig. 1: Revising memory after physical intervention. A manipulation history can support either continued execution or recovery depending on whether later evidence preserves or contradicts an earlier task relation. REMM updates the supported task state and changes the subsequent robot action without retraining the frozen executor.

Fig. 2: System and memory architecture. Left: REMM converts a long-horizon instruction and multi-view observations into the current atomic instruction for a frozen G0.5 executor, which predicts action-code tokens and action chunks for the Galaxea R1 Lite; new observations and robot state close the loop after execution. Right: a task decomposer and skill contracts define the active skill, frozen perception and visual verification produce accepted typed events, and an append-only ledger maintains an active-state projection and support graph. When new evidence supersedes a supporting fact, dependency invalidation withdraws affected goal records before REMM schedules a repair skill or issues the revised next query. Snowflakes denote frozen models.

		Memory demands			Revision demands
Task	Condition	Retention	Order	Identity	Revision	Invalidation	Recovery	Retargeting
T1a	Queued disruption	$\checkmark$	$\times$	$\times$	$\checkmark$	$\checkmark$	$\checkmark$	$\times$
T1b	Destination shift after pouring	$\checkmark$	$\times$	$\times$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$
T2	Ordered recall + disturbed repeats	$\checkmark$	$\checkmark$	$\times$	$\checkmark$	$\checkmark$	$\checkmark$	$\times$
T3	Hidden-object shuffle	$\checkmark$	$\times$	$\checkmark$	$\times$	$\times$	$\times$	$\times$

TABLE I: Evaluation coverage by task. Checks mark mechanisms tested by at least one predeclared condition. T2 combines nominal ordered recall with disturbed repeats; only the disturbed repeats test revision and recovery.

Fig. 3: Physical task families. T1a and T1b test recovery scheduling and retargeting; T2 tests ordered recall and disturbed repeats; T3 tests hidden-object identity. Blue borders mark human motion and orange borders robot execution; arrows show task progression.

Fig. 4: Physical outcomes by task. Bars show full, partial, and failed outcomes with raw counts. Assisted runs remain included; HiMe denotes our adaptation around the shared G0.5 executor.

Fig. 5: Task-level system profiles. Task-colored sectors show recorded task progress, decision accuracy, non-failure completion, and relative time and G0.5-query efficiency. Progress includes the stated missing-data fallbacks. T1 pools T1a/T1b; higher is better. Non-failure includes partial trials; efficiency is

100\times

best task-specific median cost / method median.

Fig. 6: Offline video task families. T2 shows cue demonstration and covering; T3 shows target covering and rearrangement. These diagnostics score high-level decisions without motor-policy execution.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Fig. 7: Recorded T1b memory example. The trace contains 2,408 head-camera positions for 15 identities; gaps are disconnected tracks. The joined excerpt exposes evidence-to-event, fact-supersession, and recovery references. This assisted partial rollout uses the recompute-on-demand control and is included only for mechanism inspection.

Task	Vanilla G0.5	HiMe adaptation	REMM
T1a	0 / 4 / 1	2 / 3 / 0	2 / 2 / 0
T1b	0 / 5 / 0	0 / 2 / 1	3 / 2 / 0
T2	0 / 2 / 3	0 / 5 / 0	1 / 3 / 0
T3	0 / 1 / 2	0 / 0 / 3	4 / 0 / 0
Total	0 / 12 / 6	2 / 10 / 4	10 / 7 / 0

Appendix

TABLE II: Physical outcome counts. Entries are full / partial / failure; their sum is the method’s task denominator. Assisted runs remain included.

Fig. 8: Offline video results. T2 partial averages five normalized accuracies with equal weights; segment labels give raw counts in legend order. Full T2 requires correct sequence, first object, destination, and command on the same clip (/12). T3 scores hidden-target choice (/30). No motor policy is executed.

Long-Term Memory for Embodied Agents

See all 26 papers

MemBodied: Recurrent Associative Memory for Vision-Language-Action Models
Sep 23, 2026Tej Deep Pala, Navonil Majumder, Bryce Goh +4Robotic Manipulation Vision-Language-Action Models
Nanyang Technological University · Griffin Labs · École Centrale de Lyon
T $^2$ Mem: Learning Test-Time Memory for Robotics
Sep 29, 2026Yize Liu, Huang Huang, Yining Hong +4Robot Policy Learning Test-Time Adaptation of VLA Models
Stanford University · NVIDIA · University of Michigan, Ann Arbor
MEMORA: Embodied Action Memory from Egocentric Videos for Reasoning and Planning
Jul 15, 2026Zihao Yu, Xiu Yuan, Chongjie ZhangLong-Term Memory for Embodied Agents Embodied AI
Washington University in St. Louis

Sep 23, 2026J/K move · Enter open · S save

MemBodied: Recurrent Associative Memory for Vision-Language-Action Models

Tej Deep Pala, Navonil Majumder, Bryce Goh, Raphael Yee+3

Nanyang Technological University · Griffin Labs · École Centrale de Lyon

Robotic Manipulation Vision-Language-Action Models VLMs for Robotics Long-Term Memory for Embodied Agents

Open paper AskPDF arXiv

Vision-Language-Action models provide a strong foundation for general-purpose robot control, yet a vast majority of policies do not preserve and leverage episode-level information beyond the current observation. This limitation is consequential in history-dependent manipulation tasks that depend on information available only in past observations. Retaining past observations in context can aid in recovering this information, but at the significant cost of ever-growing, bloated context and inference latency. We thus introduce MemBodied, a fixed-size episodic memory with two complementary components: an associative state that records interactions across policy calls and an episode anchor that preserves a compact representation of the initial scene as a reference. At each policy call, the model conditions action generation on the current input and the memory components, rather than directly using past observations. Across five evaluated RMBench tasks requiring memory, MemBodied achieves

7.81\times

the mean success rate of a stateless policy and

2.98\times

of vanilla recurrent memory, while outperforming the strongest memory-augmented baseline by

1.3\times

with

10\times

fewer added parameters. On the fully observable LIBERO-Long suite, it reached 90.6%, a 5.4% improvement over the stateless

π_0

policy. These findings support MemBodied as a practical alternative to expanding the policy context for history-dependent manipulation.

Abstract

Long-Term Memory for Embodied Agents

Explore similar work

MemBodied: Recurrent Associative Memory for Vision-Language-Action Models

T2^22Mem: Learning Test-Time Memory for Robotics

MEMORA: Embodied Action Memory from Egocentric Videos for Reasoning and Planning

T $^2$ Mem: Learning Test-Time Memory for Robotics