Robots often observe information that determines a future action long before that action is executed. In a shell game, for example, a robot first sees which cup hides the ball, watches the cups move, and only later needs to choose the correct cup. The final observation alone is not enough for a decision: the correct action depends on an earlier event. We refer to this temporal gap as observation-action delay. It makes memory a policy-facing problem: a policy must keep similar histories distinct, retrieve the past event relevant to the current decision, and convert that recall into an action-ready state. We call these requirements separability, addressability, and prospectiveness. We introduce Chameleon, a ~60M visuomotor policy for control-indexed prospective memory. Chameleon writes embodied event memory, preserves separable histories, retrieves control-relevant traces, and trains the resulting working state to be prospective. We also introduce Camo-Dataset, a real-robot benchmark that isolates observation-action delay by making the decision scene visually ambiguous, so the correct action must be inferred from earlier observations. Chameleon improves decision/end-to-end success on Camo-Dataset from 22.5%/21.3% to 80.8%/71.3%. On public long-horizon memory benchmarks, it achieves 87.1% +/- 0.8% on LIBERO-10, 97.3% +/- 4.5% on MemoryBench, and 75.1% +/- 1.4% on MIKASA-Robo, setting the state of the art for same-size models and exceeding multiple larger VLA baselines under the reported protocols. Probes and ablations show that Chameleon learns separable, addressable, and prospective memory, and that these properties drive its performance gains.
Figures & tables
Figure 1: Observation–action delay requires control-indexed memory. Current: Semantic summaries or similarity-based visual retrieval can select a plausible but wrong trace. Ours & Human: Effective memory requires three capabilities: separability, addressability, and prospectiveness.
Figure 2: Chameleon overview. Chameleon writes embodied event tokens, propagates them as token-grounded traces, addresses them with a learned control index, and consolidates the recalled trace into the prospective policy state ht . The state conditions a rectified-flow action policy, while Control-JEPA trains it to predict future control context.
Figure 3: Evaluation benchmarks. Left: public long-horizon and memory benchmarks. Right: Camo-Dataset real-robot tasks. The frame marked ∗ denotes the non-Markovian decision stage, where the correct action depends on the earlier episode variable z rather than the current observation alone.
Method
Clean a specified plate
Play shell game
Add various seasonings
Avg. DSR
Avg. MSR
Avg. SR
DSR/SR
DSR/SR
DSR/SR
Matched imitation baselines
Diffusion Policy Chi et al. (2023)
33.3/30.6
34.3/33.3
0.0/0.0
22.5
67.6
21.3
ACT Zhao et al. (2023)
28.0/19.4
35.5/30.6
0.0/0.0
21.2
51.8
16.7
Flow Matching Lipman et al. (2022) ; Liu et al. (2022)
30.0/25.0
25.7/25.0
0.0/0.0
18.6
62.0
16.7
Ours and mechanism ablations
Table 4
Benchmark
Protocol
Chameleon
Published context
MemoryBench Fang et al. (2025)
3 task-specific policies
97.3 ± 4.5
RVT-2 54.0 Goyal et al. (2024) ; Fang et al. (2025) ; SAM2Act 55.0 Fang et al. (2025) ; SAM2Act+ 94.3 Fang et al. (2025) ; ReMem-VLA 94.5 ∗ Li et al. (2026) .
LIBERO-10 Liu et al. (2023)
10-task mixed policy
87.1 ± 0.8
DP-T 51.0 Chi et al. (2023) ; Reuss et al. (2025) ; QueST 69.0 Reuss et al. (2025) ; DP-CNN 73.0 Chi et al. (2023) ; Reuss et al. (2025) ; MoDE 92.0 Reuss et al. (2025) ; OpenVLA 53.7 Kim et al. (2024) ; Shi et al. (2026) ; CoT-VLA 69.0 Zhao et al. (2025) ; Shi et al. (2026) ; TriVLA 73.2 Liu et al. (2025) ; Shi et al. (2026) ; π0 85.2 Black et al. (2024) ; Shi et al. (2026) ; 4D-VLA 86.5 Zhang et al. (2026) ; Shi et al. (2026) ; MemoryVLA 93.4 Shi et al. (2026) .
MIKASA-Robo Cherepanov et al. (2026)
5-task mixed policy
75.1 ± 1.4
CronusVLA 18.0 Li et al. (2025a) ; Shi et al. (2026) ; SpatialVLA 21.0 Qu et al. (2025) ; Shi et al. (2026) ; OpenVLA-OFT 28.4 Kim et al. (2025) ; Shi et al. (2026) ; π0 29.4 Black et al. (2024) ; Shi et al. (2026) ; MemoryVLA 41.2 Shi et al. (2026) ; GMP 67.8 Gao et al. (2026) .
MIKASA-Robo Cherepanov et al. (2026)
2 task-specific policies
95.6 ± 1.0
DP 19.5 Chi et al. (2023) ; Lei et al. (2026) ; DP-PTP 15.0 Lei et al. (2026) ; MaIL 19.5 Jia et al. (2024) ; Lei et al. (2026) ; DP-VPWEM 86.5 Lei et al. (2026) .
Table 2: Public long-horizon and memory benchmarks. Success rate (%). All entries follow the protocol of the cited source. Benchmark details and per-task results are provided in the Appendix.
Figure 4: Mechanistic probes on Camo-Dataset. (a) Separability : hidden-variable decoding from current tokens and memory states. (b) Addressability : counterfactual trace edits under a fixed control query. (c) Prospectiveness : future-control decoding from ht with and without Control-JEPA.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Symbol
Shape
Description
d
scalar
token / hidden dimension
V
scalar
number of camera views
Nv
scalar
patch tokens from view v
N
scalar
total event tokens, N=∑vNv+2
L
scalar
number of memory layers
H
scalar
action prediction horizon
Appendix
Table S1: Notation and tensor shapes. Shapes are per timestep t and per layer ℓ unless noted. B denotes batch size and is omitted from the shape column.
Benchmark
#Tasks
Policy
#Demos/task
#Rollouts/task
#Seeds
Protocol followed
LIBERO-10
10
mixed
50
50
3
LIBERO Liu et al. [2023]
MemoryBench
3
task-specific
100
25
4
SAM2Act Fang et al. [2025]
MIKASA-Robo (mixed)
5
mixed
250
30
3
MIKASA-Robo Cherepanov et al. [2026]
MIKASA-Robo (special.)
2
task-specific
250
30
3
DP-VPWEM Lei et al. [2026]
Appendix
Table S2: Simulation benchmark protocols. Configuration used for each public benchmark. “Policy” indicates whether a single mixed-task policy or one policy per task is trained. “Protocol” names the source whose evaluation setup we match.
Figure S1: Real-robot setup for Camo-Dataset. UR5 workspace with the 2 camera viewpoints annotated.
Task
Hidden episode variable
Aliased decision point
Diagnostic role
pe
Clean a specified plate
Which visually similar plate was used or contaminated earlier.
Candidate plates appear interchangeable when the robot selects the target.
Event-object binding under delayed relevance.
1/3
Play shell game
The hidden object’s location after occlusion and swaps.
Cups are visually identical after rearrangement.
Spatial tracking under occlusion and distractor motion.
1/3
Add various seasonings
Which subgoals have already been completed in a repeated sequence.
The workspace returns to a similar state after each seasoning action.
Sequential progress memory and prevention of repetition or omission.
1/27
Appendix
Table S3: Task design in Camo-Dataset. The dataset contains three real-robot tasks that instantiate delayed control relevance through complementary hidden variables. pe is the chance decision rate used for chance-adjusted DSR. For Add various seasonings, DSR is scored at the complete sequence level.
Component
#Params
Trainable?
Visual encoder (DP-style patch encoder)
22.9M
yes
Text encoder (DistilBERT)
66.4M
frozen
Memory layers ( L control-indexed cells)
20.3M
yes
– SSM slow / SSM fast
5.1M
yes
– Mixer / control index / attention
15.2M
yes
Rectified-flow action head
23.1M
yes
Appendix
Table S4: Parameter budget of Chameleon. Breakdown by component. The text encoder is frozen during training. “Trainable” excludes it. The “ ∼ 60M” claim in the main text refers to the trainable-parameter budget, not the total parameter count.
Model
Approx. scale
Model type
Chameleon (ours)
100–500M
policy model
Diffusion Policy Chi et al. [2023]
100–500M
policy model
ACT Zhao et al. [2023]
10–100M
policy model
Flow Matching Lipman et al. [2022] , Liu et al. [2022]
100–500M
policy model
DP-T / DP-CNN Chi et al. [2023] , Reuss et al. [2025]
100–500M
policy model
QueST Reuss et al. [2025]
100–500M
policy model
Appendix
Table S5: Reported model scale. We report approximate total/backbone model scale, not benchmark-specific trainable parameters. Ranges indicate commonly reported configurations when exact counts are implementation-dependent.
Shared model and training hyperparameters
Feature dimension d
512
Memory layers L
2
Mixer attention heads
8
SSM slow state dim
128
SSM fast state dim
32
RF head depth / heads
6 / 8
Appendix
Table S6: Hyperparameters of Chameleon. Top block: model settings shared across benchmarks; optimization rows that vary are reported in the order LIBERO-10 / MemoryBench / MIKASA / Camo. Bottom block: input/output configuration. α and K are fixed across all experiments.
Task
Chameleon (ours)
DP-family baseline
VLA baseline
MemoryBench Fang et al. [2025] – SOTA: 2 strongest baselines (SAM2Act+ Fang et al. [2025] , ReMem-VLA Li et al. [2026] )
Reopen Drawer
92.0 ± 13.5
84.0 ± 0.0
100.0
Put Block Back
100.0 ± 0.0
100.0 ± 0.0
93.0
Rearrange Block
100.0 ± 0.0
99.0 ± 2.0
99.0
Average
97.3 ± 4.5
94.3
94.5
MIKASA-Robo Cherepanov et al. [2026] (mixed, 5-task) – SOTA: 2 strongest baselines (GMP Gao et al. [2026] , MemoryVLA Shi et al. [2026] )
Appendix
Table S7: Per-task results on public benchmarks. Success rate (%). Each block uses the baselines selected per the convention in the text. Bold = best in block.
Method
Clean plate
Shell game
Seasoning
Avg. DSR / SR
Ablated property
DSR/SR
DSR/SR
DSR/SR
Chameleon (full)
91.2/86.1
86.1/86.1
65.2/41.7
80.8 / 71.3
—
w/o memory
26.7/22.2
34.4/30.6
0.0/0.0
20.4 / 17.6
separability (all memory)
similarity retrieval bank
41.4/33.3
28.6/22.2
0.0/0.0
23.3 / 18.5
addressability (vs. control-indexed)
Vanilla Mamba memory
27.6/22.2
30.0/25.0
50.0/19.4
35.9 / 22.2
separability (token-grounded traces)
w/o control index
40.7/30.6
45.8/30.6
60.0/16.7
48.8 / 26.0
addressability
Appendix
Table S8: Ablation study on Camo-Dataset. Avg. columns aggregate over tasks. The “Ablated property” column names the functional capability each variant removes.
Task
Chance
Current Zt0
Memory htL
Play shell game
33.3
46.7
83.3
Add various seasonings
3.7
37.4
98.5
Appendix
Table S9: Separability probe accuracy (%). Decoding the hidden variable z from current tokens vs. memory state at td . Chance is the per-task pe .
Full
Swap
Mask+
Mask-
Choice accuracy
93
87
40
90
Appendix
Table S10: Addressability counterfactual choice accuracy (%). The control query utℓ is fixed while the trace bank Etℓ is edited in Add various seasonings.
Vision-Language-Action models provide a strong foundation for general-purpose robot control, yet a vast majority of policies do not preserve and leverage episode-level information beyond the current observation. This limitation is consequential in history-dependent manipulation tasks that depend on information available only in past observations. Retaining past observations in context can aid in recovering this information, but at the significant cost of ever-growing, bloated context and inference latency. We thus introduce MemBodied, a fixed-size episodic memory with two complementary components: an associative state that records interactions across policy calls and an episode anchor that preserves a compact representation of the initial scene as a reference. At each policy call, the model conditions action generation on the current input and the memory components, rather than directly using past observations. Across five evaluated RMBench tasks requiring memory, MemBodied achieves 7.81× the mean success rate of a stateless policy and 2.98× of vanilla recurrent memory, while outperforming the strongest memory-augmented baseline by 1.3× with 10× fewer added parameters. On the fully observable LIBERO-Long suite, it reached 90.6%, a 5.4% improvement over the stateless π0 policy. These findings support MemBodied as a practical alternative to expanding the policy context for history-dependent manipulation.
Tej Deep Pala, Navonil Majumder, Bryce Goh +4
Nanyang Technological University · Griffin Labs · École Centrale de Lyon
A robot may lose sight of an object it must later retrieve, need to recall what a person demonstrated earlier, or track which steps of a task it has already completed. Current vision-language-action (VLA) policies often fail once the information needed for action disappears from the current observation, making memory critical for long-horizon robot behavior. Existing approaches typically provide longer histories or learn implicit memory from observation-action trajectories. But action supervision tells a policy how to act, not what to remember: it does not specify which past facts should persist or how they should change as new evidence arrives. We therefore separate maintaining an evidence-grounded account of the past from learning how to act on it. This insight motivates Explicit Concept Memory (ECoMEM), which represents task-relevant history with a reusable library of grounded concepts. An evidence-based Writer selects and updates these records, while a learned Reader turns them into memory tokens that directly condition the VLA. Across 16 RoboMME tasks, ECoMEM leads the evaluated robot policies on 15 tasks. On two new real-robot tasks, the same memory library either transfers directly or requires only one new concept, achieving 86.1% success versus 8.6% for a no-memory VLA. These results show that explicit concepts provide a reusable and extensible memory interface for robot control. Project website: https://ecomem.github.io/
Conventionally, memory in end-to-end robotic learning involves inputting a sequence of past observations into the learned policy. However, in complex multi-stage real-world tasks, the robot's memory must represent past events at multiple levels of granularity: from long-term memory that captures abstracted semantic concepts (e.g., a robot cooking dinner should remember which stages of the recipe are already done) to short-term memory that captures recent events and compensates for occlusions (e.g., a robot remembering the object it wants to pick up once its arm occludes it). In this work, our main insight is that an effective memory architecture for long-horizon robotic control should combine multiple modalities to capture these different levels of abstraction. We introduce Multi-Scale Embodied Memory (MEM), an approach for mixed-modal long-horizon memory in robot policies. MEM combines video-based short-horizon memory, compressed via a video encoder, with text-based long-horizon memory. Together, they enable robot policies to perform tasks that span up to fifteen minutes, like cleaning up a kitchen, or preparing a grilled cheese sandwich. Additionally, we find that memory enables MEM policies to intelligently adapt manipulation strategies in-context.
Marcel Torne, Karl Pertsch, Homer Walke +14
Physical Intelligence · Stanford University · UC Berkeley +1