It is widely appreciated that robotic policies require long-term memory to perform complex tasks. However, directly conditioning policies on all past observations leaves policy learning susceptible to learning spurious shortcuts, and introduces large latency at policy deployment. As such, today's memory-augmented policies curate what to remember, either by retaining only recent frames, selected keyframes, or textual summaries, or by learning latent representations of the visual stream alone. In this work, we propose an alternative approach IMPRINT, a policy-agnostic latent memory framework that augments vision-language-action (VLA) models with multi-stream memory without explicitly constraining what the policy should remember. IMPRINT compresses different streams into a latent memory that is updated incrementally at inference, enabling low latency. To avoid shortcut learning, IMPRINT adopts a two-stage training procedure, first supervising memory to reconstruct multiple streams (observations, actions, proprioception) of past information, as well as estimate current task progress, followed by end-to-end finetuning, jointly with the policy, on action prediction. Across both simulation and real-world tasks demanding memory, IMPRINT improves average success over the base VLA by at least 26%, outperforming popular alternatives, and maintaining lowest inference latency among memory-augmented VLAs. Through careful ablations, we demonstrate the utility of incorporating diverse information streams into robotic memory, validating the advantages of foregoing history curation. Project website: https://yilin-wu98.github.io/imprint_website/.
Figures & tables
Figure 2: Real World Tasks & Result Highlight
Figure 3: Average Task Success on Rmbench
Figure 4: Average Task Performance For Memory and Non-Memory Tasks in Robocasa
Figure 5: Real World Task Performance Across 3 Tasks
Figure 6: Runtime vs Success on RMBench
Figure 7: Ablations on Memory Capacity. We evaluate Imprint trained with different memory tokens against task performance (left) and supervision loss evaluated on the validation set (right).
Figure 8: Ablations of Memory Learning. We ablate different input streams (right) and varied supervision objectives (middle) and training recipes (left) of Imprint on RMBench.
Figure 9: Analysis of Memory Bottleneck. We compare E2E, frozen memory encoder, and our adapted memory with t-SNE embeddings, linear probes, and attention visualization on Cover Block .
Figure 10: Imprint Improves Different VLA Backbones with Memory.
Figure 11: Architecture of Memory Adapter for VLAs.
Setting
Value
Memory / action-expert width
1024
Query heads
4
Key-value heads
1 (grouped-query attention)
Head dimension
256
Positional encoding
Separate rotary positions for memory and action tokens
FiLM projection initialization
N(0,0.0022) , zero bias
Appendix
Table 2: Memory adapter hyperparameters for π0 and π0.5 .
Setting
Stage 1: memory pretraining
Stage 2: joint adaptation
Updates
160,000
100,000
Global batch size
64
64
GPUs
4
4
Optimizer
AdamW
AdamW
Peak learning rate
1.0×10−4
2.5×10−5
Final learning rate
1.0×10−5
2.5×10−6
Appendix
Table 3: Optimization settings for IMPRINT on RMBench.
Method
Category
Backbone
RMBench result
non-memory VLA
None
π0 , π0.5 , MolmoAct2
Trained by us
Uniform sampling
Heuristic
π0.5
Trained by us
EventVLA (visual anchors only) †
Heuristic
QwenOFT (Qwen3-VL-4B)
Yang et al. [2026a]
MemER
Hierarchical
VLM planner + VLA policy
Yang et al. [2026a]
Mem-0
Hierarchical
Qwen3-VL-8B planner + DiT policy
Chen et al. [2026b]
HAMLET
Latent
GR00T N1.6
Trained by us
Appendix
Table 4: Compared policies, their backbones, and the source of their RMBench results.
Heuristic
Hierarchical
Latent
Uniform sampling
EventVLA †
MemER
Mem-0
HAMLET
MemoryVLA
IMPRINT
What memory stores
Visual history
✓
✓
✓
✓
✓
✓
✓
Action history
✗
✗
✗
✗
✗
✗
✓
Proprio. history
✗
✗
✗
✗
✗
✗
✓
Memory form
Frames
Frames
Frames, text
Frames, text
Latent
Latent
Latent
Appendix
Table 5: Comparison of memory designs. ✓ : yes; ∼ : partially; ✗ : no. † On RMBench, EventVLA is evaluated only in its visual-anchor configuration, which keeps the initial frame and a few recent frames and omits its keyframe evidence memory [ Yang et al., 2026a ] . The reported result therefore reflects this heuristic variant rather than the full EventVLA, and the column describes this evaluated variant.
Study
Variant
Stage 1 init.
Memory updated
Change from default
Reported in
Encoding
Memory capacity
yes
yes
B∈{1,4,16,64,256,1024} (full-attention encoder)
Fig. 7
Full attention
yes
yes
Perceiver attention instead of recurrence
Fig. 6 , Sec. 5.3
Recipe
End-to-end
no
yes
Random encoder, action loss only
Fig. 8
Frozen memory
yes
no
Encoder fixed during Stage 2
Fig. 8
IMPRINT
yes
yes
None (default)
Fig. 8
Objectives
Without progress
yes
yes
λρ=0
Fig. 8
Appendix
Table 6: Configurations used in the ablation study. Stage 1 init. : whether the memory encoder starts from Stage 1 pretraining. Memory updated : whether the encoder is optimized during Stage 2. Unlisted settings match the default IMPRINT configuration.
Task
Type
Task and memory requirement
Limit
Observe and pickup
M(1)
Pick the object matching a target shown before occlusion.
250
Rearrange blocks
M(1)
Move the center block to the empty mat, press, and move the initially mat-placed block to the center.
700
Put back block
M(1)
Move the block to the center, press, and return it to its original mat.
500
Swap blocks
M(1)
Use the spare tray to exchange the blocks’ initial tray assignments, then press once.
1000
Swap T
M(1)
Exchange the initial positions and orientations of two T-shaped blocks.
600
Battery
M( n )
Use gauge feedback from tested orientation pairs to place both batteries correctly.
1000
Appendix
Table 7: RMBench tasks and episode limits, measured in simulator steps.
Task
Instruction and memory requirement
Horizon
L/S
Variation across configurations
Memory
Retrieve fruit
Place the {fruit} in the sink; recall which side of the sink it was observed on.
1600
1/1
9 fruit categories; target at the far left or right of the sink.
Retrieve oil
Pick up the olive-oil bottle; recall which of two bottles is the target after both sides are shown.
1600
1/1
Four initial/target-side combinations (LL, LR, RL, RR).
Wash and return
Wash the fruit and return it to its original container; recall the source container.
1600
1/1
Left or right source container; fruit, receptacle, and placements vary.
Wash and return to same location
Wash the fruit and return it to its exact initial location; recall that location.
1600
1/1
Initial fruit location varies.
Sweeten hot chocolate
Place {1, 2, 3} sugar cubes in the saucepan; track how many have been placed.
2100
15/17
Target counts 1/2/3 in 20/20/10 configurations; 3–5 cubes per scene, including 0–3 extras.
Appendix
Table 8: Tasks in the customized RoboCasa benchmark. Horizon : maximum number of policy-controlled simulator steps. L/S : number of distinct layouts and styles among the 50 evaluation configurations. Instructions are abbreviated.
Figure 12: Real World Tasks with Three Camera Views
Figure 13: Real World Fold Cloth Task Configuration
Figure 14: OOD Scenarios for Each Task in the Real World
M(1)
M(n)
Method
Observe / pick
Rearrange blocks
Put back block
Swap blocks
Swap T
Battery
Block ranking
Cover blocks
Press button
Average
π0.5 (no memory)
8
63
20
16
8
30
66
15
3
25.4
Uniform sampling
5
79
100
62
30
28
82
13
3
44.7
EventVLA †
21
96
95
96
87
35
81
97
3
67.8
MemER †
7
17
0
14
7
27
0
6
0
8.7
Mem-0 †
4
89
90
67
14
28
18
68
0
42.0
Appendix
Table 10: Per-task success rates on the nine RMBench memory tasks, grouped into single-event, M(1), and multi-event, M(n), tasks. The final column is the task-balanced average reported in Fig. 3 . The best result for each task and for the average is shown in bold. † Results reported by prior work (App. A.2 ).
Task
Type
π0.5
Heuristic action
Heuristic VLM
IMPRINT full
IMPRINT recurrent
Retrieve fruit
Memory
42
88
60
84
90
Retrieve oil
Memory
48
98
50
76
80
Wash and return
Memory
46
58
46
86
96
Wash and return to same location
Memory
46
36
44
62
58
Rinse cutting board
Memory
4
8
0
0
4
Sweeten hot chocolate
Memory
46
36
34
58
64
Appendix
Table 11: Per-task success rates on the customized RoboCasa benchmark (Fig. 4 ). Heuristic action supplies uniformly sampled historical frames to the action expert, whereas heuristic VLM supplies them as additional VLM-context images. The best result for each task and each average is shown in bold.
Method
Dust Table
Scan Bottle
Fold Cloth
Average
π0
6/20 (30%)
5/20 (25%)
7/32 (22%)
25.6%
HAMLET
5/20 (25%)
5/20 (25%)
6/32 (19%)
22.9%
MemER
8/20 (40%)
7/20 (35%)
12/32 (38%)
37.5%
Uniform sampling
10/20 (50%)
10/20 (50%)
15/32 (47%)
49.0%
IMPRINT
16/20 (80%)
15/20 (75%)
23/32 (72%)
75.6%
IMPRINT (OOD)
9/20 (45%)
14/20 (70%)
10/32 (31%)
48.8%
Appendix
Table 12: Real-world results (Fig. 5 ). Each cell reports successful rollouts / total rollouts, followed by the success rate in parentheses. The average is the mean of the per-task success rates. The best in-distribution result for each task and the average is shown in bold. OOD denotes the held-out object variations.
B
Total
Vision
Action
State
Progress
×10−2
×10−2
×10−4
×10−4
×10−4
1
3.649±0.295
3.599
2.37
2.43
5.13
4
3.308±0.273
3.278
1.31
1.35
3.31
16
3.406±0.277
3.368
1.50
1.55
4.41
64
3.389±0.282
3.359
1.50
1.52
2.94
256
3.338±0.280
3.311
1.27
1.31
2.81
Appendix
Table 13: Stage-1 losses across memory capacities. Total loss is mean ± SEM across tasks; component losses are unweighted. The second header row indicates scaling.
Task
Type
B=1
B=4
B=16
B=64
B=256
B=1024
RMBench
Observe / pick
Memory
7
8
8
8
9
10
Rearrange blocks
Memory
100
100
100
100
100
99
Put back block
Memory
100
100
100
100
100
100
Swap blocks
Memory
97
97
99
99
100
93
Swap T
Memory
99
99
99
99
100
99
Appendix
Table 14: Per-task success as the capacity of the full-attention memory encoder is varied (Fig. 7 ). All other architecture, training, and evaluation settings are fixed. B=1 and B=1024 are evaluated on RMBench only (–). The best result in each row is shown in bold; rows where all settings tie are left unmarked.
Training recipe
Stage 1 objective
Task
E2E
IMPRINT frozen
IMPRINT
IMPRINT
Without progress
Visual only
Action + state
Observe / pick
12
7
9
9
6
6
5
Rearrange blocks
90
99
100
100
91
78
100
Put back block
100
100
100
100
100
100
100
Swap blocks
90
99
100
100
90
89
89
Swap T
45
99
100
100
97
99
98
Appendix
Table 15: Per-task success for the training-recipe and Stage 1 objective ablations on RMBench (Fig. 8 ). The best result in each row is shown in bold separately within each ablation; rows where all variants tie are left unmarked.
π0.5
π0
MolmoAct2
Task
No memory
IMPRINT
No memory
IMPRINT
No memory
IMPRINT
Observe / pick
8
9
8
6
4
4
Rearrange blocks
63
100
13
99
52
100
Put back block
20
100
49
100
0
100
Swap blocks
16
100
4
100
11
65
Swap T
8
100
0
98
15
100
Appendix
Table 16: Per-task success with and without IMPRINT across three VLA backbones on RMBench (Fig. 10 ). For each backbone, the better result of the two variants is shown in bold; ties are left unmarked.
Memory training
Timestep accuracy
Balanced accuracy
Episode accuracy
Task success
E2E
19.94
17.29
42.00
15.00
Frozen
83.31
83.68
90.00
35.00
IMPRINT
91.53
85.95
98.00
100.00
Appendix
Table 17: Cover Blocks color-order probes and closed-loop task success (%). Probes are evaluated on held-out episode folds drawn from the memory model’s training demonstrations.
Figure 15: Inference Time Analysis For Different Bottleneck Size and Different Methods
Full attention
Recurrent
B
1 frame
1,451 frames
1 frame
1,451 frames
4
109.0
144.1
114.1
124.4
16
107.4
143.4
115.3
125.4
64
109.7
143.7
116.2
128.3
256
111.9
148.2
116.8
127.1
Appendix
Table 18: Policy-call latency in milliseconds at the shortest and longest history. All variants of our method predict 50 actions.
Historical inputs
Obs./pick
Rearrange
Put back
Swap blocks
Swap T
Battery
Ranking
Cover
Press
Avg.
Vision only
9
93
100
81
45
30
80
100
7
60.6
Vision + Action
6
100
100
100
99
38
72
83
18
68.4
Vision + State
9
94
100
98
99
28
46
70
4
60.9
Vision + Action + State (Imprint)
9
100
100
100
100
41
83
100
21
72.7
Appendix
Table 20: Historical input-stream ablation on RMBench with the full-attention encoder at B=256 . Each variant removes the listed streams from both the memory input and the Stage-1 reconstruction targets. The best result in each column is shown in bold; columns where all variants tie are left unmarked.
Figure 16: RMBench: Battery Try. Representative rollouts of Imprint , HAMLET, and Uniform Sampling.
Figure 17: RMBench: Battery Try. Representative rollouts of Imprint and the Frozen and End-to-End training variants.
Figure 18: RMBench: Block Ranking Try. Representative rollouts of Imprint , HAMLET, and Uniform Sampling.
Figure 19: RMBench: Block Ranking Try. Representative rollouts of Imprint and the Frozen and End-to-End training variants.
Figure 20: RMBench: Swap Blocks. Representative rollouts of Imprint , HAMLET, and Uniform Sampling.
Figure 21: RMBench: Swap Blocks. Representative rollouts of Imprint and the Frozen and End-to-End training variants.
Figure 22: RMBench: Swap T. Representative rollouts of Imprint , HAMLET, and Uniform Sampling.
Figure 23: RMBench: Swap T. Representative rollouts of Imprint and the Frozen and End-to-End training variants.
Figure 24: RMBench: Put Back Block. Representative rollouts of Imprint , HAMLET, and Uniform Sampling.
Figure 25: RMBench: Put Back Block. Representative rollouts of Imprint and the Frozen and End-to-End training variants.
Figure 26: RMBench: Cover Blocks. Representative rollouts of Imprint , HAMLET, and Uniform Sampling.
Figure 27: RMBench: Cover Blocks. Representative rollouts of Imprint and the Frozen and End-to-End training variants.
Figure 28: RMBench: Press Button. Representative rollouts of Imprint , HAMLET, and Uniform Sampling.
Figure 29: RMBench: Press Button. Representative rollouts of Imprint and the Frozen and End-to-End training variants.
Figure 30: RMBench: Observe and Pick Up. Representative rollouts of Imprint , HAMLET, and Uniform Sampling.
Figure 31: RMBench: Observe and Pick Up. Representative rollouts of Imprint and the Frozen and End-to-End training variants.
Figure 32: RMBench: Rearrange Blocks. Representative rollouts of Imprint , HAMLET, and Uniform Sampling.
Figure 33: RMBench: Rearrange Blocks. Representative rollouts of Imprint and the Frozen and End-to-End training variants.
Figure 34: RoboCasa: Frying Pan Adjustment. Representative rollouts of Imprint , π0.5 , and Uniform Sampling with VLM-context or action-expert conditioning.
Figure 35: RoboCasa: Sweeten Hot Chocolate. Representative rollouts of Imprint , π0.5 , and Uniform Sampling with VLM-context or action-expert conditioning.
Figure 36: RoboCasa: Pan Transfer. Representative rollouts of Imprint , π0.5 , and Uniform Sampling with VLM-context or action-expert conditioning.
Figure 37: RoboCasa: Wash and Return to the Original Plate. Representative rollouts of Imprint , π0.5 , and Uniform Sampling with VLM-context or action-expert conditioning.
Figure 38: RoboCasa: Wash and Return to the Same Location. Representative rollouts of Imprint , π0.5 , and Uniform Sampling with VLM-context or action-expert conditioning.
Figure 39: RoboCasa: Rinse Cutting Board. Representative rollouts of Imprint , π0.5 , and Uniform Sampling with VLM-context or action-expert conditioning.
Figure 40: RoboCasa: Scrub Cutting Board. Representative rollouts of Imprint , π0.5 , and Uniform Sampling with VLM-context or action-expert conditioning.
Figure 41: RoboCasa: Organize Condiments. Representative rollouts of Imprint , π0.5 , and Uniform Sampling with VLM-context or action-expert conditioning.
Figure 42: RoboCasa: Pick and Place to Sink. Representative rollouts of Imprint , π0.5 , and Uniform Sampling with VLM-context or action-expert conditioning.
Figure 43: RoboCasa: Retrieve Fruit. Representative rollouts of Imprint , π0.5 , and Uniform Sampling with VLM-context or action-expert conditioning.
Figure 44: RoboCasa: Retrieve Oil. Representative rollouts of Imprint , π0.5 , and Uniform Sampling with VLM-context or action-expert conditioning.
Figure 45: RoboCasa: Place Equal Ice Cubes. Representative rollouts of Imprint , π0.5 , and Uniform Sampling with VLM-context or action-expert conditioning.
Figure 46: Real World: Dust Table. Representative rollouts of Imprint (in-distribution and OOD objects), HAMLET, MemER, Uniform Sampling, and π0 .
Figure 47: Real World: Scan Bottle. Representative rollouts of Imprint (in-distribution and OOD objects), HAMLET, MemER, Uniform Sampling, and π0 .
Figure 48: Real World: Fold Cloth. The top row shows the in-context demonstration provided to Imprint’s memory, followed by representative rollouts of Imprint (in-distribution and OOD towel), HAMLET, MemER, Uniform Sampling, and π0 .
Mainstream Vision-Language-Action (VLA) models predict actions primarily from the current observation under a Markovian assumption, thus struggling with long-horizon, temporally dependent tasks. Existing memory-augmented VLAs either expand the observation window or retrieve history from the memory bank as auxiliary policy-side context. However, they leave memory outside the native latent embedding space of VLA reasoning, preventing historical experience from being fluidly interleaved with multimodal reasoning and action formation. To this end, we introduce LaMem-VLA, a latent-memory-native framework that reconstructs historical experience into latent memory tokens and directly interweaves them with VLA reasoning. At its core, LaMem-VLA introduces four coordinated components: (i) a curator that organizes historical experience into two complementary short-term and long-term memory vaults; (ii) a seeker that queries both vaults using the multimodal cognition to retrieve context-relevant evidence; (iii) a condenser that reconstructs the retrieved evidence into compact short-term and long-term latent memory tokens; and (iv) a weaver that injects these memory tokens with the current observation and instruction into one continuous embedding sequence. By representing, retrieving, and consuming historical experience entirely in the same continuous latent space, LaMem-VLA enables memory to directly participate in VLA reasoning and guide action generation under a bounded context. Extensive experiments on SimplerEnv and LIBERO demonstrate the superiority of our LaMem-VLA.
Hongyu Qu, Jianzhe Gao, Xiaobin Hu +6
Nanjing University of Science and Technology · Zhejiang University · National University of Singapore
How can pretrained Vision-Language-Action (VLA) models retain long-horizon visual histories with high-frequency updates without sacrificing efficiency? Existing approaches rely on external memory management, which restrains either the memory horizon or the reactiveness of pretrained policies. To this end, we present NativeMEM, a VLA policy that features long-term and real-time updated memory. At its core is an efficient memory encoding scheme, Native Memory Compression, which repurposes the VLA's own vision encoder to compress each historical frame from each camera view into a single token. Appended to the input sequence, these memory tokens enable the pretrained VLA to attend over long-term history with negligible latency overhead, requiring neither an external planner nor a freshly initialized memory module. To align the memory tokens with the pretrained policy, we first develop a generic memory tokenizer under the supervision of a frozen VLA on memory-demanding data, and then unfreeze the VLA for task-specific fine-tuning. NativeMEM consistently outperforms prior methods, boosting success rates from 32.4% to 84.0% in simulation and up to 98.7% on real robots, while maintaining low inference latency and GPU memory usage. Notably, NativeMEM exhibits high data efficiency by achieving competitive results with prior arts using only 20% of the training data.
Ziye Wang, Modi Shi, Chaojun Ni +5
The University of Hong Kong · Beihang University · Peking University +2
Vision-language-action (VLA) models have driven rapid progress in robotic manipulation, demonstrating strong fine-grained control and promising performance on long-horizon tasks. However, many existing VLAs lack explicit access to interaction history, making them vulnerable to perceptual aliasing: similar current observations and robot states at different task stages may induce action ambiguity and lower success rate. Existing methods incorporate temporal or progress cues through feature conditioning, action-prior modification, or sampling guidance. However, methods that jointly fine-tune memory modules and the base VLA incur additional policy-training costs, motivating the separation of trainable history-conditioned steering from frozen base-policy refinement. We propose ActMem-VLA, a dual-expert handover architecture that augments a frozen, fine-tuned VLA with a memory plugin comprising a Mamba-based memory module and a lightweight PreAction Expert (PAE). Specifically, Mamba encodes executed-action history into memory that conditions PAE alongside current context. With these inputs, PAE steers task progression during early, high-noise denoising, then passes the partially denoised action to the frozen Action Expert (AE) to refine action details during the remaining low-noise steps. The fine-tuned base VLA remains frozen throughout training, while only the Mamba module and PAE are jointly optimized. On LIBERO-Mem, ActMem-VLA achieves 80.8% average success across all ten tasks, compared with 65.2% for π0.5 and 49.5% for MemoryVLA, while introducing only 3.45% additional parameters. Across four real-world tasks, it improves the average success rate over π0.5 by 28.8%.
Yaxin Zhao, Dianye Huang, Chenwei Wang +2
Harbin Institute of Technology, Harbin, China. · Medical Intelligence and Robotic Cognition (MIRoC) Lab, Department of Mechanical Engineering, The University of Hong Kong (HKU), Hong Kong SAR, China. · Department of Computing, The Hong Kong Polytechnic University, Hong Kong SAR, China