Diffusion and flow-matching policies have become a standard choice for visuomotor imitation learning, owing to their ability to model multimodal action distributions. In memory-dependent manipulation, the robot must act on information that is no longer visible. Existing methods feed the observation history to the policy as an additional condition, while action generation still starts from Gaussian noise. However, the policy must then pick out the relevant target from a long and redundant history, leading to spurious correlations and unreliable decisions as the memory load grows. Additionally, the initial noise carries no information about what the robot has seen, leaving generation without guidance toward the remembered target. To address these issues, we propose RecallFlow, a flow-policy framework that builds the source distribution from episodic memory. Recall-Relocalization relates the memory to the current view through a few recall slots, so that the source reflects where the remembered targets are now. Recall Anchors then turns each slot into a Gaussian component of the source, enabling generation to start close to the intended action instead of from noise. The transport network then completes the action from the current observation alone, keeping inference nearly as cheap as a memoryless policy. We further build a suite of memory-dependent tasks grouped by memory load and compare all policies under a matched protocol. Extensive experiments show that RecallFlow achieves a state-of-the-art average success rate of 80.4% on the public MIKASA-Robo memory benchmark, and outperforms the strongest baseline by 20.3 points on our suite and by 18.8 points on four real-robot tasks.
Figures & tables
Figure 1: Start from what you remember. In memory-dependent manipulation, the evidence that decides which action to take ( cue ) disappears before the robot acts ( now ). (a) Feeding the memory to the policy as a condition still starts generation from uninformed noise. (b) RecallFlow starts generation from what it remembers: the episodic memory is the source, and the transport network sees only the present.
Figure 2: Overview of RecallFlow. The present observation is encoded into tokens Zt . Recall–Relocalization lets Zt read the episodic memory bank and decodes K recall slots, each attending to the current view through a heatmap Hk . Recall Anchors turns each slot into a Gaussian anchor in action space; with a standard Gaussian, the anchors form the source p0 . A transport network conditioned only on Zt carries x0∼p0 to the action chunk, so memory reaches the action solely through the source.
Figure 3: Evaluation tasks in simulation. (a) RecallSuite consists of five memory-dependent tasks: the cue is seen only earlier in the episode, and the candidates look identical when the robot must decide. (b), (c) Tasks from MIKASA-Robo and RoboMimic.
Single item
Multiple items
Method
MarkedPlate
CountSwing
NoRepeat
NoRepeat-4
MarkedPair
Avg. ↑
Chance
20.0
11.7
22.2
9.4
16.7
16.0
Memoryless
DP [ 9 ]
17.3
10.0
1.3
0.0
1.3
6.0
FM
16.0
6.3
13.3
2.3
0.0
7.6
History-conditioned
Table 1: Comparison on RecallSuite. We report the success rate (%) averaged over three training seeds. RecallFlow is on par with long-history DP when one item must be remembered and outperforms all baselines when several must be.
Method
SGT
IM
RC3
RC5
RC9
Avg. ↑
Octo-small [ 43 , 8 ]
46
39
45
17
11
31.6
OpenVLA ‡ [ 25 , 8 ]
47
14
59
16
6
28.4
π0 [ 2 , 8 ]
33
42
35
22
15
29.4
SpatialVLA [ 46 , 8 ]
23
27
27
17
11
21.0
CronusVLA [ 28 , 50 ]
32
5
31
13
9
18.0
MemoryVLA [ 50 ]
88
24
44
30
20
41.2
Table 2: Comparison with published methods on MIKASA-Robo [ 8 ] . We report the success rate (%) on ShellGameTouch (SGT), InterceptMedium (IM), and RememberColor with 3, 5, and 9 colors (RC3–RC9). RecallFlow, averaged over three seeds, achieves the highest average ( 80.4±1.1 ). ∗ Available only on arXiv. ‡ OpenVLA-OFT in [ 50 ] .
Figure 4: Effect of memory load, tasks without memory, and inference cost. (a) On RecallSuite, RecallFlow matches long-history DP when one item must be remembered and stays ahead of all baselines when several must be. (b) On RoboMimic, where memory is not needed, adding a long history lowers DP, whereas RecallFlow stays on par with its memoryless counterpart FM. (c) RecallFlow reaches the highest success rate on RecallSuite at nearly the inference cost of a memoryless policy. † Reported in [ 15 ] .
Figure 5: Memory decides where generation starts. (a) At a decision of a NoRepeat episode, all thirty samples of RecallFlow head to the untouched container from the source on, whereas long-history DP starts from noise and commits only in its last generation steps. (b) Over 15 decisions, the source samples of RecallFlow are already at the target. (c) RecallFlow therefore keeps its success rate with a single generation step.
Figure 6: Real-robot experiments. (a) We deploy the policies on an xArm 7 with a wrist camera and a third-person camera. (b) We design four tasks that differ in what must be remembered. (c) RecallFlow achieves the highest success rate on all four tasks. (d) The three behaviors observed on No Repeat: repeating one cup, touching one cup twice, and touching each cup once.
Variant
MarkedPlate
NoRepeat
MarkedPair
RecallFlow
95.7
97.0
67.3
W/o memory
16.0 ↓ 79.7
13.3 ↓ 83.7
0.0 ↓ 67.3
As condition
94.7 ↓ 1.0
92.0 ↓ 5.0
51.0 ↓ 16.3
W/o relocalization
98.3 ↑ 2.6
84.7 ↓ 12.3
56.3 ↓ 11.0
Flow loss only
53.3 ↓ 42.4
98.0 ↑ 1.0
9.0 ↓ 58.3
Table 3: Ablation study on RecallSuite. We report the success rate (%), averaged over three seeds, and its change when one design choice of RecallFlow is altered.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Parameter
Value
Recall slots K
4
Memory width / capacity C
256 / 32 entries
Memory-attention / slot-decoder layers
2 / 2
σmin
0.05
KL weight β , relaxed WTA ε , λ
10−3 , 0.05, 1
Warm-up
2,000 steps
Appendix
Table S1 : Hyperparameters of RecallFlow, shared across benchmarks unless noted.
Task
What must be remembered
Candidates
Episode length
Chance (%)
MarkedPlate
which plate the marker touched
5 plates
80
20.0
CountSwing
how many swings were already made
stop or swing again
216
11.7
NoRepeat
which containers were already touched
3 containers
136
22.2
NoRepeat-4
which containers were already touched
4 containers
192
9.4
MarkedPair
which two containers the marker hovered over
4 containers
152
16.7
Appendix
Table S2 : Tasks of RecallSuite. We list what each task requires the robot to remember, its candidates, the episode length in control steps, and the success rate of a memoryless policy that executes perfectly but chooses uniformly at random at every decision point (chance).
RecallSuite
RoboMimic Square
MarkedPlate
NoRepeat
ph
mh
GMP [ 15 ]
100.0
92.0
96.0
80.0
Appendix
Table S3 : GMP trained by us. We train GMP in its released configuration without the memory gate, with the budget of the rows that we train in Tabs. 1 and S8 , and report the success rate (%) of one training seed.
Method
Implementation
History
Generation
DP [ 9 ]
code of [ 15 ] , no history
current step
DDIM, from N(0,I)
FM
ours, memory disabled
current and previous step
rectified flow, from N(0,I)
Medium-history DP
code of [ 15 ]
past 16 steps
DDIM, from N(0,I)
Long-history DP
code of [ 15 ]
past 120 steps, one frame per 8 steps
DDIM, from N(0,I)
GMP [ 15 ]
released configuration, no gate
whole episode, per-chunk cross-attention
DDIM, from N(0,I)
RecallFlow
ours
episodic memory, through the source only
rectified flow, from p0
Appendix
Table S4 : Baseline configurations. All rows that we train on a benchmark share demonstrations, observations, the number of training steps on each task, and evaluation episodes.
Task
Epochs
Reported
Best
Last
ShellGameTouch
15
7
100
100
InterceptMedium
51
13
85
82
RememberColor3
51
5
100
98
RememberColor5
51
7
81
72
RememberColor9
51
11
35
26
Appendix
Table S5 : Training length on MIKASA-Robo. We report the number of epochs trained, the epoch of the reported checkpoint, and the success rate (%) at that checkpoint and at the last one. Checkpoints are saved and evaluated every 2 epochs; ShellGameTouch uses chunk length 8.
Table S6 : Protocols behind Tab. 2 , as stated by each source (“n/s”: not stated).
Method
SGT
RC3
Avg. ↑
DP [ 9 , 27 ]
–
–
19.5
DP-PTP [ 57 , 27 ]
–
–
15.0
MaIL [ 22 , 27 ]
–
–
19.5
BPP [ 51 , 27 ]
94
21
57.5
MemER [ 53 , 27 ]
82
27
54.5
DP-VPWEM [ 27 ]
91
82
86.5
Appendix
Table S7 : Comparison on ShellGameTouch and RememberColor3 of MIKASA-Robo, the two tasks on which several methods are evaluated [ 27 ] . We report the success rate (%); most sources give only the average.
Square
Method
ph
mh
Avg. ↑
Memoryless
DP [ 9 ]
98.0
73.3
85.7
FM
97.0
75.3
86.2
History-conditioned
Long-history DP †
86
69
77.5
Appendix
Table S8 : Comparison on RoboMimic [ 39 ] behind Fig. 4 . We report the success rate (%) on Square with proficient (ph) and multi-human (mh) demonstrations, averaged over three seeds. † Reported in [ 15 ] .
Figure S1 : Longer training on Square (ph). RecallFlow reaches 93% at epoch 4, the budget of Tab. S8 , and its best success rate of 97% at epoch 15.
Method
Remember Color
No Repeat
Marked Plate
Shell Game
Avg.
DP [ 9 ]
9
0
12
7
29.2
FM
5
0
8
5
18.8
Long-history DP
9
14
22
16
63.5
RecallFlow
20
17
24
18
82.3
Appendix
Table S9 : Real-robot results behind Fig. 6 . We report the number of successful trials out of 24 per task and the success rate (%) over all 96 trials.
Method
MarkedPlate
CountSwing
NoRepeat
NoRepeat-4
MarkedPair
DP [ 9 ]
15 / 18 / 19
8 / 13 / 9
2 / 0 / 2
0 / 0 / 0
3 / 0 / 1
FM
17 / 15 / 16
6 / 6 / 7
6 / 18 / 16
1 / 2 / 4
0 / 0 / 0
Medium-history DP
100 / 100 / 100
14 / 13 / 4
30 / 30 / 35
1 / 1 / 3
4 / 6 / 4
Long-history DP
100 / 100 / 100
98 / 99 / 100
68 / 81 / 63
23 / 12 / 10
42 / 46 / 22
RecallFlow
95 / 94 / 98
98 / 98 / 99
98 / 98 / 95
67 / 64 / 63
63 / 71 / 68
Appendix
Table S10 : Per-seed results on RecallSuite. We report the success rate (%) of each of the three training seeds behind Tab. 1 .
Variant
CountSwing
RecallFlow
98.0
W/o memory
6.0
As condition
100.0
W/o relocalization
92.0
Flow loss only
97.0
Appendix
Table S11 : Ablation study on CountSwing . We report the success rate (%) of one training seed for the variants of Tab. 3 .
Variant
MarkedPlate
NoRepeat
MarkedPair
RecallFlow
95 / 94 / 98
98 / 98 / 95
63 / 71 / 68
W/o memory
17 / 15 / 16
6 / 18 / 16
0 / 0 / 0
As condition
91 / 98 / 95
93 / 90 / 93
71 / 67 / 15
W/o relocalization
98 / 97 / 100
82 / 88 / 84
54 / 61 / 54
Flow loss only
38 / 68 / 54
97 / 99 / 98
7 / 12 / 8
Appendix
Table S12 : Per-seed ablations. We report the success rate (%) of each of the three training seeds behind Tab. 3 .
Steps
1
2
4
8
MarkedPlate
97.0
97.0
95.0
95.0
CountSwing
98.0
100.0
99.0
99.0
NoRepeat
95.0
96.0
96.0
98.0
Appendix
Table S13 : Fewer integration steps. We evaluate the seed-0 checkpoints of RecallFlow with 1, 2, 4, and 8 Euler steps (the default is 8) and report the success rate (%). The 8-step entry of CountSwing differs from its seed-0 entry in Tab. S10 by one episode, as this analysis was run on a different machine.
Method
Latency (ms) ↓
Memory (GB) ↓
DP [ 9 ]
148.1
1.10
FM
152.6
1.10
Medium-history DP
190.7 +38.1
1.60
Long-history DP
187.7 +35.1
1.60
GMP [ 15 ]
288.5 +135.9
1.46
RecallFlow
153.7 +1.1
1.14
Appendix
Table S14 : Inference cost behind Fig. 4 . We measure each policy on one RTX 4090 with batch size 1 and report the median latency per action chunk over 34 chunks, its increase over FM, and the peak GPU memory.
Visuomotor policies learn a direct map from raw sensory observations to robot action sequences. Policies based on Diffusion and Flow Matching capture the multimodal distribution over action sequences in an end-to-end manner. This expressivity comes at the cost of multi-step numerical integration of the learned vector field for action generation, which can be expensive and time-consuming, impeding fast control rates required in robotics applications. Furthermore, robot action sequences are usually defined on a smooth, differentiable manifold, requiring that the learned policy respects the intrinsic geometry of the robot's action space. Here, we present Riemannian MeanFlow Policy (RMFP), which learns the conditioned flow map of the probability path on the robot action manifold. Our formulation employs a flow map consistency objective grounded in the data by a Riemannian Conditional Flow Matching anchor. The flow map consistency condition is stable to train and constrains the learned model to finite-time transport, which yields on-manifold action sequence generation with as few as one network function evaluation. We present results on the spherical LASA and Push-T benchmarks, on the Tool Hang and Transport tasks of the Robomimic suite, and on the Franka Kitchen task with manifold-constrained action generation, and demonstrate that RMFP attains performance competitive with prior work at a lower sampling cost. We also employ RMFP on a real-world robotic manipulation task to demonstrate fast action generation under imperfect sensor measurements in the physical world.
S. Talha Bukhari, Austin Garrett, Yi Wei +3
Department of Computer Science, Purdue University, West Lafayette, IN 47907, USA
Imitation learning from human demonstrations has achieved significant success in robotic control, yet most visuomotor policies still condition on single-step observations or short-context histories, making them struggle with non-Markovian tasks that require long-term memory. Simply enlarging the context window incurs substantial computational and memory costs and encourages overfitting to spurious correlations, leading to catastrophic failures under distribution shift and violating real-time constraints in robotic systems. By contrast, humans can compress important past experiences into long-term memories and exploit them to solve tasks throughout their lifetime. In this paper, we propose VPWEM, a non-Markovian visuomotor policy equipped with working and episodic memories. VPWEM retains a sliding window of recent observation embeddings as short-term working memory, and introduces a Transformer-based contextual memory compressor that recursively converts out-of-window observations into a fixed number of episodic memory embeddings. The compressor uses self-attention over a cache of past summary embeddings and cross-attention over a cache of historical observations, and is trained jointly with the policy. We instantiate VPWEM on diffusion policies to exploit both short-term and episode-wide information for action generation with nearly constant memory and computation per step. Experiments demonstrate that VPWEM outperforms state-of-the-art baselines including diffusion policies and vision-language-action (VLA) models by more than 20% on the memory-intensive manipulation tasks in MIKASA and achieves an average 5% improvement on the mobile manipulation benchmark MoMaRT. Code is available at https://github.com/HarryLui98/code_vpwem.
Yuheng Lei, Zhixuan Liang, Hongyuan Zhang +1
School of Computing and Data Science, University of Hong Kong
Learning long-horizon robotic manipulation requires jointly achieving expressive behavior modeling, real-time inference, and stable execution, which remains challenging for existing generative policies. Diffusion-based approaches offer strong modeling capacity but incur high inference latency, while flow matching enables fast, near-single-step generation yet often suffers from unstable execution when operating directly in the raw action space. We propose Continuous Latent Action Flow Policy (CoLA-Flow Policy), a trajectory-level imitation learning framework that performs flow matching in a continuous latent action space. By encoding action sequences into temporally coherent latent trajectories and learning an explicit latent-space flow, CoLA-Flow Policy decouples global motion structure from low-level control noise, enabling smooth and reliable long-horizon execution. The framework further integrates geometry-aware point cloud conditioning and execution-time multimodal modulation, using visual cues as a representative modality to enhance real-world robustness. Experiments in simulation and on real robots show that CoLA-Flow Policy achieves near-single-step inference, improves trajectory smoothness by up to 93.7% and task success by up to 25 percentage points over raw action-space flow baselines, while remaining significantly faster than diffusion-based policies.
Wu Songwei, Jiang Zhiduo, Sun Wandong +4
Harbin Institute of Technology, Harbin 150001, China · Honor Device Co., Ltd., Shenzhen, China