Diffusion and flow-matching policies have become a standard choice for visuomotor imitation learning, owing to their ability to model multimodal action distributions. In memory-dependent manipulation, the robot must act on information that is no longer visible. Existing methods feed the observation history to the policy as an additional condition, while action generation still starts from Gaussian noise. However, the policy must then pick out the relevant target from a long and redundant history, leading to spurious correlations and unreliable decisions as the memory load grows. Additionally, the initial noise carries no information about what the robot has seen, leaving generation without guidance toward the remembered target. To address these issues, we propose RecallFlow, a flow-policy framework that builds the source distribution from episodic memory. Recall-Relocalization relates the memory to the current view through a few recall slots, so that the source reflects where the remembered targets are now. Recall Anchors then turns each slot into a Gaussian component of the source, enabling generation to start close to the intended action instead of from noise. The transport network then completes the action from the current observation alone, keeping inference nearly as cheap as a memoryless policy. We further build a suite of memory-dependent tasks grouped by memory load and compare all policies under a matched protocol. Extensive experiments show that RecallFlow achieves a state-of-the-art average success rate of 80.4% on the public MIKASA-Robo memory benchmark, and outperforms the strongest baseline by 20.3 points on our suite and by 18.8 points on four real-robot tasks.
Figures & tables
Figure 1: Start from what you remember. In memory-dependent manipulation, the evidence that decides which action to take ( cue ) disappears before the robot acts ( now ). (a) Feeding the memory to the policy as a condition still starts generation from uninformed noise. (b) RecallFlow starts generation from what it remembers: the episodic memory is the source, and the transport network sees only the present.
Figure 2: Overview of RecallFlow. The present observation is encoded into tokens Zt . Recall–Relocalization lets Zt read the episodic memory bank and decodes K recall slots, each attending to the current view through a heatmap Hk . Recall Anchors turns each slot into a Gaussian anchor in action space; with a standard Gaussian, the anchors form the source p0 . A transport network conditioned only on Zt carries x0∼p0 to the action chunk, so memory reaches the action solely through the source.
Figure 3: Evaluation tasks in simulation. (a) RecallSuite consists of five memory-dependent tasks: the cue is seen only earlier in the episode, and the candidates look identical when the robot must decide. (b), (c) Tasks from MIKASA-Robo and RoboMimic.
Single item
Multiple items
Method
MarkedPlate
CountSwing
NoRepeat
NoRepeat-4
MarkedPair
Avg. ↑
Chance
20.0
11.7
22.2
9.4
16.7
16.0
Memoryless
DP [ 9 ]
17.3
10.0
1.3
0.0
1.3
6.0
FM
16.0
6.3
13.3
2.3
0.0
7.6
History-conditioned
Table 1: Comparison on RecallSuite. We report the success rate (%) averaged over three training seeds. RecallFlow is on par with long-history DP when one item must be remembered and outperforms all baselines when several must be.
Method
SGT
IM
RC3
RC5
RC9
Avg. ↑
Octo-small [ 43 , 8 ]
46
39
45
17
11
31.6
OpenVLA ‡ [ 25 , 8 ]
47
14
59
16
6
28.4
π0 [ 2 , 8 ]
33
42
35
22
15
29.4
SpatialVLA [ 46 , 8 ]
23
27
27
17
11
21.0
CronusVLA [ 28 , 50 ]
32
5
31
13
9
18.0
MemoryVLA [ 50 ]
88
24
44
30
20
41.2
Table 2: Comparison with published methods on MIKASA-Robo [ 8 ] . We report the success rate (%) on ShellGameTouch (SGT), InterceptMedium (IM), and RememberColor with 3, 5, and 9 colors (RC3–RC9). RecallFlow, averaged over three seeds, achieves the highest average ( 80.4±1.1 ). ∗ Available only on arXiv. ‡ OpenVLA-OFT in [ 50 ] .
Figure 4: Effect of memory load, tasks without memory, and inference cost. (a) On RecallSuite, RecallFlow matches long-history DP when one item must be remembered and stays ahead of all baselines when several must be. (b) On RoboMimic, where memory is not needed, adding a long history lowers DP, whereas RecallFlow stays on par with its memoryless counterpart FM. (c) RecallFlow reaches the highest success rate on RecallSuite at nearly the inference cost of a memoryless policy. † Reported in [ 15 ] .
Figure 5: Memory decides where generation starts. (a) At a decision of a NoRepeat episode, all thirty samples of RecallFlow head to the untouched container from the source on, whereas long-history DP starts from noise and commits only in its last generation steps. (b) Over 15 decisions, the source samples of RecallFlow are already at the target. (c) RecallFlow therefore keeps its success rate with a single generation step.
Figure 6: Real-robot experiments. (a) We deploy the policies on an xArm 7 with a wrist camera and a third-person camera. (b) We design four tasks that differ in what must be remembered. (c) RecallFlow achieves the highest success rate on all four tasks. (d) The three behaviors observed on No Repeat: repeating one cup, touching one cup twice, and touching each cup once.
Variant
MarkedPlate
NoRepeat
MarkedPair
RecallFlow
95.7
97.0
67.3
W/o memory
16.0 ↓ 79.7
13.3 ↓ 83.7
0.0 ↓ 67.3
As condition
94.7 ↓ 1.0
92.0 ↓ 5.0
51.0 ↓ 16.3
W/o relocalization
98.3 ↑ 2.6
84.7 ↓ 12.3
56.3 ↓ 11.0
Flow loss only
53.3 ↓ 42.4
98.0 ↑ 1.0
9.0 ↓ 58.3
Table 3: Ablation study on RecallSuite. We report the success rate (%), averaged over three seeds, and its change when one design choice of RecallFlow is altered.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Parameter
Value
Recall slots K
4
Memory width / capacity C
256 / 32 entries
Memory-attention / slot-decoder layers
2 / 2
σmin
0.05
KL weight β , relaxed WTA ε , λ
10−3 , 0.05, 1
Warm-up
2,000 steps
Appendix
Table S1 : Hyperparameters of RecallFlow, shared across benchmarks unless noted.
Task
What must be remembered
Candidates
Episode length
Chance (%)
MarkedPlate
which plate the marker touched
5 plates
80
20.0
CountSwing
how many swings were already made
stop or swing again
216
11.7
NoRepeat
which containers were already touched
3 containers
136
22.2
NoRepeat-4
which containers were already touched
4 containers
192
9.4
MarkedPair
which two containers the marker hovered over
4 containers
152
16.7
Appendix
Table S2 : Tasks of RecallSuite. We list what each task requires the robot to remember, its candidates, the episode length in control steps, and the success rate of a memoryless policy that executes perfectly but chooses uniformly at random at every decision point (chance).
RecallSuite
RoboMimic Square
MarkedPlate
NoRepeat
ph
mh
GMP [ 15 ]
100.0
92.0
96.0
80.0
Appendix
Table S3 : GMP trained by us. We train GMP in its released configuration without the memory gate, with the budget of the rows that we train in Tabs. 1 and S8 , and report the success rate (%) of one training seed.
Method
Implementation
History
Generation
DP [ 9 ]
code of [ 15 ] , no history
current step
DDIM, from N(0,I)
FM
ours, memory disabled
current and previous step
rectified flow, from N(0,I)
Medium-history DP
code of [ 15 ]
past 16 steps
DDIM, from N(0,I)
Long-history DP
code of [ 15 ]
past 120 steps, one frame per 8 steps
DDIM, from N(0,I)
GMP [ 15 ]
released configuration, no gate
whole episode, per-chunk cross-attention
DDIM, from N(0,I)
RecallFlow
ours
episodic memory, through the source only
rectified flow, from p0
Appendix
Table S4 : Baseline configurations. All rows that we train on a benchmark share demonstrations, observations, the number of training steps on each task, and evaluation episodes.
Task
Epochs
Reported
Best
Last
ShellGameTouch
15
7
100
100
InterceptMedium
51
13
85
82
RememberColor3
51
5
100
98
RememberColor5
51
7
81
72
RememberColor9
51
11
35
26
Appendix
Table S5 : Training length on MIKASA-Robo. We report the number of epochs trained, the epoch of the reported checkpoint, and the success rate (%) at that checkpoint and at the last one. Checkpoints are saved and evaluated every 2 epochs; ShellGameTouch uses chunk length 8.
Table S6 : Protocols behind Tab. 2 , as stated by each source (“n/s”: not stated).
Method
SGT
RC3
Avg. ↑
DP [ 9 , 27 ]
–
–
19.5
DP-PTP [ 57 , 27 ]
–
–
15.0
MaIL [ 22 , 27 ]
–
–
19.5
BPP [ 51 , 27 ]
94
21
57.5
MemER [ 53 , 27 ]
82
27
54.5
DP-VPWEM [ 27 ]
91
82
86.5
Appendix
Table S7 : Comparison on ShellGameTouch and RememberColor3 of MIKASA-Robo, the two tasks on which several methods are evaluated [ 27 ] . We report the success rate (%); most sources give only the average.
Square
Method
ph
mh
Avg. ↑
Memoryless
DP [ 9 ]
98.0
73.3
85.7
FM
97.0
75.3
86.2
History-conditioned
Long-history DP †
86
69
77.5
Appendix
Table S8 : Comparison on RoboMimic [ 39 ] behind Fig. 4 . We report the success rate (%) on Square with proficient (ph) and multi-human (mh) demonstrations, averaged over three seeds. † Reported in [ 15 ] .
Figure S1 : Longer training on Square (ph). RecallFlow reaches 93% at epoch 4, the budget of Tab. S8 , and its best success rate of 97% at epoch 15.
Method
Remember Color
No Repeat
Marked Plate
Shell Game
Avg.
DP [ 9 ]
9
0
12
7
29.2
FM
5
0
8
5
18.8
Long-history DP
9
14
22
16
63.5
RecallFlow
20
17
24
18
82.3
Appendix
Table S9 : Real-robot results behind Fig. 6 . We report the number of successful trials out of 24 per task and the success rate (%) over all 96 trials.
Method
MarkedPlate
CountSwing
NoRepeat
NoRepeat-4
MarkedPair
DP [ 9 ]
15 / 18 / 19
8 / 13 / 9
2 / 0 / 2
0 / 0 / 0
3 / 0 / 1
FM
17 / 15 / 16
6 / 6 / 7
6 / 18 / 16
1 / 2 / 4
0 / 0 / 0
Medium-history DP
100 / 100 / 100
14 / 13 / 4
30 / 30 / 35
1 / 1 / 3
4 / 6 / 4
Long-history DP
100 / 100 / 100
98 / 99 / 100
68 / 81 / 63
23 / 12 / 10
42 / 46 / 22
RecallFlow
95 / 94 / 98
98 / 98 / 99
98 / 98 / 95
67 / 64 / 63
63 / 71 / 68
Appendix
Table S10 : Per-seed results on RecallSuite. We report the success rate (%) of each of the three training seeds behind Tab. 1 .
Variant
CountSwing
RecallFlow
98.0
W/o memory
6.0
As condition
100.0
W/o relocalization
92.0
Flow loss only
97.0
Appendix
Table S11 : Ablation study on CountSwing . We report the success rate (%) of one training seed for the variants of Tab. 3 .
Variant
MarkedPlate
NoRepeat
MarkedPair
RecallFlow
95 / 94 / 98
98 / 98 / 95
63 / 71 / 68
W/o memory
17 / 15 / 16
6 / 18 / 16
0 / 0 / 0
As condition
91 / 98 / 95
93 / 90 / 93
71 / 67 / 15
W/o relocalization
98 / 97 / 100
82 / 88 / 84
54 / 61 / 54
Flow loss only
38 / 68 / 54
97 / 99 / 98
7 / 12 / 8
Appendix
Table S12 : Per-seed ablations. We report the success rate (%) of each of the three training seeds behind Tab. 3 .
Steps
1
2
4
8
MarkedPlate
97.0
97.0
95.0
95.0
CountSwing
98.0
100.0
99.0
99.0
NoRepeat
95.0
96.0
96.0
98.0
Appendix
Table S13 : Fewer integration steps. We evaluate the seed-0 checkpoints of RecallFlow with 1, 2, 4, and 8 Euler steps (the default is 8) and report the success rate (%). The 8-step entry of CountSwing differs from its seed-0 entry in Tab. S10 by one episode, as this analysis was run on a different machine.
Method
Latency (ms) ↓
Memory (GB) ↓
DP [ 9 ]
148.1
1.10
FM
152.6
1.10
Medium-history DP
190.7 +38.1
1.60
Long-history DP
187.7 +35.1
1.60
GMP [ 15 ]
288.5 +135.9
1.46
RecallFlow
153.7 +1.1
1.14
Appendix
Table S14 : Inference cost behind Fig. 4 . We measure each policy on one RTX 4090 with batch size 1 and report the median latency per action chunk over 34 chunks, its increase over FM, and the peak GPU memory.