Imitation policies inherit the operator's pace, and re-timing the demonstrations to run faster works only until success collapses at two to three times demonstration speed. Recent methods attribute that collapse to distribution shift and answer it with finer rules for which segments may be sped up; we show it has a mechanical cause that segment selection cannot address. A low-level controller realises only a bounded displacement per control period, so merging consecutive commands produces sums outside that bound: training pairs that promise a transition the plant cannot deliver. The share of such pairs in the re-timed data, which we call the exceed share, tracks where the retrained policy collapses. The rule that follows has a single parameter: merge greedily while the sum stays realisable. Every command is then realisable by construction, the demonstrated geometry is preserved, and the speed-up lands only where the demonstrator left slack. On four simulated robomimic tasks, 200 held-out seeds per cell, a merge cap of two is at or above the frozen policy's success on all four, by 14 points on Square, and shortens the successful episodes. At matched tempo, uniform re-timing loses 9 and 16 points on Can and ToolHang, is within noise on the other two, and reaches zero on the hardest at 2x. Placement also matters: no shuffled, entropy-guided or speed-conditioned alternative beats the rule on three of the four tasks, by margins the exceed share alone does not explain; on the fourth, at a tempo the task absorbs, every placement lands in the same band. The same principle carries to a simulated humanoid whole-body tracker: re-timing 429 reference motions within its velocity envelope succeeds on 71% of clips at 2.3x, against 51% for uniform time scaling at 2x.
Figures & tables
Fig. 1: (a) Success against time to success on Can for every route, 200 held-out seeds; up and to the right is better, and the cross is the frozen policy at nominal speed. Error bars give the range over three training seeds where those exist, for uniform 1.5× and 2× and for within-limit fmax=2 and 4 ; the remaining points are a single training seed. (b) Success against tempo for the G1 whole-body tracker on 429 reference clips; dashed lines are the share of clips without a fall.
route
Lift
Can
Square
ToolHang
frozen policy, nominal speed
94% @ 45
92% @ 110
62% @ 168
16% 14–20 @ 536
uniform re-timing 1.5×
96% @ 33
87% 79–92 @ 91
72% @ 131
8% 6–10 @ 450
uniform re-timing 2×
96% @ 27
60% 54–65 @ 83
64% 59–68 @ 110
0% @ 304
within-limit re-timing, fmax=2
98% @ 33 [1.40]
96% @ 92 [1.25]
76% 74–78 @ 115 [1.49]
24% @ 339 [1.51]
within-limit re-timing, fmax=4
99% @ 30 [1.57]
96% 94–97 @ 85 [1.37]
61% 58–65 @ 101 [1.74]
23% 18–30 @ 251 [1.89]
TABLE I: Every route on the four tasks, 200 held-out seeds per cell. Cells give success @ time to success, with the training-seed range as a subscript and the tempo of the re-timed data in brackets. Uniform 3× reaches 1% on Can and 0% on ToolHang and is omitted.
route
Lift
Can
Square
ToolHang
within-limit, fmax=4
99% @ 30 [1.57]
96% 94–97 @ 85 [1.37]
61% 58–65 @ 101 [1.74]
23% 18–30 @ 251 [1.89]
shuffled placement, matched tempo
99% @ 34 [1.57]
84% @ 96 [1.37]
59% @ 119 [1.74]
10% @ 428 [1.89]
entropy-guided merging (low half)
94% @ 34 [1.36]
87% @ 93 [1.35]
76% @ 136 [1.35]
12% @ 380 [1.35]
speed-conditioned, best cell
90% @ 28 [2.00]
67% @ 118 [1.00]
74% @ 128 [1.50]
14% @ 610 [1.00]
TABLE II: Placement at comparable tempo. The shuffled row produces the same number of training pairs from the same demonstrations as the within-limit row and differs only in which steps are merged; the entropy-guided and speed-conditioned rows run at the tempo shown in brackets. The entropy row merges the low-scoring half, inverting the direction DemoSpeedup specifies; the speed-conditioned row is that policy’s best cell. The within-limit row is the same replicate set as Table I ; the control arms are a single training seed.
surviving exceedances
tempo
ρ
phase split (tr/ap/co, %)
η^ mean / p90 (mm)
success @ time
spread over all phases
1.33
0.20
tr 28, ap 8, co 65
11.3 / 27.5
90% 89–91 @ 95
none left in transport
1.33
0.20
tr 0, ap 9, co 91
13.1 / 39.0
84% 78–86 @ 101
none left in approach
1.33
0.20
tr 31, ap 0, co 69
11.7 / 29.8
91% 86–93 @ 95
none left in contact
1.33
0.20
tr 58, ap 16, co 26
9.4 / 22.2
92% @ 92
TABLE III: Placement at matched tempo and matched exceed share on Can , over three training seeds. All four arms carry the same tempo and the same per-group exceed share by construction, and differ only in which phase the surviving exceedances are left in. The replay gap η^ is the endpoint error over the four-group execution window, measured on the partitions alone and involving no policy.
Fig. 2: Final frames of one episode per arm. Top, Can : uniform re-timing at 2× puts the can in the wrong compartment, while the frozen policy and within-limit re-timing both succeed, the latter in fewer steps. Bottom, G1 on a jog with a 180-degree turn: the tracker falls under uniform time scaling at 2× and follows the within-envelope reference at 1.84× to the end.
setting
nom.
uni. 2×
within-limit fmax=4
Can , flow-matching policy
96%
33%
90% 88–91 @ 88 [1.37]
Square , flow-matching policy
85%
59%
70% 68–74 @ 124 [1.38]
Can , mixed-skill demos
55%
66%
80% @ 112 [1.72]
Lift , mixed-skill demos
77%
90%
96% @ 37 [2.49]
Lift , machine demos (no slack)
66%
30%
65% @ 60 [1.10]
Can , 50 demonstrations
73%
21%
84% @ 87 [1.35]
TABLE IV: Scope of the result. Cells give success @ time to success on 200 held-out seeds.
arm
tempo
success
no fall
uniform time scaling
1.25×
1.25 ×
95%
96%
1.5×
1.50 ×
82%
93%
2×
2.00 ×
51%
90%
3×
3.00 ×
27%
87%
within-envelope re-timing
TABLE V: Time scaling of reference motions for the G1 humanoid under the released SONIC tracker, 429 clips. Success is no fall and a tracking error within 1.25× the native-speed value.
route
tempo
ρ
per seed
success @ time
frozen, nominal speed
1.00
0.00
88/88/85
87% @ 114
uniform 2×
1.99
0.59
48/49/51
49% @ 86
within-limit, fmax=4
1.39
0.00
75/72/66
71% @ 87
TABLE VI: A pretrained vision-language-action policy on Can . SmolVLA-base fine-tuned for 12k steps on two rendered 128×128 views and proprioception, with no object pose; three training seeds of 200 episodes each. Columns give tempo, training exceed share, per-seed success and the mean with time to success.
The pace of a robot demonstration is often determined by the teleoperator rather than by the task itself. As a result, the same motion can appear at widely different speeds across demonstrations, and policies trained on such data may inherit operator-specific timing along with task behavior. We propose to learn a speed prior directly from large-scale demonstration data. Given a task instruction, a motion path and the images along it, it predicts a distribution over execution speed along the path, from which a chosen quantile defines a consistent clock for retiming each demonstration. Existing methods derive the acceleration signal from the very data they accelerate, so each new task costs another fit, annotation, or pair of predefined rates. A single trained speed prior instead covers every task in a corpus. We fit a speed prior on both robot teleoperation data and handheld demonstrations in which a human arm, rather than a robot arm, moves the gripper, and use it to retime robot demonstrations to the handheld pace before fine-tuning a vision-language-action policy. For execution, we pair the policy with a real-time optimal-control layer that tracks predicted action chunks under robot constraints using the predicted pace. On 120 RoboLab tasks, policies fine-tuned on our learned clocks succeed more often and move with up to half the jerk compared with policies fine-tuned on the original demonstration timing and uniformly accelerated to the same speed. On a real robot, our policy stacks cups in 3.50s, 1.72× faster than the default policy's 6.02s while succeeding as often, in 29 of 30 trials. Videos and further material are available at https://mllab.snu.ac.kr/projects/learned_speed_prior.
Robot manipulation policies trained through imitation learning inherit not only the demonstrated behavior but also the conservative execution tempo of robot demonstrations. Existing acceleration approaches can execute faster than the original demonstrations, but determine the appropriate acceleration primarily from robot-side information or a predefined set of tempo factors, leaving open how to obtain a task-appropriate reference for how fast each manipulation phase should progress. We introduce EgoSpeedUp, a framework that uses human manipulation as temporal supervision for robot imitation learning. Our key insight is that human demonstrations naturally reveal task-appropriate, phase-wise manipulation tempo. Given slow robot demonstrations and human demonstrations of the same task, EgoSpeedUp aligns corresponding manipulation phases, estimates their relative execution tempos from multiple human demonstrations, and transfers the resulting phase-wise tempo by retiming the robot demonstrations. The retimed demonstrations are then used for standard behavior cloning, allowing the robot to retain its executable manipulation behavior while learning to perform it at a human-informed tempo. Across two real-world manipulation tasks, EgoSpeedUp improves the task success rate by an average of 25 percentage points (pp) while reducing successful execution time by 36.5%. These results demonstrate that human manipulation tempo provides an effective temporal reference for learning faster and more reliable robot policies.
Hanbit Oh, Yukiyasu Domae, Takuma Yagi
Artificial Intelligence Research Center, National Institute of Advanced Industrial Science and Technology (AIST), Japan
Visuomotor policies trained via imitation learning often inherit the unnecessarily slow timing of teleoperated demonstrations. Yet uniform speedup is unreliable because different phases of a manipulation task tolerate acceleration differently. In this work, we introduce AdaTempo, a self-supervised method that accelerates visuomotor policies by exploiting shared relative-tempo structure in demonstrations. AdaTempo establishes phase correspondence, aggregates the aligned relative tempo into a consensus, and maps it to a continuous speedup profile used to resample demonstrations into accelerated training trajectories. Training standard policies such as ACT or Diffusion Policy on these resampled trajectories directly embeds the desired tempo in the learned behavior, without runtime tempo selection or online retiming. Extensive evaluations show that AdaTempo achieves up to a 3.57× speedup and yields a stronger success--speed trade-off than the original policies and representative acceleration baselines.