The pace of a robot demonstration is often determined by the teleoperator rather than by the task itself. As a result, the same motion can appear at widely different speeds across demonstrations, and policies trained on such data may inherit operator-specific timing along with task behavior. We propose to learn a speed prior directly from large-scale demonstration data. Given a task instruction, a motion path and the images along it, it predicts a distribution over execution speed along the path, from which a chosen quantile defines a consistent clock for retiming each demonstration. Existing methods derive the acceleration signal from the very data they accelerate, so each new task costs another fit, annotation, or pair of predefined rates. A single trained speed prior instead covers every task in a corpus. We fit a speed prior on both robot teleoperation data and handheld demonstrations in which a human arm, rather than a robot arm, moves the gripper, and use it to retime robot demonstrations to the handheld pace before fine-tuning a vision-language-action policy. For execution, we pair the policy with a real-time optimal-control layer that tracks predicted action chunks under robot constraints using the predicted pace. On 120 RoboLab tasks, policies fine-tuned on our learned clocks succeed more often and move with up to half the jerk compared with policies fine-tuned on the original demonstration timing and uniformly accelerated to the same speed. On a real robot, our policy stacks cups in 3.50s, 1.72× faster than the default policy's 6.02s while succeeding as often, in 29 of 30 trials. Videos and further material are available at https://mllab.snu.ac.kr/projects/learned_speed_prior.
Figures & tables
Figure 1: Learning the pace from demonstrations. (1) Paths are resampled every 2cm of arc length and read in windows of 17 nodes. (2) The prior predicts a distribution over the log speed of every interval. (3) Reading a source path with the target label gives a quantile of the speed along the path. (4) Integrating this speed gives the clock.
Figure 2: One execution cycle. (1) From an observation on the running plan, the policy predicts a new chunk and its pace. (2) Draft fits a smooth path within boxes around the kept waypoints and times it at the pace unless a limit binds. (3) Splice joins the running plan to this path at the earliest point it reaches on time. (4) Smooth solves one minimum-jerk program through the boxes at their scheduled times, within all limits, giving the new plan.
Figure 3: Real-world tasks and their instructions, each shown at its initial scene (1) and its goal (2). The policies for stacking cups, folding towel and cleaning trash are fine-tuned on handheld YUBI demonstrations, and placing bananas and placing cubes are performed zero-shot (Appendix H ).
Execution
Speed-up
SR ↑
TPR ↑
SPARC ↑
Peak ↓
Jerk ↓
Uniform 1×
1.00
0.462
1.45
−3.62
0.30
0.31
Uniform 2×
1.90
0.533
3.55
−3.87
0.52
1.23
Ours, q=0.95
2.00
0.577
4.39
−3.14
0.44
0.83
Uniform 2.5×
2.48
0.530
3.86
−4.11
0.69
2.41
Ours, q=0.99
2.38
0.608
5.16
−3.26
0.52
1.16
Table 1: Throughput and motion quality at matched speed-up on RoboLab. TPR is in successes per minute, Peak is the peak end-effector speed in m/s and Jerk the integrated squared jerk in 103\mathrm{m}^{2}\mathrm{/}\mathrm{s}^{5}$$ . SPARC, Peak and Jerk use the episodes that every execution of the baseline and our policy in Figure 4 completes.
Figure 4: Success rate (left) and TPR (right) against speed-up on RoboLab. Gray labels give the commanded speed-up. TPR (Appendix G ) is in successes per minute, with the time limit of each task. Bars are 95% intervals of the gain over the uniform curve, bootstrapped over tasks.
Stacking cups
Folding towel
Cleaning trash
Model
Controller
SR ↑
TPR ↑
ATR ↓
SPARC ↑
SR ↑
TPR ↑
ATR ↓
SPARC ↑
UPH ↑
SPARC ↑
Default
PD
0.93
8.25
6.48
−2.79
0.13
−4.02
6.81
−3.46
0
−5.41
Default
Ours (1x)
0.97
9.45
6.02
−2.68
0.97
10.31
5.52
−3.75
444
−4.82
Filtered BC
Ours (1x)
0.30
−1.59
7.14
−3.80
0.60
4.25
5.41
−3.46
348
−4.35
Ours
Ours (1x)
0.97
12.99
4.40
−2.70
0.97
10.59
5.38
−3.29
600
−4.49
Ours
Ours (q90)
0.97
15.68
3.65
−2.72
0.97
13.01
4.39
−3.19
732
−5.17
Table 2: Real-robot results on the YUBI tasks of Figure 3 . Ours (1x) runs our controller at α=1 , and Ours (q90), (q95) and (q99) on the tempo channels of the 0.9, 0.95 and 0.99 quantiles. TPR is in successes per minute, ATR in seconds and UPH in items per hour. Appendix G defines the metrics.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Coverage (%)
Pinball, q=0.5
Pinball, q=0.9
R2 of the median
Domain
q=0.1
0.5
0.9
0.99
prior
input-blind
prior
input-blind
within
between episodes
DROID
10.3
50.0
89.9
98.7
0.050
0.294
0.021
0.098
0.94
0.97
FastUMI-100K
9.0
49.6
90.9
99.1
0.045
0.204
0.020
0.077
0.92
0.98
Appendix
Table 3: Accuracy of the speed prior on held-out validation windows, each domain read with its own label. Coverage is the fraction of recorded speeds at or below the predicted quantile. Pinball losses are in natural log speed, for the prior and for an input-blind predictor that returns the training quantiles of the domain. R2 is the fraction of the variance of recorded log speed explained by the predicted median, within episodes after removing each episode’s mean and between episodes among the episode means, over the episodes with at least five windows. The variance between episodes is 18% of the total on DROID and 12% on FastUMI-100K, and the R2 over all windows, 0.95 and 0.93, is the average of the two columns weighted by these shares.
DROID
FastUMI-100K
Distance to gripper event
Share (%)
Recorded
Prior
Median clock
Speed-up
Share (%)
Recorded
Prior
Same interval
16.6
0.24
0.24
0.23
1.08
14.9
0.58
0.58
2cm4cm
9.1
0.45
0.44
0.57
1.57
10.5
0.67
0.67
6cm10cm
12.1
0.68
0.68
0.76
1.39
13.4
0.84
0.84
12cm16cm
8.9
0.84
0.83
0.88
1.32
10.2
0.98
0.98
Transit, none within 16cm
53.3
1.00
1.00
1.00
1.28
50.9
1.00
1.00
Appendix
Table 4: Speed around gripper events on held-out validation windows. Windows are grouped by the arc length from their center interval to the nearest interval in which the gripper opening changes by more than 0.05. Speeds are medians relative to the transit group, whose recorded and predicted medians are 228mm/s228mm/s on DROID and 274mm/s273mm/s on FastUMI-100K. In the same interval the gripper moves and keeps its source pace. The median clock reads DROID with the FastUMI-100K label, and its speed-up is the geometric mean ratio of its median to the median read with the DROID label.
Setting
YUBI
DROID zero-shot
Gripper
Yubi DYNAMIXEL
Robotiq
Open-loop horizon
8 steps
6 for ours, 8 others
Executed tempo quantiles
0.9, 0.95 and 0.99
0.99
Appendix
Table 5: Settings that differ between the setups. The DROID zero-shot tasks of Appendix H are placing bananas and placing cubes.
Setting
Ours
PD
Chunk
H=15 actions, δ=1/15s
Stiffness and damping gains, feedforward κ
4× , 2× , 1
1× , 1× , 0
Step cap
none
0.2rad
Appendix
Table 6: Settings of our controller (Ours) and the PD baseline (PD), shared by both setups. An entry that spans both columns applies to each of them.
Placing bananas
Model
Controller
Bananas ↑
Both ↑
UPH ↑
SPARC ↑
Default
PD
1.57
17
200.7
−5.28
Default
Ours (1x)
1.67
22
231.7
−4.99
Ours
Ours (1x)
1.97
29
376.7
−4.13
Ours
Ours (q99)
2.00
30
492.9
−4.53
Appendix
Table 7: Real-robot results on the placing bananas task (“put bananas into the tray”), with 30 trials per setting. PD is the baseline controller of Appendix F , Ours (1x) runs our controller at α=1 and Ours (q99) on the tempo channel of the 0.99 quantile. Bananas is the mean number of bananas placed within 30s , and Both the number of trials that place both.
Placing cubes
Model
Controller
Cubes ↑
All 9 ↑
UPH ↑
SPARC ↑
Default
PD
3.90
0
234.0
−8.19
Default
Ours (1x)
5.90
0
354.0
−6.26
Ours
Ours (1x)
7.43
2
446.7
−5.46
Ours
Ours (q99)
8.27
12
511.1
−6.18
Appendix
Table 8: Real-robot results on the placing cubes task (“put all the gray cubes into the white bin”), with 30 trials per setting. PD is the baseline controller of Appendix F , Ours (1x) runs our controller at α=1 and Ours (q99) on the tempo channel of the 0.99 quantile. Cubes is the mean number of cubes placed within 60s , and All 9 the number of trials that place all nine.
Figure 5: Mean task progress over time on the real-robot placing bananas task, over 30 trials per condition. Each banana placed in the tray within the 30s limit adds half of a trial, and curves average over trials, so curves further up and to the left progress faster.
Figure 6: Mean task progress over time on the real-robot placing cubes task, over 30 trials per condition. Each cube placed in the bin within the 60s limit adds one ninth of a trial, and curves average over trials, so curves further up and to the left progress faster.
Robot manipulation data collection has been shifting from teleoperation toward robot-free demonstrations, through interfaces such as the Universal Manipulation Interface (UMI) or directly from human hands. Vision-Language-Action (VLA) policies trained on such data inherit the demonstrator's timing. Yet human timing does not directly transfer to robots: compliant hands tolerate fast contact, whereas robots may overshoot due to actuator and tracking limitations; conversely, robots can move faster in free space. This motivates a unified approach that reconciles execution speed with contact safety. We present RoboPace, an online retiming layer that preserves the policy's geometric path while adapting its timing, respecting the target robot's kinematic and dynamic constraints. It adapts execution speed based on predicted contact, jointly accounting for contact-dependent speed limits and the robot's motion constraints. The method requires no policy retraining and operates in real time. Across three contact-rich tasks on a dual-arm robot, faster uniform execution and physical-limit-only retiming largely fail. RoboPace instead achieves higher overall success than slow uniform execution while completing four of five commands in approximately half the time, retaining the reliability of slow execution without its time cost.
Mimo Shirasaka, Takehiko Ohkawa, Takuya Okubo +3
AI Robot Association (AIRoA) · The University of Tokyo
Robot manipulation policies trained through imitation learning inherit not only the demonstrated behavior but also the conservative execution tempo of robot demonstrations. Existing acceleration approaches can execute faster than the original demonstrations, but determine the appropriate acceleration primarily from robot-side information or a predefined set of tempo factors, leaving open how to obtain a task-appropriate reference for how fast each manipulation phase should progress. We introduce EgoSpeedUp, a framework that uses human manipulation as temporal supervision for robot imitation learning. Our key insight is that human demonstrations naturally reveal task-appropriate, phase-wise manipulation tempo. Given slow robot demonstrations and human demonstrations of the same task, EgoSpeedUp aligns corresponding manipulation phases, estimates their relative execution tempos from multiple human demonstrations, and transfers the resulting phase-wise tempo by retiming the robot demonstrations. The retimed demonstrations are then used for standard behavior cloning, allowing the robot to retain its executable manipulation behavior while learning to perform it at a human-informed tempo. Across two real-world manipulation tasks, EgoSpeedUp improves the task success rate by an average of 25 percentage points (pp) while reducing successful execution time by 36.5%. These results demonstrate that human manipulation tempo provides an effective temporal reference for learning faster and more reliable robot policies.
Hanbit Oh, Yukiyasu Domae, Takuma Yagi
Artificial Intelligence Research Center, National Institute of Advanced Industrial Science and Technology (AIST), Japan
Visuomotor policies trained via imitation learning often inherit the unnecessarily slow timing of teleoperated demonstrations. Yet uniform speedup is unreliable because different phases of a manipulation task tolerate acceleration differently. In this work, we introduce AdaTempo, a self-supervised method that accelerates visuomotor policies by exploiting shared relative-tempo structure in demonstrations. AdaTempo establishes phase correspondence, aggregates the aligned relative tempo into a consensus, and maps it to a continuous speedup profile used to resample demonstrations into accelerated training trajectories. Training standard policies such as ACT or Diffusion Policy on these resampled trajectories directly embeds the desired tempo in the learned behavior, without runtime tempo selection or online retiming. Extensive evaluations show that AdaTempo achieves up to a 3.57× speedup and yields a stronger success--speed trade-off than the original policies and representative acceleration baselines.