EgoSpeedUp: Transferring Human Manipulation Tempo to Robot Policies

Hanbit Oh; Yukiyasu Domae; Takuma Yagi

cs.ROSep 24, 2026

EgoSpeedUp: Transferring Human Manipulation Tempo to Robot Policies

Authors: Hanbit Oh, Yukiyasu Domae, Takuma Yagi

Organizations: Artificial Intelligence Research Center, National Institute of Advanced Industrial Science and Technology (AIST), Japan

Abstract

Robot manipulation policies trained through imitation learning inherit not only the demonstrated behavior but also the conservative execution tempo of robot demonstrations. Existing acceleration approaches can execute faster than the original demonstrations, but determine the appropriate acceleration primarily from robot-side information or a predefined set of tempo factors, leaving open how to obtain a task-appropriate reference for how fast each manipulation phase should progress. We introduce EgoSpeedUp, a framework that uses human manipulation as temporal supervision for robot imitation learning. Our key insight is that human demonstrations naturally reveal task-appropriate, phase-wise manipulation tempo. Given slow robot demonstrations and human demonstrations of the same task, EgoSpeedUp aligns corresponding manipulation phases, estimates their relative execution tempos from multiple human demonstrations, and transfers the resulting phase-wise tempo by retiming the robot demonstrations. The retimed demonstrations are then used for standard behavior cloning, allowing the robot to retain its executable manipulation behavior while learning to perform it at a human-informed tempo. Across two real-world manipulation tasks, EgoSpeedUp improves the task success rate by an average of 25 percentage points (pp) while reducing successful execution time by 36.5%. These results demonstrate that human manipulation tempo provides an effective temporal reference for learning faster and more reliable robot policies.

Figures & tables

Fig. 1 : EgoSpeedUp transfers phase-wise manipulation tempo from human to robot demonstrations. Human demonstrations implicitly encode task-appropriate manipulation tempo. For example, humans can efficiently straighten a rope by rapidly swinging it during the motion. EgoSpeedUp aligns slow robot demonstrations with this human tempo, enabling faster robot manipulation while maintaining task success.

Fig. 2 : Overview of EgoSpeedUp. Fast human videos and slow robot demonstrations are segmented into manipulation phases and aligned through monotonic phase matching. Corresponding phase lengths provide tempo ratios that determine phase-wise target tempos for the robot demonstrations. These tempos are used to retime future robot action targets, forming the EgoSpeedUp training data for behavior cloning.

Fig. 3 : Human–robot phase correspondence and phase-wise tempo ratios for the Rope and Block tasks. Representative human and robot demonstrations are aligned into four manipulation phases ( $P0$ – $P3$ ). The corresponding mean tempo ratios are computed from phase lengths and provide the temporal reference used by EgoSpeedUp.

	Rope				Block				Total
Method	Succ.	Mean (s)	Med. (s)	Speedup	Succ.	Mean (s)	Med. (s)	Speedup	Succ.	Speedup
Nominal	12/20 (60%)	14.76	14.31	1.00 $\times$	11/20 (55%)	14.48	14.33	1.00 $\times$	23/40 (57.5%) *	1.00 $\times$ *
Global 2x	13/20 (65%)	8.54	8.39	1.73 $\times$	9/20 (45%)	7.98	8.09	1.81 $\times$	22/40 (55.0%) *	1.77 $\times$
Global 4x	11/20 (55%)	5.92	5.90	2.49 $\times$	5/20 (25%)	4.87	4.83	2.97 $\times$	16/40 (40.0%) *	2.71 $\times$
DemoSpeedup	10/20 (50%)	5.96	5.95	2.47 $\times$	6/20 (30%)	5.29	5.23	2.74 $\times$	16/40 (40.0%) *	2.60 $\times$
SpeedAug-prior	7/20 (35%)	8.71	9.00	1.70 $\times$	6/20 (30%)	7.92	8.13	1.83 $\times$	13/40 (32.5%) *	1.76 $\times$

TABLE I : Main results on Rope and Block. Each method is evaluated on 20 trials per task (two policy seeds

\times

10 initial poses). Total is the average over the two tasks. Bold indicates the highest observed success rate within each task and overall. ∗ indicates that EgoSpeedUp significantly outperforms the corresponding baseline (two-sided paired

t

-tests,

p<0.05

Method	Succ. $\uparrow$	Time (s) $\downarrow$
EgoSpeedUp	82.5%	9.25
EgoSpeedUp w/ Global Tempo	$-17.5$ pp	$+1.76$
EgoSpeedUp w/ $M=5$	$-22.5$ pp	$+0.16$
EgoSpeedUp w/ $M=10$	$-17.5$ pp	$+0.25$
EgoSpeedUp w/ Median Aggregation	$-20$ pp	$+0.18$

TABLE II : Design analysis relative to the default EgoSpeedUp configuration (phase-wise tempo,

M=40

human references, and pairwise mean aggregation). The first row reports the absolute performance of EgoSpeedUp. All subsequent rows report changes relative to this reference.

Fig. 5 : Representative failure cases and phase-associated failure counts for Rope and Block. Rope failures are categorized as grasping and straightening, and Block failures as picking and unstable placement. Stacked bars report failed trials among 20 attempts per method and task; numbers above the bars indicate total failures. Failure counts are not normalized by the number of trials that reached each phase.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Value	Hyperparameter	Value
Learning rate	$10^{-4}\!\rightarrow\!10^{-5}$	Policy backbone	DiTX
AdamW betas	$(0.9,\,0.95)$	Transformer layers	12
Weight decay	$10^{-3}$	Hidden dimension	768
Batch size	64	Attention heads	8
Visual encoder	CLIP ViT-B/16	Flow inference steps	10
Observation steps	2	Action horizon	16

Appendix

TABLE III : Common hyperparameters for all evaluated ManiFlow-based methods.

Explore similar work

SpeedTuning: Speeding Up Policy Execution with Lightweight Reinforcement Learning
Aug 10, 2026David D. Yuan, Tony Z. Zhao, Kaylee Burns +1Robotic Manipulation Policies Robotic Manipulation
Stanford University
AutoSpeed: Annotation-Free Stage-Adaptive Motion Speed Learning for Robot Manipulation
Jul 1, 2026Qingda Hu, Ziheng Qiu, Jieru Zhao +2Visuomotor Policy Robotic Manipulation
College of Intelligent Robotics and Advanced Manufacturing, Fudan University, Shanghai, China · School of Computing, Shanghai Jiao Tong University, Shanghai, China
HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos
May 24, 2026Zhi Wang, Botao He, Kelin Yu +4Human-To-Robot Transfer Egocentric Video
University of Maryland

Aug 10, 2026, cs.ROJ/K move · Enter open · S save

SpeedTuning: Speeding Up Policy Execution with Lightweight Reinforcement Learning

David D. Yuan, Tony Z. Zhao, Kaylee Burns, Chelsea Finn

Stanford University

Robotic Manipulation Policies Robotic Manipulation Imitation Smooth Execution

Open paper AskPDF arXiv

While learned robotic policies hold promise for advancing generalizable manipulation, their practical deployment is often hindered by suboptimal execution speeds. Imitation learning policies are inherently limited by hardware constraints and the speed of the operator during data collection. In addition, there are no established methods for accelerating policies learned via imitation, and the empirical relationship between execution speed and task success remains underexplored. To address these issues, we introduce SpeedTuning, a reinforcement learning framework specifically designed to enhance the speed of manipulation policies. SpeedTuning learns to predict the optimal execution speed for actions, thereby complementing a base policy without necessitating additional data collection. We provide empirical evidence that SpeedTuning achieves substantial improvements in execution speed, exceeding 2.4x speed-up, while preserving an adequate success rate compared to both the original task policy and straightforward speed-up methods such as linear interpolation at a fixed speed. We evaluate our approach across a diverse set of dynamic and precise tasks, including pouring, throwing, and picking, demonstrating its effectiveness and robustness in enhancing real-world robotic manipulation. Videos and code are available at https://daivdyuan.github.io/speed-tuning/

Abstract

Explore similar work

SpeedTuning: Speeding Up Policy Execution with Lightweight Reinforcement Learning

AutoSpeed: Annotation-Free Stage-Adaptive Motion Speed Learning for Robot Manipulation

HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos