EgoSpeedUp: Transferring Human Manipulation Tempo to Robot Policies

Hanbit Oh; Yukiyasu Domae; Takuma Yagi

cs.ROSep 24, 2026

EgoSpeedUp: Transferring Human Manipulation Tempo to Robot Policies

Authors: Hanbit Oh, Yukiyasu Domae, Takuma Yagi

Organizations: Artificial Intelligence Research Center, National Institute of Advanced Industrial Science and Technology (AIST), Japan

Abstract

Robot manipulation policies trained through imitation learning inherit not only the demonstrated behavior but also the conservative execution tempo of robot demonstrations. Existing acceleration approaches can execute faster than the original demonstrations, but determine the appropriate acceleration primarily from robot-side information or a predefined set of tempo factors, leaving open how to obtain a task-appropriate reference for how fast each manipulation phase should progress. We introduce EgoSpeedUp, a framework that uses human manipulation as temporal supervision for robot imitation learning. Our key insight is that human demonstrations naturally reveal task-appropriate, phase-wise manipulation tempo. Given slow robot demonstrations and human demonstrations of the same task, EgoSpeedUp aligns corresponding manipulation phases, estimates their relative execution tempos from multiple human demonstrations, and transfers the resulting phase-wise tempo by retiming the robot demonstrations. The retimed demonstrations are then used for standard behavior cloning, allowing the robot to retain its executable manipulation behavior while learning to perform it at a human-informed tempo. Across two real-world manipulation tasks, EgoSpeedUp improves the task success rate by an average of 25 percentage points (pp) while reducing successful execution time by 36.5%. These results demonstrate that human manipulation tempo provides an effective temporal reference for learning faster and more reliable robot policies.

Figures & tables

Fig. 1 : EgoSpeedUp transfers phase-wise manipulation tempo from human to robot demonstrations. Human demonstrations implicitly encode task-appropriate manipulation tempo. For example, humans can efficiently straighten a rope by rapidly swinging it during the motion. EgoSpeedUp aligns slow robot demonstrations with this human tempo, enabling faster robot manipulation while maintaining task success.

Fig. 2 : Overview of EgoSpeedUp. Fast human videos and slow robot demonstrations are segmented into manipulation phases and aligned through monotonic phase matching. Corresponding phase lengths provide tempo ratios that determine phase-wise target tempos for the robot demonstrations. These tempos are used to retime future robot action targets, forming the EgoSpeedUp training data for behavior cloning.

Fig. 3 : Human–robot phase correspondence and phase-wise tempo ratios for the Rope and Block tasks. Representative human and robot demonstrations are aligned into four manipulation phases ( $P0$ – $P3$ ). The corresponding mean tempo ratios are computed from phase lengths and provide the temporal reference used by EgoSpeedUp.

	Rope				Block				Total
Method	Succ.	Mean (s)	Med. (s)	Speedup	Succ.	Mean (s)	Med. (s)	Speedup	Succ.	Speedup
Nominal	12/20 (60%)	14.76	14.31	1.00 $\times$	11/20 (55%)	14.48	14.33	1.00 $\times$	23/40 (57.5%) *	1.00 $\times$ *
Global 2x	13/20 (65%)	8.54	8.39	1.73 $\times$	9/20 (45%)	7.98	8.09	1.81 $\times$	22/40 (55.0%) *	1.77 $\times$
Global 4x	11/20 (55%)	5.92	5.90	2.49 $\times$	5/20 (25%)	4.87	4.83	2.97 $\times$	16/40 (40.0%) *	2.71 $\times$
DemoSpeedup	10/20 (50%)	5.96	5.95	2.47 $\times$	6/20 (30%)	5.29	5.23	2.74 $\times$	16/40 (40.0%) *	2.60 $\times$
SpeedAug-prior	7/20 (35%)	8.71	9.00	1.70 $\times$	6/20 (30%)	7.92	8.13	1.83 $\times$	13/40 (32.5%) *	1.76 $\times$

TABLE I : Main results on Rope and Block. Each method is evaluated on 20 trials per task (two policy seeds

\times

10 initial poses). Total is the average over the two tasks. Bold indicates the highest observed success rate within each task and overall. ∗ indicates that EgoSpeedUp significantly outperforms the corresponding baseline (two-sided paired

t

-tests,

p<0.05

Method	Succ. $\uparrow$	Time (s) $\downarrow$
EgoSpeedUp	82.5%	9.25
EgoSpeedUp w/ Global Tempo	$-17.5$ pp	$+1.76$
EgoSpeedUp w/ $M=5$	$-22.5$ pp	$+0.16$
EgoSpeedUp w/ $M=10$	$-17.5$ pp	$+0.25$
EgoSpeedUp w/ Median Aggregation	$-20$ pp	$+0.18$

TABLE II : Design analysis relative to the default EgoSpeedUp configuration (phase-wise tempo,

M=40

human references, and pairwise mean aggregation). The first row reports the absolute performance of EgoSpeedUp. All subsequent rows report changes relative to this reference.

Fig. 5 : Representative failure cases and phase-associated failure counts for Rope and Block. Rope failures are categorized as grasping and straightening, and Block failures as picking and unstable placement. Stacked bars report failed trials among 20 attempts per method and task; numbers above the bars indicate total failures. Failure counts are not normalized by the number of trials that reached each phase.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Value	Hyperparameter	Value
Learning rate	$10^{-4}\!\rightarrow\!10^{-5}$	Policy backbone	DiTX
AdamW betas	$(0.9,\,0.95)$	Transformer layers	12
Weight decay	$10^{-3}$	Hidden dimension	768
Batch size	64	Attention heads	8
Visual encoder	CLIP ViT-B/16	Flow inference steps	10
Observation steps	2	Action horizon	16

Appendix

TABLE III : Common hyperparameters for all evaluated ManiFlow-based methods.

Explore similar work

Aug 10, 2026cs.RO

SpeedTuning: Speeding Up Policy Execution with Lightweight Reinforcement Learning

While learned robotic policies hold promise for advancing generalizable manipulation, their practical deployment is often hindered by suboptimal execution speeds. Imitation learning policies are inherently limited by hardware constraints and the speed of the operator during data collection. In addition, there are no established methods for accelerating policies learned via imitation, and the empirical relationship between execution speed and task success remains underexplored. To address these issues, we introduce SpeedTuning, a reinforcement learning framework specifically designed to enhance the speed of manipulation policies. SpeedTuning learns to predict the optimal execution speed for actions, thereby complementing a base policy without necessitating additional data collection. We provide empirical evidence that SpeedTuning achieves substantial improvements in execution speed, exceeding 2.4x speed-up, while preserving an adequate success rate compared to both the original task policy and straightforward speed-up methods such as linear interpolation at a fixed speed. We evaluate our approach across a diverse set of dynamic and precise tasks, including pouring, throwing, and picking, demonstrating its effectiveness and robustness in enhancing real-world robotic manipulation. Videos and code are available at https://daivdyuan.github.io/speed-tuning/

David D. Yuan, Tony Z. Zhao, Kaylee Burns +1

Stanford University

Robotic Manipulation Policies Robotic Manipulation Imitation Smooth Execution

Ask PDF

Jul 1, 2026cs.RO

AutoSpeed: Annotation-Free Stage-Adaptive Motion Speed Learning for Robot Manipulation

Different stages of manipulation tasks exhibit varying levels of difficulty, suggesting stage-dependent motion speeds and temporal prediction horizons. However, existing IL-based visuomotor policies typically imitate the execution speed of expert demonstrations and operate with a fixed temporal prediction horizon, limiting flexibility and overall task throughput. In this paper, we introduce AutoSpeed, a model-agnostic learning framework that enables existing visuomotor policies to predict trajectories with stage-adaptive motion speeds, without requiring speed or stage annotations. We treat future trajectories at different speeds as candidate optimization targets, evaluate each candidate using a composite cost that trades off prediction error against prediction horizon, and optimize the policy toward the minimum-cost candidate. With a fixed-length action sequence, speed modulation adjusts the effective temporal prediction horizon: simple stages are executed faster with a longer prediction horizon, whereas complex stages are executed more slowly with a shorter prediction horizon. Specifically, we implement speed modulation in the frequency domain via the discrete cosine transform (DCT), which enables smooth, non-integer speed scaling and thus preserves motion continuity. Extensive evaluations show that AutoSpeed substantially reduces task execution time while also improving success rates. Under the AutoSpeed framework, the inferred motion speeds exhibit a strong correspondence with task stages.

Qingda Hu, Ziheng Qiu, Jieru Zhao +2

College of Intelligent Robotics and Advanced Manufacturing, Fudan University, Shanghai, China · School of Computing, Shanghai Jiao Tong University, Shanghai, China

Visuomotor Policy Robotic Manipulation Motion Action Sequences Short-Horizon Forecasting

Ask PDF

May 24, 2026cs.RO

HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos

Human egocentric video captures rich manipulation demonstrations without any robot hardware, yet transferring these skills to robots remains challenging due to the embodiment gap between human and robot in both visual appearance and kinematics. We present HumanEgo, a framework that bridges the embodiment gap by lifting each human demonstration to an entity-level representation of hand-object interaction, and training a flow matching policy with dense auxiliary objectives that amplify supervision from every trajectory. HumanEgo is robot-data-free, hardware-agnostic, data-efficient, and zero-shot human-to-robot transferable. With only 30 minutes of human videos per task, HumanEgo achieves 92.5% average success across four real-world tasks (75% with just 15 minutes), outperforms matched-time robot teleoperation by 41%, and robustly transfers zero-shot across novel robots, cameras, and environments. We release HumanEgo as an easy-to-use, open-source framework for learning robot policies directly from human data: https://github.com/TX-Leo/HumanEgo

Zhi Wang, Botao He, Kelin Yu +4

University of Maryland

Human-To-Robot Transfer Egocentric Video Embodiment Robot Systems

Ask PDF