Fast Motion Retargeting Pipeline for Scalable Humanoid Robot Learning

Daniel Marew; Sangjun Park; Nisal Perera; Donghyun Kim

cs.ROOct 20, 2026

Fast Motion Retargeting Pipeline for Scalable Humanoid Robot Learning

Authors: Daniel Marew, Sangjun Park, Nisal Perera, Donghyun Kim

Organizations: College of Information and Computer Sciences, University of Massachusetts Amherst, MA, U.S.

Abstract

While large-scale human motion capture datasets provide a rich source of demonstrations for humanoid robot learning, retargeting the diverse motions to different humanoid robots remains a nontrivial challenge. Existing kinodynamic trajectory optimization pipelines are computationally expensive and often require manual adjustments per motion. We present FRDV (Fast Retargeting with Dynamics Validation), a scalable framework consisting of rapid constrained kinematic retargeting and an additional dynamics validation pipeline. Its kinematic stage integrates a custom Sequential Quadratic Programming solver, Real-Time Iteration, and parallel chunk-based processing, achieving up to 9,156 frames per second, approximately up to 136 times faster than GMR and ProtoMotions, while robot-specific constraints are incorporated into the optimization to reduce common retargeting artifacts, such as ground penetration and self-collision. For motions requiring additional verification, a contact implicit iLQR controller is utilized to test the retargeted reference under full rigid body dynamics. This dynamics-based validation identifies dynamically infeasible motions and refines contact-related artifacts such as foot floating and foot skating. We evaluate retargeting quality by training motion tracking reinforcement learning policies using several state-of-the-art pipelines and comparing their tracking fidelity and success rates. Furthermore, sim-to-real transfer is demonstrated on a Unitree G1 humanoid. Videos and code are available at: https://daroslab.github.io/frdv/.

Figures & tables

	FRDV (Ours)	SPIDER [ 16 ]	GMR [ 12 ]	PHC [ 17 ]	OmniRetarget [ 18 ]	ProtoMotions [ 14 ]
Kinematics	✓	– $\square$	✓	✓	✓	✓
Dynamics	✓	✓	✗	✗	✗	✗
Self Collision	✓ $\circ$	✓ $\triangle$	✗	✗	✗	✗
Ground Collision	✓ $\circ$	✓ $\triangle$	– $\diamond$	– $\square$	– $\circ$	✗
Compute	CPU	GPU	CPU	CPU	CPU	GPU
Kinematics FPS $\bullet$	9156 $\blacktriangle$ / 677 $\blacksquare$	48 $\blacklozenge$	48	36	2	67

TABLE I: Comparison of representative motion-retargeting methods. The table compares support for kinematic retargeting, dynamics validation, self- and ground-collision handling, compute platform, and single-frame kinematic retargeting throughput in frames per second. For FRDV, we report both aggregate throughput using 20-worker chunk-parallel execution and throughput using a single worker. Furthermore, for FRDV, timing results cover kinematic retargeting only and exclude CI-iLQR dynamics validation and refinement. Batched execution is excluded.

Fig. 1: LAFAN1 retargeting speed benchmark across 77 sequences. Mean FPS (log scale) with interquartile range for each method; error bars show the 25th–75th percentile range. FRDV timing results cover kinematic retargeting only and exclude CI-iLQR dynamics validation and refinement.

Method	Mean FPS per motion	Wall clock time (h)	Frames processed
ProtoMotions (PyRoki)	57.8	45.06	6,147,900
FRDV (1 worker)	709.3	7.43	19,696,008
FRDV (20 workers)	4324.4	1.08	19,696,008

TABLE II: Runtime on the AMASS SMPL dataset (13,662 motions). The lower ProtoMotions frame count results from its fixed-length representation, in which each motion is trimmed or padded to 450 frames (15 s at 30 Hz) for efficient JAX compilation and batch processing [ 13 ] . FRDV timing results cover kinematic retargeting only and exclude CI-iLQR dynamics validation and refinement.

Fig. 2: Per-frame SQP timing across three robot platforms. The time is broken down into QP preparation, QP solve, and line search. Hatched bars indicate self-collision constraints.

Fig. 3: Retargeting results across 12 different humanoid robots.

Motion (Len., s)	GMR	Proto	FRDV (Ours)
Dance1-S1 (131.5)	99.3	N/A	99.3
Dance2-S2 (225.7)	0.0	N/A	99.7
Dance2-S4 (225.7)	44.4	N/A	98.2
Dance2-S5 (225.7)	99.9	N/A	100.0
PushAndStumble1-S2 (30.0)	100.0	100.0	100.0
PushAndStumble1-S3 (30.0)	100.0	100.0	100.0

TABLE III: Per-motion policy success on LAFAN1 for the Unitree G1. The success rate is the percentage of 4096 trials in which the policy tracks the reference motion to completion without early termination. Higher is better.

Metric	Statistic	GMR	Proto	FRDV
$E_{\text{g-mpbpe}}$ , mm	Min	49.9138	63.8673	36.2905
	Median	105.8906	118.6711	100.5796
	Mean	237.1999	289.9490	156.4916
	Max	3829.2273	2003.9506	1722.3719
$E_{\text{a-mpbpe}}$ , mm	Min	20.4714	19.8689	19.1403
	Median	35.1353	33.2212	34.4500

TABLE IV: Tracking error between policy output and retargeted reference. Results use the same motion subset as Table III , with 4096 trials per motion. Statistics are pooled across all frames, trials, and motions. Lower is better.

Fig. 4: Contact-implicit iLQR verification on G1 and Dash using single-leg hopping snapshots and left-leg joint torques.

Fig. 5: Foot skating artifact on G1 . (a) Kinematic retargeting produces sliding contact. (b) Contact implicit iLQR enforces contact and eliminates sliding.

Reference	Stat.	$\boldsymbol{E_{g\text{-}\mathrm{mpbpe}}}$ (mm)	$\boldsymbol{E_{a\text{-}\mathrm{mpbpe}}}$ (mm)	$\boldsymbol{E_{\mathrm{mpboe}}}$ ( $10^{-3}$ rad)
IK only	Min.	6.4705	5.9587	38.0657
	Median	105.8434	30.4584	136.2763
	Mean	103.1215	29.2581	133.3334
	Max.	234.5076	46.1332	221.1189
CI-iLQR corrected	Min.	14.6757	6.3719	34.2465
	Median	86.1861	30.8755	129.3279

TABLE V: Tracking errors for RL policies trained on IK-only and CI-iLQR–corrected references. Lower is better.

Fig. 6: Sim-to-real transfer on Unitree G1.

Motion Retargeting

Motion retargeting transfers movement across different bodies, skeletons, or robots while preserving its intent and physical feasibility.

See all 104 papers

Direct Dynamic Retargeting for Humanoid Imitation Learning from Videos
May 22, 2026Constant Roux, Ludovic De Matteïs, Armand Jordana +4Humanoid Robot Control Motion Retargeting
LAAS-CNRS, Université de Toulouse, CNRS, Toulouse, France · IRT Saint-Exupéry, Toulouse, France · Artificial and Natural Intelligence Toulouse Institute (ANITI), Toulouse, France
A Scalable Whole-body Motion Transfer via Implicit Kinodynamic Motion Retargeting
Sep 18, 2025Xingyu Chen, Hanyu Wu, Sikai Wu +7Humanoid Robot Control Synthetic Data Generation for Robotics
1HKUST(GZ) · 4Xiaomi Robotics Lab · 2ETH Zurich +1
ReActor: Reinforcement Learning for Physics-Aware Motion Retargeting
May 7, 2026David Müller, Agon Serifi, Sammy Christen +3Cross-Embodiment Robot Learning Motion Retargeting
Disney Research, Switzerland

May 22, 2026J/K move · Enter open · S save

Direct Dynamic Retargeting for Humanoid Imitation Learning from Videos

Constant Roux, Ludovic De Matteïs, Armand Jordana, Valentin Guillet+3

LAAS-CNRS, Université de Toulouse, CNRS, Toulouse, France · IRT Saint-Exupéry, Toulouse, France · Artificial and Natural Intelligence Toulouse Institute (ANITI), Toulouse, France

Humanoid Robot Control Motion Retargeting Robot Imitation Learning Humanoid Motion Tracking Human-to-Robot Motion Imitation

Open paper AskPDF arXiv

Imitation Learning from monocular video demonstrations provides a scalable approach for teaching complex skills to humanoid robots. However, translating human motion to humanoids requires overcoming significant morphological mismatches. Standard approaches rely on Geometric Retargeting or Indirect Dynamic Retargeting pipelines. We identify that these intermediate kinematic projections introduce a geometric bias, restricting the search space and yielding suboptimal dynamic behaviors. In this paper, we propose Direct Dynamic Retargeting (DDR), a novel single-stage framework that generates high-fidelity, dynamically feasible trajectories directly from expert videos. By formulating the problem in the task space and leveraging a sampling-based Model Predictive Control solver within a physics simulator, DDR natively optimizes over complex contact sequences while mitigating input drift. Our experiments demonstrate that bypassing the geometric bias allows DDR to outperform state-of-the-art baselines in demonstration tracking accuracy. Furthermore, we establish that providing such physically viable references to RL agents accelerates training convergence and enhances the final execution of agile and balancing behaviors. Source code will be made publicly available.

Abstract

Motion Retargeting

Explore similar work

Direct Dynamic Retargeting for Humanoid Imitation Learning from Videos

A Scalable Whole-body Motion Transfer via Implicit Kinodynamic Motion Retargeting

ReActor: Reinforcement Learning for Physics-Aware Motion Retargeting