OmniTrack: General Motion Tracking via Physics-Consistent Reference

Yuhan Li; Peiyuan Zhi; Yunshen Wang; Tengyu Liu; Sixu Yan; Wenyu Liu; Xinggang Wang; Baoxiong Jia

cs.ROFeb 27, 2026

OmniTrack: General Motion Tracking via Physics-Consistent Reference

Authors: Yuhan Li, Peiyuan Zhi, Yunshen Wang, Tengyu Liu, Sixu Yan, Wenyu Liu, Xinggang Wang, Baoxiong Jia, +1 more

Organizations: Huazhong University of Science and Technology · State Key Lab of General AI, Beijing Institute for General Artificial Intelligence (BIGAI) · Shanghai Jiao Tong University

Abstract

Learning motion tracking from rich human motion data is a foundational task for achieving general control in humanoid robots, enabling them to perform diverse behaviors. However, discrepancies in morphology and dynamics between humans and robots, combined with data noise, introduce physically infeasible artifacts in reference motions, such as floating and penetration. During both training and execution, these artifacts create a conflict between following inaccurate reference motions and maintaining the robot's stability, hindering the development of a generalizable motion tracking policy. To address these challenges, we introduce OmniTrack, a general tracking framework that explicitly decouples physical feasibility from general motion tracking. In the first stage, a privileged generalist policy generates physically plausible motions that strictly adhere to the robot's dynamics via trajectory rollout in simulation. In the second stage, the general control policy is trained to track these physically feasible motions, ensuring stable and coherent control transfer to the real robot. Experiments show that OmniTrack improves tracking accuracy and demonstrates strong generalization to unseen motions. In real-world tests, OmniTrack achieves hour-long, consistent, and stable tracking, including complex acrobatic motions such as flips and cartwheels. Additionally, we show that OmniTrack supports human-style stable and dynamic online teleoperation, highlighting its robustness and adaptability to varying user inputs.

Figures & tables

Fig. 2 : Examples of physically infeasible artifacts in retargeted human motions (blue) compared with physically feasible robot motions (black/gray), including inconsistent center-of-mass motion, foot skating, floating, and ground penetration, which hinder stable humanoid control.

Fig. 3 : Overview of the OmniTrack framework. Our method adopts a two-stage pipeline that first converts raw, physically inconsistent reference motions into physics-consistent motions in simulation, and then trains a general policy to robustly track these motions under realistic conditions. The resulting system supports both offline motion tracking and online teleoperation.

Dataset	Reference	Penetration	Floating	Smoothness	MPJPE
LAFAN1	Raw Ref.	20.3%	2.52%	33.7	0.000
LAFAN1	Physical Ref.	0.0%	0.0%	31.8	21.0
AMASS	Raw Ref.	67.7%	3.29%	19.5	0.000
AMASS	Physical Ref.	0.0%	0.0%	15.8	16.0

TABLE I : Comparison of raw and physically plausible motions. Physically plausible motions eliminate penetration and floating artifacts while improving motion smoothness, with a moderate increase in MPJPE due to dynamics-consistent corrections.

Fig. 4 : Impact of physically plausible motions under varying dataset sizes. From left to right: mean reward and mean episode length during training, followed by success rate and tracking error (MPJPE) under different dataset sizes (1/8, 1/2, and full LAFAN1). Dark colors denote training with physically plausible motions, while light colors denote training with raw reference motions.

Pipeline	Seen Motions				Seen (Hard Motions)				Unseen Motions
Pipeline	SR	MPJPE	$\bm{\Delta}$ Vel	$\bm{\Delta}$ Acc	SR	MPJPE	$\bm{\Delta}$ Vel	$\bm{\Delta}$ Acc	SR	MPJPE	$\bm{\Delta}$ Vel	$\bm{\Delta}$ Acc
Dagger [ 33 ]	92.46%	40.79	6.196	2.051	67.40%	53.52	9.468	3.152	96.56%	37.34	5.113	1.598
Dagger ${}_{\text{hist}}$	93.03%	38.96	5.714	1.945	70.04%	52.15	8.808	3.027	96.52%	35.66	4.863	1.559
AAC [ 32 ]	92.78%	41.36	6.577	2.036	68.80%	55.46	10.16	3.157	96.55%	37.24	5.309	1.574
Ours	96.13%	37.61	5.709	1.934	84.81%	46.43	8.516	2.932	96.88%	34.83	4.852	1.556

TABLE II : Comparison of training pipelines for motion tracking. Our training pipeline consistently outperforms alternative training strategies across all motion categories, with particularly significant advantages on challenging motion sequences.

Fig. 5 : Diverse motion skills executed on the real humanoid robot. Our policy enables hour-long continuous and stable tracking of a wide range of human-like behaviors, demonstrating broad motion coverage, strong real-world versatility, and long-term control stability.

Method	SR	MPJPE	$\bm{\Delta}$ Vel	$\bm{\Delta}$ Acc
OmniH2O [ 10 ]	48.32%	54.58	11.40	4.293
Exbody2 [ 12 ]	58.93%	53.47	12.74	3.996
BeyondMimic [ 21 ]	70.04%	55.75	10.04	3.126
Ours	84.81%	46.43	8.516	2.932

TABLE III : Comparison with state-of-the-art humanoid motion tracking methods. Our method achieves the best overall performance, showing higher success rates and lower tracking errors compared with prior approaches.

Fig. 6 : Real-time teleoperation on the humanoid robot. Under real-time teleoperation, the robot executes dynamic, static, and contact-rich behaviors with natural, human-like motion style, demonstrating responsive control and high-fidelity whole-body coordination in the real world.

Motion	MuJoCo (Sim)			Real World
Motion	Dof	Rot	AngVel	Dof	Rot	AngVel
	(rad)	(rad)	(rad/s)	(rad)	(rad)	(rad/s)
Walking	0.0601	0.0675	0.6567	0.0710	0.0996	0.6716
Running	0.0580	0.0730	0.6781	0.0638	0.0897	0.7304
Dancing	0.0657	0.0752	0.7370	0.0708	0.0925	0.7613
Martial Arts	0.0817	0.0928	0.7794	0.0822	0.0956	0.6730

TABLE IV : Sim-to-real tracking error comparison using onboard sensor measurements. We evaluate motion tracking errors in both MuJoCo [ 38 ] simulation and the real robot using directly accessible and reliable sensor signals with noise modeling. The results show comparable error levels across domains for diverse motions, indicating effective sim-to-real alignment. Detailed definitions of the sensor signals are provided in Sec. -A .

Reward Terms	Equation	Weight
Tracking
Body pos	$\displaystyle\exp\!\Big(-\big(\tfrac{1}{\|\mathcal{B}\|}\sum_{b\in\mathcal{B}}\\|\mathbf{p}^{\mathrm{ref}}_{b}-\mathbf{p}_{b}\\|^{2}\big)/0.3^{2}\Big)$	$1.0$
Body ori	$\displaystyle\exp\!\Big(-\big(\tfrac{1}{\|\mathcal{B}\|}\sum_{b\in\mathcal{B}}\\|\log(\phi^{\mathrm{ref}}_{b}\phi_{b}^{\top})\\|^{2}\big)/0.4^{2}\Big)$	$1.0$
Body lin. vel	$\displaystyle\exp\!\Big(-\big(\tfrac{1}{\|\mathcal{B}\|}\sum_{b\in\mathcal{B}}\\|\mathbf{v}^{\mathrm{ref}}_{b}-\mathbf{v}_{b}\\|^{2}\big)/1.0^{2}\Big)$	$1.0$
Body ang. vel	$\displaystyle\exp\!\Big(-\big(\tfrac{1}{\|\mathcal{B}\|}\sum_{b\in\mathcal{B}}\\|\bm{\omega}^{\mathrm{ref}}_{b}-\bm{\omega}_{b}\\|^{2}\big)/3.14^{2}\Big)$	$1.0$
Root pos (PMG)	$\displaystyle\exp\!\Big(-\\|\mathbf{p}^{\mathrm{ref}}_{\text{root}}-\mathbf{p}_{\text{root}}\\|^{2}/0.3^{2}\Big)$	$0.5$

TABLE A.5 : General reward terms.

Physical parameters (GMT)
Domain Randomization	Sampling Distribution
Static friction coefficients	$\mu_{\text{static}}\sim\mathcal{U}[0.3,\,1.6]$
Dynamic friction coefficients	$\mu_{\text{dynamic}}\sim\mathcal{U}[0.3,\,1.2]$
Restitution coefficient	$e_{\text{rest}}\sim\mathcal{U}[0,\,0.5]$
Default joint pos [rad]	$\Delta q^{default}\!\sim\mathcal{U}[-0.01,\,0.01]$
Default root ori [rad]	$\Delta\phi_{r,roll}\!\sim\mathcal{U}[-0.02,\,0.02]$ ,

TABLE A.6 : Domain randomization parameters. (

\mathcal{U}[\cdot]

: uniform distribution)

Adaptive sampling	Value
Bin size	$1s$
Blending hyperparameter	$\alpha=0.001$
Sampling prob clamp min/max	$[0.75/N_{bin},100/N_{bin}]$

TABLE A.7 : Adaptive sampling parameters.

Input Command / Ground Truth	5 Clips				20 Clips				40 Clips
Input Command / Ground Truth	SR	MPJPE	$\Delta$ vel	$\Delta$ acc	SR	MPJPE	$\Delta$ vel	$\Delta$ acc	SR	MPJPE	$\Delta$ vel	$\Delta$ acc
Raw reference / Raw reference	95.11%	40.93	7.445	2.460	91.45%	48.01	9.363	3.0292	88.18%	51.02	9.336	2.938
Physical reference / Raw reference	98.11%	38.23	6.412	2.360	94.09%	42.86	7.905	2.799	92.57%	44.64	7.880	2.709

TABLE A.8 : Effect of Dataset Size and Reference Type on Controller Performance

Fig. A.7 : One-hour continuous outdoor motion tracking without reset. Starting from a fully charged battery, the robot continuously performs motion tracking in an outdoor environment for one hour until the battery is depleted. Please refer to the supplementary video demo for the full execution.

Fig. A.8 : VR Headset Real-time Teleoperation on the Humanoid Robot. The raw SMPL motion is captured using the Pico VR Headset [ 51 ] and then retargeted to the humanoid robot via GMR [ 2 ] . The retargeted motion is further processed by the physical motion generation stage to produce physically consistent motions, which are finally tracked by the robot through the general motion tracking stage, enabling real-time teleoperation.

Explore similar work

Feb 27, 2026

OmniXtreme: Breaking the Generality Barrier in High-Dynamic Humanoid Control

High-fidelity motion tracking serves as the ultimate litmus test for generalizable, human-level motor skills. However, current policies often hit a "generality barrier": as motion libraries scale in diversity, tracking fidelity inevitably collapses - especially for real-world deployment of high-dynamic motions. We identify this failure as the result of two compounding factors: the learning bottleneck in scaling multi-motion optimization and the physical executability constraints that arise in real-world actuation. To overcome these challenges, we introduce OmniXtreme, a scalable framework that decouples general motor skill learning from sim-to-real physical skill refinement. Our approach uses a flow-matching policy with high-capacity architectures to scale representation capacity without interference-intensive multi-motion RL optimization, followed by an actuation-aware refinement phase that ensures robust performance on physical hardware. Extensive experiments demonstrate that OmniXtreme maintains high-fidelity tracking across diverse, high-difficulty datasets. On real robots, the unified policy successfully executes multiple extreme motions, effectively breaking the long-standing fidelity-scalability trade-off in high-dynamic humanoid control.

Yunshen Wang, Shaohang Zhu, Peiyuan Zhi +7

State Key Laboratory of General Artificial Intelligence, Beijing Institute for General Artificial Intelligence (BIGAI) · Joint Laboratory of Embodied AI and Humanoid Robots, BIGAI & Unitree Robotics · Shanghai Jiao Tong University +4

Humanoid Robot Control Humanoid Motion Tracking Sim-to-Real Transfer

Ask PDF

Jul 22, 2026

What Matters in Humanoid General Motion Tracking? An Empirical Study

Humanoid general motion tracking requires policies that can follow diverse whole-body references while maintaining balance. Building such policies involves many practical design choices, and their individual effects are often hard to assess. We address this issue with an empirical study of common modeling and training factors used in recent humanoid motion-imitation pipelines. To make the study controlled and reproducible, we developed YAHMP, an open-source modular framework for training, evaluating, and deploying whole-body motion tracking policies on the Unitree G1. Within YAHMP, we define a nominal configuration and compare variants that differ in motion-command representation, observation history, action representation, actuation profile, hand-force randomization during training, and training approach. We evaluate the resulting policies on a test set of retargeted human motions and compare the nominal policy with TWIST2 as an external baseline trained on the same motion set. The results distinguish choices with clear tracking effects from choices that mainly change actuation effort, training complexity, or physical interaction capability. Finally, we deploy YAHMP policies zero-shot on the real Unitree G1, demonstrating diverse whole-body motion tracking, balance under external perturbations, and forceful interaction.

Fabio Amadio, Enrico Mingo Hoffman

HUCEBOT · Inria, Université de Lorraine, CNRS, Nancy, France

Humanoid Robot Control Robotic Policy Evaluation Robot Imitation Learning Humanoid Motion Tracking

Ask PDF

Aug 13, 2026

HumanTracker: Towards Comprehensive and Human-Aligned Motion Tracking Benchmark

Humanoid motion tracking is central to teleoperation and whole-body imitation, yet evaluation often disagrees with what people perceive in videos. Kinematic errors average per-frame pose differences but miss the physical artifacts that matter most, particularly unstable support and incorrect contacts such as foot skating and mistimed touch-downs. Meanwhile, widely used test suites are small and lack the diversity needed to stress contact-rich, long-horizon behaviors. We introduce HumanTracker to make humanoid tracking evaluation both perceptually aligned and scalable. The HumanTracker benchmark contains approximately 153 hours of optical motion trajectories from multiple professional performers, organized into four motion families with text labels for fine-grained diagnosis. We further propose HumanScore, a preference-aligned metric trained on 12K motion pairs containing 24K motions. Across representative state-of-the-art trackers, HumanScore better predicts human preferences and reveals contact and stability failures that kinematic metrics often miss.

Dairu Liu, Zekun Qi, Jiayu Zeng +11

1Nankai University · 3Galbot · 2Tsinghua University +3

Human-in-the-Loop Evaluation Humanoid Motion Tracking Preference Learning

Ask PDF