KungfuAthleteBot: Learning High-Dynamic Motion from Videos with Unified Robust Recovery

Zhongxiang Lei; Lulu Cao; Xuyang Wang; Tianyi Qian; Jinyan Liu; Xuesong Li

cs.ROFeb 14, 2026

KungfuAthleteBot: Learning High-Dynamic Motion from Videos with Unified Robust Recovery

Authors: Zhongxiang Lei, Lulu Cao, Xuyang Wang, Tianyi Qian, Jinyan Liu, Xuesong Li

Organizations: Beijing Institute of Technology · QIYUAN Lab

Abstract

Learning highly dynamic motions from video provides a scalable alternative to expensive data collection for humanoid robots, yet it is challenged by height floating, missing kinematic and force information, and limited robustness. We present KungfuAthleteBot (KAB), a unified framework that systematically addresses these issues. We first build a high-dynamic motion dataset in collaboration with professional martial artists, alleviating data scarcity while improving motion quality. To eliminate height drift and global jitter during both aerial and landing phases, we introduce a physics-guided parabolic trajectory correction that enforces physical plausibility. To bridge the discrepancy between human and robot actuation in highly dynamic behaviors, we design a three-stage curriculum that progressively enables stable acquisition of such motions. In addition, we propose a direct, teacher-free training paradigm that jointly learns disturbance rejection and fall recovery within a single policy. To the best of our knowledge, KAB achieves the fastest recovery time reported for a unified policy ( $\approx 0.7$ s). Experiments on a real humanoid robot demonstrate successful learning of highly dynamic motions from video and strong robustness, including rapid recovery under severe disturbances.

Figures & tables

Figure 1: Learning high-dynamic motion from video. (a) A dataset extracted from videos of national-level martial artists. (b) Video-derived motion carries no actuation information, so direct mimic-based tracking (e.g., BeyondMimic) fails on such references. Our curriculum (Sec. 4.3 ) and low-kinetic-energy sampling (Sec. 4.2 ) let the policy explore feasible actuation patterns. (c) A single policy performs fall recovery and motion tracking, and (d) rejects external disturbances.

Dataset	Body Lin Vel ( $w$ )	Body Ang Vel ( $w$ )	Frame Range
AMASS	$0.188\pm 0.165$	$0.510\pm 0.309$	$[2,8788]$
PHUMA	$0.193\pm 0.130$	$0.585\pm 0.368$	$[49,222]$
LAFAN1	$0.790\pm 0.251$	$1.962\pm 0.553$	$[3065,9171]$
KungfuAthlete (Ground)	$0.496\pm 0.324$	$1.537\pm 0.959$	$[24,15792]$
KungfuAthlete (Jump)	$0.948\pm 0.473$	$2.832\pm 1.431$	$[25,2842]$

Table 1: Kinematic statistics of motion datasets. Entries are means over the whole body, denoted by

w

in the column headings; frame range is given in frames.

Figure 2: Existing methods exhibit significant height floating when reconstructing jumping motions; prior work (e.g. PHUMA) therefore discards such data, wasting jump-related samples (top row). We instead enforce a physically consistent parabolic correction to refine the falling trajectories of jumping motions.

Figure 3: Overview of the training pipeline. Top-left: low-kinetic-energy (LKE) sampling draws video motion from dynamically feasible frames after a tracking failure. Bottom-left: the initial state $\mathbf{q}_{0}$ is drawn from a Bernoulli mixture over a reference frame and a gravity-based randomized fall state from $\mathcal{D}_{\mathrm{GRSI}}$ . Centre: reference-conditioned tracking rewards with recovery penalties (feet height, knee jerk, feet slip, root rotation). Right: the distributional actor–critic loop (FastSAC, Sec. 6 ).

Name	Expression	$\tau_{\mathrm{stageI}}$	$\tau_{\mathrm{stageII}}$	$\tau_{\mathrm{stageIII}}$
Anchor Z Pos. Deviation	$\mathbb{I}\Big(\\|\mathbf{p}^{\mathrm{ref}}_{anchor_{z}}-\mathbf{p}^{\mathrm{robot}}_{anchor_{z}}\\|_{2}>\tau_{\mathrm{pos_{anchor_{z}}}}\Big)$	0.5	0.3	0.5
Anchor Z Ori. Deviation	$\mathbb{I}\Big(\|(\mathbf{R}^{\mathrm{ref}}_{anchor})^{\top}\mathbf{g}-(\mathbf{R}^{\mathrm{robot}}_{anchor})^{\top}\mathbf{g}\|_{z}>\tau_{\mathrm{ori_{anchor}}}\Big)$	0.8	0.6	0.6
Anchor Ori. Deviation	$\mathbb{I}\Big(\|(\mathbf{R}^{\mathrm{ref}}_{anchor_{xy}})^{\top}\mathbf{g}-(\mathbf{R}^{\mathrm{robot}}_{anchor_{xy}})^{\top}\mathbf{g}\|_{z}>\tau_{\mathrm{ori}}\Big)$	/	0.25	0.6
Ankle Z Pos. Deviation	$\mathbb{I}\Big(\\|\mathbf{p}^{\mathrm{ref}}_{ankle_{z}}-\mathbf{p}^{\mathrm{robot}}_{ankle_{z}}\\|_{2}>\tau_{\mathrm{pos_{ankle_{z}}}}\Big)$	1.25	1.0	1.25
Wrist Z Pos. Deviation	$\mathbb{I}\Big(\\|\mathbf{p}^{\mathrm{ref}}_{wrist_{z}}-\mathbf{p}^{\mathrm{robot}}_{wrist_{z}}\\|_{2}>\tau_{\mathrm{pos_{wrist_{z}}}}\Big)$	0.5	0.5	0.5
Body Dof Deviation	$\mathbb{I}\Big(\exists b\in\mathcal{B}:\|\mathbf{p}^{\mathrm{ref}}_{k,b}-\mathbf{p}^{\mathrm{robot}}_{k,b}\|>\tau_{\mathrm{body}}\Big)$	$\pi/2$	$\pi/8$	$\pi/6$

Table 2: Bad-tracking termination conditions.

Parameter Item	Moderate	Aggressive
Push Lin. Vel.	$xy$ : $\pm 0.5$ , $z$ : $\pm 0.2$	$xy$ : $\pm 0.75$ , $z$ : $\pm 0.3$
Push Ang. Vel.	$xy$ : $\pm 0.52$ , $z$ : $\pm 0.78$	$xy$ : $\pm 0.785$ , $z$ : $\pm 1.17$
Rest Pos.	$/$	$\pm 0.15$
Rest Lin. Vel.	$/$	$xy$ : $\pm 0.75$ , $z$ : $\pm 0.3$
Rest Ang. Vel	$/$	$xy$ : $\pm 0.78$ , $z$ : $\pm 1.17$
Body Mass	1	[0.6,1.1]

Table 3: Configurations for noise and domain randomization.

Method	Video ref.	Aerial repair	Energy init.
KungfuBot Xie et al. (2025)	✓	✗	✗
OmniXtreme Wang et al. (2026)	✗	–	✗
KAB (ours)	✓	✓	✓

Table 4: Positioning on the video-to-robot pathway. Prior work assumes the processed reference is either learnable or safely discardable; we target the regime where it is neither. ✓ explicit support, ✗ absent, ‘–’ not applicable.

Motion ID	Method	$E_{\text{mpjpe}}$	$E_{\text{mpjve}}$	succ
78	TWIST	85.2 $\pm$ 23.8	18.9 $\pm$ 9.7	0/6
	GMT	68.3 $\pm$ 17.3	11.7 $\pm$ 4.8	0/6
	SONIC	71.9 $\pm$ 17.2	4.9 $\pm$ 4.6	0/6
	KAB	24.7 $\pm$ 12.1	5.4 $\pm$ 4.0	6/6
213	TWIST	87.4 $\pm$ 21.0	21.1 $\pm$ 9.8	0/6
	GMT	52.9 $\pm$ 12.8	9.7 $\pm$ 5.3	0/6

Table 5: Is the dataset challenging, and can KAB track it?

E_{\text{mpjpe}}

(

10^{-3}

rad),

E_{\text{mpjve}}

(

10^{-3}

rad/frame) and success rate over six trials. Motion IDs index clips in the released dataset. The same metrics are reported for KAB, to answer directly whether our own method can track these motions.

Panel of Figure 4 — Figure 4: (a) Reward curves of the three-stage training and the single-stage baseline on Motion 468. (b) Episode length under LKE and adaptive sampling in Stage II on Motion 278. LKE reaches usable tracking behaviour markedly earlier than error-driven adaptive sampling.

Method	Reference- free	Tracking & Recovery	No Get-Up Cmd.	Recovery Time (s)
HumanUP He et al. (2025b)	✓	✗	✗	6.708 $\pm$ 0.987
HoST Huang et al. (2025)	✓	✗	✗	2.055 $\pm$ 0.310
FIRM Xu et al. (2025)	✗	✗	✓	1.729 $\pm$ 0.424
BFM-Zero Li et al. (2025b)	✗	✓	✓	1.566 $\pm$ 0.065
Heracles Tao et al. (2026)	✗	✓	✓	2.471 $\pm$ 0.139
StableMimic Wu et al. (2026)	✗	✓	✓	n/r

Table 6: Comparison of fall recovery performance and capability unification. Within the methods listed, only KAB combines a reference-free recovery objective with unified tracking and recovery, and KAB attains the shortest recovery time under the protocol above.

Figure 5: Fall recovery from diverse fallen configurations. Top four rows: MuJoCo recovery sequences covering prone, supine and two twisted joint configurations with irregular limb postures. Bottom two rows: the corresponding real-world recovery on the Unitree G1.

Panel of Figure 6 — Figure 6: Training curves on Motion 1307. (a,b) Adding the recovery rewards does not change the episode length or the tracking error relative to the tracking-only baseline. (c,d) The full unified configuration matches the baseline’s episode length and tracking accuracy, at the cost of slower convergence ( $\sim$ 16k vs. $\sim$ 2.5k iterations). (e) Recovery components without mixed training: with no termination tolerance the episode length stays at 1, and with a 3 s tolerance it converges to only $\sim$ 150 of the full 500 steps, so the policy survives but never stands up and tracks. (f) Removing the height reward leaves partial tracking (300–400 steps) without recovery, while removing the root-orientation reward causes early collapse at $\sim$ 5k iterations.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Name	Expression	Weight
Stage-agnostic tracking reward (Stages I, II, III)
Body Dof Match	$\displaystyle\min\cos\left(\theta_{\mathrm{des}}-\theta\right)$	3.0
Penalty Action Rate	$\displaystyle\sum_{j}\\|a_{j}-a_{j}^{\mathrm{prev}}\\|^{2}$	-0.01
Limits DOF Position	$\displaystyle\sum_{j}\left[(\text{clip}(p_{j}-\text{upper}_{j},0))+(\text{clip}(\text{lower}_{j}-p_{j},0))\right]$	-10.0
Penalty Self Collision	$\mathbf{1}_{\text{self-collision}}$	-10.0
Stage II additional tracking reward

Appendix

Table 7: Physics-stability motion-tracking reward

r_{\text{mt}}

. Rows under “Stage-agnostic” are active in all stages; rows under “Stage II additional” are added on top of them in Stage II. The recovery terms are active whenever the recovery indicator is triggered.

Name	Expression	Weight
Motion Relative Body Position Error	$\displaystyle\exp\Bigg(-\frac{1}{\sigma^{2}}\,\frac{1}{B}\sum_{b}\\|\mathbf{p}_{b}^{\mathrm{rel}}-\hat{\mathbf{p}}_{b}\\|^{2}\Bigg)$	1.0
Motion Relative Body Orientation Error	$\displaystyle\exp\Bigg(-\frac{1}{\sigma^{2}}\,\frac{1}{B}\sum_{b}\\|\mathrm{quat\_error}(\mathbf{q}_{b}^{\mathrm{rel}},\hat{\mathbf{q}}_{b})\\|^{2}\Bigg)$	1.0
Motion Global Body Position Error	$\displaystyle\exp\Bigg(-\frac{1}{\sigma^{2}}\,\frac{1}{B}\sum_{b}\\|\mathbf{p}_{b}^{\mathrm{glo}}-\hat{\mathbf{p}}_{b}\\|^{2}\Bigg)$	0.5
Motion Global Body Orientation Error	$\displaystyle\exp\Bigg(-\frac{1}{\sigma^{2}}\,\frac{1}{B}\sum_{b}\\|\mathrm{quat\_error}(\mathbf{q}_{b}^{\mathrm{glo}},\hat{\mathbf{q}}_{b})\\|^{2}\Bigg)$	1.0
Motion Global Body Angular Velocity	$\displaystyle\exp\Bigg(-\frac{1}{\sigma^{2}}\,\frac{1}{B}\sum_{b}\\|\boldsymbol{\omega}_{b}-\hat{\boldsymbol{\omega}}_{b}\\|^{2}\Bigg)$	0.4
Motion Global Body Linear Velocity	$\displaystyle\exp\Bigg(-\frac{1}{\sigma^{2}}\,\frac{1}{B}\sum_{b}\\|\boldsymbol{l}_{b}-\hat{\boldsymbol{l}}_{b}\\|^{2}\Bigg)$	1.0

Appendix

Table 8: Motion tracking reward borrowed from BeyondMimic,

r_{\text{mt}}

Group	Term
Actor & critic	motion command (reference joint targets)
	reference body orientation (anchor frame)
	base angular velocity
	joint positions
	joint velocities
	previous action

Appendix

Table 9: Observation terms. Actor and critic share the reference-conditioned and proprioceptive terms; the critic additionally receives privileged terms that are unavailable at deployment. Noise is injected into actor observations only.

Figure 7: Motion 78: a 720° spin in the air, followed by a 360° kick.

Figure 8: Motion 1080: a kick followed by a roundhouse kick.

Humanoid Robotics

Humanoid robotics develops human-shaped robots that move, manipulate objects, and interact with people in human environments.

See all 143 papers

KungfuAthleteBot: learning high-dynamic humanoid motion from video with unified robust recovery
Oct 2, 2026Zhongxiang Lei, Lulu Cao, Xuyang Wang +3RL for Robotics Robot Imitation Learning
Beijing Institute of Technology · Equal contribution · QIYUAN Lab
KungfuBot: Physics-Based Humanoid Whole-Body Control for Learning Highly-Dynamic Skills
Jun 15, 2025Weiji Xie, Jinrui Han, Jiakun Zheng +6Humanoid Robot Control Robot Imitation Learning
Institute of Artificial Intelligence (TeleAI), China Telecom · Shanghai Jiao Tong University · East China University of Science and Technology +2
Humanoid Badminton: Learning Dynamic Racket Skills from Limited Human Motion Data
Sep 25, 2026Jingzhi Cui, Zhexiong Wang, Bangjie Xu +9Robot Skill Learning Robotics
Tsinghua University · Hong Kong Embodied AI Lab · Zhejiang University +3

Oct 2, 2026J/K move · Enter open · S save

KungfuAthleteBot: learning high-dynamic humanoid motion from video with unified robust recovery

Zhongxiang Lei, Lulu Cao, Xuyang Wang, Tianyi Qian+2

Beijing Institute of Technology · Equal contribution · QIYUAN Lab

RL for Robotics Robot Imitation Learning Humanoid Motion Tracking Humanoid Robotics

Open paper AskPDF arXiv

Video is an abundant, inexpensive source of human motion data that is rich in extreme athletic behaviors. Making it usable for humanoid robots, however, is not a matter of simply retargeting a reconstructed trajectory: video-derived motion is physically inconsistent, devoid of actuation information, and says nothing about failure or recovery. We present KungfuAthleteBot (KAB), a framework that treats learning high-dynamic motion from video as the central problem and resolves each of these three failure modes in turn. (C1) We build the KungfuAthlete dataset from videos of national-level martial artists and introduce a physics-guided parabolic trajectory correction that removes height floating, ground penetration, and high-frequency jitter from reconstructed aerial and landing phases. (C2) Because video carries no force information, strict tracking of a reconstructed trajectory is dynamically infeasible, and error-driven initialization keeps re-launching the policy from infeasible aerial poses. We introduce physics-driven pseudo-low-kinetic-energy (LKE) sampling, our central mechanism for making such references learnable: it biases initialization towards dynamically feasible states, letting the policy discover feasible actuation patterns instead of imitating infeasible ones. (C3) Finally, we introduce a direct training paradigm in which disturbance rejection and fall recovery are learned inside the same policy that tracks the video motion, requiring no recovery reference data and no manual mode switching. On a humanoid robot, KAB learns dynamic skills from video and recovers from arbitrary falls in about 0.7 s, the fastest reported recovery for a unified policy. Ablations on the unified policy confirm the necessity of its components, supporting the view that repairing and compensating video data, rather than only collecting more of it, is what unlocks high-dynamic humanoid skills.

Abstract

Humanoid Robotics

Explore similar work

KungfuAthleteBot: learning high-dynamic humanoid motion from video with unified robust recovery

KungfuBot: Physics-Based Humanoid Whole-Body Control for Learning Highly-Dynamic Skills

Humanoid Badminton: Learning Dynamic Racket Skills from Limited Human Motion Data