Enabling humanoid robots to perform agile and adaptive interactions has long been a core challenge in robotics. Current approaches are bottlenecked by either the scarcity of realistic interaction data or the need for meticulous, task-specific reward engineering, which limits their scalability. To narrow this gap, we present HumanX, a unified framework that compiles human video into generalizable, real-world interaction skills for humanoids, without per-task design. HumanX integrates two co-designed components: XGen, an anchor-centric data generation pipeline that yields physically plausible robot interaction data from monocular video and augments it to create a diverse, scalable training dataset; and XMimic, an interaction-prioritized learning framework to achieve generalizable interaction skills from seconds of demonstration. Evaluated across five distinct domains--basketball, football, badminton, cargo pickup, and reactive fighting--HumanX successfully acquires 9 different skills and transfers them zero-shot to a physical Unitree G1 humanoid. The learned capabilities include complex maneuvers such as pump-fake turnaround fadeaway jumpshots without any external perception, as well as interactive tasks like sustained human-robot passing sequences over 10 consecutive cycles--learned from a single video demonstration. Our experiments show that HumanX achieves higher generalization success than prior methods, demonstrating a scalable pathway within a class of anchorable interactions for learning versatile, real-world robot interactive skills.
Figures & tables
Figure 2: Overview of XGen . The pipeline begins by estimating SMPL‑based human motion from video and retargeting it to the humanoid’s morphology. The video is segmented into contact and non‑contact phases. For the contact phase, a body anchor (e.g., the midpoint between the two palms) is predefined. The object trajectory is then synthesized by transforming the object according to the anchor, followed by contact optimization to refine the robot poses. During the non‑contact phases, diverse object trajectories are generated via physical simulation. Complete interaction trajectories are obtained by concatenating and smoothly interpolating the phases.
Figure 2
Figure 5: XMimic is a unified framework for learning diverse interaction skills. To reinforce interaction and prevent overfitting to isolated body motions, it employs rewards that prioritize interaction and introduces interaction perturbations and interaction-vector termination during simulation. The training follows a two-stage pipeline: a privileged teacher policy first learns the skill using full state information, which then guides the training of a deployable student policy.
Method
Basketball Catch-Shot
Badminton Hitting
Cargo Pickup
SR ↑
GSR ↑
Eo↓ (m)
Eh↓ (cm)
SR ↑
GSR ↑
Eo↓ (m)
Eh↓ (m)
SR ↑
GSR ↑
Eo↓ (cm)
Eh↓ (cm)
SkillMimic [ 59 ]
0.0%
0.0%
1.25 ( ± 0.26)
10.3 ( ± 1.37)
68.2%
30.9%
2.94 ( ± 0.91)
0.48 ( ± 0.13)
0.4%
0.0%
25.1 ( ± 0.06)
28.3 ( ± 2.13)
OmniRetarget [ 68 ]
0.0%
0.0%
1.18 ( ± 0.15)
7.83 ( ± 0.95)
75.8%
20.4%
1.25 ( ± 1.15)
0.15 ( ± 0.10)
0.0%
0.0%
10.8 ( ± 0.00)
13.3 ( ± 1.57)
HDMI [ 61 ]
53.1%
2.4%
0.70 ( ± 0.11)
10.8 ( ± 2.41)
90.0%
25.3%
1.64 ( ± 0.66)
0.07 ( ± 0.02)
95.8%
1.8%
1.79 ( ± 0.22)
13.1 ( ± 1.28)
XMimic (Base)
93.4%
4.9%
0.69 ( ± 0.14)
6.36 ( ± 1.51)
97.1%
41.6%
1.32 ( ± 0.44)
0.12 ( ± 0.07)
100%
50.9%
1.56 ( ± 0.30)
6.58 ( ± 0.76)
+ IP
86.8%
13.5%
0.70 ( ± 0.08)
7.46 ( ± 0.46)
94.0%
60.5%
1.03 ( ± 0.57)
0.10 ( ± 0.05)
100%
94.5%
1.62 ( ± 0.03)
10.5 ( ± 1.23)
Table 1: Main Simulation Results . SR, Eo , and Eh measure the success rate on the original data, the object position tracking error, and the key-body position tracking error, respectively, while GSR measures the success rate of skill generalization.
Figure 5
Figure 8: Real Robot Experiment on Blind Basketball Skills . The policy relies solely on proprioception to control objects, achieving diverse, highly dynamic, and complex interactions.
Figure 9: Real Robot Experiment on MoCap-based Interaction Skills . When utilizing MoCap system to perceive object or human motion, our method enables sustained interaction, demonstrating high precision, agility, robustness, and generalization capability. Notably, each task shown here is learned from a single demonstration video without any task-specific reward.
Basketball Catch-Pass (MoCap)
41 / 50
Reactive Fighting (MoCap)
37 / 50
Jumpshot (NEP)
8 / 10
Cargo Pickup (MoCap)
43 / 50
Dribble (NEP)
8 / 10
Basketball Pickup (NEP)
10 / 10
Football Kicking (MoCap)
42 / 50
Pump-fake (NEP)
9 / 10
Layup (NEP)
7 / 10
Table 2: Quantitative Results on Real Robot Experiments .
Figure 9
Method
Football Kicking
Badminton Hitting
w/o Tea-Stu
74.2% GSR
52.4% GSR
w/ Tea-Stu
93.1% GSR
84.3% GSR
Table 3: Evaluation on Multi-Pattern Interaction Skills in Simulation . Each skill contains three distinct interaction patterns from three human video.
Figure 12: Diverse Skill Patterns . XMimic supports learning multiple interaction patterns for a single skill, allowing the policy to autonomously select the most suitable pattern in response to object state. ( Left ): football‑kicking patterns. ( Right ): badminton‑hitting patterns.