Reusing a demonstrated gait can reduce the work of specifying the same motion for each biped. A video captures the timing of a step, but each robot must execute it with its own limbs and actuators. MotionSkill recovers a timed reference once from monocular video and adapts it to each target in simulation. It aligns root motion and body landmarks in a common frame and records which landmarks are available at each instant. At each source timestamp, the mapper preserves limb directions, uses the target segment lengths, and solves inverse kinematics under joint limits and closure constraints. In MuJoCo, we train a separate residual policy for each of ten bipeds on its mapped reference from one four-second walking clip. Each policy uses the measured robot state to correct joint commands during contact. From initial states held out during training, MotionSkill completed 98% of trials on the original video and 26% with the upper body occluded for one second.
Figures & tables
Fig. 1: MotionSkill overview. A walking video supplies one timed motion reference. The three poses show the direction of travel. Each of the ten simulated bipeds maps that reference to its own joint geometry and tracks it with a separately trained controller.
Fig. 2: MotionSkill framework and controller adapters. Source alignment supplies root motion, body points, and time. Mapping adapts these quantities to the target geometry. Joint references carry recorded and recovered motion into physical tracking. Travel and rhythm commands support the retained controller timing study. State feedback closes the physical loop in each case. Native limits and passive joints remain in the target model.
Study
Design
Video deployment
10 bodies, 2 inputs, 5 paired reset seeds per body
Recorded motion control
10 targets × 10 initial states × 20 s = 100 trials
Supporting analyses
184 timing trials and one original/occluded video pair
TABLE I: Evaluation design. Each video condition uses a single recovered reference and the same ten target bodies. Video results use held out initial states. The longer recorded walk and native policy trials measure stepping and timing.
Fig. 3: Stepping poses and leading ankle across ten bodies. A and B mark two recorded poses of each robot during the 20 s walking trial. The strip below each pair shows which ankle is farther forward from 4 to 6 s. Teal marks the left ankle and ochre the right. Distances give mean planar displacement over ten trials per body.
Body
Clean
Mask
Alt.
Travel (m)
G1
5/5
0/5
59
8.006
T1
5/5
0/5 ∗
58
6.465
Apollo
5/5
1/5
59
10.668
Berkeley
5/5
0/5
58–59
5.176
MicroDuck
5/5
0/5
52–58
1.141
H1
5/5
4/5
57–59
9.972
TABLE II: Physical execution on ten bipeds. Clean and Mask give completions out of five held out initial states on one video. In cells marked ∗ , mapping failed before simulation. All five planned trials count as failures. For the 20 s recorded reference, Alt. gives the range of leading foot changes and Travel gives the mean planar displacement of ten trials per body. All ten trials complete on every target.
Fig. 5: Timing analysis on retained native controllers. (a) We compare native velocity (NV), geometry with a native clock (NC), and continuous source phase (M) across 120 fixed cadence trials and two friction settings. (b) Forty changing cadence trials isolate continuous reference timing. Paired comparisons use the same initial state and native controller. Points show all five seeds with mean and sample standard deviation. (c) Standardized foot separation for G1 (seed 0). The dashed gray line is the source reference. Ochre triangles (NC) and teal circles (M) show the measured response under the native clock and continuous source phase, respectively.
Fig. 6: Video source alignment and embodiment mapping. Original and occluded inputs at t=1.2 and 2.4 s are shown with aligned motion and target joint references. Read each row from the input frame through the recovered motion to the mapped joint target for G1.