Human video offers a scalable source of robot demonstrations, yet most human-to-humanoid retargeting methods assume a legged robot with human-like kinematics. This assumption does not hold for mobile-base humanoids equipped with a wheeled base, vertical lift, and two arms. Human walking must be expressed through base motion, while torso bending may require coordinated lift and arm motion. We address this mismatch with a task-conditioned framework that assigns reconstructed human motion to base, lift, and arm responsibilities before robot-specific realization. The allocator preserves the human-derived path, stabilizes heading, separates turn and translation when needed, retimes commands to satisfy base limits, and repairs lift and arm trajectories. A deployment adapter then converts the reference to 50 Hz commands using stationary-base detection, deadband and slew-rate filtering, time-consistent playback scaling, and separate linear and angular gains. We evaluate the resulting references with human-derived task-space comparisons, policy-free simulation replay, and a qualitative execution on a physical robot.
Figures & tables
Fig. 1: Embodiment-aware allocation of reconstructed human motion into arm, lift, and mobile-base intents. A single whole-body motion may require coordinated actuation across several robot subsystems.
Fig. 2: Task-conditioned embodiment-aware retargeting pipeline. Human video is reconstructed as 3D motion and decomposed into base, lift, and arm targets. These targets are optimized under feasibility, task-space, and smoothness objectives, then realized and repaired by a GMR-based adapter for simulation playback or command export.
ID
Task
Description
Primary Motion Cue
Allocation Target
01
Slow walk
Forward walking at a moderate pace
Smooth root translation
Base
02
Run
Fast forward locomotion
High root velocity
Base (speed-adaptive)
03
Side walk
Lateral walking without a dominant turn
Lateral root motion
Lateral base motion
04
Sideways-turn walk
Lateral locomotion combined with reorientation
Root translation + yaw
Base translation + rotation
05
Bend-and-reach I
Forward reach with moderate torso lowering
Torso pitch + low wrist
Lift + arms
06
Bend-and-reach II
Deeper forward reach at a lower height
Deep reach + arm extension
Lift + arms + base assist
TABLE I: TASK INDEX AND INTENDED EMBODIMENT-AWARE MOTION ALLOCATION.
Fig. 3: Example processing sequence for a bend-and-reach motion: RGB human demonstration, GVHMR 3D reconstruction, and the retargeted mobile-base humanoid reference in simulation.
Fig. 4: The same bend-and-reach reference executed in simulation and on the physical robot for Task 06. The sequence retains the intended lift and bilateral arm coordination in both environments.
Method
Locomotion
Manip./reach
All
01–04
05–09
01–09
Ours
0.355
0.295
0.322
Arm-only
1.000
0.566
0.759
Base-only
0.355
0.868
0.640
No-lift
0.355
0.440
0.402
TABLE II: Human-derived continuous task-space evaluation on scenarios 01–09. Targets are extracted from GVHMR-reconstructed SMPL motion rather than from our generated reference. Entries are normalized costs; lower is better.
Method
Loc. root
Wrist
Wrist- z
Lift short.
Primary gap
(m)
(m)
(m)
(m)
Ours
0.641
0.346
0.143
0.012
balanced
Arm-only
1.610
0.352
0.164
0.125
base/lift
Base-only
0.641
1.726
0.643
0.125
wrist/lift
No-lift
0.641
0.346
0.164
0.125
lift
TABLE III: Component-wise human-target ablation. Locomotion root is averaged over scenarios 01–04, wrist metrics over scenarios 05–09, and lift shortfall over scenarios 05, 06, and 09. Lower is better.
Group
Sc.
Completed
Body mean (m)
Anchor mean (m)
Steps
Locomotion
4
4/4
0.012
0.175
1427
Bend/reach
2
2/2
0.024
0.020
551
Base+arms
2
2/2
0.028
0.048
654
Low pick
1
1/1
0.029
0.032
315
All 01–09
9
9/9
0.020
0.097
2947
TABLE IV: Policy-free simulation replay validation. Completion means no early termination during reference playback, not learned-policy success. Lower errors are better.
Institute of Humanoid Robots, Department of Precision Machinery and Precision Instrumentation, University of Science and Technology of China, Hefei, Anhui 230026, China