Achieving autonomous and versatile whole-body loco-manipulation remains a central barrier to making humanoids practically useful. Yet existing approaches are fundamentally constrained: retargeted data are often scarce or low-quality; methods struggle to scale to large skill repertoires; and, most importantly, they rely on tracking predefined motion references rather than generating behavior from perception and high-level task specifications. To address these limitations, we propose ULTRA, a unified framework with two key components. First, we introduce a physics-driven neural retargeting algorithm that translates large-scale motion capture to humanoid embodiments while preserving physical plausibility for contact-rich interactions. Second, we learn a unified multimodal controller that supports both dense references and sparse task specifications, under sensing ranging from accurate motion-capture state to noisy egocentric visual inputs. We distill a universal tracking policy into this controller, compress motor skills into a compact latent space, and apply reinforcement learning finetuning to expand coverage and improve robustness under out-of-distribution scenarios. This enables coordinated whole-body behavior from sparse intent without test-time reference motions. We evaluate ULTRA in simulation and on a real Unitree G1 humanoid. Results show that ULTRA generalizes to autonomous, goal-conditioned whole-body loco-manipulation from egocentric perception, consistently outperforming tracking-only baselines with limited skills.
Figures & tables
Fig. 2 : ULTRA follows four stages: ( i ) Neural Retargeting: an RL policy converts MoCap data into physically feasible G1 rollouts with augmentation; ( ii ) Tracking: a privileged teacher tracks these rollouts using full state and references; ( iii ) Distillation: we distill the teacher into a multimodal student for realistic sensing and sparse goals, with additional RL finetuning; ( iv ) Deployment: the student runs under real sensing, supporting depth input or MoCap-based state estimation.
Fig. 3 : Qualitative comparison of our retargeting and OmniRetarget [ 2 ] at the same frame/sequence. Top: final frame; the baseline shows undesired standing foot placement. Bottom: a contact frame; ours yields more stable contacts.
In-Distribution (ID)
Out-of-Distribution (OOD)
Succ↑
Succ↑
Humanoid
Object
Succ↑
Succ↑
Humanoid
Object
Method
(Humanoid)
(+Object)
Eg-mpjpe↓
Empjpe↓
Ejitter↓
Epos↓
Erot↓
(Humanoid)
(+Object)
Eg-mpjpe↓
Empjpe↓
Ejitter↓
Epos↓
Erot↓
(a) Unified Multimodal Controller
ULTRA ( Ours )
67.30 ± 0.12
57.44 ± 0.40
13.49 ± 0.14
0 5.89 ± 0.02
0 6.27 ± 0.00
53.42 ± 0.21
65.44 ± 0.37
70.57 ± 0.54
52.00 ± 0.44
35.55 ± 0.23
14.67 ± 0.08
0 6.81 ± 0.01
56.52 ± 0.35
67.60 ± 0.58
(b) Privileged Teacher
ULTRA Teacher
97.57 ± 0.05
89.79 ± 0.11
12.98 ± 0.30
0 5.64 ± 0.05
14.81 ± 0.08
17.15 ± 0.03
23.28 ± 0.33
97.12 ± 0.43
81.33 ± 0.78
19.14 ± 0.48
0 7.94 ± 0.11
15.91 ± 0.08
25.57 ± 0.28
33.49 ± 0.37
TABLE I : Motion-tracking evaluation in IsaacGym. All methods are trained/evaluated on our data unless noted. Green highlights our primary tracking controller.
Fig. 4 : Zero-shot augmentation with the retargeting policy. Left: trajectory scaling. Right: object scaling. Motions remain plausible, enabling scalable data augmentation.
Table 5
Fig. 6 : Sim-to-sim comparison for egocentric goal following. Blue/green: point cloud without/with noise; yellow: object goal. Top: without finetuning. Bottom: with finetuning.
RL finetuning
ID Goals
OOD Goals
Points
Position
Points
Position
✗
16 / 20
14 / 20
5 / 20
4 / 20
✓
19 / 20
16 / 20
9 / 20
12 / 20
Δ (RL gain)
+18.8%
+14.3%
+80.0%
+200.0%
TABLE III : Sim-to-sim success rate on Mujoco across goal types with in-distributional (ID) goals from training and out-of-distribution (OOD) goals with random offsets, and across perception with egocentric point clouds or object position with no shape. Policies are trained in IsaacGym and evaluated in MuJoCo with 20 selected motion per setting.
Setting
Vertical
Lateral
Dense Reference Tracking
73% (19/26)
Sparse Goal Following (MoCap)
80% (8/10)
90% (9/10)
Sparse Goal Following (Egocentric)
50% (5/10)
60% (6/10)
TABLE IV : Real-world success rates on the OMOMO subset using a Unitree G1 humanoid. Each task is evaluated over two trials. MoCap provides object pose tracking for non-egocentric control modes, while the egocentric setting relies only on onboard sensing. MoCap is used for success evaluation in all settings. Dense reference tracking is direction-agnostic and thus reported as a single success rate.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Unitree G1 Link
SMPL-X Index
SMPL-X Body Part
left_hip_yaw_link
1
Left hip
left_knee_link
2
Left knee
left_ankle_roll_link
3
Left ankle
right_hip_yaw_link
5
Right hip
right_knee_link
6
Right knee
right_ankle_roll_link
7
Right ankle
Appendix
TABLE A: Correspondence between Unitree G1 key links and SMPL-X body indices (52-body model) used for motion retargeting.
Term
Expression
Weight
Body Tracking:
Joint position
exp(−kp⋅mean(∥pl−p^l∥22))
kp=10.0
Joint rotation
exp(−kr⋅mean(∥q−q^∥22))
kr=5.0
Body velocity
exp(−kpv⋅mean(∥p˙l−p˙^l∥22))
kpv=0.1
Joint velocity
exp(−krv⋅mean(∥q˙−q˙^∥22))
krv=0.001
Object Tracking:
Appendix
TABLE B: Tracking reward components for the teacher policy. pl,q,p˙l,q˙ denote the simulated body joint positions, joint rotation, and their velocities; p^l,q^,p˙^l,q˙^ are the corresponding reference quantities. po,qo,p˙o denote the simulated object position, rotation (quaternion), and linear velocity; p^o,q^o,p˙^o are the corresponding references. ∠(qo,q^o) is the relative rotation angle and huber(⋅) is the Huber loss. δij denotes the palm-to-surface distance between palm point i and object surface sample j , with weights wij ; hats denote reference values. cl∈{0,1} is a binary contact indicator for link l . The scalars k⋅ are temperature coefficients. All reward terms are multiplied together.
Term
Expression
Weight
Velocity Penalties:
Base linear velocity
∥vbase∥2
−0.1
Base angular velocity
∥ωbase∥22
−0.01
Joint velocity
∥q˙∥22
−0.0004
Smoothness Penalties:
Action rate
∥at−at−1∥2
−0.1
Appendix
TABLE C: Smoothness and regularization rewards for the teacher policy. All terms are penalties with negative weights wi<0 . vbase and ωbase are the base linear and angular velocities. at is the action at time t ; q˙t is the joint velocity at time t ; and ωt is the base angular velocity at time t . τ denotes joint torques and ⊙ is elementwise multiplication. qlim and τlim are per-joint position and torque limits. \mathds1contact is an indicator for foot contact; fxy and fz are the horizontal and vertical components of the contact force. dfeet and dknee are the distances between the two feet and the two knees, and clamp(⋅) clips the distance to the specified interval. g⊥feet measures foot tilt relative to gravity. ref_stand and sim_stand indicate standing phases in the reference and simulation. hterm is a swing-foot height term, c is a clearance/contact-related term, and gswing gates the swing penalty, which is active only during reference indicating swing.
Category
Feature
Dimension
Description
osim
Root height
1
Root height above ground
Local body positions p
114
39 bodies × 3 (root removed)
Local body rotations R
234
39 bodies × 6 (tan-norm)
Local body velocities p˙
117
39 bodies × 3
Local body angular vel. ω
117
39 bodies × 3
Contact indicators c
39
Binary contact flags
Appendix
TABLE D: Teacher policy observation space. At time t , ot=[otsim,otref,otΔ,otig] . otsim contains simulated quantities: root height; per-body local positions p , rotations R in 6D tan-norm, linear velocities p˙ , angular velocities ω , contact flags c∈{0,1} ; and joint states (positions q , velocities q˙ , actions a , torques τ , and history). otref contains corresponding reference targets (hats), e.g., body pose (p^,R^) and object pose/velocity (x^o,q^o,v^o,ω^o) . otΔ stores residuals between simulation and reference ( e.g., p−p^ and velocity errors), and otig stores interaction-graph features based on SDF distances between body points and the object surface and their residuals. Two observations based on next 1-frame and next 16-frame reference are concatenated, yielding total dimension 4052.
Hyperparameter
Value
Learning rate
2×10−5
Clip ratio ϵ
0.2
GAE λ (tau)
0.95
Discount γ
0.99
Horizon length
32
Mini-batch size
16384
Appendix
TABLE E: PPO hyperparameters.
Term
Definition
Weight / schedule
Total loss
L=λELE+λKLLKL+λSLS+λALA+λGLG+λPLP
see below
Expert imitation
LE=μ−aexp22
λE=1.0
Latent KL alignment
LKL=DKL(qϕ(z)∥pθ(z))
λKL=0.001+(0.1−0.001)g(s)
Latent smoothness
LS=(μp+μe)t−(μp+μe)t−122
λS=0.0001+(0.001−0.0001)g(s)
Auxiliary prediction
LA=MSE(y^aux,yaux) with mask-weighting
λA=1.0
Local-goal prediction
LG=MSE(g^local,glocal) with mask-weighting
λG=1.0
Appendix
TABLE F: Distillation-stage loss terms. μ is the student action mean from the full model forward pass, and μprior is the action mean from a prior-only forward pass (encoder disabled). aexp denotes the expert action mean (teacher target) used for supervision. The prior and privileged encoder output diagonal Gaussians over the latent z , denoted by pθ(z∣o)=N(μp,σp2) and qϕ(z∣o,opriv)=N(μe,σe2) . Let ep be the epoch index and s=clip(3000ep−500,0,1) . The cosine ramp is g(s)=21−cos(πs) .
Term
Active when
Description
Goal-reaching
target visible
Reward proximity to the sampled goal (dense).
Progress
target visible
Reward step-to-step improvement toward the goal (clipped).
Terminal bonus
success on visible targets
Add a bonus when all visible thresholds are met.
Termination
always
Same as in Table C .
Smoothness
always
Mostly same as in Table C without reference related terms
Appendix
TABLE G: RL finetuning reward components . We mask-gate goal-related terms: if a target is unobserved, we set its reward to zero and exclude it from success checks.
Category
Feature
Dimension
Description
oglobal
Root position residual (xy)
2
Horizontal position error
Heading residual (yaw)
1
Yaw angle error
ocmd
End-of-episode flag
1
Near episode end indicator
Approaching flag
1
Moving toward object
Leaving flag
1
Moving away from object
Time-to-go
1
Normalized remaining time
Appendix
TABLE H: Student policy observation space.
Category
Parameter
Range
Humanoid
Added base mass (kg)
[−3.0,3.0]
CoM offset (m)
[−0.05,0.05]
Motor strength scale
[0.8,1.2]
Environment
Ground friction
[0.5,2.0]
Gravity perturbation (m/s 2 )
[−0.1,0.1]
Gravity randomization interval
4 s
Appendix
TABLE I: Domain randomization ranges.
Parameter
Range
Object mass (kg)
[0.15,1.5]
Object mass rare (kg)
[0.05,0.15] (every 10 episodes)
Object CoM offset (m)
[−0.05,0.05]
Object inertia scale
[0.5,2.0]
Object friction
[0.2,1.2]
Object restitution
[0.0,0.3]
Appendix
TABLE J: Object domain randomization ranges.
Parameter
Value
Gaussian noise std (m)
0.02
Point dropout probability
0.15
Outlier probability
0.05
Outlier max distance (m)
0.5
Depth noise scale
0.01
Density range
[0.5, 1.0]
Appendix
TABLE K: Point cloud domain randomization.
Observation
Noise Scale
Joint positions (rad)
0.01
Joint velocities (rad/s)
0.1
Angular velocity (rad/s)
0.05
IMU
0.05
Root position (m)
0.05
Appendix
TABLE L: Observation noise scales.
Simulation
Parameter
Value
Physics
Simulation substeps
1
Control frequency inverse
17 ( ≈ 59 Hz)
Number of parallel envs
4096
PhysX solver type
1 (TGS)
Position iterations
4
Velocity iterations
1
Appendix
TABLE M: Simulation configuration.
Fig. A: ( I ) The OptiTrack camera setup used to support ULTRA control in MoCap mode. ( II ) Markers attached to the humanoid root (front and back).
Fig. B: We visualize the objects used in our real-world deployment.
Whole-body humanoid loco-manipulation requires coordinating the robot's entire kinematic chain. However, most existing systems typically decouple the upper and lower bodies into separate controllers, limiting such coordination and yielding behaviors similar to those of a wheeled dual-arm platform. In this paper, we ask what it takes to build a whole-body native vision-language-action (VLA) model that maps language and pixels directly to all of the humanoid's degrees of freedom. We conduct a systematic empirical study organized as a roadmap of one-variable-at-a-time experiments across three phases: whole-body teleoperation, VLA model design, and heterogeneous co-training. Our study yields several intriguing findings: a joint-based whole-body teleoperation interface outperforms alternatives that only partially expose the humanoid's degrees of freedom; a VLA pretrained on static and wheeled dual-arm platforms transfers surprisingly well to a humanoid's full action space; and co-training with HuMI, the humanoid analog of UMI, extends the policy to new objects and instructions without additional whole-body teleoperation on those targets. Following this roadmap yields OpenHLM, an open-source recipe for whole-body humanoid loco-manipulation. In a challenging long-horizon task that spans a wide vertical range of the humanoid, OpenHLM outperforms two state-of-the-art humanoid VLA baselines (GR00T N1.6 and Ψ0) using less than half the total demonstration time. Our code, training data, and model checkpoints are available at [https://openhlm-project.github.io/].
Yingdong Hu, Haodong Zhu, Boyuan Zheng +6
Tsinghua University · Shanghai Qi Zhi Institute · Spirit AI
Humanoid loco-manipulation of large, heavy objects demands forceful interaction across the entire body. However, such payloads shift a humanoid's center of mass and impose sustained loads across the upper body, challenging balance and command tracking. We present HULK, a whole-body control framework for forceful loco-manipulation. Using model predictive control (MPC) to guide reinforcement learning with predictions of the loaded dynamics, we train two teachers: one tracks arm motions under wrist forces, and the other locomotes while holding large objects against the body. A capture-point control barrier function augments the wrist-force teacher during training to improve balance under load. We distill both teachers into a single policy. Evaluation spans simulation and the Unitree G1. In simulation, the teacher with the barrier function achieves the lowest forward and lateral velocity tracking errors at 10 kg per arm among evaluated controllers and reduces aggregate divergent component of motion (DCM) excursion magnitude by 35.7% relative to MPC-guided reinforcement learning alone. Our wrist-force teacher withstands torso push disturbances of up to 130 N.
An Dang, Arturo Flores Alvarez, Yu-Ming Chen +5
Amazon · University of Michigan · University of California, Los Angeles +1
Humanoid robots require diverse motor skills to integrate into complex environments, but bridging the kinematic and dynamic embodiment gap from human data remains a major bottleneck. We demonstrate through Hessian analysis that traditional optimization-based retargeting is inherently non-convex and prone to local optima, leading to physical artifacts like joint jumps and self-penetration. To address this, we reformulate the targeting problem as learning data distribution rather than optimizing optimal solutions, where we propose NMR, a Neural Motion Retargeting framework that transforms static geometric mapping into a dynamics-aware learned process. We first propose Clustered-Expert Physics Refinement (CEPR), a hierarchical data pipeline that leverages VAE-based motion clustering to group heterogeneous movements into latent motifs. This strategy significantly reduces the computational overhead of massively parallel reinforcement learning experts, which project and repair noisy human demonstrations onto the robot's feasible motion manifold. The resulting high-fidelity data supervises a non-autoregressive CNN-Transformer architecture that reasons over global temporal context to suppress reconstruction noise and bypass geometric traps. Experiments on the Unitree G1 humanoid across diverse dynamic tasks (e.g., martial arts, dancing) show that NMR eliminates joint jumps and significantly reduces self-collisions compared to state-of-the-art baselines. Furthermore, NMR-generated references accelerate the convergence of downstream whole-body control policies, establishing a scalable path for bridging the human-robot embodiment gap.