Developing robust autonomous loco-manipulation skills for humanoids remains an open problem in robotics. While RL has been applied successfully to legged locomotion, applying it to complex, interaction-rich manipulation tasks is harder given long-horizon planning challenges for manipulation. A recent approach along these lines is DreamControl, which addresses these issues by leveraging off-the-shelf human motion diffusion models as a generative prior to guide RL policies during training. In this paper, we investigate the impact of DreamControl's motion prior and propose an improved framework that trains a guided diffusion model directly in the humanoid robot's motion space, aggregating diverse human and robot datasets into a unified embodiment space. We demonstrate that our approach captures a wider range of skills due to the larger training data mixture and establishes a more automated pipeline by removing the need for manual filtering interventions. Furthermore, we show that scaling the generation of reference trajectories is important for achieving robust downstream RL policies. We validate our approach through extensive experiments in simulation and on a real Unitree-G1.
Figures & tables
Figure 2: DreamControl-v2 Overview. Our four-stage pipeline enables humanoid whole-body manipulation: (1) large-scale human motion datasets are retargeted into a unified robot-space; (2) a humanoid motion diffusion model is trained to generate reference trajectories conditioned on text and spatio-temporal control signal; (3) multiple reference trajectories are used to train a physics-based RL policy; (4) The learned policy is deployed in simulation and on a real robot.
Train Data
# Samples (K)
FID ↓
R-Precision ↑
Diversity →
Control L2 ↓
Skating Ratio ↓
Downstream RL Policy ↑
Top-1
Top-2
Top-3
(3.362)
Group-1
Group-2
Zero-Shot
-
0.981
0.105
0.260
0.329
3.647
0.112
0.101
0.040
0.995
AMASS
21.8
0.305
0.110
0.184
0.266
3.174
0.104
0.142
0.030
0.985
AMASS + Nymeria
26.4
0.298
0.101
0.204
0.281
3.084
0.108
0.132
0.860
0.985
Full-Mix
29.6
0.265
0.114
0.217
0.388
3.252
0.066
0.096
0.925
0.985
Table I : Comparison Across Training Data Mixtures. We evaluate diffusion models trained on three data configurations: AMASS, AMASS + Nymeria, and Full-Mix. For downstream control, we additionally report the average RL policy success ratio (in simulation). Refer to Sec. IV-A for details related to baselines, and note that zero-shot refers to fully automated DreamControl [ 7 ] for RL evaluation, and diffusion is evaluated using OmniControl [ 7 ] . All results are evaluated on the Full-Mix test set.
Train Data
FID ↓
R-Precision ↑
Control L2 ↓
Skating Ratio ↓
AMASS Test Set
AMASS
0.328
0.291
0.070
0.100
Full-Mix
0.320
0.280
0.050
0.120
Non-AMASS Test Set
AMASS
2.720
0.117
0.065
0.539
Full-Mix
0.316
0.264
0.042
0.032
Table II : Comparison Across Test Splits. We report FID, R-Precision (Top-3), Control L2, and Skating Ratio on two distinct evaluation sets: (1) the AMASS Test Set , and (2) a Non-AMASS Test Set constructed from Nymeria (subset), GRAB, and OmniRetarget samples within the Full-Mix distribution.
Figure 3: Effect of Trajectory Scaling on RL Performance. More diffusion-generated trajectories consistently lead to higher policy success ratios across 3 different tasks. The figure on the right indicates the variance of test distribution on multiple tasks.
Task
Success Ratio ↑
Control Error ↓
Pick
0.99
0.03
Punch
0.99
0.04
Deep Squat & Pick
0.94
0.06
Drawer Open
0.96
0.02
Pour
0.92
0.03
Vertical Wipe
1.00
0.04
Table III : Task-wise Performance . We report the Success Ratio and Control Error (m) averaged over 100 randomized simulation environments for all 8 tasks in our skill library.
Figure 4: Spatial Prompting Qualitative Results. The figure depicts successful ( ) and rejected samples ( ) obtained from various spatial prompting techniques. The DreamControl-v2 model is directly prompted in robot space (blue line), trial-and-error is used for the zero-shot model (red line), and prompt calibration is indicated with green lines.
Method
FID ↓
Ctrl L2 ↓
Valid Traj.(%)
Prompt-Calib. ( AMASS )
2.010
0.180
8.0
Prompt-Calib. ( Full-Mix )
3.260
0.260
4.0
Pre-retargetting ( Full-Mix )
0.265
0.066
68.0
Table IV : Prompt calibration vs. pre-retargeting. Prompt-Calib. uses samples from the zero-shot OmniControl model [ 7 ] . FID and Ctrl L2 are evaluated on the respective AMASS and Full-Mix test sets. Valid Traj. (%) is reported for the Pick task corresponding to AMASS and the Drawer Open task for Full-Mix. Our Pre-retargeting method is evaluated on Full-Mix and Valid Traj.(%) represents the average across both tasks.
Pretrained
Iters ×103
FID ↓
R@3 ↑
Ctrl L2 ↓
Skt Ratio ↓
✗
104
0.741
0.254
0.114
0.112
✓
102
0.265
0.388
0.066
0.096
Table V : Network Initialization . We report FID, R-Precision (Top-3), Control L2 and Skating Ratio for models randomly initialized ( ✗ ) and with the pretrained weights from MDM [ 17 ] ( ✓ ).
SMPL
G1 link name
pelvis
pelvis_contour_link
left hip
left_hip_pitch_link
right hip
right_hip_pitch_link
left knee
left_knee_link
right knee
right_knee_link
left ankle
left_ankle_roll_link
Table VI : Human and G1 link correspondances for retargetting
Human
G1 link name
pelvis
pelvis contour link
left hip
left hip pitch link
right hip
right hip pitch link
spine 1
imu link
left knee
left knee link
right knee
right knee link
Table VII : SMPL and G1 link keypoint correspondances for Trajectory represenatation
Figure 4: Key-point correspondences used for retargetting human data to G1. A similar correspondence mapping is used for diffusion trajectory representation Sec. III-B1 .