While generative models have become effective at producing human-like motions from text, transferring these motions to humanoid robots for physical execution remains challenging. Existing pipelines are often limited by retargeting, where kinematic quality is undermined by physical infeasibility, contact-transition errors, and the high cost of real-world dynamical data. We present a unified latent-driven framework that bridges natural language and whole-body humanoid locomotion through a retarget-free, physics-optimized pipeline. Rather than treating generation and control as separate stages, our key insight is to couple them bidirectionally under physical constraints.We introduce a Physical Plausibility Optimization (PP-Opt) module as the coupling interface. In the forward direction, PP-Opt refines a teacher-student distillation policy with a plausibility-centric reward to suppress artifacts such as floating, skating, and penetration. In the backward direction, it converts reward-optimized simulation rollouts into high-quality explicit motion data, which is used to fine-tune the motion generator toward a more physically plausible latent distribution. This bidirectional design forms a self-improving cycle: the generator learns a physically grounded latent space, while the controller learns to execute latent-conditioned behaviors with dynamical integrity.Extensive experiments on the Unitree G1 humanoid show that our bidirectional optimization improves tracking accuracy and success rates. Across IsaacLab and MuJoCo, the implicit latent-driven pipeline consistently outperforms conventional explicit retargeting baselines in both precision and stability. By coupling diffusion-based motion generation with physical plausibility optimization, our framework provides a practical path toward deployable text-guided humanoid intelligence.
Figures & tables
Figure 1: RoboForge. aims to perform physically plausible motions. (a) performed a series of three martial arts kicks. (b) turned around and threw the javelin with great force. (c) squats to avoid, then kicks, and finally punches.
Figure 2: The RoboForge end-to-end framework. The framework is composed of four key modules: (i) Motion Generation module, (ii) Physical Plausibility Optimization Module, (iii) Training Pipeline, and (iv) Sim-to-Real Deployment.
Methods
Common MG Metrics
Physics Plausibility Metrics
R-Precision ↑
FID ↓
Div →
Penetrate ↓
Float ↓
Skate ↓
Top1 ↑
Top2 ↑
Top3 ↑
Ground-Truth
0.552
0.733
0.782
0.008
9.553
–
–
–
MLD
0.523
0.703
0.761
0.484
9.734
0.042
1.744
0.064
MLD&PP-Opt
0.531
0.709
0.766
0.462
9.682
0.000
0.713
0.061
Table 1: Evaluation of physics-optimized motion generation: MG with/without PP-Opt.
Methods
IsaacLab
MuJoCo
Succ ↑
Empjpe↓
Empkpe↓
Succ ↑
Empjpe↓
Empkpe↓
MLD
0.94
0.14
0.11
0.63
0.26
0.24
MLD&PP-Opt
0.96
0.11
0.09
0.71
0.21
0.20
Table 2: Tracking policy with/without PP-Opt training optimization.
Common MG Metrics
Physics Plausibility Metrics
MLD&PP-Opt
R-Precision ↑
FID ↓
Div →
Penetrate ↓
Float ↓
Skate ↓
Top1
Top2
Top3
One-Round
0.531
0.709
0.766
0.462
9.682
0.000
0.713
0.061
Two-Round
0.535
0.712
0.769
0.456
9.701
0.000
0.709
0.059
Three-Round
0.537
0.714
0.770
0.454
9.711
0.000
0.707
0.058
Table 3: Evaluation of the effects of multiple rounds of PP-Opt on MG.
Methods
IsaacLab
MuJoCo
Succ ↑
Empjpe↓
Empkpe↓
Succ ↑
Empjpe↓
Empkpe↓
Ours-Implicit
0.96
0.11
0.09
0.71
0.21
0.20
Ours-Explicit
0.91
0.23
0.17
0.62
0.26
0.31
Table 4: Evaluation of Implicit/Explicit interface for tracking.
Figure 3: Text-guided examples in simulation. The humanoid performs diverse motions including boxing, jumping jacks, and martial arts kicking under different text prompts.
Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI) · LIGM, École des Ponts, IP Paris, Univ Gustave Eiffel, CNRS · École polytechnique fédérale de Lausanne (EPFL)