While generative models have become effective at producing human-like motions from text, transferring these motions to humanoid robots for physical execution remains challenging. Existing pipelines are often limited by retargeting, where kinematic quality is undermined by physical infeasibility, contact-transition errors, and the high cost of real-world dynamical data. We present a unified latent-driven framework that bridges natural language and whole-body humanoid locomotion through a retarget-free, physics-optimized pipeline. Rather than treating generation and control as separate stages, our key insight is to couple them bidirectionally under physical constraints.We introduce a Physical Plausibility Optimization (PP-Opt) module as the coupling interface. In the forward direction, PP-Opt refines a teacher-student distillation policy with a plausibility-centric reward to suppress artifacts such as floating, skating, and penetration. In the backward direction, it converts reward-optimized simulation rollouts into high-quality explicit motion data, which is used to fine-tune the motion generator toward a more physically plausible latent distribution. This bidirectional design forms a self-improving cycle: the generator learns a physically grounded latent space, while the controller learns to execute latent-conditioned behaviors with dynamical integrity.Extensive experiments on the Unitree G1 humanoid show that our bidirectional optimization improves tracking accuracy and success rates. Across IsaacLab and MuJoCo, the implicit latent-driven pipeline consistently outperforms conventional explicit retargeting baselines in both precision and stability. By coupling diffusion-based motion generation with physical plausibility optimization, our framework provides a practical path toward deployable text-guided humanoid intelligence.
Figures & tables
Figure 1: RoboForge. aims to perform physically plausible motions. (a) performed a series of three martial arts kicks. (b) turned around and threw the javelin with great force. (c) squats to avoid, then kicks, and finally punches.
Figure 2: The RoboForge end-to-end framework. The framework is composed of four key modules: (i) Motion Generation module, (ii) Physical Plausibility Optimization Module, (iii) Training Pipeline, and (iv) Sim-to-Real Deployment.
Methods
Common MG Metrics
Physics Plausibility Metrics
R-Precision ↑
FID ↓
Div →
Penetrate ↓
Float ↓
Skate ↓
Top1 ↑
Top2 ↑
Top3 ↑
Ground-Truth
0.552
0.733
0.782
0.008
9.553
–
–
–
MLD
0.523
0.703
0.761
0.484
9.734
0.042
1.744
0.064
MLD&PP-Opt
0.531
0.709
0.766
0.462
9.682
0.000
0.713
0.061
Table 1: Evaluation of physics-optimized motion generation: MG with/without PP-Opt.
Methods
IsaacLab
MuJoCo
Succ ↑
Empjpe↓
Empkpe↓
Succ ↑
Empjpe↓
Empkpe↓
MLD
0.94
0.14
0.11
0.63
0.26
0.24
MLD&PP-Opt
0.96
0.11
0.09
0.71
0.21
0.20
Table 2: Tracking policy with/without PP-Opt training optimization.
Common MG Metrics
Physics Plausibility Metrics
MLD&PP-Opt
R-Precision ↑
FID ↓
Div →
Penetrate ↓
Float ↓
Skate ↓
Top1
Top2
Top3
One-Round
0.531
0.709
0.766
0.462
9.682
0.000
0.713
0.061
Two-Round
0.535
0.712
0.769
0.456
9.701
0.000
0.709
0.059
Three-Round
0.537
0.714
0.770
0.454
9.711
0.000
0.707
0.058
Table 3: Evaluation of the effects of multiple rounds of PP-Opt on MG.
Methods
IsaacLab
MuJoCo
Succ ↑
Empjpe↓
Empkpe↓
Succ ↑
Empjpe↓
Empkpe↓
Ours-Implicit
0.96
0.11
0.09
0.71
0.21
0.20
Ours-Explicit
0.91
0.23
0.17
0.62
0.26
0.31
Table 4: Evaluation of Implicit/Explicit interface for tracking.
Figure 3: Text-guided examples in simulation. The humanoid performs diverse motions including boxing, jumping jacks, and martial arts kicking under different text prompts.
Recent progress in text-conditioned human motion generation has been largely driven by diffusion models trained on large-scale human motion data. Building on this progress, recent methods attempt to transfer such models for character animation and real robot control by applying a Whole-Body Controller (WBC) that converts diffusion-generated motions into executable trajectories. While WBC trajectories become compliant with physics, they may expose substantial deviations from original motion. To address this issue, we here propose PhysMoDPO, a Direct Preference Optimization framework. Unlike prior work that relies on hand-crafted physics-aware heuristics such as foot-sliding penalties, we integrate WBC into our training pipeline and optimize diffusion model such that the output of WBC becomes compliant both with physics and original text instructions. To train PhysMoDPO we deploy physics-based and task-specific rewards and use them to assign preference to synthesized trajectories. Our extensive experiments on text-to-motion and spatial control tasks demonstrate consistent improvements of PhysMoDPO in both physical realism and task-related metrics on simulated robots. Moreover, we demonstrate that PhysMoDPO results in significant improvements when applied to zero-shot motion transfer in simulation and for real-world deployment on a G1 humanoid robot.
Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI) · LIGM, École des Ponts, IP Paris, Univ Gustave Eiffel, CNRS · École polytechnique fédérale de Lausanne (EPFL)
Text-conditioned motion generation is a promising interface for programming humanoid robots, yet current generators are often trained on human motion datasets retargeted to robot morphologies. Although such data provides rich semantic and kinematic priors, it fails to capture the nuances of whole-body tracking controllers, including balance, contact dynamics, actuation limits, and controller-specific failure modes. As a result, generated motions can be semantically plausible but difficult or impossible for the robot to execute. We introduce TEXEDO, a test-time scaling framework for humanoid motion generation that improves motion quality without requiring a stronger underlying generator. Given a text prompt, TEXEDO samples multiple candidate motions from a pretrained text-conditioned generator and selects the best motion that is both executable and task-aligned. The reward model combines a dynamic feasibility verifier, distilled from whole-body tracking rollouts to predict physical executability, with a semantic alignment verifier that measures text-motion alignment in a learned co-embedding space. Our pipeline treats dynamic feasibility as a hard constraint and semantic alignment as the selection objective within the feasible set. Through large-scale simulation studies and real-world deployment on a Unitree G1 humanoid robot, we show that TEXEDO consistently improves both tracking fidelity and text alignment. These results demonstrate that grounded verification is an effective path toward deployable language-guided humanoid motion generation. Project website: https://jianuocao.github.io/TEXEDO/
Jianuo Cao, Yuxin Chen, Yuzhen Song +3
Nanjing University · University of California, Berkeley · Southern University of Science and Technology
We present ECHO, an edge--cloud framework for language-driven whole-body control of humanoid robots. A cloud-hosted diffusion-based text-to-motion generator synthesizes motion references from natural language instructions, while an edge-deployed reinforcement-learning tracker executes them in closed loop on the robot. The two modules are bridged by a compact, robot-native 38-dimensional motion representation that encodes joint angles, root planar velocity, root height, and a continuous 6D root orientation per frame, eliminating inference-time retargeting from human body models and remaining directly compatible with low-level PD control. The generator adopts a 1D convolutional UNet with cross-attention conditioned on CLIP-encoded text features; at inference, DDIM sampling with 10 denoising steps and classifier-free guidance produces motion sequences in approximately one second on a cloud GPU. The tracker follows a Teacher--Student paradigm: a privileged teacher policy is distilled into a lightweight student equipped with an evidential adaptation module for sim-to-real transfer, further strengthened by morphological symmetry constraints and domain randomization. An autonomous fall recovery mechanism detects falls via onboard IMU readings and retrieves recovery trajectories from a pre-built motion library. We evaluate ECHO on a retargeted HumanML3D benchmark, where it achieves strong generation quality (FID 0.029, R-Precision Top-1 0.686) under a unified robot-domain evaluator, while maintaining high motion safety and trajectory consistency. Real-world experiments on a Unitree G1 humanoid demonstrate stable execution of diverse text commands with zero hardware fine-tuning.
Haozhe Jia, Jianfei Song, Yuan Zhang +5
The Hong Kong University of Science and Technology (Guangzhou). · LimX Dynamics Technology Co., Ltd. · Shandong University. +1