While velocity-commanded RL policies are effective for controlling quadrupeds, they are a limited interface for humanoids that need to perform complex whole-body motions. Furthermore, RL tracking policies for whole-body motions typically have limited information about the desired future motion, preventing them from performing controlled deviations to achieve long-term task success; such information is particularly relevant when a controller must prepare for transitions between behaviors. We address these challenges using concise motion representations that can be generated by a high-level controller, given a behavior label or text command. These representations can be fed directly into a tracking policy to provide a detailed motion reference. Our method exhibits a high degree of flexibility in performing real-world robot motions such as dancing, fighting, or walking based on conditional information. At matched capacity, our adopted stochastic motion tokenizer reconstructs motions from the Unitree G1 at a level that exceeds state-of-the-art approaches such as finite scalar quantization (FSQ) and the vector-quantized autoencoder (VQ-VAE). The resulting discrete interface conditions whole-body tracking policies that execute generated dance, fight, and walk motions lasting 16.8 s each and a 35 s fight-to-dance sequence on a physical Unitree G1.
Figures & tables
Fig. 1: The Unitree G1 executing behaviors synthesized by our three-stage, next-token-prediction framework and tracked by a single policy.
Fig. 2: A repertoire of reference motions generated by the Stage 2 TrajectoryGPT model ( pink ) and tracked by the robot during Stage 3 training ( silver robot ). The renderings are the true states of the robot and kinematics—a faithful depiction of the tracking behavior in the simulator.
Fig. 3: DAPS encode–decode on a G1 validation window: ground truth vs. reconstruction across representative channels of the standardized whole-body feature space (root state, joint angles, relative body-site positions and rotations) over 3.2 s of consecutive windows. The discrete code tracks the whole-body structure closely (overall standardized RMSE ≈ 0.20).
Figure 4
Fig. 6: Stage-3 PPO training of the deployed stitching policy (Fig. 7 ). Left : Tracking reward for joint position ( blue ) and joint velocity ( teal ). Middle : Average episode length of all environments throughout training. Right : Action-rate penalty, as part of the total environment reward. Training statistics are shown over 2 B environment steps.
Stage
Parameter
Value
Data
rate / window
30 Hz / 24 frames
Data
feature dim
162 ( 36 qpos +126 rel. sites)
Stage 1
tokens B / vocab V
16 / 1024 ( 160 bits)
Stage 1
embed / batch / steps
128 / 256 / 250 k
Stage 1
β sched. / ESS / MC
0.3→10−2 – 3×10−3 / 0.5 / 16
Stage 2
layers / embed / history
12 / 384 / N=8 chunks
TABLE I: Pipeline hyperparameters.
Fig. 7: Real-robot behavior switch on the Unitree G1: a single policy transitions from a fight motion into a dance through a wildcard bridge, during which no reference is prescribed, completing the full 35 s sequence upright.
Fig. 8: Real-robot execution of a generated dance on the Unitree G1. Six joint angles from the measured hardware state ( magenta ) are compared with the source kinematics ( blue ), linearly resampled from 30 to 50 Hz. The rightmost column shows within-chunk phase and latent-chunk progression.
Despite advances in reinforcement and imitation learning, achieving both precise control and dynamic whole-body behaviors in long-horizon tasks remains challenging. Existing approaches typically follow two paradigms: coupled whole-body policies for global coordination and decoupled policies for modular precision. However, effectively combining these two types of controllers to achieve agility, robustness, and precision remains challenging. In this work, we propose BAT, an online policyswitching framework that dynamically selects between coupled and decoupled whole-body RL generalist controllers according to the evolving motion context. BAT employs two complementary switching predictors: OpVQ-VAE provides motion-token-based predictions with strong generalization, while OpHRL provides closed-loop, state-aware predictions. A Token-Familiarity Router (TFR) selects between their predictions, favoring OpHRL for familiar token sequences and OpVQ-VAE for unfamiliar ones when their predictions disagree. BAT achieves 85.2% success on seen long-horizon motion combinations, while retaining 71.3% success on unseen motions. BAT also outperforms existing whole-body controllers on individual motions and demonstrates successful zero-shot deployment on the Unitree G1 humanoid.
Donghoon Baek, Sang-Hun Kim, Sehoon Ha
Georgia Institute of Technology, Atlanta, GA, 30308, USA · Samsung Research
We introduce Humanoid-GPT, a GPT-style Transformer with causal attention trained on a billion-scale motion corpus for whole-body control. Unlike prior shallow MLP trackers constrained by scarce data and an agility-generalization trade-off, Humanoid-GPT is pre-trained on a 2B-frame retargeted corpus that unifies all major mocap datasets with large-scale in-house recordings. Scaling both data and model capacity yields a single generative Transformer that tracks highly dynamic behaviors while achieving unprecedented zero-shot generalization to unseen motions and control tasks. Extensive experiments and scaling analyses show that our model establishes a new performance frontier, demonstrating robust zero-shot generalization to unseen tasks while simultaneously tracking highly dynamic and complex motions.
Zekun Qi, Xuchuan Chen, Dairu Liu +10
Tsinghua University · Galbot Inc. · Shanghai Jiao Tong University +2
In this report, we present HoloMotion-1, a humanoid motion foundation model for zero-shot whole-body motion tracking. A key innovation of HoloMotion-1 is to scale control-policy training with a large-scale hybrid motion corpus, where video-reconstructed motions from in-the-wild videos provide the dominant source of motion diversity, while curated motion-capture and in-house motion data provide higher-fidelity supervision and deployment-oriented coverage. This data regime enables HoloMotion-1 to move beyond conventional MoCap-only training and exposes the policy to substantially broader behaviors, capture conditions, and motion styles. Learning from such heterogeneous data introduces new challenges, including reconstruction noise, source-domain mismatch, uneven motion quality, and the need for temporal modeling under large behavioral variation. To address these challenges, HoloMotion-1 integrates large-capacity temporal modeling, a sparsely activated Mixture-of-Experts Transformer with KV-cache inference for real-time control, and a sequence-level training strategy that improves learning efficiency on extended motion sequences. Extensive experiments on multiple unseen motion benchmarks show that HoloMotion-1 generalizes robustly across diverse motion types and capture conditions, significantly improves tracking accuracy over prior methods, and transfers directly to a real humanoid robot without task-specific fine-tuning.