Learning motion priors for physics-based humanoid control is an active research topic. Existing approaches mainly include variational autoencoders (VAE) and adversarial motion priors (AMP). VAE introduces information loss, and random latent sampling may sometimes produce invalid behaviors. AMP suffers from mode collapse and struggles to capture diverse motion skills. We present the Spherical Latent Motion Prior (SLMP), a two-stage method for learning motion priors. In the first stage, we train a high-quality motion tracking controller. In the second stage, we distill the tracking controller into a spherical latent space. A combination of distillation, a discriminator, and a discriminator-guided local semantic consistency constraint shapes a structured latent action space, allowing stable random sampling without information loss. To evaluate SLMP, we collect a two-hour human combat motion capture dataset and show that SLMP preserves fine motion detail without information loss, and random sampling yields semantically valid and stable behaviors. When applied to a two-agent physics-based combat task, SLMP produces human-like and physically plausible combat behaviors only using simple rule-based rewards. Furthermore, SLMP generalizes across different humanoid robot morphologies, demonstrating its transferability beyond a single simulated avatar.
Figures & tables
Figure 1: Overview of the Spherical Latent Motion Prior (SLMP). We collect a two-hour combat motion capture dataset and convert it into SMPL motion clips for training. We train a goal-conditioned motion tracking controller, then distill it into a unit-sphere latent space using three losses: imitation, discriminator, and our discriminator-guided local semantic consistency loss ( LDLSC ). SLMP supports meaningful random sampling and drives downstream tasks such as two-agent combat via simple rewards.
Figure 2: Qualitative examples of random latent-conditioned rollouts generated by SLMP. Uniformly sampled latent codes produce diverse, stable, and physically plausible full-body motions. Additional rollouts are provided in the supplemental video.
Figure 3: Latent-space motion tracking performance. We evaluate information loss by tracking reference clips through the latent space. SLMP achieves higher success and lower MPJPE than PULSE, and approaches the expert controller.
Figure 4: Qualitative comparison of random latent-conditioned rollouts. ASE exhibits repetitive low-diversity behaviors, PULSE collapses occasionally, and SLMP generates diverse and stable motions. See supplemental video for full rollouts.
Figure 5: Random latent rollout survival curves over 1000 trials in Isaac Gym. SLMP maintains substantially higher survival rates across long horizons, whereas PULSE survival degrades due to semantic sparsity in the tails of its VAE latent space.
Figure 6: State-dependent spherical latent manifold. left: a guard stance yields diverse feasible follow-ups with multiple regions on the sphere. right: an airborne kick yields only a few feasible continuations, causing the sphere to collapse into one or two regions.
Method
Success (%) ↑
MPJPE ↓
Survival(%) ↑
VAE
99.8
33.7
10.2
VQ-VAE
99.0
36.9
12.4
Sphere
100.0
31.5
0.1
SLMP (ours)
100.0
31.7
99.8
Table 1: Ablation over latent representations. Success rate and MPJPE measure latent tracking quality. Survival measures random rollout stability under uniform sampling, defined as the percentage of 1000 randomly sampled rollouts that do not result in a fall within 10 seconds.
Figure 7: Each row shows a rollout sampled from different latent structures.
Figure 8: Survival rate at 10 seconds under random latent sampling.
Figure 9: Each row shows a rollout sampled from the latent sphere under a different training loss configuration.
Figure 10: Qualitative results on the two-agent combat task using SLMP as a motion prior. We compare dense NCP-style rewards, Smplolympics-style rewards (NCP-style rewards augmented with AMP reward), and sparse rule-based rewards. NCP-style rewards lead to degenerate stationary interactions, while Smplolympics-style rewards bias the policy toward repetitive punching behaviors. In contrast, sparse rule-based rewards with SLMP produce diverse and human-like combat behaviors.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 11: Random latent sampling on realistic humanoid robot models in simulation. Top: Unitree G1. Bottom: ENGINEAI PM01. Both models can stably generate diverse motions under random latent inputs.
Figure 12: Two-agent combat evaluation on realistic humanoid robot models in simulation. Top: Unitree G1. Bottom: ENGINEAI PM01. The robots generate human-like combat behaviors under the SLMP prior.
Category
Action Types
Attributes & Targets
Clips
Dur.
Stance
Orthodox (Idle, Guard), Weight Shift
Speed: Slow, Normal
20
11s
Footwork
Step, Pivot ( 90∘ ), Shuffle
Step Size: Small, Large
62
11s
Straight Punches
Jab, Cross
Target: Head, Body, Downward
60
11s
Hooks & Swings
Hook, Swing
Target: Head, Body; Type: Flat/Up
50
11s
Uppercuts
Uppercut (Left/Right)
Target: Head, Body
25
11s
Front Kicks
Front Kick (Left/Right)
Target: Body; Incl. Heavy Bagss
40
11s
Appendix
Table 2: Statistics of the combat motion capture dataset (502 clips). The dataset covers diverse speeds and target levels.
Component
Hidden Units
Activation
πtrack
[2048, 1536, 1024, 1024, 512, 512]
SiLU
E(⋅)
[512, 256]
ReLU
πφ
[4096, 2048, 1024, 1024, 512, 512]
SiLU
D(⋅)
[4096, 2048, 1024, 512]
ReLU
πh1,πh2
[2048, 1024, 512]
SiLU
Appendix
Table 3: Network architectures used in our framework. All networks are implemented as multilayer perceptrons (MLPs).
PPO Hyperparameters
Batch Size
Env Num
LR
γ
Clip ϵ
32768
1024
5×10−5
0.99
0.2
SLMP Hyperparameters
λdistill
λdisc
Disc LR
λDLSC
β
d
1.0
1×10−4
5×10−5
1.0
0.1
64
Appendix
Table 4: Key hyperparameters used in our experiments.
Robot motion generation creates feasible, task-directed trajectories for robots to move, manipulate objects, and interact with people and environments.
Motion priors provide powerful guidance for learning naturalistic humanoid behaviors. However, existing methods typically learn a general, task-agnostic prior from the entire reference dataset and apply it uniformly throughout policy training. As a result, the prior cannot distinguish which reference motions are relevant to the current task context, potentially providing irrelevant or conflicting guidance. We present Context-Aware Motion Priors (CMP), a framework that adapts a general motion prior to the current task context without manual skill labels, dataset partitioning, or a separate skill discovery stage. Specifically, CMP learns context-motion compatibility using high-advantage policy rollouts, while a demonstration-based objective keeps the learned relevance grounded in the reference distribution. The resulting relevance scores reweight reference supervision for training a lightweight context-conditioned adapter. To evaluate the effectiveness and generality of CMP, we instantiate it with both Adversarial Motion Priors and Score-Matching Motion Priors. Across five humanoid control tasks, CMP consistently improves task performance and sample efficiency, learns meaningful context-motion alignment, and remains robust to imbalanced reference distributions. These results show that adapting motion priors to task contexts provides more relevant guidance for humanoid policy learning.
Yunyang Mo, Yi Gu, Yangchen Zhou +2
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China
Humanoid loco-manipulation requires coordinated locomotion and manipulation informed by future scene evolution and whole-body motion, yet learning these capabilities is constrained by scarce robot demonstrations. Human video and motion datasets offer scalable supervision, but many contain only video or motion rather than paired video-motion data. Moreover, human motion does not directly specify executable robot actions. We present Being-M0.7, a latent world-action model that transfers visual-motion priors learned from mixed-modality human data to humanoid control through pre-training, robot mid-training, and action post-training. We curate a corpus from more than 10,000 hours of raw human-centric data, integrating video-only, motion-only, and paired video-motion streams to learn complementary visual dynamics and whole-body kinematic structure. Joint prediction of future latent visual states and motion encourages visual representations to encode future kinematics. Robot mid-training adapts this coarse-grained prior to robot viewpoints and body dynamics. During action post-training, an action expert combines visual predictive representations from the frozen, adapted prior with current images and proprioception through gated cross-attention, grounding predictive context in executable whole-body commands. Being-M0.7 achieves the highest aggregate success rate among the compared baselines on SIMPLE and matches the strongest baseline on real-world Unitree G1 loco-manipulation tasks.
Reinforcement learning can produce robust humanoid controllers, but each new task is typically trained as a separate policy with its own reward design and training process. Motion imitation provides an alternative source of motor competence by training policies to track retargeted human motions, yet the resulting controllers remain reference trackers and are not directly usable as task policies. We propose a three-stage pipeline that turns motion-imitation skills into a reusable hybrid motion prior (HMP) for humanoid locomotion. First, an expert policy is trained to imitate retargeted human motion-capture clips. Second, the expert is distilled into a frozen architecture composed of a proprioceptive encoder, a residual vector-quantized (RVQ) codebook, and an action decoder. Third, task-level policies are trained to solve locomotion tasks by selecting discrete codebook entries while the HMP remains frozen. We evaluate the method on velocity tracking, point-goal navigation, and fall-recovery velocity tracking in simulation, and deploy the velocity-tracking policy on a real Unitree G1 robot. The distillation process preserves the tracking behavior of the expert, while the resulting HMP can be reused without retraining as the action interface for different downstream locomotion policies. The learned HMP reveals an interpretable codebook structure in which the number of active RVQ stages modulates the available gait patterns. We further show that training the codebook with the rotation trick improves latent organization and reduces downstream falls compared with a standard straight-through estimator.