High-quality human motion data is becoming increasingly important for applications in robotics, simulation, and entertainment. Recent generative models offer a potential data source, enabling human motion synthesis through intuitive inputs like text prompts or kinematic constraints on poses. However, the small scale of public mocap datasets has limited the motion quality, control accuracy, and generalization of these models. In this work, we introduce Kimodo, an expressive and controllable kinematic motion diffusion model trained on 700 hours of optical motion capture data. Our model generates high-quality motions while being easily controlled through text and a comprehensive suite of kinematic constraints including full-body keyframes, sparse joint positions/rotations, 2D waypoints, and dense 2D paths. This is enabled through a carefully designed motion representation and two-stage denoiser architecture that decomposes root and body prediction to minimize motion artifacts while allowing for flexible constraint conditioning. Experiments on the large-scale mocap dataset justify key design decisions and analyze how the scaling of dataset size and model size affect performance.
Figures & tables
Figure 1: Controllable Motion Generation. Kimodo supports flexible and intuitive control for motion generation through text prompting combined with an extensive suite of kinematic constraints. By training on 700 hours of optical mocap data, the model achieves precise control accuracy for a large variety of behaviors. In each example, constrained joints are indicated with a red color, and generated poses at constrained frames are highlighted in yellow. Time progression is indicated by lighter to darker blue coloring.
Figure 2: Motion Authoring Demo. (Left) Our authoring interface built with Viser [ 48 ] allows intuitive control over Kimodo for motion generation. The timeline panel allows users to specify text prompts and constraints at specific frames or intervals, which are displayed in the 3D viewer. The options panel on the right side of the interface controls various generation parameters. (Right) In editing mode, users have fine-grained control to pose and translate the character at constrained frames. Editing and generation can be done on either the SOMA body skeleton [ 32 ] or Unitree G1 robot.
Figure 3: Text-to-Motion Results. (Top) Kimodo enables generating high-quality human motions for a variety of behaviors on the SOMA body skeleton. Time progression is indicated by lighter to darker blue coloring. (Middle) Motions can also be generated directly on the G1 robot to easily collect plausible demonstrations. (Bottom) The same frame is visualized from ten different generated motion samples for the same prompt, demonstrating the diversity of Kimodo outputs.
Figure 4: Multi-Prompt Generation. Longer motion sequences can be generated from multiple prompts with the demo’s timeline interface. Motions are generated sequentially with constraints between them for continuity.
Figure 5: Scaling Results. Scaling dataset size, model size, and batch size improves controllability and motion quality. Increased dataset size results in greatly improved constraint following, while model size and batch size are particularly helpful for text following (R-precision) and motion quality (FID). See Tab. 2 for full results.
Figure 6: Generating Robot Demonstrations. In these results, Kimodo is used to generate demonstration data directly on the G1 robot, which is then tracked by a physics-based humanoid policy trained with ProtoMotions [ 41 ] and deployed to a real-world robot.
Figure 7: Example Motion Data. Each mocap sequence in our training dataset includes a high-level overview description, fine-grained descriptions of sub-clips containing atomic actions, and augmented LLM paraphrases.
Figure 8: Smoothed Root Representation. For a simple walking motion, the projected pelvis path captures the sway of the hips, while the smoothed root is nearly a straight line. This smoothed trajectory offers a stable frame of reference for representing joint positions, and emulates paths drawn in practical animation tools.
Figure 9: Denoiser Architecture. (Left) Kimodo predicts clean motion given a noisy motion, pose constraints, and a text embedding. Specified pose constraints directly overwrite the noisy motion before it is given to the denoiser. (Right) The two-stage denoiser decomposes root and body motion prediction. The root denoiser first predicts the global root motion, which is transformed into a local representation as input to the body denoiser. The final output of the denoising step is the concatenation of the outputs from the two stages.
Text-Following Evaluation
Constrained Evaluation
Overview Prompt Test Set
Fine-Grained Prompt Test Set
Full-Body
End-Effector Joints
2D Root
2D Pelvis
Method
R@3 ↑
FID ↓
Skate (cm/s) ↓
Contact ↑
R@3 ↑
FID ↓
Skate (cm/s) ↓
Contact ↑
Pos (cm) ↓
Pos (cm) ↓
Rot (deg) ↓
Pos (cm) ↓
Pos@95% (cm)
Ground Truth
75.6
0.0
2.21
1.00
79.4
0.00
2.23
1.00
-
-
-
-
6.3
Full Model (Ours)
71.9
1.85
3.87
0.98
63.5
1.67
3.88
0.98
2.67
3.09
4.18
2.90
9.7
One-Stage Arch
71.5
1.65
7.59
0.94
63.5
1.51
6.80
0.95
8.37
10.19
5.19
7.74
21.1
Second Stage Global
70.3
1.87
4.17
0.98
63.2
1.66
4.07
0.98
2.97
3.39
5.67
3.25
10.2
Table 1: Ablation Study. Evaluation of text and constraint-conditioned motion generation on the Rigplay test set. The full model is compared to various baselines to justify key design decisions, including the two-stage denoiser, smoothed root representation, and dual-phase training curriculum. All models are trained using a medium batch size (8 GPU) at 20 fps. FID is multiplied ×100 for readability.
Text-Following Evaluation
Constrained Evaluation
Overview Prompt Test Set
Fine-Grained Prompt Test Set
Full-Body
End-Effector Joints
2D Root
2D Pelvis
Method
R@3 ↑
FID ↓
Skate (cm/s) ↓
Contact ↑
R@3 ↑
FID ↓
Skate (cm/s) ↓
Contact ↑
Pos (cm) ↓
Pos (cm) ↓
Rot (deg) ↓
Pos (cm) ↓
Pos@95% (cm)
Ground Truth
75.6
0.00
2.21
1.00
79.4
0.00
2.23
1.00
-
-
-
-
6.3
Full Dataset
71.5
1.84
4.23
0.97
63.0
1.07
4.15
0.98
2.77
3.31
5.36
3.29
10.7
50% Dataset
70.8
1.81
4.43
0.97
63.4
1.06
4.27
0.97
3.13
3.56
6.29
3.32
10.7
10% Dataset
71.0
2.07
5.28
0.96
62.4
1.41
5.12
0.97
4.60
6.91
10.03
4.83
15.5
Table 2: Scaling Analysis. Evaluation of text and constraint-conditioned motion generation on the Rigplay test set. (Top) Increasing the amount of training data improves motion quality and constraint accuracy due to increased diversity. (Middle) Increased model size improves performance on all metrics. (Bottom) Increasing batch size by using more GPUs generally improves performance across the board.
We present ScaleMoGen, a scale-wise autoregressive framework for text-driven human motion generation. Unlike conventional autoregressive approaches that rely on standard next-token prediction, ScaleMoGen frames motion generation as a coarse-to-fine process. We quantize 3D motions into compositional discrete tokens across multiple skeletal-emporal scales of increasing granularity, learning to generate motion by autoregressively predicting next-scale token maps. To maintain structural integrity, our motion tokenizers and quantizers are explicitly designed so that discrete tokens at every scale strictly preserve the skeletal hierarchy. Additionally, we employ bitwise quantization and prediction, which efficiently scale up the tokenizer vocabulary to preserve motion details and stabilize optimization. Extensive experiments demonstrate that ScaleMoGen achieves state-of-the-art performance, establishing an FID of 0.030 (vs. 0.045 for MoMask) on HumanML3D and a CLIP Score of 0.693 (vs. 0.685 for MoMask++) on the SnapMoGen dataset. Furthermore, we demonstrate that our skeletal-temporal multi-scale representation naturally facilitates training-free, text-guided motion editing.
Inwoo Hwang, Hojun Jang, Bing Zhou +3
Seoul National University · Snap Inc. · Meta Reality Labs
Text-to-motion generation typically relies on dataset-specific skeletal representations, with models trained separately for benchmarks with different joint layouts. We investigate whether image diffusion architectures can model 3D human motion through a common image-compatible representation that accommodates heterogeneous skeletal layouts while preserving motion-specific kinematic structure. To this end, we organize root-centered joint trajectories into topology-informed joint--time images and model global root dynamics through a complementary latent stream. A motion autoencoder conditions local-motion decoding on the root latent, while a text-conditioned Scalable Interpolant Transformer (SiT) jointly generates both latent streams, with supervision on decoded 3D joint positions further constraining generation in motion space. In addition to dataset-specific models, we train a single motion autoencoder on HumanML3D and KIT-ML, followed by a single text-conditioned SiT shared across both datasets, preserving their respective skeletal definitions without skeleton retargeting to a common joint layout. Experiments on HumanML3D and KIT-ML show that our dataset-specific models achieve state-of-the-art performance, while shared training yields further gains on both benchmarks, demonstrating the benefits of shared modeling across heterogeneous skeletal layouts.
Siyuan Yang, Chong Wang, Hao Cheng +4
Division of Robotics, Perception and Learning, KTH Royal Institute of Technology, Sweden · School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore · School of Artificial Intelligence, Hebei University of Technology, China +3
Most existing human motion generation (HMG) methods use cross-attention modules to inject text semantics, but ignore the importance of bidirectional modeling between motion and text tokens, which limits text comprehension. A straightforward idea is introducing multimodal diffusion transformers (MMDiT), which have shown effective joint text--visual modeling in vision generation, into HMG. However, we find that articulated motion is temporally coherent but weakly correlated across joints, in which directly applying an MMDiT with flow matching produces poorly coordinated and jerky motion. In this work, we propose Timo, a novel kinematics-aware MMDiT framework tailored for HMG. Timo combines fully shared multimodal attention for bidirectional text--motion modeling with flow matching, geometric and rotational-kinematics supervision that compares actual rotations and their changes over time, and a two-stage curriculum progressing from broad motion learning to detailed caption alignment. Further, we construct a benchmark of 40,025 held-out clips from six public datasets spanning diverse actions, assessing six complementary dimensions under a common evaluator and scoring protocol. Our model substantially outperforms state-of-the-art methods in both quantitative and qualitative evaluations. Remarkably, Timo surpasses Kimodo on five of six dimensions, achieving a 40.8% relative improvement in the average benchmark score. Project page: https://kyfafyd.wang/projects/timo. Demo page: https://timo.kyfafyd.wang.