cs.CVMar 16, 2026
SaveKimodo: Scaling Controllable Human Motion Generation
Organizations: NVIDIA
Abstract
High-quality human motion data is becoming increasingly important for applications in robotics, simulation, and entertainment. Recent generative models offer a potential data source, enabling human motion synthesis through intuitive inputs like text prompts or kinematic constraints on poses. However, the small scale of public mocap datasets has limited the motion quality, control accuracy, and generalization of these models. In this work, we introduce Kimodo, an expressive and controllable kinematic motion diffusion model trained on 700 hours of optical motion capture data. Our model generates high-quality motions while being easily controlled through text and a comprehensive suite of kinematic constraints including full-body keyframes, sparse joint positions/rotations, 2D waypoints, and dense 2D paths. This is enabled through a carefully designed motion representation and two-stage denoiser architecture that decomposes root and body prediction to minimize motion artifacts while allowing for flexible constraint conditioning. Experiments on the large-scale mocap dataset justify key design decisions and analyze how the scaling of dataset size and model size affect performance.
Figures & tables
Figure 1: Controllable Motion Generation. Kimodo supports flexible and intuitive control for motion generation through text prompting combined with an extensive suite of kinematic constraints. By training on 700 hours of optical mocap data, the model achieves precise control accuracy for a large variety of behaviors. In each example, constrained joints are indicated with a red color, and generated poses at constrained frames are highlighted in yellow. Time progression is indicated by lighter to darker blue coloring.
Figure 2: Motion Authoring Demo. (Left) Our authoring interface built with Viser [ 48 ] allows intuitive control over Kimodo for motion generation. The timeline panel allows users to specify text prompts and constraints at specific frames or intervals, which are displayed in the 3D viewer. The options panel on the right side of the interface controls various generation parameters. (Right) In editing mode, users have fine-grained control to pose and translate the character at constrained frames. Editing and generation can be done on either the SOMA body skeleton [ 32 ] or Unitree G1 robot.
Figure 3: Text-to-Motion Results. (Top) Kimodo enables generating high-quality human motions for a variety of behaviors on the SOMA body skeleton. Time progression is indicated by lighter to darker blue coloring. (Middle) Motions can also be generated directly on the G1 robot to easily collect plausible demonstrations. (Bottom) The same frame is visualized from ten different generated motion samples for the same prompt, demonstrating the diversity of Kimodo outputs.
Figure 4: Multi-Prompt Generation. Longer motion sequences can be generated from multiple prompts with the demo’s timeline interface. Motions are generated sequentially with constraints between them for continuity.
Figure 5: Scaling Results. Scaling dataset size, model size, and batch size improves controllability and motion quality. Increased dataset size results in greatly improved constraint following, while model size and batch size are particularly helpful for text following (R-precision) and motion quality (FID). See Tab. 2 for full results.
Figure 6: Generating Robot Demonstrations. In these results, Kimodo is used to generate demonstration data directly on the G1 robot, which is then tracked by a physics-based humanoid policy trained with ProtoMotions [ 41 ] and deployed to a real-world robot.
Figure 7: Example Motion Data. Each mocap sequence in our training dataset includes a high-level overview description, fine-grained descriptions of sub-clips containing atomic actions, and augmented LLM paraphrases.
Figure 8: Smoothed Root Representation. For a simple walking motion, the projected pelvis path captures the sway of the hips, while the smoothed root is nearly a straight line. This smoothed trajectory offers a stable frame of reference for representing joint positions, and emulates paths drawn in practical animation tools.
Figure 9: Denoiser Architecture. (Left) Kimodo predicts clean motion given a noisy motion, pose constraints, and a text embedding. Specified pose constraints directly overwrite the noisy motion before it is given to the denoiser. (Right) The two-stage denoiser decomposes root and body motion prediction. The root denoiser first predicts the global root motion, which is transformed into a local representation as input to the body denoiser. The final output of the denoising step is the concatenation of the outputs from the two stages.
| Text-Following Evaluation | Constrained Evaluation | ||||||||||||
| Overview Prompt Test Set | Fine-Grained Prompt Test Set | Full-Body | End-Effector Joints | 2D Root | 2D Pelvis | ||||||||
| Method | R@3 | FID | Skate (cm/s) | Contact | R@3 | FID | Skate (cm/s) | Contact | Pos (cm) | Pos (cm) | Rot (deg) | Pos (cm) | Pos@95% (cm) |
| Ground Truth | 75.6 | 0.0 | 2.21 | 1.00 | 79.4 | 0.00 | 2.23 | 1.00 | - | - | - | - | 6.3 |
| Full Model (Ours) | 71.9 | 1.85 | 3.87 | 0.98 | 63.5 | 1.67 | 3.88 | 0.98 | 2.67 | 3.09 | 4.18 | 2.90 | 9.7 |
| One-Stage Arch | 71.5 | 1.65 | 7.59 | 0.94 | 63.5 | 1.51 | 6.80 | 0.95 | 8.37 | 10.19 | 5.19 | 7.74 | 21.1 |
| Second Stage Global | 70.3 | 1.87 | 4.17 | 0.98 | 63.2 | 1.66 | 4.07 | 0.98 | 2.97 | 3.39 | 5.67 | 3.25 | 10.2 |
Table 1: Ablation Study. Evaluation of text and constraint-conditioned motion generation on the Rigplay test set. The full model is compared to various baselines to justify key design decisions, including the two-stage denoiser, smoothed root representation, and dual-phase training curriculum. All models are trained using a medium batch size (8 GPU) at 20 fps. FID is multiplied for readability.
| Text-Following Evaluation | Constrained Evaluation | ||||||||||||
| Overview Prompt Test Set | Fine-Grained Prompt Test Set | Full-Body | End-Effector Joints | 2D Root | 2D Pelvis | ||||||||
| Method | R@3 | FID | Skate (cm/s) | Contact | R@3 | FID | Skate (cm/s) | Contact | Pos (cm) | Pos (cm) | Rot (deg) | Pos (cm) | Pos@95% (cm) |
| Ground Truth | 75.6 | 0.00 | 2.21 | 1.00 | 79.4 | 0.00 | 2.23 | 1.00 | - | - | - | - | 6.3 |
| Full Dataset | 71.5 | 1.84 | 4.23 | 0.97 | 63.0 | 1.07 | 4.15 | 0.98 | 2.77 | 3.31 | 5.36 | 3.29 | 10.7 |
| 50% Dataset | 70.8 | 1.81 | 4.43 | 0.97 | 63.4 | 1.06 | 4.27 | 0.97 | 3.13 | 3.56 | 6.29 | 3.32 | 10.7 |
| 10% Dataset | 71.0 | 2.07 | 5.28 | 0.96 | 62.4 | 1.41 | 5.12 | 0.97 | 4.60 | 6.91 | 10.03 | 4.83 | 15.5 |
Table 2: Scaling Analysis. Evaluation of text and constraint-conditioned motion generation on the Rigplay test set. (Top) Increasing the amount of training data improves motion quality and constraint accuracy due to increased diversity. (Middle) Increased model size improves performance on all metrics. (Bottom) Increasing batch size by using more GPUs generally improves performance across the board.