Timo: Taming Multimodal Diffusion Transformers for Human Motion Generation

Zhao Wang; Jiangtao Hu; Jack Yu; Tao Yu

cs.CVSep 25, 2026

Timo: Taming Multimodal Diffusion Transformers for Human Motion Generation

Authors: Zhao Wang, Jiangtao Hu, Jack Yu, Tao Yu

Organizations: LimX Dynamics

Abstract

Most existing human motion generation (HMG) methods use cross-attention modules to inject text semantics, but ignore the importance of bidirectional modeling between motion and text tokens, which limits text comprehension. A straightforward idea is introducing multimodal diffusion transformers (MMDiT), which have shown effective joint text--visual modeling in vision generation, into HMG. However, we find that articulated motion is temporally coherent but weakly correlated across joints, in which directly applying an MMDiT with flow matching produces poorly coordinated and jerky motion. In this work, we propose Timo, a novel kinematics-aware MMDiT framework tailored for HMG. Timo combines fully shared multimodal attention for bidirectional text--motion modeling with flow matching, geometric and rotational-kinematics supervision that compares actual rotations and their changes over time, and a two-stage curriculum progressing from broad motion learning to detailed caption alignment. Further, we construct a benchmark of $40{,}025$ held-out clips from six public datasets spanning diverse actions, assessing six complementary dimensions under a common evaluator and scoring protocol. Our model substantially outperforms state-of-the-art methods in both quantitative and qualitative evaluations. Remarkably, Timo surpasses Kimodo on five of six dimensions, achieving a $40.8$ % relative improvement in the average benchmark score. Project page: https://kyfafyd.wang/projects/timo. Demo page: https://timo.kyfafyd.wang.

Figures & tables

Figure 1: Comparison on our six-dataset benchmark. Timo and four released systems: MotionMillion ( Fan et al., 2025 ) , HY-Motion ( Wen et al., 2025 ) , GENMO ( Li et al., 2025 ) and Kimodo ( Rempe et al., 2026 ) .

Figure 2: Kinematics-aware multimodal generation. (a) Text and motion tokens enter one $24$ -block stack as a single sequence predicting clean body parameters $\hat{\mathbf{x}}_{0}$ directly. (b) A block shares QKV, MLP and adaLN ( Peebles and Xie, 2023 ) weights across modalities. (c) Reconstruction, rotational kinematics and decoded geometry provide complementary supervision. At right, predictions with equal reconstruction loss but different temporal errors, distinguished by $\mathcal{L}_{\text{kin}}$ .

Figure 3: Corpus composition. (a) Motion categories over all $1{,}983$ hours; the three rings are family, subfamily and action, and arc is hours. (b) Caption length over all corpora, weighted by hours. (c) Our own captions’ vocabulary. Hours include mirrored augmentation and held-out splits.

Method	Venue	Params	Nat. $\uparrow$	Match $\uparrow$	Div. $\uparrow$	Smth. $\uparrow$	Speed $\uparrow$	Plaus. $\uparrow$	Avg. $\uparrow$
MotionMillion ( Fan et al., 2025 )	ICCV’25	$7.8$ B	42.3	30.5	51.7	49.5	45.5	72.1	48.6
HY-Motion ( Wen et al., 2025 )	arXiv’25	$1.0$ B	38.0	52.6	67.3	54.3	31.4	71.4	52.5
GENMO ( Li et al., 2025 )	ICCV’25	$523$ M	44.2	28.0	52.3	79.8	20.6	68.1	48.8
Kimodo ( Rempe et al., 2026 )	arXiv’26	$283$ M	36.8	51.0	59.1	79.0	49.5	93.4	61.5
Timo (ours)		$495$ M	86.4	86.8	89.3	90.7	81.4	85.0	86.6

Table 1: Comparison on our six-dataset benchmark. Released multi-corpus systems are re-evaluated, not quoted from their papers. Training data and representations differ; ours includes in-house capture. Axes are custom

0

–

100

scores, higher better; differences and relative gains refer to these scores, not the underlying raw metrics. Params excludes frozen text encoders. Avg. is the unweighted mean of the six scores.

Figure 4: Qualitative comparison. One model per row, one prompt per column, moments left to right, shaded light to dark as time advances; a column shares one camera, so a body that sits lower or reaches further has really done so. Fig. S2 carries two further prompts.

Table 6

Table 7

Figure 5: Long-sequence application: twelve prompts over $36$ seconds. Travelling and in-place prompts alternate, and Timo generates them as a single motion, drawn in one scene with a common camera and scale. Displayed poses are aligned to the floor and shaded light to dark with time. Arrows span the travelling prompts.

Figure 6: Execution on two physical humanoids. Left: LimX Luna performs a guiding gesture. Right: LimX Oli walks, squats and stands. Generated references are retargeted and tracked by the robot controller; Fig. S4 provides the full examples.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Value
Backbone	$24$ blocks, width $1024$ , $16$ heads, $495$ M parameters
Feed-forward	SwiGLU, hidden width $2752$
Motion	$151$ features at $30$ fps; $240$ frames per window
Text	At most $50$ token embeddings of width $1024$
Optimiser	AdamW ( Loshchilov and Hutter, 2019 ) , learning rate $2\times 10^{-4}$
Schedule	$100$ k steps per stage; rate halved at $50$ k and $80$ k

Appendix

Table S1: Model and optimisation settings shared by both training stages.

Group	Term	Weight
Reconstruction	Clean-motion error; min-SNR truncation $\gamma=5$	$1$
Root rotation	Absolute / velocity / acceleration	$5\,/\,10\,/\,10$
Body rotation	Velocity / acceleration	$1\,/\,1$
Geometry	3D joints / vertices	$500\,/\,500$
	2D reprojection	$1000$
	Translation / contact BCE	$1\,/\,1$

Appendix

Table S2: Objective weights. The objective is unchanged between stages. Huber terms use

\beta=0.05

; contact labels use a foot-velocity threshold of

0.15

m/s.

Dataset	Source	Caption	Tier	Hours †
HumanML3D ( Guo et al., 2022 )	studio	clip	fine	52.8
BABEL ( Punnakkal et al., 2021 )	studio	clip	fine	77.7
AIST++ ( Li et al., 2021 )	studio	clip	fine	0.4
GRAB ( Taheri et al., 2020 )	studio	clip	fine	7.5
PerMo ( Kim et al., 2025 )	studio	clip	fine	17.0
BONES ( Luo et al., 2026 )	studio	clip	fine	152.3

Appendix

Table S3: Corpora. Source is the acquisition route, studio capture or in-the-wild video reconstruction, and Caption what the text describes: a clip, a vision–language model’s label, or a multi-minute take. StudioTake and StudioClip are our own in-house capture ( Fig. 3 ); every other row is a public corpus. † Hours include left–right mirroring across train and held-out splits; BONES already contains mirror twins and is counted once.

Method	R@1 $\uparrow$	R@2 $\uparrow$	R@3 $\uparrow$	Seg. FID $\downarrow$	Trans. FID $\downarrow$	MM-Dist $\downarrow$	Pen. $\downarrow$
MotionMillion	25.3	36.9	44.3	0.201	0.409	1.237	13.31
HY-Motion	38.9	54.0	62.7	0.233	0.478	1.187	13.90
GENMO	22.7	34.6	43.0	0.212	0.338	1.236	16.83
Kimodo	37.4	53.1	62.6	0.319	0.364	1.199	1.42
Timo	62.1	77.6	85.2	0.053	0.034	1.055	4.93

Appendix

Table S4: Raw metrics underlying the main comparison. R@1/2/3 are percentages within

32

-candidate retrieval groups. Fréchet distances (FID) and text–motion distance (MM-Dist) use the same six-corpus evaluator for every row, so they are not directly comparable to numbers from a differently trained evaluator. Pen. is the capsule-interpenetration proxy, multiplied by

10^{3}

for readability, not a collision frequency.

Panel of Figure 14 — Figure S1: Additional long-sequence examples. Top: five prompts over approximately $20$ seconds. Bottom: twelve prompts over $36$ seconds. As in Fig. 5 , poses share a scene and darken with time; each displayed pose is aligned to the floor. The lower sequence also includes motion postprocessing ( Section D.1 ).

Figure S2: Additional qualitative comparisons. Waving and stepping prompts across five methods, with six displayed poses per result and a common body scale.

Figure S3: Timo across fourteen prompts : six moments each, shaded light to dark. One panel height is fixed for the whole figure, so the $0.5$ m grid is the same size in every cell and amplitude compares across prompts as well as within one.

Figure S4: Four motions executed on physical humanoids. Samples from Timo, retargeted to the robot’s rig and tracked by the whole-body controller of Fu et al. (2026) ; eight moments per row. The first two rows are a LimX Luna ( LimX Dynamics, 2026 ) and the last two a LimX Oli ( LimX Dynamics, 2025 ) . The generator is shared across platforms; retargeting and tracking adapt its outputs to each robot. Robot images are not shown at a common physical scale.

Explore similar work

Dec 15, 2025

MoLingo: Motion-Language Alignment for Text-to-Human Motion Generation

We introduce MoLingo, a text-to-motion (T2M) model that generates realistic, lifelike human motion by denoising in a continuous latent space. Recent works perform latent space diffusion, either on the whole latent at once or auto-regressively over multiple latents. In this paper, we study how to make diffusion on continuous motion latents work best. We focus on two questions: (1) how to build a semantically aligned latent space so diffusion becomes more effective, and (2) how to best inject text conditioning so the motion follows the description closely. We propose a semantic-aligned motion encoder trained with frame-level text labels so that latents with similar text meaning stay close, which makes the latent space more diffusion-friendly. We also compare single-token conditioning with a multi-token cross-attention scheme and find that cross-attention gives better motion realism and text-motion alignment. With semantically aligned latents, auto-regressive generation, and cross-attention text conditioning, our model sets a new state of the art in human motion generation on standard metrics and in a user study. We will release our code and models for further research and downstream usage.

Yannan He, Garvita Tiwari, Xiaohan Zhang +4

University of Tübingen, Germany · Tübingen AI Center, Germany · Max Planck Institute for Informatics, Saarland Informatics Campus, Germany +2

Latent Diffusion Models Text-to-Motion Generation Cross-Modal Alignment Motion Diffusion Models Human Motion Generation Diffusion Models for Motion Generation

Ask PDF

Mar 19, 2026

OpenT2M: No-frill Motion Generation with Open-source,Large-scale, High-quality Data

Text-to-motion (T2M) generation aims to create realistic human movements from text descriptions, with promising applications in animation and robotics. Despite recent progress, current T2M models perform poorly on unseen text descriptions due to the small scale and limited diversity of existing motion datasets. To address this problem, we introduce OpenT2M, a million-level, high-quality, and open-source motion dataset containing over 2800 hours of human motion. Each sequence undergoes rigorous quality control through physical feasibility validation and multi-granularity filtering, with detailed second-wise text annotations. We also develop an automated pipeline for creating long-horizon sequences, enabling complex motion generation. Building upon OpenT2M, we introduce MonoFrill, a pretrained motion model that achieves compelling T2M results without complicated designs or technique tricks as "frills". Its core component is 2D-PRQ, a novel motion tokenizer that captures spatiotemporal dependencies by dividing the human body into biology parts. Experiments show that OpenT2M significantly improves generalization of existing T2M models, while 2D-PRQ achieves superior reconstruction and strong zero-shot performance. We expect OpenT2M and MonoFrill will advance the T2M field by addressing longstanding data quality and benchmarking challenges.

Bin Cao, Sipeng Zheng, Hao Luo +3

CASIA · UCAS · BAAI +3

Text-to-Motion Generation Human Motion Generation

Ask PDF

Sep 19, 2026

MixiMotion: One-Step Text-to-Motion Generation via Asymmetric Set Distillation

Iterative text-to-motion generation delivers high-quality and semantically aligned motions but requires multiple network evaluations, resulting in substantial inference latency. We present MixiMotion, a strict one-step text-to-motion generation framework based on offline set distillation. Instead of distilling a single teacher trajectory for each text prompt, MixiMotion constructs an offline bank of multiple teacher motions and aligns teacher and student sample sets through asymmetric bidirectional matching. The teacher-to-student direction promotes coverage of diverse teacher-supported motions, while the student-to-teacher direction suppresses unsupported generations. We further introduce differentiable decoded-space kinematic supervision to complement normalized representation matching with constraints in the decoded motion space. At inference, MixiMotion generates a complete motion sequence with a single network evaluation, without teacher queries, iterative sampling, or candidate ranking. On ViMoGen, MixiMotion achieves a semantic alignment score of

0.835

, outperforming the evaluated one-step baselines and approaching the

0.858

score of its 50-step HY-Motion-1.0-Lite teacher. In blinded human evaluation, MixiMotion obtains an overall rating of

4.33

, compared with

4.50

for the teacher, while outperforming the evaluated one-/few-step baselines. Meanwhile, generation latency is reduced from

829.58

,ms to

9.30

,ms, corresponding to an

89.2\times

speedup. These results demonstrate an effective quality--efficiency trade-off for strict one-step text-to-motion generation.

Hung Dinh, Binh Mai, Tran Quoc Bao Le +2

Posts and Telecommunications Institute of Technology, Hanoi, Vietnam

Text-to-Motion Generation Human Motion Generation One-Step Generative Modeling Knowledge Distillation

Ask PDF