Organizations: School of Computer Science and Engineering, Southeast University, Nanjing, China. · Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China. · State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing, China.
Text-to-Motion (T2M) generation aims to synthesize realistic human motion sequences from natural language descriptions. While two-stage frameworks leveraging discrete motion representations have advanced T2M research, they often neglect cross-sequence temporal consistency, i.e., the shared temporal structures present across different instances of the same action. This leads to semantic misalignments and physically implausible motions. To address this limitation, we propose TCA-T2M, a framework for temporal consistency-aware T2M generation. Our approach introduces a temporal consistency-aware spatial VQ-VAE (TCaS-VQ-VAE) for cross-sequence temporal alignment, coupled with a masked motion transformer for text-conditioned motion generation. Additionally, a kinematic constraint block mitigates discretization artifacts to ensure physical plausibility. Experiments on HumanML3D and KIT-ML benchmarks demonstrate that TCA-T2M achieves state-of-the-art performance, highlighting the importance of temporal consistency in robust and coherent T2M generation.
Figures & tables
Figure 1: Illustration of temporal consistency across three distinct human action sequences. (a) A person walks forward; (b) A person walks and sits down; (c) A person sits down and stands up. Despite differences in kinematic details, these sequences exhibit shared temporal structures. Enforcing temporal alignment in the latent space ensures that motion representation encoder E maps corresponding action phases across sequences to similar representations. This constraint enables the learned motion representation to capture semantic information while preserving temporal consistency, which is essential for the subsequent text-conditioned motion generation stage in T2M.
Figure 2: Method overview. (a) Temporal consistency-aware spatial VQ-VAE employs hierarchical residual quantization to discretize motion features, incorporates cycle-consistency constraints to enforce temporal coherence, and utilizes a kinematic constraint block to refine motion details. (b) Masked motion transformer adopts a dual-transformer structure for cross-modal text-motion synthesis. Specifically, the motion transformer restores masked motion tokens under CLIP text guidance, while the residual transformer predicts successive residual tokens using textual context and preceding token sequences.
Method
Year
R-Precision ↑
FID ↓
MM Dist ↓
Diversity →
MModality ↑
Top 1
Top 2
Top 3
Real
–
0.511 ±.003
0.703 ±.003
0.797 ±.002
0.002 ±.000
2.974 ±.008
9.503 ±.065
-
T2M-GPT [ 21 ]
2023
0.492 ±.003
0.679 ±.002
0.775 ±.002
0.141 ±.005
3.121 ±.009
9.722 ±.082
1.831 ±.048
MotionGPT [ 45 ]
2023
0.492 ±.003
0.681 ±.003
0.778 ±.002
0.232 ±.008
3.096 ±.008
9.528 ±.071
2.008 ±.084
MotionGPT [ 46 ]
2024
0.364 ±.005
0.533 ±.003
0.629 ±.004
0.805 ±.002
3.914 ±.013
9.972 ±.026
2.473 ±.041
TM2T [ 14 ]
2022
0.424 ±.003
0.618 ±.003
0.729 ±.002
1.501 ±.017
3.467 ±.011
9.920 ±.083
2.424 ±.093
Table 1: Comparison of text-to-motion generation performance on the HumanML3D dataset. The arrows ↑ , ↓ , and → indicate that higher, lower, and closer-to-real-motion values are better, respectively. Bold and underline indicate the best and the second best result. The model marked with * represents results obtained by reproducing the code experiments from the original paper. FID assesses global motion quality. R-Precision measures text-motion alignment. MM-Dist quantifies cross-modal coherence between motion features and text embeddings. MModality evaluates intra-modal diversity, while Diversity measures global diversity.
Method
Year
R-Precision ↑
FID ↓
MM Dist ↓
Diversity →
MModality ↑
Top 1
Top 2
Top 3
Real
–
0.424 ±.005
0.649 ±.006
0.779 ±.006
0.031 ±.004
2.788 ±.012
11.08 ±.097
-
T2M-GPT [ 21 ]
2023
0.402 ±.006
0.619 ±.005
0.737 ±.006
0.717 ±.041
3.053 ±.026
10.86 ±.094
1.912 ±.036
MotionGPT [ 45 ]
2023
0.366 ±.005
0.558 ±.004
0.680 ±.005
0.510 ±.016
3.527 ±.021
10.350 ±.084
2.328 ±.117
MotionGPT [ 46 ]
2024
0.340 ±.002
0.570 ±.003
0.660 ±.004
0.868 ±.032
3.721 ±.018
9.972 ±.026
2.296 ±.022
TM2T [ 14 ]
2022
0.280 ±.005
0.463 ±.006
0.587 ±.005
3.599 ±.153
4.591 ±.026
9.473 ±.117
3.292 ±.081
Table 2: Comparison of text-to-motion generation performance on the KIT-ML dataset.
Figure 3: Qualitative comparisons between MDM [ 47 ] and our method across representative motion from the HumanML3D dataset. Key frames highlight critical motion details. The visual comparisons underscore our method’s strength in semantic comprehension of textual prompts and consistent action execution across multi-step sequences with dynamic environment adaptation.
Method
Reconstruction
Generation
FID ↓
FID ↓
MM-Dist ↓
Evaluation on the KIT-ML dataset
M2DM [ 59 ]
0.413 ±.009
0.515 ±.029
3.015 ±.017
T2M-GPT [ 21 ]
0.472 ±.011
0.514 ±.029
3.007 ±.023
Momask [ 8 ]
0.184 ±.002
0.228 ±.009
3.021 ±.025
Ours
0.137 ±.003
0.198 ±.011
2.760 ±.016
Table 3: Comparison of motion reconstruction and generation performance. We present an analysis of the temporal cycle-consistency constraint (TCC), the kinematic constraint block (KCB) and the residual quantization(RQ) on the HumanML3D and KIT-ML datasets.
Figure 4: Visualizations of long motion generation and zero-shot motion generation. (a) Long motion generation. We integrate three text prompts—“a person walks forward then turns right”,“a person crawling from right to left” and“the person is walking in a counterclockwise circl”—to generate a long-sequence motion. (b) Zero-shot generation. We separately test the text prompts“A person is climbing a ladde” and“A person is rolling” to generate zero-shot motion sequences.
Ablations
Type
Reconstruction
Generation
FID ↓
FID ↓
MM-Dist ↓
TCC loss
Classification
0.113 ±.001
0.134 ±.004
3.031 ±.008
Regression(MSE)
0.025 ±.000
0.068 ±.003
2.951 ±.007
Regression(Huber)
0.040 ±.000
0.082 ±.003
2.932 ±.007
Cycle length
2
0.025 ±.000
0.068 ±.003
2.951 ±.007
3
0.030 ±.000
0.103 ±.004
2.997 ±.008
Table 4: Ablation study on the impact of loss type, cycle length for the temporal cycle-consistency constraint, and codebook size ( K×d ) on HumanML3D. K denotes the number of codes, and d is the embedding dimension. See Section 4.5 for details.
Method
Kendall’s tau ↑
Training time (s)
HumanML3D
KIT-ML
Ours w/o TCC
0.1757
0.3524
0.26
Ours
0.2571
0.4820
0.06
Table 5: Evaluation of temporal consistency for human motion generation. The training time is reported as the average time per iteration on an NVIDIA RTX 4090 GPU.
Method
Model size (M)
FID ↓
Inference time (s)
Tokenizer
Total
TM2T [ 14 ]
36.71
131.01
1.50
0.46
T2M [ 1 ]
3.44
40.29
1.09
0.02
T2M-GPT [ 21 ]
19.44
32.86
0.12
0.23
Ours
44.90
71.37
0.07
0.09
Table 6: Comparison of model parameters, inference efficiency, and motion generation quality.
Figure 5: Visualizations of failure cases of our approach. (a) shows that a person walks up stairs. Then the person turns right and walks back down stairs. (b) shows a man walking backwards. Then, he punches and kicks. (c) shows a man getting down on his hands and feet and crawling forward. Then, he turns around and crawls back before standing up again.
Text-to-motion (T2M) generation aims to create realistic human movements from text descriptions, with promising applications in animation and robotics. Despite recent progress, current T2M models perform poorly on unseen text descriptions due to the small scale and limited diversity of existing motion datasets. To address this problem, we introduce OpenT2M, a million-level, high-quality, and open-source motion dataset containing over 2800 hours of human motion. Each sequence undergoes rigorous quality control through physical feasibility validation and multi-granularity filtering, with detailed second-wise text annotations. We also develop an automated pipeline for creating long-horizon sequences, enabling complex motion generation. Building upon OpenT2M, we introduce MonoFrill, a pretrained motion model that achieves compelling T2M results without complicated designs or technique tricks as "frills". Its core component is 2D-PRQ, a novel motion tokenizer that captures spatiotemporal dependencies by dividing the human body into biology parts. Experiments show that OpenT2M significantly improves generalization of existing T2M models, while 2D-PRQ achieves superior reconstruction and strong zero-shot performance. We expect OpenT2M and MonoFrill will advance the T2M field by addressing longstanding data quality and benchmarking challenges.
Text-to-motion generation, which converts motion language descriptions into coherent 3D human motion sequences, has attracted increasing attention in fields, such as avatar animation and humanoid robotic interaction. Though existing models have achieved significant fidelity, they still suffer from two core limitations: (i) They treat motion periodicity and keyframe saliency as independent factors, overlooking their coupling and causing generation drift in long sequences. (ii) They are fragile to semantically equivalent paraphrases, where minor synonym substitutions distort textual embeddings, propagating through the decoder and producing unstable or erroneous motions. In this work, we propose T2M Mamba to address these limitations by (i) proposing Periodicity-Saliency Aware Mamba, which utilizes novel algorithms for keyframe weight estimation via enhanced Density Peaks Clustering and motion periodicity estimation via FFT-accelerated autocorrelation to capture coupled dynamics with minimal computational overhead, and (ii) constructing a Periodic Differential Cross-modal Alignment Module (PDCAM) to enhance robust alignment of textual and motion embeddings. Extensive experiments on HumanML3D and KIT-ML datasets have been conducted, confirming the effectiveness of our approach, achieving an FID of 0.068 and consistent gains on all other metrics.
Xingzu Zhan, Chen Xie, Honghang Chen +2
Shenzhen University, Shenzhen, China · Jinan University, Guangzhou, China
Text-to-motion generation is driven by learning motion representations for semantic alignment with language. Existing methods rely on either continuous or discrete motion representations. However, continuous representations entangle semantics with dynamics, while discrete representations lose fine-grained motion details. In this context, we propose FlowCoMotion, a novel motion generation framework that unifies both treatments from a modeling perspective. Specifically, FlowCoMotion employs token-latent coupling to capture both semantic content and high-fidelity motion details. In the latent branch, we apply multi-view distillation to regularize the continuous latent space, while in the token branch we use discrete temporal resolution quantization to extract high-level semantic cues. The motion latent is then obtained by combining the representations from the two branches through a token-latent coupling network. Subsequently, a velocity field is predicted based on the textual conditions. An ODE solver integrates this velocity field from a simple prior, thereby guiding the sample to the potential state of the target motion. Extensive experiments show that FlowCoMotion achieves competitive performance on text-to-motion benchmarks, including HumanML3D and SnapMoGen.
Dawei Guan, Di Yang, Chengjie Jin +1
School of Artificial Intelligence and Data Science, University of Science and Technology of China · Suzhou Institute for Advanced Research, University of Science and Technology of China