cs.CVFeb 20, 2026
SaveTemporal Consistency-Aware Text-to-Motion Generation
Organizations: School of Computer Science and Engineering, Southeast University, Nanjing, China. · Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China. · State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing, China.
Abstract
Text-to-Motion (T2M) generation aims to synthesize realistic human motion sequences from natural language descriptions. While two-stage frameworks leveraging discrete motion representations have advanced T2M research, they often neglect cross-sequence temporal consistency, i.e., the shared temporal structures present across different instances of the same action. This leads to semantic misalignments and physically implausible motions. To address this limitation, we propose TCA-T2M, a framework for temporal consistency-aware T2M generation. Our approach introduces a temporal consistency-aware spatial VQ-VAE (TCaS-VQ-VAE) for cross-sequence temporal alignment, coupled with a masked motion transformer for text-conditioned motion generation. Additionally, a kinematic constraint block mitigates discretization artifacts to ensure physical plausibility. Experiments on HumanML3D and KIT-ML benchmarks demonstrate that TCA-T2M achieves state-of-the-art performance, highlighting the importance of temporal consistency in robust and coherent T2M generation.
Figures & tables
Figure 1: Illustration of temporal consistency across three distinct human action sequences. (a) A person walks forward; (b) A person walks and sits down; (c) A person sits down and stands up. Despite differences in kinematic details, these sequences exhibit shared temporal structures. Enforcing temporal alignment in the latent space ensures that motion representation encoder maps corresponding action phases across sequences to similar representations. This constraint enables the learned motion representation to capture semantic information while preserving temporal consistency, which is essential for the subsequent text-conditioned motion generation stage in T2M.
Figure 2: Method overview. (a) Temporal consistency-aware spatial VQ-VAE employs hierarchical residual quantization to discretize motion features, incorporates cycle-consistency constraints to enforce temporal coherence, and utilizes a kinematic constraint block to refine motion details. (b) Masked motion transformer adopts a dual-transformer structure for cross-modal text-motion synthesis. Specifically, the motion transformer restores masked motion tokens under CLIP text guidance, while the residual transformer predicts successive residual tokens using textual context and preceding token sequences.
| Method | Year | R-Precision | FID | MM Dist | Diversity | MModality | ||
| Top 1 | Top 2 | Top 3 | ||||||
| Real | – | 0.511 ±.003 | 0.703 ±.003 | 0.797 ±.002 | 0.002 ±.000 | 2.974 ±.008 | 9.503 ±.065 | - |
| T2M-GPT [ 21 ] | 2023 | 0.492 ±.003 | 0.679 ±.002 | 0.775 ±.002 | 0.141 ±.005 | 3.121 ±.009 | 9.722 ±.082 | 1.831 ±.048 |
| MotionGPT [ 45 ] | 2023 | 0.492 ±.003 | 0.681 ±.003 | 0.778 ±.002 | 0.232 ±.008 | 3.096 ±.008 | 9.528 ±.071 | 2.008 ±.084 |
| MotionGPT [ 46 ] | 2024 | 0.364 ±.005 | 0.533 ±.003 | 0.629 ±.004 | 0.805 ±.002 | 3.914 ±.013 | 9.972 ±.026 | 2.473 ±.041 |
| TM2T [ 14 ] | 2022 | 0.424 ±.003 | 0.618 ±.003 | 0.729 ±.002 | 1.501 ±.017 | 3.467 ±.011 | 9.920 ±.083 | 2.424 ±.093 |
Table 1: Comparison of text-to-motion generation performance on the HumanML3D dataset. The arrows , , and indicate that higher, lower, and closer-to-real-motion values are better, respectively. Bold and underline indicate the best and the second best result. The model marked with * represents results obtained by reproducing the code experiments from the original paper. FID assesses global motion quality. R-Precision measures text-motion alignment. MM-Dist quantifies cross-modal coherence between motion features and text embeddings. MModality evaluates intra-modal diversity, while Diversity measures global diversity.
| Method | Year | R-Precision | FID | MM Dist | Diversity | MModality | ||
| Top 1 | Top 2 | Top 3 | ||||||
| Real | – | 0.424 ±.005 | 0.649 ±.006 | 0.779 ±.006 | 0.031 ±.004 | 2.788 ±.012 | 11.08 ±.097 | - |
| T2M-GPT [ 21 ] | 2023 | 0.402 ±.006 | 0.619 ±.005 | 0.737 ±.006 | 0.717 ±.041 | 3.053 ±.026 | 10.86 ±.094 | 1.912 ±.036 |
| MotionGPT [ 45 ] | 2023 | 0.366 ±.005 | 0.558 ±.004 | 0.680 ±.005 | 0.510 ±.016 | 3.527 ±.021 | 10.350 ±.084 | 2.328 ±.117 |
| MotionGPT [ 46 ] | 2024 | 0.340 ±.002 | 0.570 ±.003 | 0.660 ±.004 | 0.868 ±.032 | 3.721 ±.018 | 9.972 ±.026 | 2.296 ±.022 |
| TM2T [ 14 ] | 2022 | 0.280 ±.005 | 0.463 ±.006 | 0.587 ±.005 | 3.599 ±.153 | 4.591 ±.026 | 9.473 ±.117 | 3.292 ±.081 |
Table 2: Comparison of text-to-motion generation performance on the KIT-ML dataset.
Figure 3: Qualitative comparisons between MDM [ 47 ] and our method across representative motion from the HumanML3D dataset. Key frames highlight critical motion details. The visual comparisons underscore our method’s strength in semantic comprehension of textual prompts and consistent action execution across multi-step sequences with dynamic environment adaptation.
| Method | Reconstruction | Generation | |
| FID | FID | MM-Dist | |
| Evaluation on the KIT-ML dataset | |||
| M2DM [ 59 ] | 0.413 ±.009 | 0.515 ±.029 | 3.015 ±.017 |
| T2M-GPT [ 21 ] | 0.472 ±.011 | 0.514 ±.029 | 3.007 ±.023 |
| Momask [ 8 ] | 0.184 ±.002 | 0.228 ±.009 | 3.021 ±.025 |
| Ours | 0.137 ±.003 | 0.198 ±.011 | 2.760 ±.016 |
Table 3: Comparison of motion reconstruction and generation performance. We present an analysis of the temporal cycle-consistency constraint (TCC), the kinematic constraint block (KCB) and the residual quantization(RQ) on the HumanML3D and KIT-ML datasets.
Figure 4: Visualizations of long motion generation and zero-shot motion generation. (a) Long motion generation. We integrate three text prompts—“a person walks forward then turns right”,“a person crawling from right to left” and“the person is walking in a counterclockwise circl”—to generate a long-sequence motion. (b) Zero-shot generation. We separately test the text prompts“A person is climbing a ladde” and“A person is rolling” to generate zero-shot motion sequences.
| Ablations | Type | Reconstruction | Generation | |
| FID | FID | MM-Dist | ||
| TCC loss | Classification | 0.113 ±.001 | 0.134 ±.004 | 3.031 ±.008 |
| Regression(MSE) | 0.025 ±.000 | 0.068 ±.003 | 2.951 ±.007 | |
| Regression(Huber) | 0.040 ±.000 | 0.082 ±.003 | 2.932 ±.007 | |
| Cycle length | 2 | 0.025 ±.000 | 0.068 ±.003 | 2.951 ±.007 |
| 3 | 0.030 ±.000 | 0.103 ±.004 | 2.997 ±.008 | |
Table 4: Ablation study on the impact of loss type, cycle length for the temporal cycle-consistency constraint, and codebook size ( ) on HumanML3D. denotes the number of codes, and is the embedding dimension. See Section 4.5 for details.
| Method | Kendall’s tau | Training time (s) | |
| HumanML3D | KIT-ML | ||
| Ours w/o TCC | 0.1757 | 0.3524 | 0.26 |
| Ours | 0.2571 | 0.4820 | 0.06 |
Table 5: Evaluation of temporal consistency for human motion generation. The training time is reported as the average time per iteration on an NVIDIA RTX 4090 GPU.
| Method | Model size (M) | FID | Inference time (s) | |
| Tokenizer | Total | |||
| TM2T [ 14 ] | 36.71 | 131.01 | 1.50 | 0.46 |
| T2M [ 1 ] | 3.44 | 40.29 | 1.09 | 0.02 |
| T2M-GPT [ 21 ] | 19.44 | 32.86 | 0.12 | 0.23 |
| Ours | 44.90 | 71.37 | 0.07 | 0.09 |
Table 6: Comparison of model parameters, inference efficiency, and motion generation quality.
Figure 5: Visualizations of failure cases of our approach. (a) shows that a person walks up stairs. Then the person turns right and walks back down stairs. (b) shows a man walking backwards. Then, he punches and kicks. (c) shows a man getting down on his hands and feet and crawling forward. Then, he turns around and crawls back before standing up again.