OpenT2M: No-frill Motion Generation with Open-source,Large-scale, High-quality Data

Bin Cao; Sipeng Zheng; Hao Luo; Boyuan Li; Jing Liu; Zongqing Lu

cs.CVMar 19, 2026

OpenT2M: No-frill Motion Generation with Open-source,Large-scale, High-quality Data

Authors: Bin Cao, Sipeng Zheng, Hao Luo, Boyuan Li, Jing Liu, Zongqing Lu

Organizations: CASIA · UCAS · BAAI · BeingBeyond · PKU · RUC

Abstract

Text-to-motion (T2M) generation aims to create realistic human movements from text descriptions, with promising applications in animation and robotics. Despite recent progress, current T2M models perform poorly on unseen text descriptions due to the small scale and limited diversity of existing motion datasets. To address this problem, we introduce OpenT2M, a million-level, high-quality, and open-source motion dataset containing over 2800 hours of human motion. Each sequence undergoes rigorous quality control through physical feasibility validation and multi-granularity filtering, with detailed second-wise text annotations. We also develop an automated pipeline for creating long-horizon sequences, enabling complex motion generation. Building upon OpenT2M, we introduce MonoFrill, a pretrained motion model that achieves compelling T2M results without complicated designs or technique tricks as "frills". Its core component is 2D-PRQ, a novel motion tokenizer that captures spatiotemporal dependencies by dividing the human body into biology parts. Experiments show that OpenT2M significantly improves generalization of existing T2M models, while 2D-PRQ achieves superior reconstruction and strong zero-shot performance. We expect OpenT2M and MonoFrill will advance the T2M field by addressing longstanding data quality and benchmarking challenges.

Figures & tables

	#Clips	#Hours	#Avg. Length	#long-horizon	#physically-feasible	#vision
BABEL [ 30 ]	52.9K	33.2	2.3s	✓	✓	✓
KIT [ 29 ]	5.7K	11.2	9.5s	✗	✓	✗
HumanML3D [ 11 ]	29.2K	28.6	7.1s	✗	✗	✗
Motion-X [ 21 ]	81.1K	144.2	6.4s	✗	✗	✓
MotionLib [ 38 ]	1.2M	1456.4	-	✗	✓	✓
MotionMillion [ 7 ]	2M	-	-	✗	✗	✓

Table 1: Comparison with existing human motion datasets, where “#physically-feasible” refers to the motion sequences that comply with physical laws and “#long-horizon” denotes the dataset that can serve as a long-horizon benchmark.

#Model	#training data	R@1 $\uparrow$	R@2 $\uparrow$	R@3 $\uparrow$	FID $\downarrow$	MMDist $\downarrow$	DIV $\uparrow$
Real	-	0.316	0.495	0.621	-	3.771	7.749
MDM	HumanML3D	0.065	0.126	0.180	51.307	7.642	3.040
MDM	Motion-X	0.055	0.107	0.160	56.257	8.008	3.019
MDM	OpenT2M	0.194	0.338	0.447	8.153	4.889	7.136
T2M-GPT	HumanML3D	0.070	0.130	0.186	62.036	8.093	2.586
T2M-GPT	Motion-X	0.063	0.120	0.173	53.464	7.770	2.957

Table 2: Comparison of zero-shot performance on OpenT2M zero using different datasets for training. Models trained on OpenT2M consistently present significant OOD improvements.

#Model	#LLM backbone	#pretrain	R@1 $\uparrow$	R@2 $\uparrow$	R@3 $\uparrow$	FID $\downarrow$	MMDist $\downarrow$	DIV $\uparrow$
Real	-	-	0.519	0.710	0.801	-	3.176	10.954
MonoFrill	GPT2-medium	-	0.078	0.148	0.212	61.809	8.803	4.810
MonoFrill	LlaMA2-7B	-	0.472	0.645	0.741	0.619	3.572	11.226
MonoFrill	LlaMA3-8B	-	0.503	0.694	0.792	0.546	3.224	11.104
MonoFrill	GPT2-medium	✓	0.215	0.316	0.377	17.91	7.129	8.372
MonoFrill	LlaMA2-7B	✓	0.485	0.676	0.773	0.435	3.386	11.373

Table 3: Comparison of motion instruction tuning on HumanML3D. We apply a limited number of training steps to avoid overfitting. Models with #pretrain consistently achieve significant improvements across diverse #LLM backbones .

Model	#text refinement	#long-horizon	R@1 $\uparrow$	R@2 $\uparrow$	R@3 $\uparrow$	FID $\downarrow$	MMDist $\downarrow$	DIV $\uparrow$
Real	✓	✓	0.573	0.740	0.822	-	2.842	10.450
MonoFrill	-	-	0.091	0.165	0.226	36.837	7.976	5.871
MonoFrill	-	✓	0.484	0.648	0.738	0.430	3.520	10.682
MonoFrill	✓	✓	0.510	0.677	0.765	0.297	3.322	10.748

Table 4: Comparison on OpenT2M long , where “#text refinement” refers to converting raw texts into cleaned user commands, "#long-horizon" denotes incorporating long-horizon motion data into OpenT2M .

#text refinement	R@1 $\uparrow$	R@2 $\uparrow$	R@3 $\uparrow$
-	0.520	0.709	0.801
✓	0.533	0.720	0.808

Table 5: Ablation of text refinement on HumanML3D

		HumanML3D		Motion-X		OpenT2M
Motion Tokenizer	Codebook Size	FID $\downarrow$	MPJPE $\downarrow$	FID $\downarrow$	MPJPE $\downarrow$	FID $\downarrow$	MPJPE $\downarrow$
$\text{VQ-VAE}_{1}$	1024	0.358	83.902	0.127	115.382	3.130	178.534
$\text{FSQ}_{1}$	65536	0.151	70.480	0.828	110.021	1.962	165.084
$\text{RQ-VAE}_{6}$	1024	0.031	48.696	0.013	67.390	0.080	96.753
$\text{RQ-VAE}_{8}$	1024	0.021	45.633	0.020	65.484	0.062	84.655
$\text{PRQ}_{4}$	1024	0.003	28.703	0.012	73.989	0.094	95.743

Table 6: Comparison of motion reconstruction on three benchmarks. Subscripts denote the number of quantization layers.

Model	LLM	R@1 $\uparrow$	R@2 $\uparrow$	R@3 $\uparrow$	FID $\downarrow$	MMDist $\downarrow$	DIV $\uparrow$
$\text{{MonoFrill}-VQ}_{1}$	GPT2-medium	0.257	0.410	0.513	11.226	5.146	7.393
$\text{{MonoFrill}-VQ}_{1}$	LlaMA2-7B	0.345	0.534	0.656	3.005	3.955	8.463
$\text{{MonoFrill}-VQ}_{1}$	LlaMA3-8B	0.345	0.534	0.656	2.979	3.960	8.437
$\text{{MonoFrill}-2D-PRQ}_{4}$	GPT2-medium	0.357	0.534	0.645	8.880	4.316	7.905
$\text{{MonoFrill}-2D-PRQ}_{4}$	LlaMA2-7B	0.491	0.675	0.777	0.475	2.962	9.450
$\text{{MonoFrill}-2D-PRQ}_{4}$	LlaMA3-8B	0.478	0.668	0.777	0.552	3.012	8.901

Table 7: Comparison of T2M on OpenT2M under different model parameters and motion tokenizers.

	HumanML3D		Motion-X
Motion Tokenizer	FID $\downarrow$	MPJPE $\downarrow$	FID $\downarrow$	MPJPE $\downarrow$
$\text{VQ-VAE}_{1}$	25.525	237.702	44.889	293.301
$\text{PRQ}_{4}$	2.169	135.964	5.020	167.508
$\text{2D-PRQ}_{4}$	0.107	77.695	1.606	108.921

Table 8: Zero-shot comparison of motion tokenizers.

Panel of Figure 4 — Figure 4: Statistics of the OpenT2M dataset. (a) Motion sequence distribution (log scale). (b) Average motion length distribution.

Figure 5: Length distribution comparison between OpenT2M long and BABEL datasets.

Explore similar work

Re $^2$ MoGen: Open-Vocabulary Motion Generation via LLM Reasoning and Physics-Aware Refinement
Apr 20, 2026Jiakun Zheng, Ting Xiao, Shiqin Cao +3Text-to-Motion Generation Human Motion Generation
1East China University of Science and Technology · 2Hong Kong University of Science and Technology · Institute of Artificial Intelligence (TeleAI), China Telecom
Plan, Don't Pose: Long Composite Motion Generation with Text-Aligned BFM
May 28, 2026Nikolay Shvetsov, Maksim Bobrin, Nazar Buzun +2Text-to-Motion Generation Human Motion Generation
AvaCapo, Potsdam, Germany · Potsdam University, Potsdam, Germany · Applied AI Institute, Computational Imaging Lab, Moscow, Russia +2
MoLingo: Motion-Language Alignment for Text-to-Human Motion Generation
Dec 15, 2025Yannan He, Garvita Tiwari, Xiaohan Zhang +4Latent Diffusion Models Text-to-Motion Generation
University of Tübingen, Germany · Tübingen AI Center, Germany · Max Planck Institute for Informatics, Saarland Informatics Campus, Germany +2

Apr 20, 2026J/K move · Enter open · S save

Re $^2$ MoGen: Open-Vocabulary Motion Generation via LLM Reasoning and Physics-Aware Refinement

Jiakun Zheng, Ting Xiao, Shiqin Cao, Xinran Li+2

1East China University of Science and Technology · 2Hong Kong University of Science and Technology · Institute of Artificial Intelligence (TeleAI), China Telecom

Text-to-Motion Generation Human Motion Generation

Open paper AskPDF arXiv

Text-to-motion (T2M) generation aims to control the behavior of a target character via textual descriptions. Leveraging text-motion paired datasets, existing T2M models have achieved impressive performance in generating high-quality motions within the distribution of their training data. However, their performance deteriorates notably when the motion descriptions differ significantly from the training texts. To address this issue, we propose Re

^2

MoGen, a Reasoning and Refinement open-vocabulary Motion Generation framework that leverages enhanced Large Language Model (LLM) reasoning to generate an initial motion planning and then refine its physical plausibility via reinforcement learning (RL) post-training. Specifically, Re

^2

MoGen consists of three stages: We first employ Monte Carlo tree search to enhance the LLM's reasoning ability in generating reasonable keyframes of the motion based on text prompts, specifying only the root and several key joints' positions to ease the reasoning process. Then, we apply a human pose model as a prior to optimize the full-body poses based on the planned keyframes and use the resulting incomplete motion to supervise fine-tuning a pre-trained motion generator via a dynamic temporal matching objective, enabling spatiotemporal completion. Finally, we use post-training with physics-aware reward to refine motion quality to eliminate physical implausibility in LLM-planned motions. Extensive experiments demonstrate that our framework can generate semantically consistent and physically plausible motions and achieve state-of-the-art performance in open-vocabulary motion generation.

Abstract

Explore similar work

Re2^22MoGen: Open-Vocabulary Motion Generation via LLM Reasoning and Physics-Aware Refinement

Plan, Don't Pose: Long Composite Motion Generation with Text-Aligned BFM

MoLingo: Motion-Language Alignment for Text-to-Human Motion Generation

Re $^2$ MoGen: Open-Vocabulary Motion Generation via LLM Reasoning and Physics-Aware Refinement