Next-Scale Autoregressive Models for Text-to-Motion Generation

Zhiwei Zheng; Shibo Jin; Lingjie Liu; Mingmin Zhao

cs.CVApr 4, 2026

Next-Scale Autoregressive Models for Text-to-Motion Generation

Authors: Zhiwei Zheng, Shibo Jin, Lingjie Liu, Mingmin Zhao

Organizations: University of Pennsylvania

Abstract

Autoregressive (AR) models offer stable and efficient training, but standard next-token prediction is not well aligned with the temporal structure required for text-conditioned motion generation. We introduce MoScale, a next-scale AR framework that generates motion hierarchically from coarse to fine temporal resolutions. By providing global semantics at the coarsest scale and refining them progressively, MoScale establishes a causal hierarchy better suited for long-range motion structure. To improve robustness under limited text-motion data, we further incorporate cross-scale hierarchical refinement for improving per-scale initial predictions and in-scale temporal refinement for selective bidirectional re-prediction. MoScale achieves SOTA text-to-motion performance with high training efficiency, scales effectively with model size, and generalizes zero-shot to diverse motion generation and editing tasks.

Figures & tables

Figure 2: Overview of MoScale. (a) MoScale encodes motion sequences into discrete tokens from coarse to fine through multi-scale quantization. (b) It autoregressively predicts tokens at the next scale, conditioned on the prefix and text inputs, using hierarchical scale-wise causal attention. (c) Within each scale, MoScale performs temporal refinement to further improve token quality and consistency.

Method	Structure	Top1 $\uparrow$	Top2 $\uparrow$	Top3 $\uparrow$	FID $\downarrow$	MM-Dist $\downarrow$	Diversity $\rightarrow$	MModality $\uparrow$
Real Motion	-	$0.511^{\scriptsize.003}$	$0.703^{\scriptsize.003}$	$0.797^{\scriptsize.002}$	$0.002^{\scriptsize.000}$	$2.974^{\scriptsize.008}$	$9.503^{\scriptsize.065}$	-
TM2T [ 12 ]	Next-Token AR	$0.424^{\scriptsize.003}$	$0.618^{\scriptsize.003}$	$0.729^{\scriptsize.002}$	$1.501^{\scriptsize.017}$	$3.467^{\scriptsize.011}$	$8.589^{\scriptsize.076}$	$2.090^{\scriptsize.083}$
T2M-GPT [ 55 ]		$0.492^{\scriptsize.003}$	$0.679^{\scriptsize.002}$	$0.775^{\scriptsize.002}$	$0.141^{\scriptsize.005}$	$3.121^{\scriptsize.009}$	$9.722^{\scriptsize.082}$	$2.424^{\scriptsize.093}$
AttT2M [ 60 ]		$0.499^{\scriptsize.003}$	$0.690^{\scriptsize.002}$	$0.786^{\scriptsize.002}$	$0.112^{\scriptsize.006}$	$3.038^{\scriptsize.007}$	$9.700^{\scriptsize.090}$	$1.831^{\scriptsize.048}$
ParCo [ 61 ]		$0.515^{\scriptsize.003}$	$0.706^{\scriptsize.003}$	$0.801^{\scriptsize.002}$	$0.109^{\scriptsize.005}$	$2.927^{\scriptsize.008}$	$9.576^{\scriptsize.088}$	$\underline{2.452}^{\scriptsize.051}$
MDM [ 45 ]	Diffusion	$0.418^{\scriptsize.005}$	$0.604^{\scriptsize.005}$	$0.707^{\scriptsize.004}$	$0.489^{\scriptsize.025}$	$3.630^{\scriptsize.023}$	${9.450}^{\scriptsize.066}$	$1.382^{\scriptsize.060}$

Table 1: Performance on HumanML3D. We report the average result over 20 runs with 95% confidence interval. Bold for the best and underline for the second.

\rightarrow

indicates that values closer to real motion correspond to better results.

S

is the total steps across all scales.

Method	Structure	Top1 $\uparrow$	Top2 $\uparrow$	Top3 $\uparrow$	FID $\downarrow$	MM-Dist $\downarrow$	Diversity $\rightarrow$	MModality $\uparrow$
Real Motion	-	$0.424^{\scriptsize.005}$	$0.649^{\scriptsize.006}$	$0.779^{\scriptsize.006}$	$0.031^{\scriptsize.004}$	$2.788^{\scriptsize.012}$	$11.080^{\scriptsize.097}$	-
TM2T [ 12 ]	Next-token AR	$0.280^{\scriptsize.005}$	$0.463^{\scriptsize.006}$	$0.587^{\scriptsize.005}$	$3.599^{\scriptsize.153}$	$4.591^{\scriptsize.026}$	$9.473^{\scriptsize.117}$	$\textbf{3.292}^{\scriptsize.081}$
T2M-GPT [ 55 ]		$0.416^{\scriptsize.006}$	$0.627^{\scriptsize.006}$	$0.745^{\scriptsize.006}$	$0.514^{\scriptsize.029}$	$3.007^{\scriptsize.023}$	$10.921^{\scriptsize.108}$	$1.570^{\scriptsize.039}$
AttT2M [ 60 ]		$0.413^{\scriptsize.006}$	$0.632^{\scriptsize.006}$	$0.751^{\scriptsize.006}$	$0.870^{\scriptsize.039}$	$3.039^{\scriptsize.021}$	$10.960^{\scriptsize.123}$	$\underline{2.281}^{\scriptsize.047}$
ParCo [ 61 ]		$0.430^{\scriptsize.004}$	$0.649^{\scriptsize.007}$	$0.772^{\scriptsize.006}$	$0.453^{\scriptsize.027}$	$2.820^{\scriptsize.028}$	$10.950^{\scriptsize.094}$	$1.245^{\scriptsize.022}$
MDM [ 45 ]	Diffusion	-	-	$0.396^{\scriptsize.004}$	$0.497^{\scriptsize.021}$	$9.191^{\scriptsize.022}$	$10.847^{\scriptsize.109}$	$1.907^{\scriptsize.214}$

Table 2: Performance on KIT-ML test set. Bold for the best result and underline for the second best.

Method	FULL	MEDIUM+HIGH	HIGH
ParCo	0.801	0.778	0.709
MoMask++	$0.811_{\scriptsize.010\uparrow}$	$0.802_{\scriptsize.024\uparrow}$	$0.762_{\scriptsize.053\uparrow}$
Ours	$\textbf{0.817}_{\scriptsize.016\uparrow}$	$\textbf{0.812}_{\scriptsize.034\uparrow}$	$\textbf{0.775}_{\scriptsize.066\uparrow}$

Table 3: Quantitative comparison on the HumanML3D test set using the Top-3 accuracy. Test samples are ranked by text complexity, and each method is evaluated on progressively more challenging subsets. Subscripts denote absolute improvements over ParCo.

Figure 3: Comparison of Top-1 text alignment and training time on HumanML3D [ 11 ] . We provide four model sizes, tiny, small, medium, and large, for our method and T2M-GPT [ 55 ] .

Figure 4: Motion editing results. MoScale achieves better instruction adherence and retains unedited motion (shown in gray).

Ablation	Top-1 $\uparrow$	FID $\downarrow$	MM-Dist $\downarrow$
Base	$0.481^{\scriptsize.003}$	$0.176^{\scriptsize.007}$	$3.136^{\scriptsize.009}$
HR	$0.534^{\scriptsize.002}$	$0.090^{\scriptsize.005}$	$2.853^{\scriptsize.006}$
TR	$0.497^{\scriptsize.003}$	$0.129^{\scriptsize.005}$	$3.043^{\scriptsize.008}$
HR & TR	$0.540^{\scriptsize.002}$	$0.046^{\scriptsize.002}$	$2.830^{\scriptsize.005}$
$\gamma_{\max}$ = 0.2	$0.529^{\scriptsize.003}$	$0.096^{\scriptsize.004}$	$2.858^{\scriptsize.008}$
$\gamma_{\max}$ = 0.4	$0.535^{\scriptsize.002}$	$0.075^{\scriptsize.004}$	$2.837^{\scriptsize.005}$

Table 4: Ablation studies. 1) hierarchical and temporal refinement (HR and TR), 2) corruption rate, and 3) transformer depth.

Explore similar work

ScaleMoGen: Autoregressive Next-Scale Prediction for Human Motion Generation
May 12, 2026Inwoo Hwang, Hojun Jang, Bing Zhou +3Text-to-Motion Generation Human Motion Generation
Seoul National University · Snap Inc. · Meta Reality Labs
Coordinate-Based Dual-Constrained Autoregressive Motion Generation
Apr 9, 2026Kang Ding, Hongsong Wang, Jie Gui +1Text-to-Motion Generation Human Motion Generation
School of Cyber Science and Engineering, Southeast University, Nanjing 210096, China · School of Computer Science and Engineering, Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications, Ministry of Education, Southeast University, Nanjing 210096, China · School of Cyber Science and Engineering, Southeast University, also with Purple Mountain Laboratories, Nanjing 211111, China, and also with Engineering Research Center of Blockchain Application, Supervision And Management (Southeast University), Ministry of Education, Southeast University, Nanjing 210096, China +1
MotionHiFlow: Text-to-motion via hierarchical flow matching
Apr 25, 2026Heng Li, Xiaotong Lin, Ling-An Zeng +3Flow Matching Text-to-Motion Generation
Sun Yat-sen University · Shandong University · Guangdong Province Key Laboratory of Information Security Technology, China +1

May 12, 2026J/K move · Enter open · S save

ScaleMoGen: Autoregressive Next-Scale Prediction for Human Motion Generation

Inwoo Hwang, Hojun Jang, Bing Zhou, Jian Wang+2

Seoul National University · Snap Inc. · Meta Reality Labs

Text-to-Motion Generation Human Motion Generation Motion Representation Learning

Open paper AskPDF arXiv

We present ScaleMoGen, a scale-wise autoregressive framework for text-driven human motion generation. Unlike conventional autoregressive approaches that rely on standard next-token prediction, ScaleMoGen frames motion generation as a coarse-to-fine process. We quantize 3D motions into compositional discrete tokens across multiple skeletal-emporal scales of increasing granularity, learning to generate motion by autoregressively predicting next-scale token maps. To maintain structural integrity, our motion tokenizers and quantizers are explicitly designed so that discrete tokens at every scale strictly preserve the skeletal hierarchy. Additionally, we employ bitwise quantization and prediction, which efficiently scale up the tokenizer vocabulary to preserve motion details and stabilize optimization. Extensive experiments demonstrate that ScaleMoGen achieves state-of-the-art performance, establishing an FID of 0.030 (vs. 0.045 for MoMask) on HumanML3D and a CLIP Score of 0.693 (vs. 0.685 for MoMask++) on the SnapMoGen dataset. Furthermore, we demonstrate that our skeletal-temporal multi-scale representation naturally facilitates training-free, text-guided motion editing.

Abstract

Explore similar work

ScaleMoGen: Autoregressive Next-Scale Prediction for Human Motion Generation

Coordinate-Based Dual-Constrained Autoregressive Motion Generation

MotionHiFlow: Text-to-motion via hierarchical flow matching