Discrete motion representations have substantially advanced autoregressive text-to-motion generation. However, most motion tokenizers are optimized for reconstruction and do not explicitly allocate capacity according to semantic role. Action-level meaning and fine-grained kinematic detail must therefore be encoded through the same reconstruction-driven hierarchy. We introduce SeMoCo, a semantic-first motion codec, together with a dual-axis motion generator for language-conditioned motion generation. Each motion token contains one semantic token and a residual sequence of kinematic tokens. The generator models semantic progression across time and autoregressively refines the residual entries. We also construct Ω-MotionVerse, a large-scale, multi-source human-motion dataset unified under the SOMA representation. Across the reported comparisons, SeMoCo achieves the best reconstruction accuracy among the compared codecs, while strong text-to-motion results demonstrate the effectiveness of its motion tokens for downstream generation.
Figures & tables
Figure 1: SeMoCo tokenizer. Given an input motion sequence, the encoder produces a continuous latent that is factorized into a semantic stream and a kinematic residual stream. The semantic stream is vector-quantized and aligned with the TMR-SOMA embedding space, whereas RVQ progressively quantizes the residual stream to preserve kinematic detail. Their quantized outputs are summed at the decoder input to reconstruct the motion; semantic supervision is used only during codec training.
Figure 2: Dual-axis motion language model. The temporal Transformer models language-conditioned motion packets across time, while the MTP module completes each packet from its semantic code to the kinematic residual codes. The generated packets are decoded into full-body motion.
Figure 3: Ω -MotionVerse. Composition of the supporting human motion dataset, comprising around 1,000 hours of text-paired motion sequences.
Figure 4: Qualitative text-to-motion comparison. Generated motion sequences from HY-Motion, Kimodo, and Ours-Base for three representative text prompts, visualized at matched timestamps.
Figure 5: Per-sequence reconstruction error. Cumulative distribution of MPJPE, with each method evaluated on its own test set: ours on the HumanML3D portion of our split ( n=2,108 ), MoMask and MotionGPT3 on the official HumanML3D test set ( n=4,372 ), and MotionMillion on its own test set ( n=4,042 ). Left: 0–100 mm. Right: the upper 12% of each distribution, where the dashed line marks the range of the left panel.
Figure 6: Per-frame vertical acceleration of the right wrist and right ankle on the most dynamic clip of our test set, shown over its busiest 150-frame window with all methods resampled to 30 fps and pelvis aligned. MotionMillion is not included.
Component
Ours-Lite
Ours-Base
Interpretation
Trainable parameters
188.0M
391.1M
Frozen text encoder excluded
Temporal layers
12
24
Base doubles temporal depth
Temporal hidden width
768
1,024
Base uses a wider temporal state
Temporal query/KV heads
6 / 3
8 / 4
128-D heads and 2:1 GQA
Temporal FFN width
2,048
2,730
Scaled with backbone width
Code-predictor width/layers
1,024 / 5
1,024 / 5
Fixed across sizes
Table 14: Ours-Lite and Ours-Base instantiate the same dual-axis generator at different temporal capacities. Parameter counts cover the trainable generator and exclude the frozen text encoder. Both variants are trained for 100,000 updates; reported results use the best validation checkpoint among the 1,000-step validation saves, which for both variants is step 99,000.
Setting
Value
Setting
Value
Optimizer
AdamW
Learning rate
2×10−4
Weight decay
0.01
Adam betas
(0.9,0.95)
LR schedule
4,000-step warmup + cosine
Gradient clipping
1.0
Precision
bf16
Batch size
32 per process
Gradient accumulation
None
Optimization steps
100,000
Data-parallel processes
1 (Lite) / 2 (Base)
Global batch size
32 (Lite) / 64 (Base)
Table 15: Generator training configuration. Ours-Lite and Ours-Base share the same data, objective, optimizer settings, update count, tokenizer, code predictor, and text encoder, and use the same per-process batch of 32. Ours-Lite is trained on one data-parallel process and Ours-Base on two, giving global batch sizes of 32 and 64.
Text-to-motion generation is driven by learning motion representations for semantic alignment with language. Existing methods rely on either continuous or discrete motion representations. However, continuous representations entangle semantics with dynamics, while discrete representations lose fine-grained motion details. In this context, we propose FlowCoMotion, a novel motion generation framework that unifies both treatments from a modeling perspective. Specifically, FlowCoMotion employs token-latent coupling to capture both semantic content and high-fidelity motion details. In the latent branch, we apply multi-view distillation to regularize the continuous latent space, while in the token branch we use discrete temporal resolution quantization to extract high-level semantic cues. The motion latent is then obtained by combining the representations from the two branches through a token-latent coupling network. Subsequently, a velocity field is predicted based on the textual conditions. An ODE solver integrates this velocity field from a simple prior, thereby guiding the sample to the potential state of the target motion. Extensive experiments show that FlowCoMotion achieves competitive performance on text-to-motion benchmarks, including HumanML3D and SnapMoGen.
Dawei Guan, Di Yang, Chengjie Jin +1
School of Artificial Intelligence and Data Science, University of Science and Technology of China · Suzhou Institute for Advanced Research, University of Science and Technology of China
We present ScaleMoGen, a scale-wise autoregressive framework for text-driven human motion generation. Unlike conventional autoregressive approaches that rely on standard next-token prediction, ScaleMoGen frames motion generation as a coarse-to-fine process. We quantize 3D motions into compositional discrete tokens across multiple skeletal-emporal scales of increasing granularity, learning to generate motion by autoregressively predicting next-scale token maps. To maintain structural integrity, our motion tokenizers and quantizers are explicitly designed so that discrete tokens at every scale strictly preserve the skeletal hierarchy. Additionally, we employ bitwise quantization and prediction, which efficiently scale up the tokenizer vocabulary to preserve motion details and stabilize optimization. Extensive experiments demonstrate that ScaleMoGen achieves state-of-the-art performance, establishing an FID of 0.030 (vs. 0.045 for MoMask) on HumanML3D and a CLIP Score of 0.693 (vs. 0.685 for MoMask++) on the SnapMoGen dataset. Furthermore, we demonstrate that our skeletal-temporal multi-scale representation naturally facilitates training-free, text-guided motion editing.
Inwoo Hwang, Hojun Jang, Bing Zhou +3
Seoul National University · Snap Inc. · Meta Reality Labs
Autoregressive (AR) models offer stable and efficient training, but standard next-token prediction is not well aligned with the temporal structure required for text-conditioned motion generation. We introduce MoScale, a next-scale AR framework that generates motion hierarchically from coarse to fine temporal resolutions. By providing global semantics at the coarsest scale and refining them progressively, MoScale establishes a causal hierarchy better suited for long-range motion structure. To improve robustness under limited text-motion data, we further incorporate cross-scale hierarchical refinement for improving per-scale initial predictions and in-scale temporal refinement for selective bidirectional re-prediction. MoScale achieves SOTA text-to-motion performance with high training efficiency, scales effectively with model size, and generalizes zero-shot to diverse motion generation and editing tasks.