Human motion generation plays an important role in applications such as character animation, virtual environments, and embodied interaction. While existing approaches have achieved remarkable progress, many of them are developed for individual tasks, including text-to-motion, pose-conditioned generation, and trajectory control. Although these tasks involve different types of conditions, a unified framework capable of handling them within a common representation would greatly simplify motion generation systems. We observe that diverse motion conditions can be naturally formulated as different observation patterns over motion sequences, where each task corresponds to a specific masking strategy. Based on this insight, we introduce MotionMaestro, a unified motion generation framework that learns a shared representation for complete motions and heterogeneous partial observations through masked motion tokenization. MotionMaestro employs a three-stage training strategy that first learns a masked motion tokenizer, then refines its reconstruction ability on clean motions, and finally trains a conditional flow-matching generator in the learned latent space. Furthermore, we introduce an observation map and an observation loss to explicitly preserve provided motion conditions during generation. With this unified representation and conditioning mechanism, MotionMaestro supports text-guided and unconditional synthesis, pose conditioning and partial completion, temporal interpolation, trajectory control, and motion continuation. Experiments on the large-scale RoMo and MotionMillion datasets show state-of-the-art performance across diverse motion generation tasks.
Figures & tables
Figure 2: Motion observation patterns. Temporal, spatial, and feature-group masks describe the conditions for different motion generation tasks. Orange marks observed features, while gray denotes masked regions to be generated.
Figure 3: Overview of MotionMaestro. Stage 1 trains the tokenizer’s encoder and decoder through masked reconstruction. Stage 2 refines only the decoder on clean sequences with the encoder frozen. Stage 3 freezes the entire tokenizer and trains a conditional flow-matching generator. Complete targets and masked conditions share the same encoder, while an observation map identifies available motion information. The observation loss adds motion-space supervision on observed features.
Figure 4: Task-aligned masking for motion tokenization. Anatomical tube, temporal stride, feature-group, and short-window masking expose the tokenizer to the motion conditions associated with different tasks. Pink regions denote observed inputs, while hatched regions are masked.
Method
Venue
Uncond. FID ↓
T2M FID ↓
P2M FID ↓
TP2M FID ↓
FLF FID ↓
FI MPJPE ↓
Extrap. FID ↓
Partial MPJPE ↓
Trajectory FID ↓
Long FID ↓
MDM ( Tevet et al., 2022 )
ICLR2023
507.6
450.7
518.4
456.1
523.3
117.6
342.8
278.9
–
510.9
OmniControl ( Xie et al., 2024 )
ICLR2024
429.1
199.4
771.0
312.8
753.2
875.9
740.3
627.3
283.7
398.6
MotionLab ( Guo et al., 2025 )
ICCV2025
76.7
101.5
53.5
46.5
30.0
12.7
28.1
71.8
89.3
123.5
MotionMaestro (Ours)
–
20.4
14.9
34.8
28.7
20.8
6.7
19.4
61.0
59.2
26.2
Table 1: Unified motion generation on RoMo. Each method is evaluated on its supported tasks. FI and partial completion report MPJPE (mm) over unobserved frames and joints, respectively; the remaining tasks report FID. Lower is better. Bold and underlined values mark the best and second-best results per task. Results for MotionMaestro-5B, scaled up from our default 1.3B model, are provided in the Appendix .
Method
Uncond. FID ↓
T2M FID ↓
P2M FID ↓
TP2M FID ↓
FLF FID ↓
FI MPJPE ↓
Extrap. FID ↓
Partial MPJPE ↓
Trajectory FID ↓
Long FID ↓
Inference Time (s)
Memory GB
Params Billion (B)
MDM
427.7
297.6
523.8
334.4
530.6
122.8
370.4
206.8
–
336.2
4.69
0.4
0.081
OmniControl
376.3
153.9
366.8
193.8
365.5
375.4
551.3
349.6
276.6
177.9
61.58
0.5
0.102
MotionLab
135.1
61.4
74.4
49.4
50.7
12.3
38.8
71.1
94.4
108.6
0.97
2.6
0.367
MotionMaestro
67.2
36.2
74.3
52.8
44.5
3.8
36.9
61.3
118.2
46.4
1.28
11.7
1.285
Table 2: Unified motion generation and computational cost on MotionMillion. Evaluation settings, metrics, and notation follow Table 1 . We additionally report computational cost.
Figure 5: Qualitative comparison and zero-shot motion editing. We compare MotionMaestro with MDM and MotionLab on frame interpolation, partial completion, long sequence generation, and text-guided partial editing. The partial editing example demonstrates MotionMaestro’s editing capability without editing-specific training or fine-tuning. See the Appendix for additional results.
Table 7
Method
Uncond. FID ↓
T2M FID ↓
P2M FID ↓
TP2M FID ↓
FLF FID ↓
FI MPJPE ↓
Extrap. FID ↓
Partial MPJPE ↓
Trajectory FID ↓
Long FID ↓
Tokenizer training (Stage 1)
Clean sequences only
35.5
18.3
38.3
31.0
24.2
7.8
22.1
61.8
72.9
41.2
Random masking
22.3
15.6
37.8
30.6
21.0
7.5
21.3
63.7
59.7
36.2
Generator training (Stage 3)
w/o observation map Oc
18.6
15.0
36.9
29.6
20.9
6.2
19.6
61.3
57.5
30.9
w/o Lobs
21.8
15.2
35.9
29.8
20.1
6.8
19.9
61.1
58.9
28.7
Table 5: Ablations of tokenizer training and observation conditioning on RoMo. The upper block compares training on clean sequences only, random masking, and structured masking (MotionMaestro, bottom row). The lower block uses the same tokenizer trained with structured masking and separately removes the generator’s observation map input Oc or observation loss Lobs .
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
Section
Contents
Section A
Additional ablations and discussion
Section B
Additional experiments
Section C
Data and preprocessing
Section D
Architecture and training details
Section E
Evaluation protocol and baseline adaptations
Appendix
Table 6: Overview of the Appendix .
Tokenizer
MPJPE local (mm) ↓
MPJPE world (mm) ↓
Rotation (deg) ↓
Root path (mm) ↓
Final pos. (mm) ↓
Velocity (mm) ↓
Clean-only AE
2.339
3.887
0.407
1.872
4.500
0.216
Random-mask AE
2.440
5.392
0.423
3.339
7.469
0.924
Structured-mask AE (Ours)
3.392
6.199
0.455
3.807
8.899
0.740
Appendix
Table 7: Clean-motion reconstruction after Stage 1 on RoMo. We evaluate complete-motion reconstruction from unmasked inputs before decoder refinement. Lower is better for all metrics. Bold and underlined values denote the best and second-best results in each column, respectively.
Tokenizer
MPJPE local (mm) ↓
MPJPE world (mm) ↓
Rotation (deg) ↓
Root path (mm) ↓
Final pos. (mm) ↓
Velocity (mm) ↓
Clean-only AE
1.876
2.400
0.388
0.800
1.706
0.207
Random-mask AE
1.897
3.392
0.360
1.811
4.134
0.833
Structured-mask AE (Ours)
2.148
3.127
0.394
1.284
2.822
0.642
Appendix
Table 8: Clean-motion reconstruction after Stage 2 on RoMo. We evaluate the refined tokenizers used for downstream generation on unmasked inputs. Metrics and notation follow Table 7 .
Tokenizer
Anatomical tube ↓
Temporal stride ↓
Feature-group ↓
Mixed ↓
Clean-only AE
257.8
207.4
273.1
249.4
Random-mask AE
201.7
9.3
306.7
199.8
Structured-mask AE (Ours)
46.2
7.0
158.7
43.8
Appendix
Table 9: Masked-motion reconstruction after Stage 2. All tokenizers are evaluated on the same four masking families using their refined decoders, without the latent generator. We report MPJPE (mm) over hidden regions only. Lower is better; bold and underlined values denote the best and second-best results, respectively.
RoMo
MotionMillion
Stage
Local MPJPE (mm) ↓
Rotation error ( ∘ ) ↓
Path error (mm) ↓
Local MPJPE (mm) ↓
Rotation error ( ∘ ) ↓
Path error (mm) ↓
Stage 1
3.392
0.455
3.807
3.160
0.460
4.358
Stage 2
2.148
0.394
1.284
2.063
0.363
1.909
Appendix
Table 10: Effect of decoder refinement. MPJPE uses all valid joints and frames of clean inputs, and path error uses all valid frames. All metrics are lower-is-better; bold marks the better stage within each dataset.
Variant
FLF ↓
FI ↓
Extrap. ↓
Partial ↓
P2M ↓
TP2M ↓
Trajectory ↓
w/o observation map
5.50
4.32
2.92
3.56
6.25
5.92
8.06
w/o Lobs
5.78
3.78
2.98
3.74
6.40
5.94
7.93
MotionMaestro (Ours)
4.85
3.70
2.91
3.49
5.06
4.63
7.51
Appendix
Table 11: Observation preservation on RoMo. The first six columns report observed-joint MPJPE; Trajectory reports observed root- xz error. All values are in millimeters on 512 clips per task. All variants use the same frozen tokenizer. Bold and underlined values mark the lowest and second-lowest point estimates, not statistical significance.
Variant
FID ↓
Paired Δ vs. full model
Clean sequences only
17.910 ± 0.360
+3.335 ± 0.161
Random masking
15.928 ± 0.576
+1.352 ± 0.749
w/o observation map
14.918 ± 0.366
+0.342 ± 0.496
w/o Lobs
15.056 ± 0.047
+0.480 ± 0.324
MotionMaestro (Ours)
14.575 ± 0.283
–
Appendix
Table 12: RoMo T2M sampling variability. Mean and sample standard deviation over three sampling seeds. Paired Δ subtracts the full-model FID at the same seed; positive values indicate higher FID.
Method
Venue
Uncond. FID ↓
T2M FID ↓
P2M FID ↓
TP2M FID ↓
FLF FID ↓
FI MPJPE ↓
Extrap. FID ↓
Partial MPJPE ↓
Trajectory FID ↓
Long FID ↓
MDM ( Tevet et al., 2022 )
ICLR2023
507.6
450.7
518.4
456.1
523.3
117.6
342.8
278.9
–
510.9
OmniControl ( Xie et al., 2024 )
ICLR2024
429.1
199.4
771.0
312.8
753.2
875.9
740.3
627.3
283.7
398.6
MotionLab ( Guo et al., 2025 )
ICCV2025
76.7
101.5
53.5
46.5
30.0
12.7
28.1
71.8
89.3
123.5
MotionMaestro (Ours)
–
20.4
14.9
34.8
28.7
20.8
6.7
19.4
61.0
59.2
26.2
MotionMaestro-5B (Ours)
–
20.3
17.2
31.3
28.6
18.4
5.8
17.9
47.2
59.0
19.0
Appendix
Table 13: Unified motion generation including MotionMaestro-5B on RoMo. FI and Partial report MPJPE (mm) over unobserved frames and joints, respectively; the remaining tasks report FID. Lower is better; bold and underlined mark the best and second-best results per task.
Method
Uncond. FID ↓
T2M FID ↓
P2M FID ↓
TP2M FID ↓
FLF FID ↓
FI MPJPE ↓
Extrap. FID ↓
Partial MPJPE ↓
Trajectory FID ↓
Long FID ↓
Inference Time (s)
Memory GB
Params Billion (B)
MDM
427.7
297.6
523.8
334.4
530.6
122.8
370.4
206.8
–
336.2
4.69
0.4
0.081
OmniControl
376.3
153.9
366.8
193.8
365.5
375.4
551.3
349.6
276.6
177.9
61.58
0.5
0.102
MotionLab
135.1
61.4
74.4
49.4
50.7
12.3
38.8
71.1
94.4
108.6
0.97
2.6
0.367
MotionMaestro
67.2
36.2
74.3
52.8
44.5
3.8
36.9
61.3
118.2
46.4
1.28
11.7
1.285
MotionMaestro-5B
67.9
34.7
51.3
42.8
32.5
3.2
24.3
45.2
109.5
35.8
1.31
30.6
5.096
Appendix
Table 14: Unified motion generation and computational cost on MotionMillion. Evaluation settings, metrics, and notation follow Table 13 . We additionally report computational cost.
Method
RoMo
MotionMillion
s=2
s=8
s=16
s=32
s=2
s=8
s=16
s=32
MPJPE ↓
FID ↓
MPJPE ↓
FID ↓
MPJPE ↓
FID ↓
MPJPE ↓
FID ↓
MPJPE ↓
FID ↓
MPJPE ↓
FID ↓
MPJPE ↓
FID ↓
MPJPE ↓
FID ↓
MDM
59.3
61.9
252.2
282.5
375.2
403.6
431.3
464.0
75.4
128.7
203.5
301.0
285.9
401.1
330.4
454.8
OmniControl
803.2
747.8
929.2
782.7
943.3
776.1
930.6
745.8
383.9
503.6
372.0
473.5
373.9
457.7
373.6
426.0
MotionLab
11.1
10.7
18.9
14.1
35.4
21.4
57.5
30.4
11.6
9.4
15.7
11.4
26.1
16.9
47.0
27.8
MotionMaestro
4.8
3.9
12.7
4.7
30.1
12.9
54.1
18.6
2.8
1.5
7.5
1.8
20.3
5.9
43.1
15.2
Appendix
Table 15: Frame interpolation with varying observation stride on RoMo and MotionMillion. MPJPE (mm) is computed over unobserved frames; FID uses complete outputs. Lower is better; bold and underlined mark the best and second-best results per column.
Method
RoMo
MotionMillion
torso
head + hands
one arm
upper
head + hands
one arm
MPJPE ↓
FID ↓
MPJPE ↓
FID ↓
MPJPE ↓
FID ↓
MPJPE ↓
FID ↓
MPJPE ↓
FID ↓
MPJPE ↓
FID ↓
MDM
301.4
553.8
346.0
516.8
313.5
561.4
206.8
401.9
247.9
469.3
230.6
470.0
OmniControl
535.3
477.5
627.3
631.5
297.4
293.1
349.6
448.5
349.6
448.5
252.5
287.0
MotionLab
378.1
379.3
73.8
35.3
81.2
32.7
71.1
24.3
68.6
47.7
74.7
40.5
MotionMaestro
109.3
44.7
68.5
31.8
79.5
39.0
61.3
23.5
73.7
67.0
79.7
62.2
Appendix
Table 16: Partial completion under different observed body parts on RoMo and MotionMillion. We report MPJPE (mm) over unobserved joints and FID. Observed body parts are specified separately for each dataset. Lower is better; bold and underlined mark the best and second-best results, respectively.
RoMo
MotionMillion
Method
Foot skate ratio
GMD skate (mm/s)
Penetration (%)
Float (%)
Peak jerk
Accel. mean
Foot skate ratio
GMD skate (mm/s)
Penetration (%)
Float (%)
Peak jerk
Accel. mean
Ground Truth
0.611
76.5
0.77
25.31
561.5
4.83
0.509
63.1
0.66
25.42
225.9
2.17
MDM ( Tevet et al., 2022 )
0.961
163.5
1.59
40.17
2306.8
17.76
0.959
351.9
1.48
40.04
1675.7
11.23
OmniControl ( Xie et al., 2024 )
0.877
117.4
1.23
37.12
1023.3
7.98
0.877
130.4
0.91
30.73
622.6
5.18
MotionLab ( Guo et al., 2025 )
0.932
121.9
1.68
38.60
760.9
6.78
0.915
123.2
1.82
39.53
388.1
3.87
MotionMaestro (Ours)
0.841
82.4
0.75
26.97
571.4
6.55
0.760
69.9
0.59
25.71
289.9
3.60
Appendix
Table 17: Physical plausibility on RoMo and MotionMillion. Metrics are computed directly from joint positions without a learned evaluator. Foot skate measures the fraction of contact frames with foot movement. Penetration and float measure the percentage of frames whose lowest joint is more than 10 mm below or 15 mm above the floor, respectively. Bold and underlined mark the closest and second-closest generated results to ground truth per metric and dataset. Mean acceleration is displayed with a 10−3 scale factor.
RoMo
MotionMillion
Method
w
T2M FID ↓
T2M R@1 ↑
TP2M FID ↓
Long FID ↓
Long R@1 ↑
T2M FID ↓
T2M R@1 ↑
TP2M FID ↓
Long FID ↓
Long R@1 ↑
Ground truth
–
–
0.363
–
–
0.370
–
0.854
–
–
0.897
MDM
2.5
450.7
0.205
456.1
510.9
0.220
297.6
0.323
334.4
336.2
0.281
OmniControl
2.5
199.4
0.338
312.8
398.6
0.278
153.9
0.484
193.8
177.9
0.493
MotionLab
5.75
101.5
0.424
46.5
123.5
0.380
61.4
0.706
49.4
108.6
0.662
MotionMaestro
1.0 †
14.9
0.293
28.7
26.2
0.258
36.2
0.401
52.8
46.4
0.436
Appendix
Table 18: Effect of text guidance scale w on RoMo and MotionMillion. The null branch removes text while retaining all motion conditions. w=1 ( † ) is used for all our main comparisons; baselines retain their published scales. Bold marks each model’s lowest FID per task and dataset across the grid, including ties. These test-set minima are not used to select the main-table setting. Caption-free tasks are unaffected by w .
Setting
RoMo
MotionMillion
Training clips (train + validation)
773,240
744,906
Official test clips
40,677
131,812
Motion features per frame
272
272
Stage-3 maximum training length
384
300
Standard evaluation lengths (frames)
24–128
56–256
Long-sequence evaluation lengths (frames)
129–300
129–300
Appendix
Table 19: Data and sequence lengths. Counts refer to the cleaned splits used in our experiments, not the complete source releases. Standard evaluation lengths are the actual exported lengths after deterministic bucketing and center cropping.
Setting
Stage 1
Stage 2
Stage 3
Trainable components
Encoder + decoder
Decoder
Generator
Updates
100,000
20,000
200,000
Peak learning rate
10−4
2×10−5
10−4
Learning-rate warmup
2,000
500
2,000
Final / peak learning rate
0.1
0.1
0.1
Global padded-frame budget
49,152
49,152
8,192
Appendix
Table 20: Three-stage optimization settings. Settings for the default MotionMaestro model are shared across datasets unless specified in the text. Frame and token budgets are global upper budgets; the realized batch size depends on sequence length.
Task
Motion observation
Text
Main metric
Uncond.
None
No
FID
T2M
None
Yes
FID
P2M
First pose
No
FID
TP2M
First pose
Yes
FID
FLF
First and last poses
No
FID
FI
Periodic poses at spacing s=4
No
Hidden MPJPE
Appendix
Table 21: The ten main-paper evaluation tasks. A pose includes joint positions and rotations together with root placement and heading. Text is provided only for T2M, TP2M, and Long. MPJPE is evaluated on unobserved joint positions; FID uses the whole output sequence.
Figure 6: Additional qualitative comparison (1/5). We compare MDM, MotionLab, and the 1.3B and 5B variants of MotionMaestro. Rows show frame interpolation, partial completion, long-sequence generation, and text-guided partial editing, respectively. Orange denotes supplied motion observations. The rightmost column shows ground-truth motions for the first three rows and the source motion for partial editing. The editing prompt requests placing both hands on the hips.
Figure 7: Additional qualitative comparison (2/5). Methods, row order, and color conventions follow Figure 6 . The editing prompt requests crossing both arms over the chest while retaining the supplied lower-body motion.
Figure 8: Additional qualitative comparison (3/5). Methods, row order, and color conventions follow Figure 6 . The editing prompt requests placing both hands on the hips while retaining the supplied lower-body motion.
Figure 9: Additional qualitative comparison (4/5). Methods, row order, and color conventions follow Figure 6 . The editing prompt requests crossing both arms over the chest while retaining the supplied lower-body motion.
Figure 10: Additional qualitative comparison (5/5). Methods, row order, and color conventions follow Figure 6 . The editing prompt requests raising both arms overhead while retaining the supplied lower-body motion.
Generative models excel at motion synthesis for a fixed number of agents but struggle to generalize with variable agents. Based on limited, domain-specific data, existing methods employ autoregressive models to generate motion recursively, which suffer from inefficiency and error accumulation. We propose Unified Motion Flow (UMF), which consists of Pyramid Motion Flow (P-Flow) and Semi-Noise Motion Flow (S-Flow). UMF decomposes the number-free motion generation into a single-pass motion prior generation stage and multi-pass reaction generation stages. Specifically, UMF utilizes a unified latent space to bridge the distribution gap between heterogeneous motion datasets, enabling effective unified training. For motion prior generation, P-Flow operates on hierarchical resolutions conditioned on different noise levels, thereby mitigating computational overheads. For reaction generation, S-Flow learns a joint probabilistic path that adaptively performs reaction transformation and context reconstruction, alleviating error accumulation. Extensive results and user studies demonstrate UMF' s effectiveness as a generalist model for multi-person motion generation from text. Project page: https://githubhgh.github.io/umf/.
Text-to-motion generation is driven by learning motion representations for semantic alignment with language. Existing methods rely on either continuous or discrete motion representations. However, continuous representations entangle semantics with dynamics, while discrete representations lose fine-grained motion details. In this context, we propose FlowCoMotion, a novel motion generation framework that unifies both treatments from a modeling perspective. Specifically, FlowCoMotion employs token-latent coupling to capture both semantic content and high-fidelity motion details. In the latent branch, we apply multi-view distillation to regularize the continuous latent space, while in the token branch we use discrete temporal resolution quantization to extract high-level semantic cues. The motion latent is then obtained by combining the representations from the two branches through a token-latent coupling network. Subsequently, a velocity field is predicted based on the textual conditions. An ODE solver integrates this velocity field from a simple prior, thereby guiding the sample to the potential state of the target motion. Extensive experiments show that FlowCoMotion achieves competitive performance on text-to-motion benchmarks, including HumanML3D and SnapMoGen.
Dawei Guan, Di Yang, Chengjie Jin +1
School of Artificial Intelligence and Data Science, University of Science and Technology of China · Suzhou Institute for Advanced Research, University of Science and Technology of China
We present ScaleMoGen, a scale-wise autoregressive framework for text-driven human motion generation. Unlike conventional autoregressive approaches that rely on standard next-token prediction, ScaleMoGen frames motion generation as a coarse-to-fine process. We quantize 3D motions into compositional discrete tokens across multiple skeletal-emporal scales of increasing granularity, learning to generate motion by autoregressively predicting next-scale token maps. To maintain structural integrity, our motion tokenizers and quantizers are explicitly designed so that discrete tokens at every scale strictly preserve the skeletal hierarchy. Additionally, we employ bitwise quantization and prediction, which efficiently scale up the tokenizer vocabulary to preserve motion details and stabilize optimization. Extensive experiments demonstrate that ScaleMoGen achieves state-of-the-art performance, establishing an FID of 0.030 (vs. 0.045 for MoMask) on HumanML3D and a CLIP Score of 0.693 (vs. 0.685 for MoMask++) on the SnapMoGen dataset. Furthermore, we demonstrate that our skeletal-temporal multi-scale representation naturally facilitates training-free, text-guided motion editing.
Inwoo Hwang, Hojun Jang, Bing Zhou +3
Seoul National University · Snap Inc. · Meta Reality Labs