Organizations: The Hong Kong University of Science and Technology (Guangzhou) · Shanghai Jiao Tong University · Duke University · University of California, Berkeley
Dance motion combines slowly varying body configurations with rapid local movements. We investigate how explicit temporal-frequency decomposition can guide the temporal resolution and codebook capacity of discrete motion representations. We introduce frequency-factorized motion tokenization (FFMT), which separates root-relative motion into a low-frequency choreographic scaffold and a complementary high-frequency residual. FFMT uses a lower token rate and a smaller codebook for the low-frequency component, and a higher token rate and a larger codebook for the high-frequency residual. This representation further induces a plan-then-articulate autoregressive factorization: low-frequency tokens establish coarse choreography, while high-frequency tokens realize local motion details conditioned on the generated plan and music. Experiments on AIST++ and FineDance show strong distributional fidelity while preserving realistic diversity and music--motion synchronization. Ablations consistently support frequency factorization, asymmetric temporal allocation, and low-to-high conditioning.
Figures & tables
Figure 1: Overview of FFMT. (a) Frequency-factorized tokenization assigns different temporal resolutions and codebook sizes to low-frequency structure and high-frequency residuals, with separate upper/lower-body codebooks. (b) Plan-then-articulate generation predicts low-frequency tokens first and conditions high-frequency generation on the aligned low-frequency context and music. The resulting tokens are decoded into rotations and root motion, with a training-derived prior used during inference.
Method
FID k↓
FID g↓
Div k→
Div g→
BAS →
AIST++
Ground Truth
–
–
8.19
7.45
0.2374
Bailando++ [ 11 ]
21.29
11.56
9.31
6.20
0.2330
EDGE [ 12 ]
42.16
22.12
3.96
4.61
0.2334
Lodge [ 5 ]
37.09
18.79
5.58
4.85
0.2423
TokenDance [ 13 ]
21.55
11.85
8.05
7.12
0.2313
Table 1: Comparison on AIST++ and FineDance. ↓ : lower is better; → : closer to GT is better. Bold/underline indicate the best/second-best reported values for each dataset.
AIST++
FineDance
Variant
FID k
FID g
Div k
Div g
BAS
FID k
FID g
Div k
Div g
BAS
w/o Freq.
28.74
10.43
6.06
6.08
0.216
56.82
49.68
7.38
10.17
0.202
Same Rate
18.92
9.88
8.84
6.26
0.260
48.01
36.34
8.87
6.82
0.219
Uniform K
15.16
9.25
8.39
6.18
0.249
39.46
30.07
8.74
7.05
0.220
w/o L → H
55.27
24.56
8.71
6.39
0.256
140.77
48.93
4.95
11.27
0.224
w/o Prior
13.14
7.34
8.63
6.48
0.253
35.39
27.84
9.10
7.78
0.215
Table 2: Generation ablations on AIST++ and FineDance. Lower FID is better; diversity and BAS are better when closer to ground truth.
AIST++
FineDance
Variant
MPJPE ↓
EH↓
EA↓
MPJPE ↓
EH↓
EA↓
w/o Freq.
96.647
0.7338
17.144
95.903
0.4280
7.332
Same Rate
96.797
0.7175
17.287
93.101
0.4013
7.248
Uniform K
95.518
0.6862
16.934
92.637
0.3743
6.719
Full
94.459
0.5514
15.070
88.770
0.3190
5.256
Table 3: Tokenizer reconstruction ablations. MPJPE: mm; EH : dimensionless; EA : m/s 2 . Lower is better; bold marks the best values.