ChoreoStudio: Planning What, When, and Who in Music-Driven Group Choreography

Jing Xu; Cunjian Chen; Qiuhong Ke

cs.SDOct 23, 2026

ChoreoStudio: Planning What, When, and Who in Music-Driven Group Choreography

Authors: Jing Xu, Cunjian Chen, Qiuhong Ke

Organizations: Department of Data Science and Artificial Intelligence, Faculty of Information Technology, Monash University, Australia

Abstract

Music-driven group dance generation must decide not only how each dancer moves, but also what formation the group should form, when it should change, and who should execute each transition. Existing methods can generate orderly and rhythmically synchronized group motion, but remain limited in formation evolution and inter-dancer interaction. In human choreography, formation transitions often align with changes in high-level musical structure, such as the progression from an introduction to a chorus or from a chorus to a climax. We present ChoreoStudio, which makes choreography planning explicit. A music disentangler first decomposes the audio into frame-level rhythmic features and segment-level structural features. Given the segment-level features and sparse labels, a frozen omni-modal model acts as a scriptwriter, producing a global semantic description that guides the Director's choreography planning. Based on this context, the Director produces three coupled interaction decisions: a centroid-relative point set specifies what formation to produce, a transition gate determines when it should change, and a mobility allocation indicates who should execute the transition. The transition gate and mobility allocation are further conditioned on the planned geometric change, coupling formation targets with their temporal and dancer-specific execution. At a finer temporal scale, frame-aligned music features condition the diffusion generator to control detailed body motion and preserve beat-level alignment. The segment-level interaction plan is injected in parallel through frame-aligned instruction tokens and cross-dancer attention biases, coordinating how the generator realizes each transition. Experiments on AIOZ-GDance demonstrate that ChoreoStudio follows the planned interactions while achieving state-of-the-art performance in motion quality, group organization, and formation dynamics.

Figures & tables

Figure 1: Overview of ChoreoStudio. Multi-scale music features and a frozen omni-modal model provide rhythmic, structural, and semantic context. The Director jointly constructs a directed inter-dancer relation graph and a centroid-relative formation point set, then uses the planned geometric change to schedule transition timing and dancer mobility. The resulting plan conditions the Dance Generator through frame-aligned dancer-specific instructions and relation-derived spatial attention biases.

Figure 2: ChoreoStudio architecture. Segment-level structure and semantics guide the Director, while frame-level rhythm conditions motion synthesis; a mask marks unavailable motion. Through Perceive, Plan, Schedule, and Encode, the Director couples formation, timing, and mobility, encoding $\mathbf{P}$ as frame-aligned instructions $\mathbf{U}$ and relation-derived cross-dancer attention bias $\mathbf{B}$ .

Figure 3: State-aware perception in the Director. Exchangeable actor seeds distinguish dancers without assigning roles to their indices, while later cycles use the generated geometry and motion state. State tokens attend to segment-level music, the high-level description, and optional feedback to produce dancer-segment representations for choreography planning.

Method	1 Dancer			2 Dancers			3 Dancers			4 Dancers
Method	FID $\downarrow$	Div $\uparrow$	PFC $\downarrow$	GMR $\downarrow$	GMC $\uparrow$	TIF $\downarrow$	GMR $\downarrow$	GMC $\uparrow$	TIF $\downarrow$	GMR $\downarrow$	GMC $\uparrow$	TIF $\downarrow$
GCD Le et al. (2023a)	39.24	9.64	2.53	34.39	80.32	0.17	30.22	80.22	0.19	36.28	81.82	0.13
CoDancers Yang et al. (2024)	23.98	9.48	3.53	24.55	72.52	0.08	26.34	74.22	0.08	26.44	75.34	0.10
TCDiff++ Dai et al. (2026)	22.42	17.43	1.32	15.77	80.87	0.11	11.57	82.31	0.10	13.63	81.56	0.13
ST - GDance Xu et al. (2025b)	28.87	12.82	0.97	19.42	80.52	0.12	14.76	81.24	0.11	14.02	81.42	0.10
ChoreoStudio	22.34	14.35	0.52	13.56	81.64	0.10	10.42	83.23	0.08	11.94	82.41	0.09

Table 1: Single and Group dance performance on AIOZ-GDance. Best results are bold ; second-best results are underlined .

	IDDV	Jitter	FMR	BAS
GT	0.302	0.045	0.832	0.363
Best baseline	0.181	0.051	0.717	0.224
ChoreoStudio	0.267	0.048	0.848	0.351

Table 2: Formation dynamics and music alignment score relative to real data.

Figure 4: Qualitative comparison on matched music segments. We compare local motion and long-timescale formation evolution across methods using the same music and temporal extent.

	GMR $\downarrow$	GMC $\uparrow$	TIF $\downarrow$	FMR $\uparrow$
ChoreoStudio	10.42	83.23	0.08	0.848
w/o Semantic Description	17.68	79.34	0.09	0.819
w/o Verifier	16.10	78.68	0.10	0.803
w/o Director	15.23	78.34	0.11	0.783
w/o Role Assignment	12.52	81.53	0.09	0.795
w/o Transition Timing	14.32	79.32	0.10	0.803

Table 3: Ablation study for group dance generation (3 dancers).

Intervention	Error reduction $\uparrow$	Dir. $\uparrow$	$\Delta$ TIF $\downarrow$	$\Delta$ Jitter $\downarrow$
Formation geometry	0.40	78.54%	0.014	0.003
Transition timing	0.37	74.85%	0.018	0.004
Dancer mobility	0.29	68.68%	0.015	0.005

Table 4: Controlled plan interventions. Error reduction measures improvements toward the edited target.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Formation dynamics and music alignment. Bars report IDDV, Jitter, FMR, and BAS for TCDiff, ST-GDance, and ChoreoStudio. Dashed lines indicate the corresponding real-data references. All methods are evaluated using the same sequences, temporal windows, and metric implementations.

Method	$E_{\mathrm{form}}\downarrow$	$E_{\mathrm{pair}}\downarrow$	Trans. F1 $\uparrow$	Offset (s) $\downarrow$	$E_{\mathrm{mob}}\downarrow$
ChoreoStudio	0.13	0.09	0.80	0.32	0.16

Appendix

Table 5: Plan realization on the 150-frame three-dancer setting. Formation and pairwise errors are measured in meters; lower is better except for transition F1.

Term	Weight	Term	Weight
$\mathcal{L}_{\mathrm{rec}}$	0.636	$\mathcal{L}_{\mathrm{vel}}$	2.964
$\mathcal{L}_{\mathrm{pos}}$	0.646	$\mathcal{L}_{\mathrm{contact}}$	0.5
$\mathcal{L}_{\mathrm{dir}}$	0.3	$\mathcal{L}_{\mathrm{real}}$	1.0
$\mathcal{L}_{\mathrm{space}}$	1.0	$\mathcal{L}_{\mathrm{jump}}$	1.0
$\mathcal{L}_{\mathrm{struct}}$	1.0	$\mathcal{L}_{\mathrm{struct\mbox{-}inv}}$	0.3

Appendix

Table 6: Loss weights for the main ChoreoStudio model.

Method	1 Dancer			2 Dancers			3 Dancers			4 Dancers
Method	FID $\downarrow$	Div $\uparrow$	PFC $\downarrow$	GMR $\downarrow$	GMC $\uparrow$	TIF $\downarrow$	GMR $\downarrow$	GMC $\uparrow$	TIF $\downarrow$	GMR $\downarrow$	GMC $\uparrow$	TIF $\downarrow$
GCD Le et al. (2023a)	51.25	7.24	3.52	38.43	80.76	0.21	40.68	79.25	0.28	42.52	80.32	0.20
CoDancers Yang et al. (2024)	43.62	5.48	4.23	31.67	68.32	0.12	35.20	70.53	0.14	36.32	72.52	0.13
TCDiff++ Dai et al. (2026)	22.47	16.19	1.53	16.52	79.63	0.12	14.67	81.64	0.15	13.78	80.43	0.14
ST - GDance Xu et al. (2025b)	31.42	14.35	3.21	32.56	75.32	0.14	34.25	74.42	0.15	33.61	76.26	0.13
ChoreoStudio	21.52	16.32	1.03	14.48	83.42	0.11	15.63	82.64	0.12	14.58	81.37	0.12

Appendix

Table 7: Dance generation performance in the 720-frame long-term setting. Best results are bold ; second-best results are underlined .

Variant	GMR $\downarrow$	GMC $\uparrow$	TIF $\downarrow$	FMR
Full ChoreoStudio	10.42	83.23	0.08	0.848
Relation-only	15.29	81.89	0.10	0.792
Latent bypass	14.52	80.45	0.09	0.819
w/o relation-formation feedback	15.73	79.79	0.11	0.738
w/o music structure	14.33	82.71	0.09	0.798

Appendix

Table 8: Additional planning diagnostics. Results are evaluated using the same 150-frame three-dancer protocol as the main experiments. FMR is interpreted by proximity to the real-data reference.

Setting	NFE	GMR $\downarrow$	GMC $\uparrow$	TIF $\downarrow$	FMR
$R=1$ , 50 steps	50	15.43	79.32	0.10	0.810
$R=1$ , 100 steps	100	14.74	80.94	0.09	0.805
Best-of-2, 50 steps	100	13.62	81.29	0.09	0.826
$R=2$ , 50 steps	100	10.36	83.20	0.10	0.847

Appendix

Table 9: Denoising-budget-matched refinement analysis. NFE denotes the total number of denoising evaluations. Best-of-two samples are independently generated and ranked by the same frozen Verifier. FMR is ranked by proximity to the real-data reference of 0.832.

Description edit	Target metric	Target change	Direction $\uparrow$	$\Delta$ GMR	$\Delta$ TIF
Compact to expansive	Spread	+0.34 m	82.6%	+0.28	-0.006
Stable to dynamic	IDDV	+0.052	77.4%	+0.46	+0.004

Appendix

Table 10: Semantic control under paired generation. Changes are computed as edited minus original. Direction accuracy reports the percentage of pairs whose target metric changes in the requested direction.

Figure 6: User study results. Normalized blind-ranking scores across four perceptual dimensions. ChoreoStudio is consistently preferred, with the largest margins in Music Alignment and Inter-Dancer Coordination.

Panel of Figure 7 — Figure 7: Qualitative analysis of explicit choreography planning. (a) Removing the Director, semantic description, or Verifier produces distinct degradations in group organization, motion variation, and formation transitions. (b) Editing formation geometry, transition timing, or dancer mobility produces the requested motion changes while fixing the music, initial noise, and remaining plan variables.

Human Motion Generation

Human motion generation creates articulated movement from language, audio, video, spatial constraints, or interaction goals.

See all 292 papers

Oct 8, 2026

Breaking the Group Size Barrier: Parameter-Efficient Group Dance Generation with Chain-of-Dancers

Group dance generation aims to synthesize coordinated multi-dancer choreography from music, with broad applications in animation and interactive content creation. This task requires modeling dense inter-person dependencies to ensure spatial coordination, while naturally preserving individual dancer identities. Existing approaches model all dancers jointly with end-to-end transformers, which tie the architecture to a fixed group size and entangle per-dancer identities across frames. We propose ChainDance, a scalable framework that reformulates group dance generation as a Chain-of-Dancers: a sequential decomposition over per-dancer conditional distributions, allowing a single model to scale across variable group sizes without retraining and naturally preserving per-dancer identity. Built on a frozen single-dancer diffusion backbone, ChainDance introduces two lightweight modules: a Role-Aware Text Encoder (RATE) for per-dancer semantic conditioning, and a Group-Aware Motion Encoder (GAME) that aggregates previously generated dancers via a distance-weighted graph convolutional network, and incorporates a training-free noise optimization procedure at inference time to enforce global spatial coherence. Experiments on AIOZ-GDance demonstrate that ChainDance achieves state-of-the-art motion quality and group coordination while structurally preserving per-dancer identity, with

3

4\times

fewer parameters and requiring

3

6\times

less training time compared to prior approaches.

Jing Xu, Cunjian Chen, Qiuhong Ke

Department of Data Science and Artificial Intelligence, Monash University, Australia

Human Motion Generation Motion Diffusion Models

Ask PDF

Jul 15, 2026

Music-to-Dance Generation via Atomic Movements

Music-driven dance generation aims to produce human motion that is both rhythmically synchronized and semantically consistent with music. While recent neural approaches have achieved impressive visual realism, they typically model motion as a continuous signal and neglect its compositional nature, making generated dances structurally incoherent and difficult to control. In this work, we introduce a structure-aware framework that models choreography as a sequence of atomic movements-semantically interpretable motion events that serve as the building blocks of dance. To construct this atomic movement vocabulary, we first segment large-scale dance data and cluster them into atomic movement groups. We then employ a large language model to semantically relabel and refine the clusters, yielding a set of interpretable and reusable atomic movements. Based on these atomic movement annotations, we design a two-stage generation framework that mirrors the human choreography process. In the atomic movement planning stage, the model predicts the type, duration, and timing of atomic movements conditioned on the input music, forming a symbolic dance allocation. In the completion stage, a transition-aware generator synthesizes smooth and stylistically coherent motion conditioned on the planned structure. Extensive experiments demonstrate that our method produces dances with significantly improved structural coherence, rhythmic alignment, and perceptual naturalness compared to existing baselines, while providing enhanced interpretability and controllable editing through explicit structural representation.

Xinhao Cai, Yixuan Sun, Minghang Zheng +4

Wangxuan Institute of Computer Technology, Peking University · School of Electronics Engineering and Computer Science, Peking University · National Institute of Health Data Science, Peking University +3

Human Motion Generation

Ask PDF

Mar 20, 2026

ST-GDance++: A Scalable Spatial-Temporal Diffusion for Long-Duration Group Choreography

Group dance generation from music requires synchronizing multiple dancers while maintaining spatial coordination, making it highly relevant to applications such as film production, gaming, and animation. Recent group dance generation models have achieved promising generation quality, but they remain difficult to deploy in interactive scenarios due to bidirectional attention dependencies. As the number of dancers and the sequence length increase, the attention computation required for aligning music conditions with motion sequences grows quadratically, leading to reduced efficiency and increased risk of motion collisions. Effectively modeling dense spatial-temporal interactions is therefore essential, yet existing methods often struggle to capture such complexity, resulting in limited scalability and unstable multi-dancer coordination. To address these challenges, we propose ST-GDance++, a scalable framework that decouples spatial and temporal dependencies to enable efficient and collision-aware group choreography generation. For spatial modeling, we introduce lightweight distance-aware graph convolutions to capture inter-dancer relationships while reducing computational overhead. For temporal modeling, we design a diffusion noise scheduling strategy together with an efficient temporal-aligned attention mask, enabling stream-based generation for long motion sequences and improving scalability in long-duration scenarios. Experiments on the AIOZ-GDance dataset show that ST-GDance++ achieves competitive generation quality with significantly reduced latency compared to existing methods.

Jing Xu, Weiqiang Wang, Cunjian Chen +2

Faculty of Information Technology, Monash University, Melbourne, VIC, 3800, Australia . · School of Computing and Communications, Lancaster University, Lancaster, LA1 4YW, UK .

Human Motion Generation Diffusion Models for Motion Generation

Ask PDF

Abstract

Human Motion Generation

Explore similar work

Breaking the Group Size Barrier: Parameter-Efficient Group Dance Generation with Chain-of-Dancers

Music-to-Dance Generation via Atomic Movements

ST-GDance++: A Scalable Spatial-Temporal Diffusion for Long-Duration Group Choreography