CoMPAS3D: A Dataset and Benchmark for Interactive Motion

Bermet Burkanova; Yasaman Etesam; Payam Jome Yazdian; Trinity Evans; Chuxuan Zhang; Zoe Stanley; Paige Tuttösí; Angelica Lim

cs.LGJul 25, 2025

CoMPAS3D: A Dataset and Benchmark for Interactive Motion

Authors: Bermet Burkanova, Yasaman Etesam, Payam Jome Yazdian, Trinity Evans, Chuxuan Zhang, Zoe Stanley, Paige Tuttösí, Angelica Lim

Organizations: School of Computing Science, Simon Fraser University, Burnaby, BC, Canada

Abstract

Socially interactive humanoids aim to interact with humans through their bodies, adapting in real time to a partner's movement, intent, and abilities. This requires models that understand both how bodies move and what their movements mean. Yet, existing evaluation frameworks for interactive motion generation do not assess whether generated reactive motions are valid or appropriate within a shared movement vocabulary. Existing frameworks rely on kinematic metrics such as FID and beat alignment that cannot capture this semantic property, while improvised interaction datasets lack fine-grained move annotations. Inspired by computational evaluations of games such as Go and Chess, we propose Salsa as an embodied interaction evaluation domain. Salsa is interactive, dyadic, and governed by a move vocabulary and community-validated judging criteria covering timing, musicality, technique, difficulty, partnering, and originality. We present CoMPAS3D, a motion capture dataset of improvised partner salsa dance and an evaluation framework covering kinematic quality, two objective metrics (move legibility and proficiency appropriateness), and six competition-based subjective dimensions. The dataset comprises 3 hours of improvisation by 18 dancers spanning beginner, intermediate, and professional skill levels, with over 2,800 expert-annotated segments covering move types, execution errors, and stylistic elements. We define three benchmark tasks: move classification, proficiency estimation, and follower generation. Fine-tuned vision-language models perform well on ground-truth sequences and correlate with subjective expert judgments, suggesting their promise as measurement tools. Applied to Duolando and InterGen, our objective metrics reveal a large gap between ground-truth and generated dances, corroborated by expert human evaluations across six judging dimensions.

Figures & tables

Figure 1: CoMPAS3D comprises 3 hours of improvised salsa dance with beginner (top), intermediate (middle) and professional (bottom) pairs with synchronised music and fine-grained annotations.

Dataset	# Participants	# Genres	Pairs/Genre	Audio	Experience	$\bar{T}$ /s	$T$	# Frames	Improvised	Mocap	Annotated	Representation
ReMoCap	9	n/a	n/a	✗	Pro	Varies	2.04h	275.7K	✗	✗	✗	3D Skeletons
ExPI	4	1	2	✗	Pro	10.4	0.33h	30K	✗	✓	✓	3D Meshes
InterHuman	30	n/a	n/a	✗	Pro	10	6.56h	1.7M	✗	✓	✓	SMPL
InterDance	Unknown	15	Unknown	✓	Pro	142.7	3.93h	1.7M	✓	✓	✗	SMPL-X
DD100	10	10	0.5	✓	Pro	70.2	1.95h	210K	✓	✓	✗	SMPL-X
CoMPAS3D	18	1	9	✓	Beg, Int., Pro	150	3.0h	1.3M	✓	✓	✓	SMPL-X

Table 1: Comparison of publicly available dance datasets capturing human-human interaction (HHI).

\bar{T}

/s represents the average duration per sequence in seconds. Pairs/Genre highlights the depth of coverage within a single movement vocabulary: DD100 captures 0.5 pairs per genre across 10 ballroom styles, and InterDance’s pairs per genre is unknown across 15 genres, whereas CoMPAS3D dedicates all 9 pairs to a single genre, similar to a richly annotated dataset in English rather than shallow coverage across multiple languages.

Figure 2: Distribution over the 30 move classes (sorted by beginner move frequency) in CoMPAS3D for beginner, intermediate and pro pairs. Beginners tend to primarily use “basic step”, which professionals use less. Pros use a wider variety of moves, e.g. left turns and copa.

Figure 3: Proposed benchmark tasks for the CoMPAS3D dataset: (1) move classification (dyadic and on solo follower moves), (2) proficiency estimation and (3) follower generation. Objective evaluation of follower dance generation uses (1) and (2).

	Solo Metrics				Interactive Metrics			Rhythmic
Method	FID k ( $\downarrow$ )	FID g ( $\downarrow$ )	Div k ( $\uparrow$ )	Div g ( $\uparrow$ )	FID cd ( $\downarrow$ )	Div cd ( $\uparrow$ )	BED( $\uparrow$ )	BAS( $\rightarrow$ )
Groundtruth	$0.00_{{\scriptstyle\pm.00}}$	$0.00_{{\scriptstyle\pm.00}}$	$8.74$	$0.81$	$0.00_{{\scriptstyle\pm.00}}$	$2.01$	$0.42$	$0.16$
InterGen	$24.58_{{\scriptstyle\pm.75}}$	$13.57_{{\scriptstyle\pm.45}}$	$8.59$	$0.72$	$18.62_{{\scriptstyle\pm.28}}$	$2.07$	$0.32$	$0.20$
Duolando	$12.46_{{\scriptstyle\pm.45}}$	$15.54_{{\scriptstyle\pm.25}}$	$7.96$	$0.78$	$20.14_{{\scriptstyle\pm.23}}$	$1.93$	$0.30$	$0.22$

Table 3: Quantitative comparison on CoMPAS3D. We include ground truth as reference and two generative baselines. We present solo, interactive, and motion–music alignment metrics.

\pm

denotes a 95% bootstrap confidence interval over 100 runs. Arrows indicate higher (

\uparrow

), lower (

\downarrow

) or (

\rightarrow

) closer to groundtruth is better. Among generative methods (excluding ground truth), the best value in each column is shown in bold .

		Legibility $\uparrow$		Appropriateness $\uparrow$
	Method	Move Acc.	Move F1	Profic. Acc.	Profic. F1
	Groundtruth	69.88	53.55	51.51	51.64
FT	InterGen	18.86	7.69	49.77	46.55
FT	Duolando	14.47	6.70	45.47	39.30
0-shot	InterGen	14.47	4.09	32.22	18.00
0-shot	Duolando	25.00	4.86	33.57	19.80

Table 4: Objective legibility (using InternVL3) and proficiency appropriateness (using Qwen2.5-VL) evaluations (in bold ) on generated follower motions on CoMPAS3D. Ground truth from Table 2 is included as reference.

Dimension	ICC(C,k)	95% CI
Timing	0.48 ∗	[0.10, 0.72]
Musicality	0.60 ∗∗	[0.30, 0.78]
Technique	0.47 ∗	[0.08, 0.71]
Difficulty	0.56 ∗∗	[0.23, 0.76]
Partner Coord.	0.76 ∗∗	[0.58, 0.87]
Originality	0.55 ∗∗	[0.22, 0.76]

Table 5: Expert inter-rater agreement (ICC). Two-way mixed-effects, consistency, average measures across three expert salsa dance annotators.

Panel of Figure 4 — Figure 4: Human evaluation of generated salsa dance sequences across six subjective dimensions, rated by (left) three expert salsa dancers (mean 9.0 years of experience, range 5–12) and (right) 31 non-expert participants. Bars show mean ratings (1–5 Likert scale) for ground truth (GT), InterGen (IG), and Duolando (DU). ${}^{*}p<0.05$ , ${}^{**}p<0.01$ , ${}^{***}p<0.001$

Figure 5: Correlation between objective and subjective measures.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Pair	Proficiency	Annotations	Test Set
Pair 1	Beginner	100%	Song1_Take1
Pair 2	Intermediate	100%	Song1_Take2
Pair 3	Beginner	100%	Song2_Take1
Pair 4	Intermediate	100%	Song2_Take2
Pair 5	Professional	50%	Song3_Take1
Pair 6	Intermediate	n/a	Song3_Take2

Appendix

Table 6: Pair proficiency levels, public annotations and corresponding sequences held out for testing for generative models.

Figure 6: ELAN annotation tool used for segmenting and labeling dance moves in animated SMPL-X representation files. The annotation includes four tracks: Together – when dancers execute the move as a pair; Separate_Leader – when the leader dances solo or adds "Man Styling" to the base move; Separate_Follower – when the follower dances solo or incorporates "Lady Styling"; and Errors – for marking mistakes.

Figure 7: The annotation validation task of the salsa move identification built in Gorilla. Each trial presented a short video clip of a base move, followed by two dropdown menus prompting the second annotator to label the primary and secondary moves observed.

Name	Category	Description
Arm lock	Move	A locking arm movement often used to create tension or highlight transitions.
Basic step	Move	Fundamental salsa step with variations including side, cross-back, and back basic steps.
Body shake	Move	A rapid shaking movement emphasizing torso dynamics.
Body roll	Move	A fluid, wave-like motion passing through the body.
Change of Directions	Move	Transition step involving directional changes, including position swaps.
Check	Move	A checking step used to halt or redirect movement.

Appendix

Table 7: Comprehensive Overview of Move, Styling, and Error Annotations in the CoMPAS3D Dataset. This table categorizes the various elements annotated during the dataset creation process, specifying whether each element pertains to a dance move, styling, or error classification.

Song	Artist	Title	Tempo (BPM)
Song 1	Tito Rojas	Lo que te queda	90
Song 2	Louie Ramirez, Ray de La Paz	Lluvia	105
Song 3	Leoni Torres	Idilio	95
Song 4	Johnny Ventura	Dilema	93

Appendix

Table 8: Songs used in the CoMPAS3D dataset with artist names and tempos.

Figure 8: The human evaluation interface presented to Prolific participants. Participants rated each video on six dimensions using 5-point Likert scales.

Dimension	Definition shown to participants
Timing	The majority of the routine should show recognizable timing in the dance genre (i.e. Salsa: 1-2-3, 5-6-7).
Partner Coordination	Reflected in the lead/follow aspect. Are partners truly connected, or merely executing choreography while holding hands? Synchronicity during side-by-sides and shines also counts.
Musicality	The couple’s ability to creatively work with the timing of the music, including through patterns, footwork, hits, tricks, etc.
Technique	Reflected through balance, placement and line; sometimes called latin motion or sabor. Movement should be grounded, with clarity of weight changes, and appear both clear and effortless.
Difficulty	Reflected by difficulty of patterns, turns, shines, and level of tricks, dips, drops and flares. Movement must be successfully executed to receive credit.
Originality	The couple’s interpretation of the music. Includes clever turn patterns, shines, musical choreography, and good use of space.

Appendix

Table 9: Evaluation dimensions and their definitions, as displayed to participants in the human study interface. Adapted from Salsa Congress Rules [ 2 ] . Compared to the competition rules, only "showmanship", which rates costumes and good sportsmanship, was omitted. The competition item entitled "choreography" was renamed to originality (due to the improvised nature of the dataset), but the definition remains similar.

Panel of Figure 9 — Figure 9: Sample frames from the two video configurations used in our experiments: (1) follower-only (single person) ( left ) and (2) follower and leader (dyadic) ( right ).

Panel of Figure 10 — Figure 10: Confusion matrices for move classification using fine-tuned models, reported in Table 2 .

Panel of Figure 11 — Figure 11: Confusion matrices for move classification using zero-shot models, reported in Table 2 .

Panel of Figure 12 — Figure 12: Confusion matrices for proficiency classification using fine-tuned models, reported in Table 2 .

Panel of Figure 13 — Figure 13: Confusion matrices for proficiency classification using zero-shot models, reported in Table 2 .

Panel of Figure 14 — Figure 14: Confusion matrices for legibility using both zero-shot and fine-tuned InternVL3 models, reported in Table 4 .

Panel of Figure 15 — Figure 15: Confusion matrices for appropriateness using both zero-shot and fine-tuned Qwen2.5-VL models, reported in Table 4 .

Human Motion Generation

Human motion generation creates articulated movement from language, audio, video, spatial constraints, or interaction goals.

See all 301 papers

SalsaAgent: A multimodal embodied language model for interactive dance generation
May 28, 2026Payam Jome Yazdian, Zoe Stanley, Angelica LimMultimodal Large Language Models Human Motion Generation
Simon Fraser University, Burnaby, Canada
The GENEA Challenge 2026: A Large-Scale Disentangled Evaluation of Speech-Driven Gesture Generation on the Seamless Interaction Dataset
Aug 11, 2026Rajmund Nagy, Silvia Arellano García, Hendric Voss +4Human Motion Generation Co-Speech Gesture Generation
KTH Royal Institute of Technology · Bielefeld University · Independent researcher +3
Social Structure Matters in 3D Human-Human Interaction Generation
Jun 23, 2026Zhongju Wang, Beier Wang, Yatao Bian +6Text-to-Motion Generation Human Motion Generation
University of New South Wales · National University of Singapore · NVIDIA +3

May 28, 2026J/K move · Enter open · S save

SalsaAgent: A multimodal embodied language model for interactive dance generation

Payam Jome Yazdian, Zoe Stanley, Angelica Lim

Simon Fraser University, Burnaby, Canada

Multimodal Large Language Models Human Motion Generation

Open paper AskPDF arXiv

Embodied interaction with humanoids depends on bidirectional nonverbal reactivity, coordination, and synchrony to convey cues and move with a partner. For socially interactive embodied agents, reactive motion generation requires expressive full-body motion that remains contextually appropriate while maintaining spatial and temporal synchrony. We present SalsaAgent, a language model that generates expressive, full-body salsa follower motions in reaction to a human leader and music. We formulate partner interaction as nonverbal token passing, extending the vocabulary of a large language model (LLM) to process discrete motion tokens, pairwise relation tokens, and audio tokens. Our method introduces full-body and pairwise-relation tokenizers, aligns language and motion tokens with automatically derived text descriptions of skeleton dynamics, and applies a two-stage token-to-diffusion pipeline. Subjective and objective evaluations show improved motion quality, two-person spatial coordination, and music and partner coordination relative to prior baselines.

Abstract

Human Motion Generation

Explore similar work

SalsaAgent: A multimodal embodied language model for interactive dance generation

The GENEA Challenge 2026: A Large-Scale Disentangled Evaluation of Speech-Driven Gesture Generation on the Seamless Interaction Dataset

Social Structure Matters in 3D Human-Human Interaction Generation