Learning Quantised Structure-Preserving Motion Representations for Dance Fingerprinting

Arina Kharlamova; Bowei He; Chen Ma; Xue Liu

cs.CVApr 1, 2026

Learning Quantised Structure-Preserving Motion Representations for Dance Fingerprinting

Authors: Arina Kharlamova, Bowei He, Chen Ma, Xue Liu

Organizations: Department of Computer Science MBZUAI, Abu Dhabi, UAE · Department of Computer Science City University of Hong Kong, Hong Kong, China

Abstract

We present DANCEMATCH, an end-to-end framework for motion-based dance retrieval, the task of identifying semantically similar choreographies directly from raw video, defined as DANCE FINGERPRINTING. While existing motion analysis and retrieval methods can compare pose sequences, they rely on continuous embeddings that are difficult to index, interpret, or scale. In contrast, DANCEMATCH constructs compact, discrete motion signatures that capture the spatio-temporal structure of dance while enabling efficient large-scale retrieval. Our system integrates Skeleton Motion Quantisation (SMQ) with Spatio-Temporal Transformers (STT) to encode human poses, extracted via Apple CoMotion, into a structured motion vocabulary. We further design DANCE RETRIEVAL ENGINE (DRE), which performs sub-linear retrieval using a histogram-based index followed by re-ranking for refined matching. To facilitate reproducible research, we release DANCETYPESBENCHMARK, a pose-aligned dataset annotated with quantised motion tokens. Experiments demonstrate robust retrieval across diverse dance styles and strong generalisation to unseen choreographies, establishing a foundation for scalable motion fingerprinting and quantitative choreographic analysis.

Figures & tables

Figure 1: Illustration of the proposed end-to-end framework for motion-based dance retrieval. The system processes an input dance video into a structured motion representation and retrieves top-matching dances through alignment-based similarity.

Figure 2: Overview of the DanceMatch framework. The Spatio-Temporal Transformer (STT) encoder extracts per-joint latent embeddings from skeletal motion sequences. Latent patches are discretised via vector quantisation into motion tokens using a learnable codebook. The STT decoder reconstructs the motion sequence from the quantised representation. Immediate dead-code revival and EMA-based updates ensure stable codebook utilisation and prevent mode collapse.

Figure 3: Two-stage DRE architecture: histogram-based indexing (Stage 1) provides a shortlist for re-ranking via multi-metric temporal alignment (Stage 2). Complexity per stage: $O(NK)$ vs. $O(LM_{q}\bar{M})$ .

Method	Mean	Match Rate	Rank1 (%)
Histogram Index	0.487	61.9	38.1
Brute-Force	0.509	65.3	39.0

Table 1: Retrieval accuracy on AIST (ten genres,

\geq 2

samples per class).

Video Retrieval

Video retrieval matches natural-language, visual, or multimodal queries to relevant videos, clips, and moments.

See all 61 papers

Fine-grained Motion Retrieval via Joint-Angle Motion Images and Token-Patch Late Interaction
Mar 10, 2026Yao Zhang, Zhuchenyang Liu, Yanlan He +2Representation Learning Feature Interaction Modeling
Aalto University, Espoo, Finland · Fudan University, Shanghai, China · Georgia Institute of Technology, Atlanta GA, USA
MoCHA: Denoising Caption Supervision for Motion-Text Retrieval
Mar 24, 2026Nikolai Warner, Cameron Ethan Taylor, Irfan Essa +1Cross-Modal Retrieval Multimodal Contrastive Learning
Georgia Institute of Technology, Atlanta, GA, USA · Amazon, Sunnyvale, CA, USA
Music-to-Dance Generation via Atomic Movements
Jul 15, 2026Xinhao Cai, Yixuan Sun, Minghang Zheng +4Human Motion Generation
Wangxuan Institute of Computer Technology, Peking University · School of Electronics Engineering and Computer Science, Peking University · National Institute of Health Data Science, Peking University +3

Mar 10, 2026J/K move · Enter open · S save

Fine-grained Motion Retrieval via Joint-Angle Motion Images and Token-Patch Late Interaction

Yao Zhang, Zhuchenyang Liu, Yanlan He, Thomas Ploetz+1

Aalto University, Espoo, Finland · Fudan University, Shanghai, China · Georgia Institute of Technology, Atlanta GA, USA

Representation Learning Feature Interaction Modeling Cross-Modal Retrieval

Open paper AskPDF arXiv

Text-motion retrieval aims to learn a semantically aligned latent space between natural language descriptions and 3D human motion skeleton sequences, enabling bidirectional search across the two modalities. Most existing methods use a dual-encoder framework that compresses motion and text into global embeddings, discarding fine-grained local correspondences, and thus reducing accuracy. Additionally, these global-embedding methods offer limited interpretability of the retrieval results. To overcome these limitations, we propose an interpretable, joint-angle-based motion representation that maps joint-level local features into a structured pseudo-image, compatible with pre-trained Vision Transformers. For text-to-motion retrieval, we employ MaxSim, a token-wise late interaction mechanism, and enhance it with Masked Language Modeling regularization to foster robust, interpretable text-motion alignment. Extensive experiments on HumanML3D and KIT-ML show that our method outperforms state-of-the-art text-motion retrieval approaches while offering interpretable fine-grained correspondences between text and motion. The code is available in the supplementary material.

Abstract

Video Retrieval

Explore similar work

Fine-grained Motion Retrieval via Joint-Angle Motion Images and Token-Patch Late Interaction

MoCHA: Denoising Caption Supervision for Motion-Text Retrieval

Music-to-Dance Generation via Atomic Movements