Variable-Length Audio Fingerprinting

Hongjie Chen; Hanyu Meng; Huimin Zeng; Ryan A. Rossi; Lie Lu; Josh Kimball

cs.SDMar 25, 2026

Variable-Length Audio Fingerprinting

Authors: Hongjie Chen, Hanyu Meng, Huimin Zeng, Ryan A. Rossi, Lie Lu, Josh Kimball

Organizations: Dolby Laboratories, Atlanta, USA · The University of New South Wales, Sydney, Australia · University of Illinois Urbana-Champaign, Champaign, USA · Adobe Research, San Jose, USA ·

Abstract

Audio fingerprinting converts audio to much lower-dimensional representations, allowing distorted recordings to still be recognized as their originals through similar fingerprints. Existing deep learning approaches rigidly fingerprint fixed-length audio segments, thereby neglecting temporal dynamics during segmentation. To address limitations due to this rigidity, we propose Variable-Length Audio FingerPrinting (VLAFP), a novel method that supports variable-length fingerprinting. To the best of our knowledge, VLAFP is the first deep audio fingerprinting model capable of processing audio of variable length, for both training and testing. Our experiments show that VLAFP outperforms existing state-of-the-arts in live audio identification and audio retrieval across three real-world datasets.

Figures & tables

Figure 1. Limitations of fixed-length audio fingerprinting. The top row shows a $3$ -second excerpt (containing both interlude and verse) in its original form and with a $1.5\times$ speed-up. The middle and bottom rows compare fixed-length segmentation and variable-length segmentation. Fixed-length segmentation suffers from three issues: Loss of Natural Boundaries. Segments cut across semantic units ((a)-2), complicating interpretation. Redundant or Noisy Context. Segments oversimplify ((a)-1) or combine too much information ((a)-2). Distortion Incompatibility. Time-stretch prevents exact matching; no subfigure in (b) aligns perfectly with (a)’s. Variable-length segmentation addresses them through segments aligned with semantic boundaries.

Figure 2. The architecture of our proposed VLAFP model. (a) Initial Projection: Audio

\boldsymbol{\mathrm{A}}

in its spectrogram representation is projected through a linear layer. (b) Inter-frame Self-Attention: Multi-head self-attention layers learn inter-frame relationships. (c) Frame-to-segment Cross-Attention: Multi-head cross-attention layers model the frame-to-segment relationships. (d) Segment Embedding Initialization: Replicas of segment embeddings are initialized through frame-to-segment pooling. (e) Fingerprint Summarization: Replicas of segment embeddings are aggregated and L2-normalized to generate an audio fingerprint

\mathbf{z}

Task	Fingerprint database source	Query (distorted)	Segmentation		Distortion			Objective	Dataset	Metric
Task	Fingerprint database source	Query (distorted)	VL	FL (1 sec)	TS	BG	IR	Objective	Dataset	Metric
CBR	a commercial	distorted broadcast	VLAFP	baselines	✓	✓	✓	identify commercial	all	precision, recall, F1
DTR	dummy + original target	distorted target		all methods		✓	✓	retrieve original	FMA	top-1 hit rate

Table 1. Setup of Commercial-Broadcast Retrieval (CBR) and Dummy-Target Retrieval (DTR) .

Figure 3. Diagram of Commercial-Broadcast Retrieval (CBR).

Figure 4. Diagram of Dummy-Target Retrieval (DTR).

Method	FMA			LibriSpeech			AudioSet
Method	Precision	Recall	F1	Precision	Recall	F1	Precision	Recall	F1
wav2vec2	$5.76$	$32.78$	$9.79$	$3.88$	$26.09$	$6.75$	$5.32$	$\mathbf{40.63}$	$9.40$
HuBERT	$4.87$	$\mathbf{98.94}$	$9.29$	$4.53$	$15.48$	$7.01$	$6.96$	$25.25$	$10.91$
AST	$9.62$	$30.88$	$14.68$	$3.80$	$25.03$	$6.59$	$12.80$	$26.24$	$17.21$
AMG	$25.81$	$31.37$	$28.32$	$22.51$	$28.82$	$25.28$	$17.48$	$27.72$	$21.44$
NAFP	$\mathit{75.84}$	$64.23$	$\mathit{69.55}$	$\mathit{44.74}$	$\mathit{33.37}$	$\mathit{38.23}$	$\mathbf{55.95}$	$34.32$	$\mathit{42.54}$

Table 2. CBR results on FMA , LibriSpeech , and AudioSet . Values are reported as percentages (

\%

) for precision , recall , and F1-score , with the best scores highlighted in bold and second best in italics .

Table 7

Table 8

Figure 5. Segment count and average length.

Figure 6. Segment length distribution.

Segmentation Threshold $\theta$	VLAFP on FMA
Segmentation Threshold $\theta$	Precision	Recall	F1
$0$	$79.28$	$\mathbf{71.63}$	$\mathbf{75.26}$
$1$	$\mathbf{81.00}$	$70.15$	$75.19$
$2$	$74.86$	$65.24$	$69.72$
$3$	$74.16$	$61.31$	$67.12$
$4$	$71.99$	$64.95$	$68.29$

Table 7. CBR results of VLAFP on FMA when trained with different segmentation threshold

\theta\in\left\{0,1,2,3,4,+\infty\right\}

. Smaller

\theta

(0 or 1) with more shorter segments yields better results.

Segmentation Method	VLAFP on FMA
Segmentation Method	Precision	Recall	F1
( main )	$\mathbf{81.00}$	$\mathbf{70.15}$	$\mathbf{75.19}$
No Silence	$76.29$	$67.86$	$71.83$
Pelt	$69.84$	$60.31$	$64.73$
Waveform	$78.1$	$68.60$	$73.07$

Table 8. CBR results when trained with different segmentation methods, including our proposed segmentation method ( main ) and three other methods. main yields the best results.

Figure 7. DTR results of VLAFP on FMA with different

\theta

Figure 8. DTR results on FMA with different segmentations.

Augmentation			VLAFP on FMA
TS	BG	IR	Precision	Recall	F1
No Augmentation			$26.54$	$24.62$	$25.54$
$\checkmark$			$51.68$	$39.87$	$45.01$
	$\checkmark$		$66.16$	$60.70$	$63.31$
		$\checkmark$	$76.27$	$66.17$	$70.86$
$\checkmark$	$\checkmark$		$67.95$	$63.45$	$65.62$

Table 9. CBR results on FMA with different augmentations: time stretching (TS), background mixing (BG), and impulse response convolution (IR). The best results are obtained when using all augmentation types.

#pos	VLAFP on FMA
#pos	Precision	Recall	F1
$1$	$\mathbf{76.38}$	$66.46$	$71.07$
$2$	$74.93$	$\mathbf{67.63}$	$\mathbf{71.09}$
$3$	$\mathbf{76.38}$	$64.56$	$69.97$
$4$	$75.41$	$65.49$	$70.10$
$5$	$74.55$	$63.77$	$68.74$

Table 10. CBR results of VLAFP on FMA when trained with different numbers of positive samples

\text{\#pos}\in\left\{1,2,3,4,5\right\}

per anchor. Fewer positive samples yield better result.

Figure 9. DTR results on FMA with different augmentations.

Figure 10. DTR results on number of positive samples.

Symbol	Meaning	Symbol	Meaning	Symbol	Meaning
$a$	audio signal	$n$	audio length	$\mathbf{z}$	audio fingerprint
$\boldsymbol{\mathrm{A}}$	audio in time-frequency rep.	$T$	audio frame length	$F$	number of frequency bins
$d$	feature dimension	$d_{h}$	head dimension	$H$	number of attention heads
$L$	number of blocks	$h^{0}$	initial embedding	$h^{l},\tilde{h}^{l}$	frame-level embedding
$x_{q}^{l}$	query variable	$x_{kv}^{l}$	key/value variable	$s^{l}$	seg.-level embedding
$a$	anchor sample	$a^{+}$	positive sample to $a$	$P(a)$	set of positive samples

Table 11. A summary of notations.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Split	#Audio	Duration (seconds / hours)					Type
Dataset	Split	#Audio	Min.	Med.	Avg.	Max.	Total	Type
FMA	Train	$10,000$	$6.2$	$30.0$	$30.0$	$30.0$	$83.3$ h	Music
FMA	Test	$500$	$30.0$	$30.0$	$30.0$	$30.0$	$4.2$ h	Music
LibriSpeech	Train	$5,459$	$1.4$	$13.8$	$12.3$	$17.3$	$25.9$ h	Speech
LibriSpeech	Test	$2,620$	$1.3$	$5.8$	$7.4$	$35.0$	$5.4$ h	Speech
AudioSet	Train	$10,000$	$1.0$	$9.9$	$10.0$	$10.0$	$27.5$ h	General

Appendix

Table 12. A summary of dataset statistics. The train / test split is configured using the original dataset splits.

Figure 11. Details of database construction and broadcast construction & query in Commercial-Broadcast Retrieval (CBR).

Figure 12. Details of database construction and query in Dummy-Target Retrieval (DTR).

Figure 13. A sensitivity study of hyperparameter sensitivity of VLAFP. Each row is for a specific number of heads

H\in\left\{8,16,32\right\}

and each column is for a specific hidden dimension

d\in\left\{64,128,256\right\}

. In each figure, each line is for a specific learning rate

\eta\in\left\{10^{-5},10^{-4},10^{-3}\right\}

, and each dot is for a specific number of blocks

L\in\left\{4,8,16\right\}

. The average accuracy across query duration is reported. See Sec. D for detailed discussions.

Method	dim $d$	CBR Commercial		CBR Broadcast		DTR Dummy		DTR Target
Method	dim $d$	#seg	size	#seg	size	#seg	size	#seg	size
wav2vec2	1568	15k	87M	289k	1.7GB	581k	3.6GB	30k	177MB
HuBERT	1568	15k	87M	289k	1.7GB	581k	3.6GB	30k	177MB
AST	527	15k	29M	289k	581MB	581k	1.3GB	30k	60MB
AMG	128	15k	7M	289k	141MB	581k	299MB	30k	15MB
NAFP	128	15k	7M	289k	141MB	581k	299MB	30k	15MB

Appendix

Table 13. A comparison of storage across methods for CBR and DTR on the FMA dataset. The best (lowest) values are highlighted in bold and second best in italics .

Table 26

Table 27

Music Information Retrieval

Music Information Retrieval develops computational methods for finding, classifying, describing, and analyzing music from audio, scores, and metadata.

See all 153 papers

Applicability and Generalization of Audio Fingerprinting Methods to Speech
Oct 19, 2026Kemal Altwlkany, Elmedin Selmanovic, Sead Delalić +1
Infobip · Faculty of Science, University of Sarajevo
POLARIS: Training-Free Audio Fingerprinting with Saliency-Based Landmarks and Delaunay Grouping
Sep 13, 2026Jiheng LiMusic Information Retrieval
Department of Computer Science, Vanderbilt University, Nashville, TN, 37235
Alethia: A Foundational Encoder for Voice Deepfakes
Apr 30, 2026Yi Zhu, Brahmi Dwivedi, Jayaram Raghuram +1Audio Representation Learning Audio Deepfake Detection
Reality Defender Inc.

Oct 19, 2026J/K move · Enter open · S save

Applicability and Generalization of Audio Fingerprinting Methods to Speech

Kemal Altwlkany, Elmedin Selmanovic, Sead Delalić, Emanuel Lacic

Infobip · Faculty of Science, University of Sarajevo

Open paper AskPDF arXiv

Audio fingerprinting methods identify audio based on its content, even when the content has been possibly corrupted. Despite their applicability and usage in the speech domain, research and advancements in audio fingerprinting traditionally focus on music and are almost exclusively evaluated on music. We provide insights into speech-oriented audio fingerprinting by benchmarking the best performing audio fingerprinting models on speech data while taking into consideration the effects of noise, reverberation, temporal changes, pitch modifications, and lossy compression. Our findings indicate that audio fingerprinting methods based on self-supervised learning outperform classical approaches on speech data, while the performance of these models on speech is inferior compared to the performance that they achieve on music. Results also include an analysis of false positives, showing that fingerprinting methods are more likely to match with an audio of the same speaker rather than an audio of the same text, when mismatches occur.

Abstract

Music Information Retrieval

Explore similar work

Applicability and Generalization of Audio Fingerprinting Methods to Speech

POLARIS: Training-Free Audio Fingerprinting with Saliency-Based Landmarks and Delaunay Grouping

Alethia: A Foundational Encoder for Voice Deepfakes