Automatic setlist identification aims to identify the songs performed in live concert recordings by matching noisy and variable-quality audio against a database of reference tracks. This task presents challenges beyond conventional music version identification, including substantial arrangement differences, background noise, and the need to process continuous recordings efficiently at large scale. In this work, we evaluate modern music version retrieval methods for setlist identification using the ASID benchmark, comparing the established Re-MOVE system with CLEWS and NAVE, an internal neural audio version embedding model. Our results show that NAVE and CLEWS substantially outperform Re-MOVE. Notably, NAVE uses 256-dimensional embeddings, compared with 1024 dimensions for CLEWS, providing a four-fold reduction in embedding storage while achieving better retrieval performance. We further evaluate music/non-music gating and find modest improvements in detection and temporal coverage, while qualitative analysis suggests that such preprocessing can improve recognition boundaries beyond what the standard ASID metrics capture. Overall, the results demonstrate that segment-level audio embeddings provide a strong basis for scalable setlist identification and that compact representations can achieve strong accuracy under realistic live-concert conditions at much lower storage and serving cost.
Figures & tables
System
Input
Embedding
Dim.
Re-MOVE
cremaPCP chroma
track / window vector
256
CLEWS
audio
segment sequence
1024
NAVE
audio
segment sequence
256
Table 1: Embedding dimensionality of the compared systems.
System
Filter
DAP
DLP
TP
FP
FN
Re-MOVE (paper)
none
0.803
0.605
—
—
—
Re-MOVE (paper)
SVM
0.678
0.569
—
—
—
Re-MOVE (ours)
none
0.779
0.573
860
1053
240
Re-MOVE (ours)
SVM
0.601
0.507
652
115
432
CLEWS
none
0.903
0.689
975
272
109
NAVE recognition
none
0.903
0.697
973
280
105
Table 2: Overall results on ASID eval set (62 concerts, 120 s / 30 s sliding window, postprocessed). Re-MOVE numbers, both with/without SVM classifier variant. NAVE and CLEWS are evaluated without SVM.
Quality
DAP
TP
FP
FN
n concerts
AQ-A (broadcast)
0.897
505
165
58
31
AQ-B (professional rec.)
0.971
199
47
6
16
AQ-C (smartphone)
0.862
254
63
41
14
Table 3: Eval DAP by audio quality tier (NAVE recognition, no SVM).
CLEWS
NAVE recog.
NAVE pooled
Genre
DAP
DLP
DAP
DLP
DAP
DLP
Pop
0.929
0.829
0.933
0.839
0.921
0.832
Rock/Metal
0.886
0.799
0.903
0.816
0.897
0.811
Indie/Alt.
0.943
0.796
0.943
0.822
0.953
0.830
Hip-hop/Rap
0.845
0.278
0.810
0.273
0.796
0.271
Electronic
0.891
0.686
0.873
0.691
0.982
0.791
Table 4: Eval DAP and DLP by genre (postprocessed, no SVM).
Strategy
DAP
DLP
TP
FP
FN
Unfiltered
0.903
0.697
973
280
105
overlap_mean
0.909
0.705
984
331
99
music_beats_speech
0.909
0.705
984
313
99
conservative_gate
0.908
0.707
983
287
100
music_island_loose
0.839
0.640
924
243
177
Table 5: Effect of TVSM music-gating strategies on NAVE recognition (eval, 62 concerts).
Figure 1: Full version identification output using NAVE recognition filtered vs unfiltered for Adele concert.
Figure 2: Version identification output using NAVE recognition filtered vs unfiltered, and score of the music/non-music classifier. Zoom in on first part of Adele concert.
Berlin School of Library and Information Science, Humboldt-Universität zu Berlin, Germany · Music Technology Group, Universitat Pompeu Fabra, Barcelona, Spain