Geometric Consistency Based Label-Free Validation for Checkpoint Selection in Audio Active Learning

Shiqi Zhang; Tuomas Virtanen

cs.SDOct 26, 2026

Geometric Consistency Based Label-Free Validation for Checkpoint Selection in Audio Active Learning

Authors: Shiqi Zhang, Tuomas Virtanen

Organizations: Faculty of Information Technology and Communication Sciences, Tampere University, Tampere, Finland

Abstract

Audio active learning (AL) trains a classifier at low annotation cost by labeling only an informative fraction of an unlabeled collection over successive rounds. Each round picks one of the checkpoints saved during training. Mainstream AL protocols do so with an assumed validation set, but in practice its labels also count toward the annotation cost, shrinking the advantage of an already small annotation budget. We propose geometric consistency, a label-free validation rule for checkpoint selection. Both classifiers, nearest-class-mean and k-nearest-neighbor, are built from the labeled set in a frozen pretrained embedding space. They then predict the unlabeled data to model its geometry. The rule scores each checkpoint by its agreement with these pseudo-labels and its cross-entropy on the labeled set. It works with any query strategy, has one hyperparameter, insensitive in practice, and adds only one batched forward pass over the unlabeled data per round. At matched label budgets across datasets, encoders, and query strategies, it attains the best mean rank in area under the labeling curve (AULC) among methods without extra labels and is on par with an out-of-budget validation set. To our knowledge, this is the first systematic study of label-free validation for checkpoint selection in audio AL.

Figures & tables

Figure 1: (a) Pool-based AL with a frozen encoder and a shallow head trained from scratch each round. A stopping method picks one of

E

candidate checkpoints as the model of the round. (b) Three checkpoint selection methods using the same per-round label budget. Hold-out validation leaves some labels for validation.

K

-fold refit does the validation with the same labels it then retrains on, which biases the selection, and requires extra retrainings. The proposed geometric consistency requires neither held-out labels nor retrainings and adds one batched pool forward. (c) Geometric consistency scores each checkpoint by the agreement of its predictions on the pool with the nearest-class-mean (NCM) and

k

-nearest-neighbor (

k

-NN) pseudo-labels and by its cross-entropy on the labeled set.

Method	Final acc (%)	AULC (%)	Rank $\downarrow$
GWA	74.53 $\pm$ 0.24	69.94 $\pm$ 0.30	4.17
LabelWave	75.13 $\pm$ 0.34	69.82 $\pm$ 0.34	3.69
holdout-v	74.99 $\pm$ 0.24	69.19 $\pm$ 0.31	5.25
holdout-v-refit	75.35 $\pm$ 0.33	70.10 $\pm$ 0.29	4.10
kfold-refit	76.03 $\pm$ 0.27	70.99 $\pm$ 0.20	2.71
Geo-Con (ours)	76.78 $\pm$ 0.17	71.99 $\pm$ 0.19	1.08

Table 1: Aggregate results over 48 settings

\times

10 seeds (mean

\pm

std across seeds, settings averaged within each seed). Rank averages within-setting AULC ranks of the six deployable methods. Gray rows are non-deployable references.

	ESC-50		BSD10K		DB3V		UrbanSound8K
Method	CLAP	PANNs	CLAP	PANNs	CLAP	PANNs	CLAP	PANNs
GWA	91.66 $\pm$ 0.41	84.10 $\pm$ 0.43	54.28 $\pm$ 0.44	47.83 $\pm$ 1.08	60.87 $\pm$ 1.14	56.02 $\pm$ 1.06	83.56 $\pm$ 0.60	81.20 $\pm$ 0.61
LabelWave	91.91 $\pm$ 0.63	84.83 $\pm$ 0.40	53.74 $\pm$ 0.73	48.46 $\pm$ 0.84	56.98 $\pm$ 1.10	57.51 $\pm$ 0.72	83.22 $\pm$ 1.06	81.89 $\pm$ 0.66
holdout-v	90.25 $\pm$ 0.69	82.97 $\pm$ 0.81	51.04 $\pm$ 0.78	46.83 $\pm$ 0.66	60.62 $\pm$ 1.17	57.11 $\pm$ 1.01	83.38 $\pm$ 0.90	81.31 $\pm$ 0.40
holdout-v-refit	91.38 $\pm$ 0.67	83.64 $\pm$ 0.50	53.31 $\pm$ 0.78	47.96 $\pm$ 0.70	62.33 $\pm$ 0.76	57.01 $\pm$ 0.92	83.81 $\pm$ 1.01	81.35 $\pm$ 0.50
kfold-refit	92.28 $\pm$ 0.40	84.64 $\pm$ 0.32	53.84 $\pm$ 0.57	48.23 $\pm$ 0.85	63.80 $\pm$ 0.34	59.03 $\pm$ 0.81	84.16 $\pm$ 0.67	81.89 $\pm$ 0.42

Table 2: Test macro AULC (%) per dataset

\times

encoder block (mean

\pm

std across seeds, strategies averaged within each seed). Best deployable method per column in bold; gray rows are non-deployable references.

Method	rand.	entr.	marg.	core.	mherd.	badge
GWA	4.00	4.38	4.38	4.00	3.75	4.50
LabelWave	3.88	4.38	3.50	3.50	3.63	3.25
holdout-v	5.38	5.25	5.38	5.63	4.75	5.13
holdout-v-refit	4.13	3.50	4.13	3.38	5.13	4.38
kfold-refit	2.63	2.50	2.50	3.25	2.63	2.75
Geo-Con (ours)	1.00	1.00	1.13	1.25	1.13	1.00

Table 3: Mean AULC rank of the six deployable methods under each acquisition strategy (8 settings per strategy).

Variant	Final acc (%)	AULC (%)	Rank $\downarrow$
full rule, Eq. ( 3 )	76.78 $\pm$ 0.17	71.99 $\pm$ 0.19	1.08
no fit term $\ell$	75.09 $\pm$ 0.28	71.22 $\pm$ 0.21	1.77
no $k$ -NN term $a^{\mathrm{knn}}$	76.64 $\pm$ 0.25	71.93 $\pm$ 0.20	1.10
no NCM term $a^{\mathrm{ncm}}$	76.74 $\pm$ 0.17	71.82 $\pm$ 0.19	1.31
$a^{\mathrm{ncm}}$ only	74.44 $\pm$ 0.32	70.77 $\pm$ 0.21	2.27
$a^{\mathrm{knn}}$ only	76.26 $\pm$ 0.16	71.50 $\pm$ 0.19	1.77

Table 4: Ablations over the full matrix (48 settings

\times

10 seeds, mean

\pm

std across seeds). Each variant replaces Geo-Con in the ranking against the five deployable baselines. Top, component ablation, best per column in bold; bottom, sensitivity to

k

(default

k{=}10

is the full rule).

Active Learning

Active learning selects the examples, labels, features, or experiments most useful for improving a model under limited budgets.

See all 290 papers

Cover First, Disagree Softly: Rethinking Mismatch-First Active Learning for Frame-Level Audio Classification
Jul 15, 2026Shiqi Zhang, Tuomas VirtanenMulti-Label Classification Active Learning
Audio Research Group, Tampere University, Tampere, Finland
Should Pseudo-Labels Share Mini-Batches with Labeled Examples? Semi-Supervised Continual Active Learning with Limited Training Cost
Oct 26, 2026Shiqi Zhang, Tuomas VirtanenActive Learning Pseudo-Labeling
Faculty of Information Technology and Communication Sciences Tampere University, Tampere, Finland
BioDCASE: Active Learning for Bioacoustics
Sep 14, 2026Ben McEwen, Rupa Kurinchi-Vendhan, Shiqi Zhang +3Active Learning Bioacoustics
University of Amsterdam, Amsterdam, Netherlands · Massachusetts Institute of Technology, Cambridge, Massachusetts, USA · Tampere University, Finland +2

Jul 15, 2026J/K move · Enter open · S save

Cover First, Disagree Softly: Rethinking Mismatch-First Active Learning for Frame-Level Audio Classification

Shiqi Zhang, Tuomas Virtanen

Audio Research Group, Tampere University, Tampere, Finland

Multi-Label Classification Active Learning Sound Event Detection

Open paper AskPDF arXiv

Sound event detection relies on frame-level strong labels whose annotation is expensive. Active learning addresses this problem by selecting the audio segments whose labels help the classifier most. One of the prevailing acquisition strategies for this task, mismatch-first farthest-traversal (MFFT), combines the disagreement between two classifiers and the diversity of the selected segments through hard sequential decisions. It selects whole groups of high-disagreement segments first and spreads only the remaining budget by farthest traversal. On two multi-label datasets we show that this design is blind to the similarity among the selected segments and fails under low budgets, with every mismatch-first variant ending below the plain geometric strategy it builds on. We propose mismatch-weighted facility location (MW-FL), which spends the entire budget through a disagreement-weighted coverage objective that penalizes similarity among the selected segments. The disagreement signal from MFFT is used to obtain the nonnegative weights of this facility-location objective, without introducing hyperparameters. Experiments across two geometric mechanisms with three ways of using disagreement show that coverage of the selected segments is the dominant factor, hard disagreement gating of selection is harmful on both mechanisms, and soft disagreement weighting helps on top of coverage. MW-FL attains the best area under the learning curve on both datasets.

Abstract

Active Learning

Explore similar work

Cover First, Disagree Softly: Rethinking Mismatch-First Active Learning for Frame-Level Audio Classification

Should Pseudo-Labels Share Mini-Batches with Labeled Examples? Semi-Supervised Continual Active Learning with Limited Training Cost

BioDCASE: Active Learning for Bioacoustics