Should Pseudo-Labels Share Mini-Batches with Labeled Examples? Semi-Supervised Continual Active Learning with Limited Training Cost

Shiqi Zhang; Tuomas Virtanen

cs.LGOct 26, 2026

Should Pseudo-Labels Share Mini-Batches with Labeled Examples? Semi-Supervised Continual Active Learning with Limited Training Cost

Authors: Shiqi Zhang, Tuomas Virtanen

Organizations: Faculty of Information Technology and Communication Sciences Tampere University, Tampere, Finland

Abstract

Active learning (AL) reduces annotation by querying informative examples. Continual active learning (CAL) also cuts the training cost, the total number of gradient steps, by training one model across AL rounds with a fixed number of mini-batch steps per round. Pseudo-labels add supervision without annotation, but they increase training cost. In CAL, where every round trains on a fixed number of mini-batches, whether and how pseudo-labeled examples should share these mini-batches with labeled examples is therefore open. We provide the first formalization of this problem as two decisions after each query, namely (i) which unlabeled examples should enter training with a pseudo-label and (ii) what share of each mini-batch they should take. Our method, semi-supervised continual active learning (SSCAL), works with any acquisition function. It admits pseudo-labels based on distances from unlabeled examples to labeled neighbors, caps their share, and reweights the loss on labeled examples to keep their expected gradient contribution unchanged. On frozen embeddings, its pseudo-labeler updates incrementally without extra model forward passes. Across two audio datasets, two frozen encoders, six acquisition functions and ten seeds, SSCAL ranks first on average among four protocols with equal training steps, answering the question affirmatively.

Figures & tables

Setting	uniform	equal	new-only	SSCAL	epoch †
BSD10K / CLAP	57.79 $\pm$ 0.67 (2.08)	56.92 $\pm$ 0.53 (2.90)	51.20 $\pm$ 0.18 (4.00)	59.79 $\pm$ 0.53 (1.02) ∗	59.01 $\pm$ 0.42 (–)
BSD10K / PANNs	50.70 $\pm$ 0.55 (1.65)	49.66 $\pm$ 0.39 (2.77)	39.40 $\pm$ 0.36 (4.00)	50.86 $\pm$ 0.62 (1.58) ns	52.21 $\pm$ 0.65 (–)
FSD50K / CLAP	44.40 $\pm$ 0.60 (2.27)	43.94 $\pm$ 0.55 (2.73)	36.15 $\pm$ 0.60 (4.00)	48.95 $\pm$ 0.58 (1.00) ∗	56.16 $\pm$ 0.33 (–)
FSD50K / PANNs	45.69 $\pm$ 0.21 (2.01)	45.42 $\pm$ 0.26 (2.88)	36.43 $\pm$ 0.38 (4.00)	47.15 $\pm$ 0.17 (1.12) ∗	43.46 $\pm$ 0.15 (–)
First / 240	34	4	0	202	–

Table 1: AULC (

\%

, higher is better), mean

\pm

s.d. over ten seeds, each averaging six acquisition functions. Parentheses give mean ranks among four fixed-step protocols within paired runs. ∗ marks

p<0.001

and ns marks

p=0.42

(two-sided paired Wilcoxon, 60 pairs per setting, unadjusted). The epoch † protocol runs

10.5\times

as many training steps, for reference only, and has no rank (–).

Acquisition	uniform	equal	new-only	SSCAL	epoch †
random	51.44 $\pm$ 0.48 (1.98)	50.84 $\pm$ 0.41 (2.90)	44.64 $\pm$ 0.34 (4.00)	52.64 $\pm$ 0.50 (1.13)	53.00 $\pm$ 0.48 (–)
entropy	45.28 $\pm$ 0.47 (2.08)	44.77 $\pm$ 0.55 (2.68)	33.17 $\pm$ 0.39 (4.00)	48.68 $\pm$ 0.65 (1.25)	50.32 $\pm$ 0.39 (–)
margin	46.20 $\pm$ 0.50 (2.15)	45.77 $\pm$ 0.51 (2.65)	35.65 $\pm$ 0.80 (4.00)	49.64 $\pm$ 0.63 (1.20)	51.35 $\pm$ 0.63 (–)
Core-set	49.86 $\pm$ 0.39 (1.78)	48.60 $\pm$ 0.33 (2.98)	40.07 $\pm$ 0.28 (4.00)	51.23 $\pm$ 0.53 (1.25)	52.36 $\pm$ 0.43 (–)
TypiClust	53.22 $\pm$ 0.33 (1.85)	52.52 $\pm$ 0.37 (3.00)	46.00 $\pm$ 0.24 (4.00)	54.35 $\pm$ 0.33 (1.15)	54.63 $\pm$ 0.33 (–)
BADGE	51.86 $\pm$ 0.46 (2.19)	51.42 $\pm$ 0.24 (2.71)	45.23 $\pm$ 0.24 (4.00)	53.58 $\pm$ 0.29 (1.10)	54.60 $\pm$ 0.41 (–)

Table 2: AULC (

\%

), mean

\pm

s.d. over ten seeds, each averaging four settings. The overall row also averages all six acquisition functions within each seed. Values combine BSD10K macro accuracy and FSD50K macro mAP. Parentheses give mean within-run ranks. The epoch † protocol runs

10.5\times

as many training steps, for reference only, and has no rank (–).

Figure 1: Aggregated test learning curves for the four settings against the labeled fraction of the training set. Each seed first averages six acquisition functions. Lines and shading show the mean and one standard deviation of these curves across ten seeds. The epoch † protocol runs

10.5\times

as many training steps and is shown for reference only.

$\rho$	$\beta=0.1$	$\beta=0.2$	$\beta=0.4$
0.6	+0.15 $\pm$ 0.11 (1.25)	+0.16 $\pm$ 0.11 (1.38)	+0.12 $\pm$ 0.14 (1.38)
0.8	+0.48 $\pm$ 0.09 (1.08)	+0.61 $\pm$ 0.08 (1.17)	+0.70 $\pm$ 0.11 (1.21)
1.0	+1.01 $\pm$ 0.12 (1.13)	+1.50 $\pm$ 0.15 (1.21)	+2.04 $\pm$ 0.15 (1.13)

Table 3: Paired

\Delta

AULC versus uniform (points), mean

\pm

s.d. across ten seeds after averaging all 24 configurations within each seed. Ranks in parentheses compare seed means against uniform, equal and new-only within configurations (Tables 1 – 2 use run ranks). Bold marks the best validation AULC.

Setting	Prec.	Cov.	$\|P_{t}\|$	$\Delta$ Wall (s)
				SSCAL	epoch †
B / CLAP	83.2	52.8	4092	+1.36	+7.43
B / PANNs	75.4	48.0	3730	+0.94	+7.91
F / CLAP	95.2	85.7	28035	+1.36	+9.85
F / PANNs	87.2	71.3	23331	+0.25	+15.95

Table 4: Default SSCAL, with B and F for BSD10K and FSD50K. Prec. is pooled positive-label precision and Cov. is

|P_{t}|/|U_{t}|

, both in

\%

, averaged with

|P_{t}|

over AL rounds and runs.

\Delta

Wall is training-side wall time (training + policy) over initialization and 40 AL rounds in excess of uniform (0.95–2.27 s), averaged over 60 runs. SSCAL and uniform run 820 steps, epoch † 8,610 (

10.5\times

, reference only).

Active Learning

Active learning selects the examples, labels, features, or experiments most useful for improving a model under limited budgets.

See all 290 papers

Geometric Consistency Based Label-Free Validation for Checkpoint Selection in Audio Active Learning
Oct 26, 2026Shiqi Zhang, Tuomas VirtanenActive Learning Audio Classification
Faculty of Information Technology and Communication Sciences, Tampere University, Tampere, Finland
Batch Bayesian Active Learning with Partial Batch Label Sampling
Oct 10, 2025Kangping Hu, Stephen MussmannActive Learning
Department of Computer Science, Georgia Institute of Technology, Atlanta, GA 30332
Active Learning with Imperfect Labels: Optimal Labeler Assignment and Sample Selection
Dec 14, 2025Pouya Ahadi, Blair Winograd, Camille Zaug +3Human-in-the-Loop Annotation Active Learning
School of Industrial and Systems Engineering Georgia Institute of Technology Atlanta, GA 30332, USA · Ford Motor Company

Oct 26, 2026J/K move · Enter open · S save

Geometric Consistency Based Label-Free Validation for Checkpoint Selection in Audio Active Learning

Shiqi Zhang, Tuomas Virtanen

Faculty of Information Technology and Communication Sciences, Tampere University, Tampere, Finland

Active Learning Audio Classification

Open paper AskPDF arXiv

Audio active learning (AL) trains a classifier at low annotation cost by labeling only an informative fraction of an unlabeled collection over successive rounds. Each round picks one of the checkpoints saved during training. Mainstream AL protocols do so with an assumed validation set, but in practice its labels also count toward the annotation cost, shrinking the advantage of an already small annotation budget. We propose geometric consistency, a label-free validation rule for checkpoint selection. Both classifiers, nearest-class-mean and k-nearest-neighbor, are built from the labeled set in a frozen pretrained embedding space. They then predict the unlabeled data to model its geometry. The rule scores each checkpoint by its agreement with these pseudo-labels and its cross-entropy on the labeled set. It works with any query strategy, has one hyperparameter, insensitive in practice, and adds only one batched forward pass over the unlabeled data per round. At matched label budgets across datasets, encoders, and query strategies, it attains the best mean rank in area under the labeling curve (AULC) among methods without extra labels and is on par with an out-of-budget validation set. To our knowledge, this is the first systematic study of label-free validation for checkpoint selection in audio AL.

Abstract

Active Learning

Explore similar work

Geometric Consistency Based Label-Free Validation for Checkpoint Selection in Audio Active Learning

Batch Bayesian Active Learning with Partial Batch Label Sampling

Active Learning with Imperfect Labels: Optimal Labeler Assignment and Sample Selection