Batch Bayesian Active Learning with Partial Batch Label Sampling

Kangping Hu; Stephen Mussmann

cs.LGOct 10, 2025

Batch Bayesian Active Learning with Partial Batch Label Sampling

Authors: Kangping Hu, Stephen Mussmann

Organizations: Department of Computer Science, Georgia Institute of Technology, Atlanta, GA 30332

Abstract

Over the past couple of decades, many active learning acquisition functions have been proposed, leaving practitioners with an unclear choice of which to use. Bayesian active learning offers principled objectives with explainable intuition, including Expected Error Reduction (EER), Expected Predictive Information Gain (EPIG), and Bayesian Active Learning by Disagreements (BALD). A key challenge of such methods is the difficulty of extending to the batch setting, leading to either computational challenges (BatchBALD) or dramatic performance drops (top- $B$ selection). In this work, we extend active learning's Bayesian Decision Theory framework to derive Partial Batch Label Sampling (ParBaLS) for the EPIG algorithm. We show experimentally that ParBaLS EPIG gives superior performance for a variety of datasets with different budgets using Bayesian logistic regression on embeddings from large pre-trained models. Our code is available at https://github.com/ADDAPT-ML/ParBaLS.

Figures & tables

Figure 1: ParBaLS overview, with an example snapshot when the batch size is $B=4$ , the currently labeled points are $L=\{7,8,9\}$ , and the chosen but unlabeled points in the partial batch are $S=\{2,4\}$ . We see how the point $\{5\}$ is selected by the inner loop and added to $S$ . After this update, we assume another inner loop to select the point $\{1\}$ and achieve $|S|=4=B$ . Then we execute the outer loop of sending the batch $S$ to the oracle annotator for ground-truth labels, which are added to $L$ for the next iteration.

Algorithm	Bayesian Tables				Heuristics Tables
	Highest Mean (HM)	Ties with HM	Wins $-$ Losses	Highest Loss Rate $\downarrow$	Highest Mean (HM)	Ties with HM	Wins $-$ Losses	Highest Loss Rate $\downarrow$
Random	-	-	-	-	1.65%	36.09%	-1.08	61.71%
Confidence	-	-	-	-	14.33%	71.35%	+1.56	25.62%
GLISTER	-	-	-	-	2.20%	26.72%	-3.07	75.76%
CoreSet	-	-	-	-	1.93%	28.93%	-4.03	71.90%
BADGE	-	-	-	-	2.75%	39.12%	-0.95	61.98%

Table 1: Leaderboard across all 11 datasets and three budget settings, evaluated separately at each iteration. Highest Mean (HM) and Ties with HM show the percentages of available dataset-budget-iteration evaluations. Wins

-

Losses is the sum of normalized pairwise win rates in a strategy’s Figure 8 row minus the corresponding column sum. Highest Loss Rate is the worst loss rate of each algorithm in Figure 8 , where lower is better. BatchBALD is evaluated only for the

B=10

budget setting.

Figure 2: Learning curves of Bayesian AL algorithms with Bayesian Logistic Regression, where each of the 10 iterations has a labeling budget of 20 samples, starting with random initialization of 100 samples.

Figure 3: Learning curves of ParBaLS and heuristics baselines with Bayesian Logistic Regression, where each of the 10 iterations has a labeling budget of 20 samples, starting with random initialization of 100 samples.

Figure 4: Comparison between batch acquisition and single selection with a total of 300 labeling budget, starting with random initialization of 100 samples.

Appendix figures & tables39 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: An illustrative diagram for Active Learning.

Figure 6: An illustrative diagram for ParBaLS.

Figure 7: An illustration of the proposed method, ParBaLS.

Dataset	Encoder	Before PCA	After PCA
CIFAR-10	DINOv2-ViT-S/14	384	348
iWildCam	CLIP-ViT-B/32	512	305
fMoW	CLIP-ViT-B/32	512	342
AG News	bert-base-uncased	768	210
Yelp Review Full	bert-base-uncased	768	168
Civil Comments	bert-base-uncased	768	187

Appendix

Table 2: Encoders used for each dataset and the embedding dimensions before and after 99% PCA.

Figure 8: Pairwise statistical-significance win matrices across all acquisition rounds.

Metric	Mean improvement	95% interval
Final test accuracy	$+1.54$ pp	$[+0.46,+2.62]$
Iteration-averaged test accuracy	$+1.25$ pp	$[+0.60,+1.91]$
Final test NLL reduction	0.0112	$[0.0052,0.0172]$
Iteration-averaged test NLL reduction	0.0086	$[0.0057,0.0115]$

Appendix

Table 3: Paired global comparison of ParBaLS with Top-

B

EPIG across 10 seeds, 11 datasets, and 3 budget settings.

Initial Budget	Acquisition Budget	Bayesian Methods	Heuristic Methods
100	$10\times 20$	Table 7	Table 10
20	$10\times 20$	Table 8	Table 11
100	$10\times 10$	Table 9	Table 12

Appendix

Table 4: Guide to the main active learning result tables. “Initial Budget” denotes the number of randomly labeled samples used for initialization. “Acquisition Budget” gives the number of active learning iterations and the per-iteration labeling budget.

Datasets	Airline Passenger	Credit Card
Algorithm	Satisfaction	Fraud	CIFAR-10
ParBaLS-MAP EPIG	89.34 $\pm$ 0.46	93.45 $\pm$ 0.29	93.40 $\pm$ 1.19
ParBaLS EPIG ( $m=5$ )	89.45 $\pm$ 0.58	93.37 $\pm$ 0.13	93.80 $\pm$ 1.18
ParBaLS EPIG ( $m=10$ )	89.35 $\pm$ 0.60	93.45 $\pm$ 0.22	94.90 $\pm$ 0.54
ParBaLS EPIG ( $m=20$ )	89.70 $\pm$ 0.55	93.49 $\pm$ 0.20	94.17 $\pm$ 0.72

Appendix

Table 5: Final test accuracy of different variants of ParBaLS with Bayesian Logistic Regression, where each of the 10 iterations has a labeling budget of 20 samples, except for the first iteration that starts with 100 samples.

Dataset	Initial Budget	Acquisition Budget	Bayesian Methods	Heuristic Methods
CIFAR-10	20	$10\times 20$	Table 13	Table 15
CIFAR-10	100	$10\times 20$	Table 14	Table 16
iWildCam	20	$10\times 20$	Table 17	Table 19
iWildCam	100	$10\times 20$	Table 18	Table 20
fMoW	20	$10\times 20$	Table 21	Table 23
fMoW	100	$10\times 20$	Table 22	Table 24

Appendix

Table 6: Guide to the one-vs-all active learning result tables. “Initial Budget” denotes the number of randomly labeled samples used for initialization. “Acquisition Budget” gives the number of active learning iterations and the per-iteration labeling budget.

Datasets	Tabular Datasets					Text Datasets			Image Datasets
Algorithm	Airline	Credit Card	Magic	Satellite	Vowels	Civil Comments	AG News	Yelp	CIFAR-10	iWildCam	fMoW
Top- $B$ BALD	87.07 $\pm$ 0.83	93.41 $\pm$ 0.18	68.73 $\pm$ 2.09	82.59 $\pm$ 0.28	43.46 $\pm$ 0.20	82.19 $\pm$ 2.49	80.19 $\pm$ 1.80	71.58 $\pm$ 1.41	88.25 $\pm$ 0.93	78.62 $\pm$ 2.83	91.78 $\pm$ 3.84
PowerBALD	88.43 $\pm$ 0.46	93.18 $\pm$ 0.20	65.09 $\pm$ 1.33	82.69 $\pm$ 0.37	43.64 $\pm$ 0.43	86.67 $\pm$ 0.89	85.17 $\pm$ 1.02	78.51 $\pm$ 1.14	90.06 $\pm$ 0.75	82.93 $\pm$ 4.62	91.42 $\pm$ 1.18
SoftmaxBALD	87.42 $\pm$ 0.74	92.75 $\pm$ 0.24	65.11 $\pm$ 2.36	82.03 $\pm$ 0.61	43.23 $\pm$ 0.68	85.25 $\pm$ 0.98	85.29 $\pm$ 0.75	78.11 $\pm$ 1.22	88.90 $\pm$ 0.79	81.99 $\pm$ 4.63	90.44 $\pm$ 1.51
SoftRankBALD	88.65 $\pm$ 0.45	93.34 $\pm$ 0.13	65.66 $\pm$ 1.82	82.41 $\pm$ 0.23	43.59 $\pm$ 0.37	86.35 $\pm$ 1.64	84.47 $\pm$ 1.82	75.39 $\pm$ 1.28	91.74 $\pm$ 0.86	85.27 $\pm$ 1.23	94.12 $\pm$ 2.02
Top- $B$ EPIG	89.05 $\pm$ 0.44	93.35 $\pm$ 0.14	65.45 $\pm$ 2.26	82.70 $\pm$ 0.19	43.51 $\pm$ 0.19	88.41 $\pm$ 0.95	87.54 $\pm$ 0.79	79.30 $\pm$ 0.99	93.82 $\pm$ 1.02	85.80 $\pm$ 3.00	97.06 $\pm$ 0.72

Appendix

Table 7: Bayesian AL comparison:

100+10\times 20

. The reported number is final test accuracy averaged over 10 seeds with

\pm

defining the

95\%

confidence interval using Student’s

t

-distribution. The highest mean accuracy is underlined, and all methods with means that are not significantly different (

p

-value greater than or equal to

0.05

) according to Welch’s

t

-test are bolded.

Datasets	Tabular Datasets					Text Datasets			Image Datasets
Algorithm	Airline	Credit Card	Magic	Satellite	Vowels	Civil Comments	AG News	Yelp	CIFAR-10	iWildCam	fMoW
Top- $B$ BALD	84.50 $\pm$ 2.10	93.07 $\pm$ 0.42	67.72 $\pm$ 4.32	82.52 $\pm$ 0.21	41.30 $\pm$ 0.61	70.83 $\pm$ 1.87	69.93 $\pm$ 2.01	63.60 $\pm$ 1.22	77.93 $\pm$ 1.06	66.73 $\pm$ 4.06	75.47 $\pm$ 2.40
PowerBALD	87.62 $\pm$ 0.62	93.09 $\pm$ 0.33	67.40 $\pm$ 1.76	82.29 $\pm$ 0.49	41.88 $\pm$ 1.90	82.40 $\pm$ 1.52	82.21 $\pm$ 1.08	77.86 $\pm$ 0.89	83.22 $\pm$ 0.72	79.84 $\pm$ 4.44	86.60 $\pm$ 2.14
SoftmaxBALD	86.70 $\pm$ 0.59	93.08 $\pm$ 0.30	66.66 $\pm$ 2.28	81.81 $\pm$ 0.66	41.60 $\pm$ 1.43	83.03 $\pm$ 1.39	81.96 $\pm$ 1.41	76.98 $\pm$ 1.26	83.86 $\pm$ 1.11	81.31 $\pm$ 4.93	87.06 $\pm$ 1.78
SoftRankBALD	87.88 $\pm$ 0.46	93.43 $\pm$ 0.28	65.01 $\pm$ 2.20	81.71 $\pm$ 0.37	41.84 $\pm$ 0.74	82.89 $\pm$ 2.58	78.80 $\pm$ 1.93	69.87 $\pm$ 1.31	82.67 $\pm$ 0.95	73.53 $\pm$ 3.80	82.98 $\pm$ 2.15
Top- $B$ EPIG	88.31 $\pm$ 0.33	93.26 $\pm$ 0.28	64.97 $\pm$ 3.08	81.89 $\pm$ 0.28	41.28 $\pm$ 1.31	86.37 $\pm$ 1.20	84.33 $\pm$ 1.29	77.31 $\pm$ 0.84	87.17 $\pm$ 0.97	75.71 $\pm$ 6.66	92.03 $\pm$ 0.95

Appendix

Table 8: Bayesian AL comparison:

20+10\times 20

. The reported number is final test accuracy averaged over 10 seeds with

\pm

defining the

95\%

confidence interval using Student’s

t

-distribution. The highest mean accuracy is underlined, and all methods with means that are not significantly different (

p

-value greater than or equal to

0.05

) according to Welch’s

t

-test are bolded.

Datasets	Tabular Datasets					Text Datasets			Image Datasets
Algorithm	Airline	Credit Card	Magic	Satellite	Vowels	Civil Comments	AG News	Yelp	CIFAR-10	iWildCam	fMoW
Top- $B$ BALD	86.94 $\pm$ 0.75	93.28 $\pm$ 0.20	66.97 $\pm$ 1.74	81.44 $\pm$ 0.56	41.80 $\pm$ 0.87	79.63 $\pm$ 1.71	77.04 $\pm$ 1.65	69.61 $\pm$ 1.46	79.35 $\pm$ 1.12	74.41 $\pm$ 3.02	81.79 $\pm$ 1.46
PowerBALD	87.14 $\pm$ 0.54	93.12 $\pm$ 0.25	66.66 $\pm$ 2.36	81.75 $\pm$ 0.48	41.28 $\pm$ 1.14	81.18 $\pm$ 1.36	81.49 $\pm$ 1.13	76.37 $\pm$ 1.09	82.20 $\pm$ 1.04	79.49 $\pm$ 5.06	84.31 $\pm$ 1.30
SoftmaxBALD	87.01 $\pm$ 0.71	92.69 $\pm$ 0.34	65.27 $\pm$ 1.18	81.66 $\pm$ 0.66	41.65 $\pm$ 1.80	81.47 $\pm$ 1.69	80.90 $\pm$ 1.49	76.16 $\pm$ 1.04	82.04 $\pm$ 0.95	81.72 $\pm$ 4.30	84.18 $\pm$ 1.37
SoftRankBALD	87.89 $\pm$ 0.51	93.51 $\pm$ 0.17	66.05 $\pm$ 1.83	81.57 $\pm$ 0.39	41.65 $\pm$ 1.32	82.56 $\pm$ 2.04	80.66 $\pm$ 1.70	72.54 $\pm$ 1.00	82.62 $\pm$ 1.24	77.03 $\pm$ 3.68	82.52 $\pm$ 1.43
BatchBALD	86.69 $\pm$ 0.72	93.27 $\pm$ 0.09	67.98 $\pm$ 1.89	81.39 $\pm$ 0.77	41.80 $\pm$ 0.91	82.63 $\pm$ 2.68	78.66 $\pm$ 1.43	70.82 $\pm$ 0.87	81.90 $\pm$ 1.39	76.92 $\pm$ 3.27	84.52 $\pm$ 1.45

Appendix

Table 9: Bayesian AL comparison:

100+10\times 10

. The reported number is final test accuracy averaged over 10 seeds with

\pm

defining the

95\%

confidence interval using Student’s

t

-distribution. The highest mean accuracy is underlined, and all methods with means that are not significantly different (

p

-value greater than or equal to

0.05

) according to Welch’s

t

-test are bolded.

Datasets	Tabular Datasets					Text Datasets			Image Datasets
Algorithm	Airline	Credit Card	Magic	Satellite	Vowels	Civil Comments	AG News	Yelp	CIFAR-10	iWildCam	fMoW
Random	87.24 $\pm$ 0.52	92.84 $\pm$ 0.29	65.77 $\pm$ 2.10	82.40 $\pm$ 0.48	43.55 $\pm$ 0.52	84.80 $\pm$ 1.25	83.92 $\pm$ 0.75	77.61 $\pm$ 0.96	88.39 $\pm$ 0.48	82.30 $\pm$ 3.90	90.02 $\pm$ 1.49
Confidence	89.35 $\pm$ 0.33	92.71 $\pm$ 0.41	66.71 $\pm$ 1.74	82.44 $\pm$ 0.59	43.33 $\pm$ 0.36	89.67 $\pm$ 0.74	87.87 $\pm$ 0.78	80.11 $\pm$ 1.01	96.46 $\pm$ 0.44	90.24 $\pm$ 3.00	98.96 $\pm$ 0.61
GLISTER	87.43 $\pm$ 0.73	93.42 $\pm$ 0.35	62.25 $\pm$ 4.61	80.86 $\pm$ 0.65	43.16 $\pm$ 0.62	84.13 $\pm$ 1.53	82.12 $\pm$ 0.89	73.07 $\pm$ 1.02	91.65 $\pm$ 1.01	79.29 $\pm$ 2.99	91.69 $\pm$ 3.28
CoreSet	85.87 $\pm$ 0.91	93.05 $\pm$ 0.24	66.28 $\pm$ 1.53	81.02 $\pm$ 0.43	43.46 $\pm$ 0.32	71.58 $\pm$ 1.42	74.59 $\pm$ 1.46	69.26 $\pm$ 1.12	73.75 $\pm$ 0.71	71.95 $\pm$ 3.12	73.35 $\pm$ 3.33
BADGE	87.88 $\pm$ 0.72	92.77 $\pm$ 0.30	66.31 $\pm$ 1.58	82.35 $\pm$ 0.55	43.40 $\pm$ 0.24	85.04 $\pm$ 1.64	83.84 $\pm$ 0.89	77.94 $\pm$ 1.23	88.19 $\pm$ 0.73	85.95 $\pm$ 2.69	92.54 $\pm$ 1.46

Appendix

Table 10: Heuristics AL comparison:

100+10\times 20

. The reported number is final test accuracy averaged over 10 seeds with

\pm

defining the

95\%

confidence interval using Student’s

t

-distribution. The highest mean accuracy is underlined, and all methods with means that are not significantly different (

p

-value greater than or equal to

0.05

) according to Welch’s

t

-test are bolded.

Datasets	Tabular Datasets					Text Datasets			Image Datasets
Algorithm	Airline	Credit Card	Magic	Satellite	Vowels	Civil Comments	AG News	Yelp	CIFAR-10	iWildCam	fMoW
Random	86.17 $\pm$ 0.65	92.41 $\pm$ 0.38	65.92 $\pm$ 1.35	80.99 $\pm$ 0.72	42.01 $\pm$ 1.34	82.63 $\pm$ 1.60	80.69 $\pm$ 0.89	76.47 $\pm$ 0.83	83.47 $\pm$ 1.16	78.58 $\pm$ 2.95	85.47 $\pm$ 1.33
Confidence	88.61 $\pm$ 0.52	92.22 $\pm$ 0.49	66.02 $\pm$ 2.90	81.50 $\pm$ 0.60	43.42 $\pm$ 1.40	87.28 $\pm$ 1.55	85.34 $\pm$ 0.84	78.69 $\pm$ 1.09	90.29 $\pm$ 1.35	86.65 $\pm$ 2.30	94.69 $\pm$ 1.61
GLISTER	86.02 $\pm$ 1.11	92.40 $\pm$ 0.89	65.97 $\pm$ 4.94	76.05 $\pm$ 2.55	40.54 $\pm$ 1.94	72.36 $\pm$ 2.50	72.12 $\pm$ 2.32	65.88 $\pm$ 1.50	78.71 $\pm$ 2.03	67.11 $\pm$ 3.71	75.89 $\pm$ 4.56
CoreSet	80.57 $\pm$ 2.04	93.28 $\pm$ 0.17	68.86 $\pm$ 3.71	80.02 $\pm$ 1.14	38.98 $\pm$ 0.71	57.48 $\pm$ 0.76	63.18 $\pm$ 1.11	58.95 $\pm$ 1.53	62.60 $\pm$ 0.56	60.13 $\pm$ 1.95	57.75 $\pm$ 2.46
BADGE	87.09 $\pm$ 0.58	93.18 $\pm$ 0.23	64.85 $\pm$ 2.31	81.42 $\pm$ 0.52	41.84 $\pm$ 1.54	83.07 $\pm$ 1.42	81.27 $\pm$ 0.85	75.50 $\pm$ 0.82	83.59 $\pm$ 0.77	83.97 $\pm$ 2.22	85.81 $\pm$ 1.48

Appendix

Table 11: Heuristics AL comparison:

20+10\times 20

. The reported number is final test accuracy averaged over 10 seeds with

\pm

defining the

95\%

confidence interval using Student’s

t

-distribution. The highest mean accuracy is underlined, and all methods with means that are not significantly different (

p

-value greater than or equal to

0.05

) according to Welch’s

t

-test are bolded.

Datasets	Tabular Datasets					Text Datasets			Image Datasets
Algorithm	Airline	Credit Card	Magic	Satellite	Vowels	Civil Comments	AG News	Yelp	CIFAR-10	iWildCam	fMoW
Random	86.65 $\pm$ 0.81	92.44 $\pm$ 0.27	66.16 $\pm$ 1.99	81.95 $\pm$ 0.41	40.71 $\pm$ 2.00	80.88 $\pm$ 1.46	80.64 $\pm$ 1.41	75.39 $\pm$ 1.12	82.78 $\pm$ 1.19	78.31 $\pm$ 5.05	84.70 $\pm$ 1.42
Confidence	88.15 $\pm$ 0.66	92.82 $\pm$ 0.41	66.48 $\pm$ 1.99	81.14 $\pm$ 0.66	42.47 $\pm$ 1.56	86.43 $\pm$ 1.79	83.78 $\pm$ 1.09	77.31 $\pm$ 1.10	87.31 $\pm$ 0.87	83.55 $\pm$ 3.11	91.78 $\pm$ 1.34
GLISTER	87.14 $\pm$ 0.64	93.30 $\pm$ 0.39	63.07 $\pm$ 4.32	80.88 $\pm$ 0.68	39.55 $\pm$ 1.64	80.19 $\pm$ 2.57	77.92 $\pm$ 1.93	71.04 $\pm$ 1.19	82.78 $\pm$ 1.22	73.29 $\pm$ 4.52	81.78 $\pm$ 2.11
CoreSet	86.18 $\pm$ 0.63	93.32 $\pm$ 0.30	64.94 $\pm$ 2.37	81.16 $\pm$ 0.86	40.71 $\pm$ 0.75	72.33 $\pm$ 1.45	73.23 $\pm$ 1.58	68.88 $\pm$ 1.44	72.26 $\pm$ 0.78	69.24 $\pm$ 2.68	72.79 $\pm$ 2.19
BADGE	86.94 $\pm$ 0.46	92.88 $\pm$ 0.48	65.70 $\pm$ 2.09	82.26 $\pm$ 0.51	41.80 $\pm$ 2.02	79.76 $\pm$ 1.79	79.44 $\pm$ 1.33	75.19 $\pm$ 1.08	81.44 $\pm$ 0.75	80.40 $\pm$ 3.32	82.56 $\pm$ 1.49

Appendix

Table 12: Heuristics AL comparison:

100+10\times 10

. The reported number is final test accuracy averaged over 10 seeds with

\pm

defining the

95\%

confidence interval using Student’s

t

-distribution. The highest mean accuracy is underlined, and all methods with means that are not significantly different (

p

-value greater than or equal to

0.05

) according to Welch’s

t

-test are bolded.

Figure 9: Learning curves of Bayesian AL algorithms with Bayesian Logistic Regression, where each of the 10 iterations has a labeling budget of 20 samples, starting with random initialization of 20 samples.

Figure 10: Learning curves of ParBaLS and heuristics baselines with Bayesian Logistic Regression, where each of the 10 iterations has a labeling budget of 20 samples, starting with random initialization of 20 samples.

Figure 11: Learning curves of Bayesian AL algorithms with Bayesian Logistic Regression, where each of the 10 iterations has a labeling budget of 10 samples, starting with random initialization of 100 samples.

Figure 12: Learning curves of ParBaLS and heuristics baselines with Bayesian Logistic Regression, where each of the 10 iterations has a labeling budget of 10 samples, starting with random initialization of 100 samples.

Datasets	One-vs-all CIFAR-10
Algorithm	0	1	2	3	4	5	6	7	8	9
Top- $B$ BALD	77.93 $\pm$ 1.06	82.13 $\pm$ 0.89	76.68 $\pm$ 1.52	77.13 $\pm$ 1.02	82.07 $\pm$ 1.14	79.52 $\pm$ 0.91	79.18 $\pm$ 0.92	83.30 $\pm$ 0.79	80.92 $\pm$ 1.11	80.79 $\pm$ 0.85
PowerBALD	83.22 $\pm$ 0.72	84.28 $\pm$ 0.48	84.06 $\pm$ 0.94	81.79 $\pm$ 0.74	82.14 $\pm$ 1.45	82.03 $\pm$ 0.69	84.20 $\pm$ 1.07	84.97 $\pm$ 0.73	84.42 $\pm$ 0.64	84.32 $\pm$ 0.60
SoftmaxBALD	83.86 $\pm$ 1.11	83.43 $\pm$ 0.54	83.34 $\pm$ 0.72	81.38 $\pm$ 1.26	81.64 $\pm$ 1.16	82.76 $\pm$ 0.76	84.00 $\pm$ 0.95	84.07 $\pm$ 1.11	84.04 $\pm$ 0.88	83.87 $\pm$ 0.84
SoftRankBALD	82.67 $\pm$ 0.95	85.70 $\pm$ 0.94	81.82 $\pm$ 0.91	80.57 $\pm$ 0.71	84.24 $\pm$ 1.02	82.67 $\pm$ 0.89	83.24 $\pm$ 0.95	86.18 $\pm$ 1.19	84.28 $\pm$ 1.10	84.96 $\pm$ 0.73
Top- $B$ EPIG	87.17 $\pm$ 0.97	87.41 $\pm$ 0.84	86.06 $\pm$ 1.22	85.17 $\pm$ 0.91	85.42 $\pm$ 0.73	86.60 $\pm$ 0.69	88.00 $\pm$ 0.75	88.65 $\pm$ 1.09	88.25 $\pm$ 1.04	87.60 $\pm$ 0.83

Appendix

Table 13: Final test accuracy of Bayesian AL algorithms with Bayesian Logistic Regression on One-vs-all CIFAR-10, where each of the 10 iterations has a labeling budget of 20 samples, starting with random initialization of 20 samples.

Datasets	One-vs-all CIFAR-10
Algorithm	0	1	2	3	4	5	6	7	8	9
Top- $B$ BALD	88.25 $\pm$ 0.93	92.87 $\pm$ 0.64	86.98 $\pm$ 0.95	85.79 $\pm$ 1.10	91.68 $\pm$ 1.04	89.07 $\pm$ 1.04	89.23 $\pm$ 0.59	94.05 $\pm$ 1.17	91.70 $\pm$ 0.65	91.84 $\pm$ 0.83
PowerBALD	90.06 $\pm$ 0.75	90.22 $\pm$ 0.89	89.17 $\pm$ 1.39	87.22 $\pm$ 1.09	88.71 $\pm$ 0.87	87.95 $\pm$ 1.36	89.53 $\pm$ 1.15	90.54 $\pm$ 0.87	90.95 $\pm$ 1.00	89.13 $\pm$ 0.70
SoftmaxBALD	88.90 $\pm$ 0.79	89.88 $\pm$ 1.07	88.78 $\pm$ 0.96	87.46 $\pm$ 0.71	87.58 $\pm$ 1.30	88.07 $\pm$ 1.14	89.12 $\pm$ 0.54	89.72 $\pm$ 1.21	90.37 $\pm$ 0.78	89.96 $\pm$ 1.07
SoftRankBALD	91.74 $\pm$ 0.86	94.65 $\pm$ 0.49	91.00 $\pm$ 0.92	88.83 $\pm$ 0.64	91.84 $\pm$ 1.14	91.46 $\pm$ 0.99	91.65 $\pm$ 0.71	94.99 $\pm$ 0.55	93.45 $\pm$ 0.98	93.11 $\pm$ 0.63
Top- $B$ EPIG	93.82 $\pm$ 1.02	94.50 $\pm$ 0.95	93.23 $\pm$ 1.09	91.24 $\pm$ 0.71	92.12 $\pm$ 0.85	92.64 $\pm$ 0.62	94.84 $\pm$ 0.41	94.48 $\pm$ 0.88	94.98 $\pm$ 0.52	94.74 $\pm$ 0.87

Appendix

Table 14: Final test accuracy of Bayesian AL algorithms with Bayesian Logistic Regression on One-vs-all CIFAR-10, where each of the 10 iterations has a labeling budget of 20 samples, starting with random initialization of 100 samples.

Datasets	One-vs-all CIFAR-10
Algorithm	0	1	2	3	4	5	6	7	8	9
Random	83.47 $\pm$ 1.16	83.93 $\pm$ 0.75	82.99 $\pm$ 1.16	81.90 $\pm$ 0.75	82.58 $\pm$ 1.05	82.60 $\pm$ 1.21	83.94 $\pm$ 0.97	84.36 $\pm$ 0.89	83.70 $\pm$ 0.90	83.51 $\pm$ 1.03
Confidence	90.29 $\pm$ 1.35	91.16 $\pm$ 0.97	89.48 $\pm$ 0.70	87.04 $\pm$ 1.07	88.47 $\pm$ 0.67	88.55 $\pm$ 0.96	90.85 $\pm$ 0.73	91.32 $\pm$ 0.93	91.66 $\pm$ 0.55	90.63 $\pm$ 0.82
GLISTER	78.71 $\pm$ 2.03	81.66 $\pm$ 1.70	77.62 $\pm$ 1.38	78.59 $\pm$ 0.85	79.34 $\pm$ 1.03	79.14 $\pm$ 2.04	80.82 $\pm$ 2.01	80.57 $\pm$ 0.91	81.00 $\pm$ 1.89	79.81 $\pm$ 2.81
CoreSet	62.60 $\pm$ 0.56	63.46 $\pm$ 0.62	61.54 $\pm$ 0.76	62.10 $\pm$ 0.53	63.35 $\pm$ 0.64	63.05 $\pm$ 0.46	64.33 $\pm$ 0.66	62.83 $\pm$ 0.32	63.24 $\pm$ 0.71	62.94 $\pm$ 0.59
BADGE	83.59 $\pm$ 0.77	83.15 $\pm$ 0.99	82.45 $\pm$ 1.21	81.40 $\pm$ 0.94	82.07 $\pm$ 0.92	82.39 $\pm$ 1.08	83.25 $\pm$ 1.12	84.31 $\pm$ 0.94	84.25 $\pm$ 1.02	84.02 $\pm$ 1.28

Appendix

Table 15: Final test accuracy of ParBaLS and heuristic baselines with Bayesian Logistic Regression on One-vs-all CIFAR-10, where each of the 10 iterations has a labeling budget of 20 samples, starting with random initialization of 20 samples.

Datasets	One-vs-all CIFAR-10
Algorithm	0	1	2	3	4	5	6	7	8	9
Random	88.39 $\pm$ 0.48	89.21 $\pm$ 0.91	87.62 $\pm$ 0.84	86.23 $\pm$ 0.55	87.60 $\pm$ 0.87	86.86 $\pm$ 0.91	88.78 $\pm$ 1.09	88.69 $\pm$ 0.87	89.14 $\pm$ 0.77	88.60 $\pm$ 1.06
Confidence	96.46 $\pm$ 0.44	97.55 $\pm$ 0.43	95.44 $\pm$ 0.64	93.21 $\pm$ 0.74	94.75 $\pm$ 0.53	94.69 $\pm$ 0.60	96.97 $\pm$ 0.22	97.82 $\pm$ 0.44	97.87 $\pm$ 0.40	96.82 $\pm$ 0.33
GLISTER	91.65 $\pm$ 1.01	92.88 $\pm$ 0.73	89.61 $\pm$ 1.92	89.22 $\pm$ 1.21	91.22 $\pm$ 1.14	89.46 $\pm$ 1.33	92.36 $\pm$ 1.12	93.78 $\pm$ 1.26	92.01 $\pm$ 1.73	92.75 $\pm$ 0.99
CoreSet	73.75 $\pm$ 0.71	75.39 $\pm$ 0.90	73.36 $\pm$ 0.74	72.20 $\pm$ 0.95	74.11 $\pm$ 0.67	74.42 $\pm$ 0.93	74.87 $\pm$ 0.89	74.44 $\pm$ 0.90	74.69 $\pm$ 0.67	73.87 $\pm$ 0.87
BADGE	88.19 $\pm$ 0.73	89.96 $\pm$ 0.98	87.66 $\pm$ 0.99	86.47 $\pm$ 0.93	88.06 $\pm$ 0.99	88.12 $\pm$ 0.61	88.87 $\pm$ 0.67	90.70 $\pm$ 0.89	89.95 $\pm$ 1.00	89.09 $\pm$ 0.44

Appendix

Table 16: Final test accuracy of ParBaLS and heuristic baselines with Bayesian Logistic Regression on One-vs-all CIFAR-10, where each of the 10 iterations has a labeling budget of 20 samples, starting with random initialization of 100 samples.

Datasets	One-vs-all iWildCam
Algorithm	0	1	2	3	4
Top- $B$ BALD	66.73 $\pm$ 4.06	74.30 $\pm$ 2.91	76.66 $\pm$ 3.20	78.34 $\pm$ 2.10	75.41 $\pm$ 3.17
PowerBALD	79.84 $\pm$ 4.44	83.63 $\pm$ 4.00	82.14 $\pm$ 3.03	86.92 $\pm$ 4.52	82.66 $\pm$ 3.74
SoftmaxBALD	81.31 $\pm$ 4.93	82.49 $\pm$ 3.18	82.78 $\pm$ 2.12	85.52 $\pm$ 4.76	82.28 $\pm$ 3.63
SoftRankBALD	73.53 $\pm$ 3.80	83.37 $\pm$ 3.51	83.00 $\pm$ 4.28	84.90 $\pm$ 3.68	83.49 $\pm$ 2.48
Top- $B$ EPIG	75.71 $\pm$ 6.66	78.56 $\pm$ 5.40	74.86 $\pm$ 4.80	83.96 $\pm$ 4.15	80.81 $\pm$ 4.45

Appendix

Table 17: Final test accuracy of Bayesian AL algorithms with Bayesian Logistic Regression on One-vs-all iWildCam, where each of the 10 iterations has a labeling budget of 20 samples, starting with random initialization of 20 samples.

Datasets	One-vs-all iWildCam
Algorithm	0	1	2	3	4
Top- $B$ BALD	78.62 $\pm$ 2.83	87.80 $\pm$ 2.69	89.33 $\pm$ 3.67	93.68 $\pm$ 2.35	90.42 $\pm$ 1.99
PowerBALD	82.93 $\pm$ 4.62	86.18 $\pm$ 3.40	83.73 $\pm$ 5.29	91.36 $\pm$ 2.64	87.60 $\pm$ 2.58
SoftmaxBALD	81.99 $\pm$ 4.63	85.45 $\pm$ 3.94	84.44 $\pm$ 5.00	91.02 $\pm$ 3.04	87.43 $\pm$ 3.46
SoftRankBALD	85.27 $\pm$ 1.23	89.87 $\pm$ 2.99	90.97 $\pm$ 3.45	94.98 $\pm$ 4.25	91.25 $\pm$ 3.03
Top- $B$ EPIG	85.80 $\pm$ 3.00	87.68 $\pm$ 2.75	81.24 $\pm$ 4.56	91.66 $\pm$ 3.95	86.56 $\pm$ 4.49

Appendix

Table 18: Final test accuracy of Bayesian AL algorithms with Bayesian Logistic Regression on One-vs-all iWildCam, where each of the 10 iterations has a labeling budget of 20 samples, starting with random initialization of 100 samples.

Datasets	One-vs-all iWildCam
Algorithm	0	1	2	3	4
Random	78.58 $\pm$ 2.95	81.47 $\pm$ 4.14	80.23 $\pm$ 5.51	85.29 $\pm$ 4.27	83.42 $\pm$ 3.90
Confidence	86.65 $\pm$ 2.30	92.47 $\pm$ 1.76	90.27 $\pm$ 2.09	95.02 $\pm$ 3.61	92.41 $\pm$ 2.02
GLISTER	67.11 $\pm$ 3.71	70.73 $\pm$ 4.06	72.56 $\pm$ 5.24	74.70 $\pm$ 3.83	74.15 $\pm$ 3.04
CoreSet	60.13 $\pm$ 1.95	63.37 $\pm$ 3.28	57.61 $\pm$ 5.08	60.50 $\pm$ 4.09	53.88 $\pm$ 3.31
BADGE	83.97 $\pm$ 2.22	83.01 $\pm$ 2.69	80.29 $\pm$ 3.62	87.80 $\pm$ 3.63	86.10 $\pm$ 3.91

Appendix

Table 19: Final test accuracy of ParBaLS and heuristic baselines with Bayesian Logistic Regression on One-vs-all iWildCam, where each of the 10 iterations has a labeling budget of 20 samples, starting with random initialization of 20 samples.

Datasets	One-vs-all iWildCam
Algorithm	0	1	2	3	4
Random	82.30 $\pm$ 3.90	83.00 $\pm$ 4.90	81.70 $\pm$ 5.83	88.41 $\pm$ 4.18	85.87 $\pm$ 2.98
Confidence	90.24 $\pm$ 3.00	96.69 $\pm$ 2.03	94.12 $\pm$ 1.71	99.78 $\pm$ 0.02	96.51 $\pm$ 1.68
GLISTER	79.29 $\pm$ 2.99	85.97 $\pm$ 2.72	88.90 $\pm$ 3.35	92.28 $\pm$ 2.48	89.59 $\pm$ 2.13
CoreSet	71.95 $\pm$ 3.12	73.24 $\pm$ 3.24	71.84 $\pm$ 5.41	73.57 $\pm$ 4.55	66.75 $\pm$ 4.65
BADGE	85.95 $\pm$ 2.69	87.23 $\pm$ 4.01	86.00 $\pm$ 5.70	92.35 $\pm$ 3.98	88.90 $\pm$ 3.14

Appendix

Table 20: Final test accuracy of ParBaLS and heuristic baselines with Bayesian Logistic Regression on One-vs-all iWildCam, where each of the 10 iterations has a labeling budget of 20 samples, starting with random initialization of 100 samples.

Datasets	One-vs-all fMoW
Algorithm	0	1	2	3	4
Top- $B$ BALD	75.47 $\pm$ 2.40	74.41 $\pm$ 1.92	73.43 $\pm$ 2.37	71.15 $\pm$ 1.24	74.55 $\pm$ 1.91
PowerBALD	86.60 $\pm$ 2.14	84.77 $\pm$ 1.35	85.61 $\pm$ 1.80	83.91 $\pm$ 1.64	86.96 $\pm$ 1.52
SoftmaxBALD	87.06 $\pm$ 1.78	84.85 $\pm$ 1.37	85.04 $\pm$ 1.38	83.85 $\pm$ 1.40	86.21 $\pm$ 1.54
SoftRankBALD	82.98 $\pm$ 2.15	81.86 $\pm$ 1.66	81.18 $\pm$ 2.22	79.76 $\pm$ 2.60	83.88 $\pm$ 2.52
Top- $B$ EPIG	92.03 $\pm$ 0.95	90.12 $\pm$ 0.64	91.27 $\pm$ 0.99	87.99 $\pm$ 1.24	91.91 $\pm$ 0.90

Appendix

Table 21: Final test accuracy of Bayesian AL algorithms with Bayesian Logistic Regression on One-vs-all fMoW, where each of the 10 iterations has a labeling budget of 20 samples, starting with random initialization of 20 samples.

Datasets	One-vs-all fMoW
Algorithm	0	1	2	3	4
Top- $B$ BALD	91.78 $\pm$ 3.84	88.81 $\pm$ 2.49	89.80 $\pm$ 2.10	84.56 $\pm$ 2.98	91.57 $\pm$ 2.80
PowerBALD	91.42 $\pm$ 1.18	89.52 $\pm$ 1.91	90.24 $\pm$ 0.94	88.55 $\pm$ 1.37	91.11 $\pm$ 1.19
SoftmaxBALD	90.44 $\pm$ 1.51	88.33 $\pm$ 1.36	89.09 $\pm$ 1.17	88.08 $\pm$ 1.54	90.64 $\pm$ 1.06
SoftRankBALD	94.12 $\pm$ 2.02	90.59 $\pm$ 2.79	92.58 $\pm$ 1.52	88.80 $\pm$ 2.16	94.44 $\pm$ 2.08
Top- $B$ EPIG	97.06 $\pm$ 0.72	94.67 $\pm$ 0.68	96.25 $\pm$ 0.28	93.80 $\pm$ 0.88	96.96 $\pm$ 0.59

Appendix

Table 22: Final test accuracy of Bayesian AL algorithms with Bayesian Logistic Regression on One-vs-all fMoW, where each of the 10 iterations has a labeling budget of 20 samples, starting with random initialization of 100 samples.

Datasets	One-vs-all fMoW
Algorithm	0	1	2	3	4
Random	85.47 $\pm$ 1.33	83.47 $\pm$ 1.38	84.42 $\pm$ 1.09	82.61 $\pm$ 1.91	85.39 $\pm$ 1.33
Confidence	94.69 $\pm$ 1.61	91.84 $\pm$ 1.06	93.09 $\pm$ 1.44	91.44 $\pm$ 1.41	94.65 $\pm$ 1.41
GLISTER	75.89 $\pm$ 4.56	74.36 $\pm$ 4.60	73.75 $\pm$ 4.43	71.15 $\pm$ 4.58	75.09 $\pm$ 3.97
CoreSet	57.75 $\pm$ 2.46	56.98 $\pm$ 2.28	58.71 $\pm$ 2.14	58.43 $\pm$ 2.21	57.29 $\pm$ 2.11
BADGE	85.81 $\pm$ 1.48	83.89 $\pm$ 1.11	85.21 $\pm$ 1.21	82.97 $\pm$ 1.82	86.48 $\pm$ 0.94

Appendix

Table 23: Final test accuracy of ParBaLS and heuristic baselines with Bayesian Logistic Regression on One-vs-all fMoW, where each of the 10 iterations has a labeling budget of 20 samples, starting with random initialization of 20 samples.

Datasets	One-vs-all fMoW
Algorithm	0	1	2	3	4
Random	90.02 $\pm$ 1.49	88.00 $\pm$ 2.00	89.04 $\pm$ 1.77	86.66 $\pm$ 2.10	89.89 $\pm$ 1.44
Confidence	98.96 $\pm$ 0.61	97.42 $\pm$ 0.82	98.20 $\pm$ 0.43	95.84 $\pm$ 1.38	99.20 $\pm$ 0.27
GLISTER	91.69 $\pm$ 3.28	89.51 $\pm$ 3.01	89.75 $\pm$ 3.36	87.01 $\pm$ 3.99	92.18 $\pm$ 3.25
CoreSet	73.35 $\pm$ 3.33	72.14 $\pm$ 3.12	73.52 $\pm$ 2.69	71.78 $\pm$ 3.13	72.91 $\pm$ 2.95
BADGE	92.54 $\pm$ 1.46	90.66 $\pm$ 0.95	91.13 $\pm$ 1.01	88.84 $\pm$ 1.41	91.50 $\pm$ 1.04

Appendix

Table 24: Final test accuracy of ParBaLS and heuristic baselines with Bayesian Logistic Regression on One-vs-all fMoW, where each of the 10 iterations has a labeling budget of 20 samples, starting with random initialization of 100 samples.

Datasets	One-vs-all AG News
Algorithm	0	1	2	3
Top- $B$ BALD	69.93 $\pm$ 2.01	76.70 $\pm$ 1.37	69.21 $\pm$ 1.30	68.83 $\pm$ 1.10
PowerBALD	82.21 $\pm$ 1.08	88.81 $\pm$ 1.09	77.43 $\pm$ 1.50	78.93 $\pm$ 1.16
SoftmaxBALD	81.96 $\pm$ 1.41	87.83 $\pm$ 1.36	78.30 $\pm$ 0.98	79.47 $\pm$ 0.85
SoftRankBALD	78.80 $\pm$ 1.93	86.64 $\pm$ 1.13	75.52 $\pm$ 1.11	77.73 $\pm$ 1.40
Top- $B$ EPIG	84.33 $\pm$ 1.29	91.11 $\pm$ 1.02	78.61 $\pm$ 0.88	81.27 $\pm$ 0.80

Appendix

Table 25: Final test accuracy of Bayesian AL algorithms with Bayesian Logistic Regression on One-vs-all AG News, where each of the 10 iterations has a labeling budget of 20 samples, starting with random initialization of 20 samples.

Datasets	One-vs-all AG News
Algorithm	0	1	2	3
Top- $B$ BALD	80.19 $\pm$ 1.80	88.53 $\pm$ 1.30	78.79 $\pm$ 1.17	78.47 $\pm$ 1.49
PowerBALD	85.17 $\pm$ 1.02	91.46 $\pm$ 0.81	80.03 $\pm$ 0.96	82.53 $\pm$ 0.67
SoftmaxBALD	85.29 $\pm$ 0.75	91.45 $\pm$ 0.82	79.31 $\pm$ 1.14	81.87 $\pm$ 1.01
SoftRankBALD	84.47 $\pm$ 1.82	92.72 $\pm$ 1.27	80.89 $\pm$ 1.18	82.63 $\pm$ 0.93
Top- $B$ EPIG	87.54 $\pm$ 0.79	94.45 $\pm$ 0.68	81.79 $\pm$ 0.98	84.14 $\pm$ 0.71

Appendix

Table 26: Final test accuracy of Bayesian AL algorithms with Bayesian Logistic Regression on One-vs-all AG News, where each of the 10 iterations has a labeling budget of 20 samples, starting with random initialization of 100 samples.

Datasets	One-vs-all AG News
Algorithm	0	1	2	3
Random	80.69 $\pm$ 0.89	87.59 $\pm$ 1.17	76.29 $\pm$ 1.03	78.02 $\pm$ 1.32
Confidence	85.34 $\pm$ 0.84	92.55 $\pm$ 0.69	80.02 $\pm$ 1.19	82.35 $\pm$ 1.22
GLISTER	72.12 $\pm$ 2.32	77.76 $\pm$ 3.07	71.09 $\pm$ 1.86	70.69 $\pm$ 2.01
CoreSet	63.18 $\pm$ 1.11	65.31 $\pm$ 0.79	61.13 $\pm$ 1.11	63.44 $\pm$ 0.78
BADGE	81.27 $\pm$ 0.85	86.75 $\pm$ 0.86	75.79 $\pm$ 1.54	78.40 $\pm$ 0.85

Appendix

Table 27: Final test accuracy of ParBaLS and heuristic baselines with Bayesian Logistic Regression on One-vs-all AG News, where each of the 10 iterations has a labeling budget of 20 samples, starting with random initialization of 20 samples.

Datasets	One-vs-all AG News
Algorithm	0	1	2	3
Random	83.92 $\pm$ 0.75	90.92 $\pm$ 0.77	79.14 $\pm$ 1.04	80.90 $\pm$ 1.58
Confidence	87.87 $\pm$ 0.78	94.82 $\pm$ 0.76	82.49 $\pm$ 1.35	85.08 $\pm$ 0.78
GLISTER	82.12 $\pm$ 0.89	90.22 $\pm$ 1.06	79.19 $\pm$ 1.32	80.24 $\pm$ 1.14
CoreSet	74.59 $\pm$ 1.46	79.80 $\pm$ 1.14	71.15 $\pm$ 1.67	73.64 $\pm$ 1.42
BADGE	83.84 $\pm$ 0.89	90.65 $\pm$ 1.06	79.81 $\pm$ 0.94	81.15 $\pm$ 1.31

Appendix

Table 28: Final test accuracy of ParBaLS and heuristic baselines with Bayesian Logistic Regression on One-vs-all AG News, where each of the 10 iterations has a labeling budget of 20 samples, starting with random initialization of 100 samples.

Datasets	One-vs-all Yelp
Algorithm	0	1	2	3	4
Top- $B$ BALD	63.60 $\pm$ 1.22	58.83 $\pm$ 1.17	60.13 $\pm$ 1.04	60.87 $\pm$ 1.04	65.27 $\pm$ 0.79
PowerBALD	77.86 $\pm$ 0.89	68.33 $\pm$ 1.43	67.15 $\pm$ 1.46	68.47 $\pm$ 2.03	74.82 $\pm$ 1.17
SoftmaxBALD	76.98 $\pm$ 1.26	67.85 $\pm$ 1.42	67.68 $\pm$ 1.01	68.42 $\pm$ 1.10	74.47 $\pm$ 0.82
SoftRankBALD	69.87 $\pm$ 1.31	64.46 $\pm$ 1.81	64.86 $\pm$ 1.32	65.79 $\pm$ 0.68	73.24 $\pm$ 1.63
Top- $B$ EPIG	77.31 $\pm$ 0.84	68.76 $\pm$ 1.02	69.06 $\pm$ 0.83	68.82 $\pm$ 0.96	77.09 $\pm$ 0.93

Appendix

Table 29: Final test accuracy of Bayesian AL algorithms with Bayesian Logistic Regression on One-vs-all Yelp, where each of the 10 iterations has a labeling budget of 20 samples, starting with random initialization of 20 samples.

Datasets	One-vs-all Yelp
Algorithm	0	1	2	3	4
Top- $B$ BALD	71.58 $\pm$ 1.41	64.94 $\pm$ 1.34	65.26 $\pm$ 1.33	65.78 $\pm$ 1.07	71.63 $\pm$ 0.93
PowerBALD	78.51 $\pm$ 1.14	68.51 $\pm$ 1.27	69.00 $\pm$ 0.83	68.79 $\pm$ 1.24	76.16 $\pm$ 1.10
SoftmaxBALD	78.11 $\pm$ 1.22	70.08 $\pm$ 1.19	69.26 $\pm$ 0.95	69.58 $\pm$ 1.31	76.28 $\pm$ 0.68
SoftRankBALD	75.39 $\pm$ 1.28	68.48 $\pm$ 1.32	68.68 $\pm$ 1.30	69.57 $\pm$ 1.22	76.63 $\pm$ 1.07
Top- $B$ EPIG	79.30 $\pm$ 0.99	71.30 $\pm$ 0.86	71.07 $\pm$ 0.79	71.09 $\pm$ 0.63	78.56 $\pm$ 0.90

Appendix

Table 30: Final test accuracy of Bayesian AL algorithms with Bayesian Logistic Regression on One-vs-all Yelp, where each of the 10 iterations has a labeling budget of 20 samples, starting with random initialization of 100 samples.

Datasets	One-vs-all Yelp
Algorithm	0	1	2	3	4
Random	76.47 $\pm$ 0.83	68.73 $\pm$ 1.32	67.67 $\pm$ 1.42	66.99 $\pm$ 1.15	74.00 $\pm$ 1.12
Confidence	78.69 $\pm$ 1.09	69.85 $\pm$ 1.55	70.32 $\pm$ 0.87	70.35 $\pm$ 0.76	76.80 $\pm$ 1.18
GLISTER	65.88 $\pm$ 1.50	60.10 $\pm$ 2.48	61.08 $\pm$ 2.40	62.66 $\pm$ 1.59	66.71 $\pm$ 1.91
CoreSet	58.95 $\pm$ 1.53	55.78 $\pm$ 0.62	55.85 $\pm$ 1.23	56.79 $\pm$ 0.86	58.90 $\pm$ 1.12
BADGE	75.50 $\pm$ 0.82	68.10 $\pm$ 0.96	66.29 $\pm$ 0.87	67.83 $\pm$ 1.17	74.18 $\pm$ 0.90

Appendix

Table 31: Final test accuracy of ParBaLS and heuristic baselines with Bayesian Logistic Regression on One-vs-all Yelp, where each of the 10 iterations has a labeling budget of 20 samples, starting with random initialization of 20 samples.

Datasets	One-vs-all Yelp
Algorithm	0	1	2	3	4
Random	77.61 $\pm$ 0.96	69.03 $\pm$ 1.00	68.31 $\pm$ 0.95	69.45 $\pm$ 1.31	76.51 $\pm$ 0.85
Confidence	80.11 $\pm$ 1.01	71.63 $\pm$ 1.03	71.04 $\pm$ 0.84	71.44 $\pm$ 1.00	78.72 $\pm$ 0.97
GLISTER	73.07 $\pm$ 1.02	68.39 $\pm$ 1.41	66.46 $\pm$ 1.92	68.57 $\pm$ 1.01	74.49 $\pm$ 1.26
CoreSet	69.26 $\pm$ 1.12	63.23 $\pm$ 1.27	64.07 $\pm$ 1.29	64.56 $\pm$ 1.17	69.09 $\pm$ 1.23
BADGE	77.94 $\pm$ 1.23	68.74 $\pm$ 0.80	68.96 $\pm$ 1.39	68.40 $\pm$ 0.92	75.88 $\pm$ 1.29

Appendix

Table 32: Final test accuracy of ParBaLS and heuristic baselines with Bayesian Logistic Regression on One-vs-all Yelp, where each of the 10 iterations has a labeling budget of 20 samples, starting with random initialization of 100 samples.

Active Learning

Active learning selects the examples, labels, features, or experiments most useful for improving a model under limited budgets.

See all 280 papers

Loss-Driven Bayesian Active Learning
Apr 13, 2026Zhuoyue Huang, Freddie Bickford Smith, Tom RainforthActive Learning
University of Oxford
Labeled TrustSet Guided: Batch Active Learning with Reinforcement Learning
Apr 14, 2026Guofeng Cui, Yang Liu, Pichao Wang +4Active Learning
Prime Video, Amazon Seattle, USA
Active Learning with Imperfect Labels: Optimal Labeler Assignment and Sample Selection
Dec 14, 2025Pouya Ahadi, Blair Winograd, Camille Zaug +3Human-in-the-Loop Annotation Active Learning
School of Industrial and Systems Engineering Georgia Institute of Technology Atlanta, GA 30332, USA · Ford Motor Company

Apr 13, 2026J/K move · Enter open · S save

Loss-Driven Bayesian Active Learning

Zhuoyue Huang, Freddie Bickford Smith, Tom Rainforth

University of Oxford

Active Learning

Open paper AskPDF arXiv

The central goal of active learning is to gather data that maximises downstream predictive performance, but popular approaches have limited flexibility in customising this data acquisition to different downstream problems and losses. We propose a rigorous loss-driven approach to Bayesian active learning that allows data acquisition to directly target the loss associated with a given decision problem. In particular, we show how any loss can be used to derive a unique objective for optimal data acquisition. Critically, we then show that any loss taking the form of a weighted Bregman divergence permits analytic computation of a central component of its corresponding objective, making the approach applicable in practice. In regression and classification experiments with a range of different losses, we find our approach reduces test losses relative to existing techniques.

Abstract

Active Learning

Explore similar work

Loss-Driven Bayesian Active Learning

Labeled TrustSet Guided: Batch Active Learning with Reinforcement Learning

Active Learning with Imperfect Labels: Optimal Labeler Assignment and Sample Selection