Labeled TrustSet Guided: Batch Active Learning with Reinforcement Learning

Guofeng Cui; Yang Liu; Pichao Wang; Hankai Hsu; Xiaohang Sun; Xiang Hao; Zhu Liu

cs.LGApr 14, 2026

Labeled TrustSet Guided: Batch Active Learning with Reinforcement Learning

Authors: Guofeng Cui, Yang Liu, Pichao Wang, Hankai Hsu, Xiaohang Sun, Xiang Hao, Zhu Liu

Organizations: Prime Video, Amazon Seattle, USA

Abstract

Batch active learning (BAL) is a crucial technique for reducing labeling costs and improving data efficiency in training large-scale deep learning models. Traditional BAL methods often rely on metrics like Mahalanobis Distance to balance uncertainty and diversity when selecting data for annotation. However, these methods predominantly focus on the distribution of unlabeled data and fail to leverage feedback from labeled data or the model's performance. To address these limitations, we introduce TrustSet, a novel approach that selects the most informative data from the labeled dataset, ensuring a balanced class distribution to mitigate the long-tail problem. Unlike CoreSet, which focuses on maintaining the overall data distribution, TrustSet optimizes the model's performance by pruning redundant data and using label information to refine the selection process. To extend the benefits of TrustSet to the unlabeled pool, we propose a reinforcement learning (RL)-based sampling policy that approximates the selection of high-quality TrustSet candidates from the unlabeled data. Combining TrustSet and RL, we introduce the Batch Reinforcement Active Learning with TrustSet (BRAL-T) framework. BRAL-T achieves state-of-the-art results across 10 image classification benchmarks and 2 active fine-tuning tasks, demonstrating its effectiveness and efficiency in various domains.

Figures & tables

Figure 1: An overview of the BRAL-T framework, which consists of two main processes: (1) Active Learning for data selection. (2) Reinforcement Learning to approximate TrustSet selection.

Figure 2: Details of BRAL-T. (a) Active Learning Process: $\pi_{\phi_{i}}$ selects the subset $S_{i}$ from $U_{i}$ for the oracle to annotate, and the model $M_{\theta_{L_{i+1}}}$ is trained on $S_{i}\cup L_{i}$ . (b) RL process: we sample $\mathcal{L}$ and $\mathcal{U}$ from $L_{i+1}$ to train policy $\pi_{\phi_{i+1}}$ .

Figure 3: Reinforcement learning process. We use $\mathcal{U}_{0}$ as an example for process in the figure.

Methods	FashionMNIST		EMNIST		CIFAR10		CIFAR100
Methods	AUBC	F-acc	AUBC	F-acc	AUBC	F-acc	AUBC	F-acc
LossPrediction	0.859	0.888	0.762	0.793	0.837	0.911	0.481	0.655
WAAL	0.861	0.891	0.808	0.831	0.842	0.883	0.460	0.594
RandomSample	0.844	0.874	0.804	0.828	0.832	0.902	0.517	0.650
BRAL-T	0.863	0.894	0.813	0.833	0.847	0.916	0.525	0.662

Table 1: Experiment results of image classification task on 8 benchmarks.

Figure 4: Visualization of experiment results on Cifar10, Cifar10-imb, Cifar100 and FashionMNIST.

Methods	Cifar10-LT-r10		Cifar10-LT-r20		Cifar100-LT-r10		Cifar100-LT-r20
Methods	AUBC	F-acc	AUBC	F-acc	AUBC	F-acc	AUBC	F-acc
TiDAL	0.510	0.619	0.435	0.583	0.398	0.450	0.365	0.420
SIMILAR	0.507	0.654	0.431	0.566	0.405	0.514	0.366	0.460
LossPrediction	0.478	0.679	0.413	0.585	0.424	0.516	0.380	0.460
WAAL	0.510	0.623	0.435	0.589	0.381	0.458	0.342	0.412
RandomSample	0.476	0.686	0.397	0.587	0.382	0.463	0.343	0.410

Table 2: Experiment results on Cifar10-LT and Cifar100-LT datasets.

Table 7

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmarks	$\|L_{0}\|$	$\|U_{0}\|$	$Q$	$b$	$\#e$	$C$
FashionMNIST	500	59,500	10,000	250	40	10
EMNIST	1,000	696,932	50,000	500	40	62
CIFAR10	1,000	49,000	40,000	500	50	10
CIFAR100	1,000	49,000	40,000	500	60	100
CIFAR10-imb	1,000	27,239	20,000	500	50	10
CIFAR10-LT	1,000	-	$\|D\|$	100	50	10

Appendix

Table 5: Setting of benchmarks. Where

|L_{0}|

refers to size of initial labeled set,

|U_{0}|

refers to size of initial unlabeled data pool,

Q

refers to budget,

b

refers to batch size for target model training,

\#e

refers to number of epoch for target model training and

C

refers to number of clusters from unlabeled data pool. For all the benchmarks, the number of clusters

M

from labeled dataset is set to be the same as

C

and number of candidate action for

U_{c}

is set to be 5.

Methods	FashionMNIST		EMNIST		CIFAR10		CIFAR100
Methods	AUBC	F-acc	AUBC	F-acc	AUBC	F-acc	AUBC	F-acc
LossPrediction	$\pm$ 0.002	$\pm$ 0.038	$\pm$ 0.016	$\pm$ 0.022	$\pm$ 0.006	$\pm$ 0.012	$\pm$ 0.019	$\pm$ 0.012
WAAL	$\pm$ 0.002	$\pm$ 0.015	$\pm$ 0.012	$\pm$ 0.015	$\pm$ 0.006	$\pm$ 0.009	$\pm$ 0.006	$\pm$ 0.011
RandomSample	$\pm$ 0.001	$\pm$ 0.009	$\pm$ 0.004	$\pm$ 0.007	$\pm$ 0.003	$\pm$ 0.011	$\pm$ 0.003	$\pm$ 0.008
BRAL-T	$\pm$ 0.001	$\pm$ 0.008	$\pm$ 0.005	$\pm$ 0.014	$\pm$ 0.003	$\pm$ 0.006	$\pm$ 0.004	$\pm$ 0.009

Appendix

Table 6: Confidence Interval of Experiment results of image classification task.

Figure 5: Time Cost of BRAL-T and baselines.

Figure 6: Pairwise Comparison of BRAL-T, VAAL, SAAL and BAIT.

Method	AUBC	F-Acc
LossPrediction	0.748	0.848
WAAL	0.752	0.799
RandomSample	0.710	0.810
BRAL-T	0.762	0.851
BRAL-T (two agents)	0.755	0.837

Appendix

Table 7: BRAL-T reusing two RL agents.

Figure 7: Accuracy-budget curve of Different Initial Labeled Set Quality.

Initial Method	AUBC	F-Acc
Random	0.762	0.851
Twisted Main	0.750	0.855
Twisted Rare	0.756	0.855

Appendix

Table 8: Experiment Result of Different Initial Labeled Set Quality.

Figure 8: Comparison between BRAL-T with RL policy and Ground Truth Labels.

# Actions	AUBC	F-Acc
5	0.762	0.851
10	0.763	0.853
50	0.755	0.851
100	0.758	0.854

Appendix

Table 9: Ablation Study on Different Candidate Action Number.

$\lambda$	AUBC	F-Acc
0.25	0.762	0.851
1.00	0.752	0.842
2.00	0.749	0.830

Appendix

Table 10: Impact of

\lambda

value on CIFAR10-imb dataset.

Active Learning

Active learning selects the examples, labels, features, or experiments most useful for improving a model under limited budgets.

See all 280 papers

Batch Bayesian Active Learning with Partial Batch Label Sampling
Oct 10, 2025Kangping Hu, Stephen MussmannActive Learning
Department of Computer Science, Georgia Institute of Technology, Atlanta, GA 30332
Active Learning with Imperfect Labels: Optimal Labeler Assignment and Sample Selection
Dec 14, 2025Pouya Ahadi, Blair Winograd, Camille Zaug +3Human-in-the-Loop Annotation Active Learning
School of Industrial and Systems Engineering Georgia Institute of Technology Atlanta, GA 30332, USA · Ford Motor Company
Energy-Based Open-Set Active Learning for Object Classification
Apr 22, 2026Zongyao Lyu, William J. BeksiActive Learning Open-Set Recognition
The University of Texas at Arlington, Arlington TX 76019, USA

Oct 10, 2025J/K move · Enter open · S save

Batch Bayesian Active Learning with Partial Batch Label Sampling

Kangping Hu, Stephen Mussmann

Department of Computer Science, Georgia Institute of Technology, Atlanta, GA 30332

Active Learning

Open paper AskPDF arXiv

Over the past couple of decades, many active learning acquisition functions have been proposed, leaving practitioners with an unclear choice of which to use. Bayesian active learning offers principled objectives with explainable intuition, including Expected Error Reduction (EER), Expected Predictive Information Gain (EPIG), and Bayesian Active Learning by Disagreements (BALD). A key challenge of such methods is the difficulty of extending to the batch setting, leading to either computational challenges (BatchBALD) or dramatic performance drops (top-

B

selection). In this work, we extend active learning's Bayesian Decision Theory framework to derive Partial Batch Label Sampling (ParBaLS) for the EPIG algorithm. We show experimentally that ParBaLS EPIG gives superior performance for a variety of datasets with different budgets using Bayesian logistic regression on embeddings from large pre-trained models. Our code is available at https://github.com/ADDAPT-ML/ParBaLS.

Abstract

Active Learning

Explore similar work

Batch Bayesian Active Learning with Partial Batch Label Sampling

Active Learning with Imperfect Labels: Optimal Labeler Assignment and Sample Selection

Energy-Based Open-Set Active Learning for Object Classification