Diversity-Based Active Learning: An Evaluation of Metric Spaces for Active Learning Selection

Siddharth Chilamkur; Dorit S. Hochbaum

cs.LGAug 24, 2026

Diversity-Based Active Learning: An Evaluation of Metric Spaces for Active Learning Selection

Authors: Siddharth Chilamkur, Dorit S. Hochbaum

Organizations: University of California, Berkeley

Abstract

With rapid advancement over the last few years, many different methods are now widely used for classification. However, training these models requires substantial labeled data. Active Learning is a potential solution to this problem. Pool-based active learning minimizes costs by querying only the most informative samples from an unlabeled dataset. Diversity-based approaches, on the other hand, attempt to select a representative subset of the data. There are many different objectives for determining the selection process, including exact K-center, exact K-median, and Greedy K-center. In this paper, we will focus on evaluating the performance of Greedy K-center across a variety of metric spaces: the raw feature space, a Linear Discriminant Analysis (LDA) space, and a model-derived probability space (with and without entropy-based weighting). Using Random Forest classifiers as a baseline evaluator, our empirical results on synthetic and real-world datasets demonstrate that mapping unlabeled instances into a predictive probability space and weighting the result by entropy often dominates the other options for active learning selection with Greedy K-center.

Figures & tables

Dataset	Size	RF Time (s)	NN Time (s)
Iris	150	0.1617	0.2926
Wine	178	0.1657	0.6458
Spambase	4,601	0.2362	2.6099
Phoneme	5,404	0.2459	8.5601
Satimage	6,435	0.2832	9.0500

TABLE I: Fully Supervised Baselines: Runtime Comparison for RF vs. NN (L4 GPU)

Panel of Figure 2 — Fig. 1: Learning curves for large percentage batches (10%-50%). The charts demonstrate the performance advantage of mapping queries into the Probability Space, even as the complexity of the dataset increases (e.g. Synthetic 2C, Synthetic 3C).

Dataset	Size	Baseline	Random	Feature Space	LDA	Prob	Prob+Ent
Iris	150	0.951 $\pm$ 0.037	0.948 $\pm$ 0.043	0.947 $\pm$ 0.038	0.942 $\pm$ 0.045	0.951 $\pm$ 0.038	0.949 $\pm$ 0.039
Wine	178	0.981 $\pm$ 0.022	0.969 $\pm$ 0.032	0.966 $\pm$ 0.031	0.972 $\pm$ 0.024	0.980 $\pm$ 0.023	0.981 $\pm$ 0.022
Digits	1,797	0.973 $\pm$ 0.008	0.961 $\pm$ 0.010	0.974 $\pm$ 0.007	0.974 $\pm$ 0.008	0.978 $\pm$ 0.007	0.977 $\pm$ 0.007
Synthetic 1C	2,500	0.969 $\pm$ 0.014	0.961 $\pm$ 0.016	0.963 $\pm$ 0.015	0.960 $\pm$ 0.018	0.972 $\pm$ 0.012	0.971 $\pm$ 0.013
Synthetic 2C	2,500	0.925 $\pm$ 0.018	0.907 $\pm$ 0.023	0.910 $\pm$ 0.022	0.900 $\pm$ 0.024	0.927 $\pm$ 0.019	0.930 $\pm$ 0.018
Synthetic 3C	2,500	0.887 $\pm$ 0.022	0.865 $\pm$ 0.026	0.866 $\pm$ 0.025	0.857 $\pm$ 0.026	0.884 $\pm$ 0.026	0.887 $\pm$ 0.024

TABLE II: Mean Accuracy at Final Query Step (50%) (The best results in each row are bolded , and the second-best results are underlined ).

Dataset	Size	Random	Feature Space	LDA	Prob	Prob+Ent
Iris	150	37.468 $\pm$ 1.502	37.643 $\pm$ 1.507	37.352 $\pm$ 1.663	37.628 $\pm$ 1.578	37.637 $\pm$ 1.571
Wine	178	37.776 $\pm$ 1.273	37.679 $\pm$ 1.236	37.788 $\pm$ 1.254	38.193 $\pm$ 1.063	38.249 $\pm$ 0.994
Digits	1,797	37.597 $\pm$ 0.382	38.029 $\pm$ 0.317	38.053 $\pm$ 0.322	38.398 $\pm$ 0.254	38.462 $\pm$ 0.287
Synthetic 1C	2,500	759.608 $\pm$ 15.231	761.548 $\pm$ 14.128	758.682 $\pm$ 16.418	766.178 $\pm$ 13.350	766.352 $\pm$ 13.667
Synthetic 2C	2,500	708.258 $\pm$ 20.482	710.314 $\pm$ 20.896	701.114 $\pm$ 21.719	717.530 $\pm$ 18.973	718.698 $\pm$ 18.998
Synthetic 3C	2,500	670.652 $\pm$ 22.658	672.282 $\pm$ 21.561	661.988 $\pm$ 22.685	680.222 $\pm$ 21.924	680.862 $\pm$ 21.291

TABLE III: Area Under Curve (AUC) for 10%-50% Batches (The best results in each row are bolded , and the second-best results are underlined ).

Panel of Figure 5 — Fig. 2: Learning curves (100-400 samples) demonstrating the improvement in performance of using the Probability Space over raw Feature Space and LDA space in the Greedy $K$ -center algorithm.

Dataset	Size	Random	Feature Space	LDA	Prob	Prob+Ent
Digits	1,797	0.954 $\pm$ 0.009	0.965 $\pm$ 0.007	0.971 $\pm$ 0.005	0.981 $\pm$ 0.004	0.982 $\pm$ 0.004
Segment	2,310	0.940 $\pm$ 0.008	0.943 $\pm$ 0.008	0.945 $\pm$ 0.008	0.969 $\pm$ 0.004	0.969 $\pm$ 0.004
1 Cluster Syn	2,500	0.944 $\pm$ 0.022	0.948 $\pm$ 0.021	0.941 $\pm$ 0.025	0.968 $\pm$ 0.017	0.973 $\pm$ 0.014
2 Cluster Syn	2,500	0.875 $\pm$ 0.030	0.879 $\pm$ 0.030	0.856 $\pm$ 0.034	0.900 $\pm$ 0.027	0.909 $\pm$ 0.027
3 Cluster Syn	2,500	0.822 $\pm$ 0.033	0.832 $\pm$ 0.032	0.803 $\pm$ 0.036	0.844 $\pm$ 0.035	0.853 $\pm$ 0.033
4 Cluster Syn	2,500	0.792 $\pm$ 0.027	0.792 $\pm$ 0.030	0.768 $\pm$ 0.032	0.807 $\pm$ 0.031	0.813 $\pm$ 0.033

TABLE IV: Mean Accuracy at 400 Samples (The best results in each row are bolded , and the second-best results are underlined ).

Dataset	Size	Random	Feature Space	LDA	Prob	Prob+Ent
Digits	1,797	278.251 $\pm$ 3.268	279.356 $\pm$ 2.112	281.919 $\pm$ 2.162	286.673 $\pm$ 1.407	287.086 $\pm$ 1.400
Segment	2,310	277.127 $\pm$ 2.608	278.377 $\pm$ 2.534	278.982 $\pm$ 2.003	285.340 $\pm$ 1.372	285.158 $\pm$ 1.632
1 Cluster Syn	2,500	278.283 $\pm$ 7.539	279.770 $\pm$ 6.906	277.384 $\pm$ 8.824	284.644 $\pm$ 6.711	286.439 $\pm$ 6.102
2 Cluster Syn	2,500	254.503 $\pm$ 9.648	255.605 $\pm$ 9.808	248.800 $\pm$ 11.208	259.561 $\pm$ 9.396	261.957 $\pm$ 9.252
3 Cluster Syn	2,500	238.549 $\pm$ 10.123	240.371 $\pm$ 10.122	233.694 $\pm$ 10.800	242.694 $\pm$ 10.739	244.407 $\pm$ 10.680
4 Cluster Syn	2,500	227.725 $\pm$ 9.596	228.457 $\pm$ 9.671	222.446 $\pm$ 9.735	230.420 $\pm$ 9.664	232.262 $\pm$ 9.299

TABLE V: Area Under Curve (AUC) for 100-400 Samples (The best results in each row are bolded , and the second-best results are underlined ).

Fig. 3: The Madelon dataset, an exception to the Probability Space dominance in extreme data scarcity.

Active Learning

Active learning selects the examples, labels, features, or experiments most useful for improving a model under limited budgets.

See all 280 papers

May 4, 2026

Gradient-Discrepancy Acquisition for Pool-Based Active Learning

The effectiveness of active learning hinges on the choice of the acquisition criterion by which a learning algorithm selects potentially informative data points whose label is subsequently queried. This paper proposes a novel gradient-based acquisition criterion, derived from a generalization bound introduced by Luo et al. (2022). This criterion can be applied in lieu of uncertainty measures in uncertainty sampling, or incorporated into diversity-based methods that consider the spread of sampled points in addition to the uncertainty of their labels. We provide a theoretical justification of the proposed acquisition criterion, and demonstrate its effectiveness in an empirical evaluation.

Mohamadsadegh Khosravani, Sandra Zilles

Department of Computer Science University of Regina · University of Regina

Active Learning

Ask PDF

Jul 1, 2026

Group-invariant Coresets for Data-efficient Active Learning

Active learning reduces labeling cost by querying the most informative unlabeled samples, but standard coreset methods ignore known data symmetries and can waste budget on transformed versions of the same instance. We propose GRINCO, a group-invariant coreset framework that performs acquisition in the quotient space induced by a transformation group, so that selection operates on orbits rather than raw samples. The method uses either canonical representatives or learned orbit-separating invariant embeddings to define practical quotient metrics, and combines quotient-space k-center selection with invariant training through an orbit-averaged loss. We further derive a generalization bound that relates excess orbit-averaged risk to quotient-space coverage, label uncertainty, and intra-orbit variability. Experiments on synthetic scale-invariant data and image benchmarks with rotation-induced redundancy show that GRINCO improves orbit coverage and achieves stronger label efficiency than conventional coreset baselines, especially when group-induced redundancy is substantial.

L. C. Ayres, J. C. M. Bermudez, S. J. M. de Almeida +1

Universidade Federal de Santa Catarina, Florianópolis, SC, Brazil · Universidade Católica de Pelotas, Pelotas, RS, Brazil · Université de Lorraine, CNRS, CRAN, Vandoeuvre-lès-Nancy, France

Active Learning Coreset Selection Data-Efficient Learning

Ask PDF

Mar 11, 2026

Adaptive Active Learning for Regression via Reinforcement Learning

Active learning for regression reduces labeling costs by selecting the most informative samples. Improved Greedy Sampling is a prominent method that balances feature-space diversity and output-space uncertainty using a static, multiplicative rule. We propose Weighted improved Greedy Sampling (WiGS), which replaces this framework with a dynamic, additive criterion. We formulate weight selection as a reinforcement learning problem, enabling an agent to adapt the exploration-investigation balance throughout learning. Experiments on 18 benchmark datasets and a synthetic environment show WiGS outperforms iGS and other baseline methods in both accuracy and labeling efficiency, particularly in domains with irregular data density where the baseline's multiplicative rule ignores high-error samples in dense regions.

Simon D. Nguyen, Troy Russo, Kentaro Hoffman +1

Department of Statistics, University of Washington · Department of Sociology, University of Washington

Active Learning Adaptive Sampling

Ask PDF

Abstract

Active Learning

Explore similar work

Gradient-Discrepancy Acquisition for Pool-Based Active Learning

Group-invariant Coresets for Data-efficient Active Learning

Adaptive Active Learning for Regression via Reinforcement Learning