Diversity-Based Active Learning: An Evaluation of Metric Spaces for Active Learning Selection

Siddharth Chilamkur; Dorit S. Hochbaum

cs.LGAug 24, 2026

Diversity-Based Active Learning: An Evaluation of Metric Spaces for Active Learning Selection

Authors: Siddharth Chilamkur, Dorit S. Hochbaum

Organizations: University of California, Berkeley

Abstract

With rapid advancement over the last few years, many different methods are now widely used for classification. However, training these models requires substantial labeled data. Active Learning is a potential solution to this problem. Pool-based active learning minimizes costs by querying only the most informative samples from an unlabeled dataset. Diversity-based approaches, on the other hand, attempt to select a representative subset of the data. There are many different objectives for determining the selection process, including exact K-center, exact K-median, and Greedy K-center. In this paper, we will focus on evaluating the performance of Greedy K-center across a variety of metric spaces: the raw feature space, a Linear Discriminant Analysis (LDA) space, and a model-derived probability space (with and without entropy-based weighting). Using Random Forest classifiers as a baseline evaluator, our empirical results on synthetic and real-world datasets demonstrate that mapping unlabeled instances into a predictive probability space and weighting the result by entropy often dominates the other options for active learning selection with Greedy K-center.

Figures & tables

Dataset	Size	RF Time (s)	NN Time (s)
Iris	150	0.1617	0.2926
Wine	178	0.1657	0.6458
Spambase	4,601	0.2362	2.6099
Phoneme	5,404	0.2459	8.5601
Satimage	6,435	0.2832	9.0500

TABLE I: Fully Supervised Baselines: Runtime Comparison for RF vs. NN (L4 GPU)

Panel of Figure 2 — Fig. 1: Learning curves for large percentage batches (10%-50%). The charts demonstrate the performance advantage of mapping queries into the Probability Space, even as the complexity of the dataset increases (e.g. Synthetic 2C, Synthetic 3C).

Dataset	Size	Baseline	Random	Feature Space	LDA	Prob	Prob+Ent
Iris	150	0.951 $\pm$ 0.037	0.948 $\pm$ 0.043	0.947 $\pm$ 0.038	0.942 $\pm$ 0.045	0.951 $\pm$ 0.038	0.949 $\pm$ 0.039
Wine	178	0.981 $\pm$ 0.022	0.969 $\pm$ 0.032	0.966 $\pm$ 0.031	0.972 $\pm$ 0.024	0.980 $\pm$ 0.023	0.981 $\pm$ 0.022
Digits	1,797	0.973 $\pm$ 0.008	0.961 $\pm$ 0.010	0.974 $\pm$ 0.007	0.974 $\pm$ 0.008	0.978 $\pm$ 0.007	0.977 $\pm$ 0.007
Synthetic 1C	2,500	0.969 $\pm$ 0.014	0.961 $\pm$ 0.016	0.963 $\pm$ 0.015	0.960 $\pm$ 0.018	0.972 $\pm$ 0.012	0.971 $\pm$ 0.013
Synthetic 2C	2,500	0.925 $\pm$ 0.018	0.907 $\pm$ 0.023	0.910 $\pm$ 0.022	0.900 $\pm$ 0.024	0.927 $\pm$ 0.019	0.930 $\pm$ 0.018
Synthetic 3C	2,500	0.887 $\pm$ 0.022	0.865 $\pm$ 0.026	0.866 $\pm$ 0.025	0.857 $\pm$ 0.026	0.884 $\pm$ 0.026	0.887 $\pm$ 0.024

TABLE II: Mean Accuracy at Final Query Step (50%) (The best results in each row are bolded , and the second-best results are underlined ).

Dataset	Size	Random	Feature Space	LDA	Prob	Prob+Ent
Iris	150	37.468 $\pm$ 1.502	37.643 $\pm$ 1.507	37.352 $\pm$ 1.663	37.628 $\pm$ 1.578	37.637 $\pm$ 1.571
Wine	178	37.776 $\pm$ 1.273	37.679 $\pm$ 1.236	37.788 $\pm$ 1.254	38.193 $\pm$ 1.063	38.249 $\pm$ 0.994
Digits	1,797	37.597 $\pm$ 0.382	38.029 $\pm$ 0.317	38.053 $\pm$ 0.322	38.398 $\pm$ 0.254	38.462 $\pm$ 0.287
Synthetic 1C	2,500	759.608 $\pm$ 15.231	761.548 $\pm$ 14.128	758.682 $\pm$ 16.418	766.178 $\pm$ 13.350	766.352 $\pm$ 13.667
Synthetic 2C	2,500	708.258 $\pm$ 20.482	710.314 $\pm$ 20.896	701.114 $\pm$ 21.719	717.530 $\pm$ 18.973	718.698 $\pm$ 18.998
Synthetic 3C	2,500	670.652 $\pm$ 22.658	672.282 $\pm$ 21.561	661.988 $\pm$ 22.685	680.222 $\pm$ 21.924	680.862 $\pm$ 21.291

TABLE III: Area Under Curve (AUC) for 10%-50% Batches (The best results in each row are bolded , and the second-best results are underlined ).

Panel of Figure 5 — Fig. 2: Learning curves (100-400 samples) demonstrating the improvement in performance of using the Probability Space over raw Feature Space and LDA space in the Greedy $K$ -center algorithm.

Dataset	Size	Random	Feature Space	LDA	Prob	Prob+Ent
Digits	1,797	0.954 $\pm$ 0.009	0.965 $\pm$ 0.007	0.971 $\pm$ 0.005	0.981 $\pm$ 0.004	0.982 $\pm$ 0.004
Segment	2,310	0.940 $\pm$ 0.008	0.943 $\pm$ 0.008	0.945 $\pm$ 0.008	0.969 $\pm$ 0.004	0.969 $\pm$ 0.004
1 Cluster Syn	2,500	0.944 $\pm$ 0.022	0.948 $\pm$ 0.021	0.941 $\pm$ 0.025	0.968 $\pm$ 0.017	0.973 $\pm$ 0.014
2 Cluster Syn	2,500	0.875 $\pm$ 0.030	0.879 $\pm$ 0.030	0.856 $\pm$ 0.034	0.900 $\pm$ 0.027	0.909 $\pm$ 0.027
3 Cluster Syn	2,500	0.822 $\pm$ 0.033	0.832 $\pm$ 0.032	0.803 $\pm$ 0.036	0.844 $\pm$ 0.035	0.853 $\pm$ 0.033
4 Cluster Syn	2,500	0.792 $\pm$ 0.027	0.792 $\pm$ 0.030	0.768 $\pm$ 0.032	0.807 $\pm$ 0.031	0.813 $\pm$ 0.033

TABLE IV: Mean Accuracy at 400 Samples (The best results in each row are bolded , and the second-best results are underlined ).

Dataset	Size	Random	Feature Space	LDA	Prob	Prob+Ent
Digits	1,797	278.251 $\pm$ 3.268	279.356 $\pm$ 2.112	281.919 $\pm$ 2.162	286.673 $\pm$ 1.407	287.086 $\pm$ 1.400
Segment	2,310	277.127 $\pm$ 2.608	278.377 $\pm$ 2.534	278.982 $\pm$ 2.003	285.340 $\pm$ 1.372	285.158 $\pm$ 1.632
1 Cluster Syn	2,500	278.283 $\pm$ 7.539	279.770 $\pm$ 6.906	277.384 $\pm$ 8.824	284.644 $\pm$ 6.711	286.439 $\pm$ 6.102
2 Cluster Syn	2,500	254.503 $\pm$ 9.648	255.605 $\pm$ 9.808	248.800 $\pm$ 11.208	259.561 $\pm$ 9.396	261.957 $\pm$ 9.252
3 Cluster Syn	2,500	238.549 $\pm$ 10.123	240.371 $\pm$ 10.122	233.694 $\pm$ 10.800	242.694 $\pm$ 10.739	244.407 $\pm$ 10.680
4 Cluster Syn	2,500	227.725 $\pm$ 9.596	228.457 $\pm$ 9.671	222.446 $\pm$ 9.735	230.420 $\pm$ 9.664	232.262 $\pm$ 9.299

TABLE V: Area Under Curve (AUC) for 100-400 Samples (The best results in each row are bolded , and the second-best results are underlined ).

Fig. 3: The Madelon dataset, an exception to the Probability Space dominance in extreme data scarcity.

Active Learning

Active learning selects the examples, labels, features, or experiments most useful for improving a model under limited budgets.

See all 280 papers

Gradient-Discrepancy Acquisition for Pool-Based Active Learning
May 4, 2026Mohamadsadegh Khosravani, Sandra ZillesActive Learning
Department of Computer Science University of Regina · University of Regina
Group-invariant Coresets for Data-efficient Active Learning
Jul 1, 2026L. C. Ayres, J. C. M. Bermudez, S. J. M. de Almeida +1Active Learning Coreset Selection
Universidade Federal de Santa Catarina, Florianópolis, SC, Brazil · Universidade Católica de Pelotas, Pelotas, RS, Brazil · Université de Lorraine, CNRS, CRAN, Vandoeuvre-lès-Nancy, France
Adaptive Active Learning for Regression via Reinforcement Learning
Mar 11, 2026Simon D. Nguyen, Troy Russo, Kentaro Hoffman +1Active Learning Adaptive Sampling
Department of Statistics, University of Washington · Department of Sociology, University of Washington

May 4, 2026J/K move · Enter open · S save

Gradient-Discrepancy Acquisition for Pool-Based Active Learning

Mohamadsadegh Khosravani, Sandra Zilles

Department of Computer Science University of Regina · University of Regina

Active Learning

Open paper AskPDF arXiv

The effectiveness of active learning hinges on the choice of the acquisition criterion by which a learning algorithm selects potentially informative data points whose label is subsequently queried. This paper proposes a novel gradient-based acquisition criterion, derived from a generalization bound introduced by Luo et al. (2022). This criterion can be applied in lieu of uncertainty measures in uncertainty sampling, or incorporated into diversity-based methods that consider the spread of sampled points in addition to the uncertainty of their labels. We provide a theoretical justification of the proposed acquisition criterion, and demonstrate its effectiveness in an empirical evaluation.

Abstract

Active Learning

Explore similar work

Gradient-Discrepancy Acquisition for Pool-Based Active Learning

Group-invariant Coresets for Data-efficient Active Learning

Adaptive Active Learning for Regression via Reinforcement Learning