In the classification context where revealing the label of a sample is costly, we propose a new Active Learning (AL) strategy named k-Volume (kVol). It randomly selects a sample to be labeled according to the volume of the smallest ball containing its k nearest neighbors. kVol is highly versatile: it can be applied either in the input space or in the output space, and it enjoys the desirable properties of the three main families of AL strategies: density, diversity, and (when performed in the output space) uncertainty. From a theoretical perspective, we establish that kVol is asymptotically equivalent to uniform sampling over the support of the (input or output) data distribution. Based on experiments on various synthetic and real-life tabular datasets and images, we observe that kVol is surprisingly stable: whatever the classifier or the experimental setup, it still lies among the best AL strategies unlike other competitors that fail dramatically. In particular, kVol outperforms state-of-the-art competitors such as CoreSet, TypiClust or Uncertainty on several real-life datasets, at a low computational cost.
Figures & tables
Figure 1: Illustration of \kvolk with k=4 .
Scenario
Balanced
Dimension
No. 1
Yes
2
No. 2
No
2
No. 3
Yes
20
No. 4
No
20
Table 1: Parameters for the numerical experiments on synthetic data.
Figure 2: Visualization of the synthetic training set in the 2D scenarios.
Figure 3: Accuracy comparison during active training. Initial batch size b0=50 , step batch size b=10 and budget B=500 . The dimension of the data space is 2.
Dataset Id
Name
Number of classes
Number of points
Input dimension
46
splice
3
3190
60
182
satimage
6
6430
36
846
elevators
2
16599
18
Table 2: Characteristics of the tabular datasets used in Section 4.2 .
Figure 4: Accuracy comparison during active training. Initial batch size b0=10 , step batch size b=10 and budget B=500 . The dataset id is 182.
Name
Number of classes
Number of points
Image dimension
Embedding dimension
Reference
MNIST
10
60000
784
2048
[ 24 ]
CIFAR10
10
50000
3072
2048
[ 23 ]
CIFAR100
100
50000
3072
2048
[ 23 ]
Table 3: Characteristics of the datasets of images used in Section 4.3 .
Figure 5: Accuracy comparison during active training. Initial batch size b0=10 , step batch size b=50 and budget B=760 .
Figure 6: CIFAR100. Accuracy comparison during active training. Initial batch size b0=500 , step batch size b=500 and budget B=5000 .
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Difficult case for the uncertainty family. The red line represents the decision boundary of the trained classifier N , the colors represent the probabilities of each class, and the shape of the points represent the true labels.
Figure 8: Accuracy comparison during active training. Initial batch size b0=50 , step batch size b=10 and budget B=500 . The dimension of the data space is 20.
Figure 9: Accuracy comparison during active training. Initial batch size b0=10 , step batch size b=10 and budget B=500 .
Figure 10: Accuracy comparison during active training. Initial batch size b0=100 , step batch size b=10 and budget B=500 .
Figure 11: Accuracy comparison during active training. Initial batch size b0=10 , step batch size b=10 and budget B=500 . The dataset id is 182.
Figure 12: Accuracy comparison during active training. Initial batch size b0=10 , step batch size b=10 and budget B=500 . The dataset id is 46.
Figure 13: Accuracy comparison during active training. Initial batch size b0=10 , step batch size b=10 and budget B=500 . The dataset id is 846.
Figure 14: Accuracy comparison during active training. Initial batch size b0=500 , step batch size b=500 and budget B=10000 .
Figure 15: Accuracy comparison during active training for different values of k . Initial batch size b0=10 , step batch size b=50 and budget B=760 . The dataset is CIFAR10.
Uniform
Entropy
CoreSet
TypiClust
kVolOut ( k=4 )
kVolIn ( k=4 )
Figure 6
1.70e+03s
1.71e+03s
3.06e+03s
1.07e+04s
1.73e+03s
1.90e+03s
Figure 14
6.54e+03s
6.54e+03s
-
-
6.58e+03s
6.87e+03s
kVolOut1
kVolOut10
kVolOut100
kVolOut500
Figure 15
1.14e+02s
1.37e+02s
2.01e+02s
3.43e+02s
Appendix
Table 4: Average computing time per seed (over 10 repetitions) for each strategy (in seconds).
With rapid advancement over the last few years, many different methods are now widely used for classification. However, training these models requires substantial labeled data. Active Learning is a potential solution to this problem. Pool-based active learning minimizes costs by querying only the most informative samples from an unlabeled dataset. Diversity-based approaches, on the other hand, attempt to select a representative subset of the data. There are many different objectives for determining the selection process, including exact K-center, exact K-median, and Greedy K-center. In this paper, we will focus on evaluating the performance of Greedy K-center across a variety of metric spaces: the raw feature space, a Linear Discriminant Analysis (LDA) space, and a model-derived probability space (with and without entropy-based weighting). Using Random Forest classifiers as a baseline evaluator, our empirical results on synthetic and real-world datasets demonstrate that mapping unlabeled instances into a predictive probability space and weighting the result by entropy often dominates the other options for active learning selection with Greedy K-center.
Active Learning (AL) is commonly used in applications where labeling data is expensive or time-consuming. In practice, however, labels are often noisy due to varying labeler expertise and annotation uncertainty, especially for complex or ambiguous samples. Learning from such imperfectly labeled data can degrade classifier performance. We propose an AL framework that explicitly accounts for label noise by optimally assigning labelers and selecting samples to minimize labeling error. Our approach, called OLAS (Optimal Labeler Assignment and Sampling), uses a noise model that depends on both labeler accuracy and model uncertainty to guide these decisions. We develop two tractable optimization formulations: one for assigning samples to labelers to minimize worst-case noise, and another for selecting samples while controlling overall label noise. Theoretical results provide closed-form solutions under mild conditions. Empirical evaluations on benchmark datasets and a real-world warranty claim classification problem show that OLAS achieves the highest or near-highest classification accuracy among existing AL strategies across most settings, using only a single label per sample.
Pouya Ahadi, Blair Winograd, Camille Zaug +3
School of Industrial and Systems Engineering Georgia Institute of Technology Atlanta, GA 30332, USA · Ford Motor Company
Knowledge distillation (KD) is a widely used framework for training compact, task-specific models by transferring the knowledge from teacher models. However, its application to active learning (AL), which aims to minimize annotation costs through iterative sample selection, remains underexplored. This gap stems from the fact that KD typically assumes access to sufficient labeled data, whereas AL operates in data-scarce scenarios where task-specific teacher models are often unavailable. In this paper, we first introduce ActiveKD, a framework that integrates AL with KD by leveraging the zero- and few-shot capabilities of large vision-language models (VLMs). A key aspect of ActiveKD is the structured prediction bias of VLMs--i.e., their predictions form clusters in the probability space. We regard this structure as an inductive bias of the teacher model, capturing generalizable output patterns beneficial to student learning. To exploit this bias, we propose Probabilistic CoreSet (PCoreSet), a selection strategy that maximizes coverage in the probability space rather than the feature space. PCoreSet strategically selects probabilistically diverse unlabeled samples, facilitating more efficient transfer of teacher knowledge under limited annotation budgets. Extensive evaluations on 11 datasets show that ActiveKD consistently improves every selection method over the no-distillation baseline (e.g., +29.07% on ImageNet, averaged over methods). Under ActiveKD, several widely-used AL selectors underperform Random and none consistently beats it; in contrast, PCoreSet is the only selector that consistently outperforms Random across 5 students and 3 teachers, always achieving the best performance except for the first 2 AL rounds. Our code is available at https://github.com/erjui/PCoreSet.