Revisiting Unknowns: Towards Effective and Efficient Open-Set Active Learning

Chen-Chen Zong; Yu-Qi Chi; Xie-Yang Wang; Yan Cui; Sheng-Jun Huang

cs.CVMar 9, 2026

Revisiting Unknowns: Towards Effective and Efficient Open-Set Active Learning

Authors: Chen-Chen Zong, Yu-Qi Chi, Xie-Yang Wang, Yan Cui, Sheng-Jun Huang

Organizations: Nanjing University of Aeronautics and Astronautics, Nanjing, 211106, China

Abstract

Open-set active learning (OSAL) aims to identify informative samples for annotation when unlabeled data may contain previously unseen classes-a common challenge in safety-critical and open-world scenarios. Existing approaches typically rely on separately trained open-set detectors, introducing substantial training overhead and overlooking the supervisory value of labeled unknowns for improving known-class learning. In this paper, we propose E $^2$ OAL (Effective and Efficient Open-set Active Learning), a unified and detector-free framework that fully exploits labeled unknowns for both stronger supervision and more reliable querying. E $^2$ OAL first uncovers the latent class structure of unknowns through label-guided clustering in a frozen contrastively pre-trained feature space, optimized by a structure-aware F1-product objective. To leverage labeled unknowns, it employs a Dirichlet-calibrated auxiliary head that jointly models known and unknown categories, improving both confidence calibration and known-class discrimination. Building on this, a logit-margin purity score estimates the likelihood of known classes to construct a high-purity candidate pool, while an OSAL-specific informativeness metric prioritizes partially ambiguous yet reliable samples. These components together form a flexible two-stage query strategy with adaptive precision control and minimal hyperparameter sensitivity. Extensive experiments across multiple OSAL benchmarks demonstrate that E $^2$ OAL consistently surpasses state-of-the-art methods in accuracy, efficiency, and query precision, highlighting its effectiveness and practicality for real-world applications. The code is available at github.com/chenchenzong/E2OAL.

Figures & tables

Figure 1: Per-round and mean test accuracy on CIFAR-100 (40 known / 60 unknown) using ResNet-18.

C_{known}

excludes labeled unknowns ,

C_{known{+}1}

collapses them into a single “unknown” class, and

C_{all}

leverages their true labels.

C_{all}

consistently performs best, suggesting that preserving the latent structure of unknown classes benefits known-class learning.

Figure 2: Overview of the proposed E 2 OAL framework. Each AL round consists of two stages: (1) Adaptive class estimation and calibration-aware training, where latent unknown classes are discovered via label-guided clustering and incorporated into model learning through Dirichlet-based auxiliary supervision; (2) Flexible two-stage query selection, where a high-purity candidate pool is first constructed using a purity score guided by a target query precision, followed by informativeness-driven sample selection.

Figure 3: Test accuracy across AL rounds under varying mismatch ratios on CIFAR-10/100 and Tiny-ImageNet.

Figure 4: Mean query precision vs. mean test accuracy across rounds under varying mismatch ratios on CIFAR-10/100 and Tiny-ImageNet.

Method	CIFAR-10	CIFAR-100	Tiny-ImageNet
Ours*	95.94	67.54	60.44
EAOA	95.88	67.14	57.31
BUAL	95.04	63.73	56.09
EOAL	93.64	63.69	56.13

Table 1: Final-round test accuracy (%) under fixed mismatch ratios (30% for CIFAR-10/100 and 15% for Tiny-ImageNet). “Ours*” denotes a variant trained without leveraging labeled unknowns .

Method	CIFAR-10	CIFAR-100	Tiny-ImageNet
$\mathcal{L}_{\mathrm{EDL}}$	9495	7934	5844
$\mathcal{L}_{\mathrm{CE}}$	9394	7814	5661

Table 2: Purity comparison under the most challenging mismatch ratios: 20% for CIFAR-10/100 and 10% for Tiny-ImageNet. Sampling by Eq. ( 6 ) with CE and EDL losses on the auxiliary head, evaluated by the total number of queried known-class samples.

Method	CIFAR-10	CIFAR-100	Tiny-ImageNet
Ours	95.95	65.73	48.63
EAOA	94.90	61.95	44.73
Uncertainty	95.37	61.28	44.73

Table 3: Informativeness comparison under the most challenging mismatch ratios: 20% for CIFAR-10/100 and 10% for Tiny-ImageNet. Final test accuracy (%) when retaining only the informativeness component from “Ours”, “EAOA”, and “Uncertainty”.

Method	CIFAR-10	CIFAR-100	Tiny-ImageNet
Ours	97.52	72.10	64.02
w/o ClassExp	97.17	70.73	62.67
$S_{\text{purity}}$	96.73	72.00	61.93
$S_{\text{info}}$	96.00	68.20	57.60

Table 4: Final-round test accuracy (%) of different ablation variants under intermediate mismatch ratios on CIFAR-10/100 and Tiny-ImageNet. “

S_{\text{info}}

” and “

S_{\text{purity}}

” denote sampling based solely on informativeness or purity. “w/o ClassExp” disables class expansion by collapsing all labeled unknowns into a single class.

Mismatch ratio	$p^{*}=0.4$	$0.5$	$0.6$	$0.7$
$20\%$	81.47	81.67	82.20	82.18
$30\%$	71.90	72.96	72.10	73.17
$40\%$	66.68	68.03	67.98	66.78

Table 5: Final-round test accuracy (%) on CIFAR-100 under varying target query precisions and mismatch ratios.

Figure 5: Total training time (hours) on CIFAR-100 under a 40% mismatch ratio. Bars indicate actual training time with the average query precision annotated; the dashed line shows the approximate projection assuming a linear relationship between query precision and time, aligned to the precision level of “Uncertainty”.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 11 — Figure C1 : Per-round and mean test accuracy on CIFAR-100 under varying class splits and network architectures. $C_{known}$ excludes labeled unknowns , $C_{known{+}1}$ collapses them into a single class, and $C_{all}$ leverages their ground-truth labels.

Figure D1 : Test accuracy across rounds under mismatch ratio 30% on CIFAR-10/100 and 15% on Tiny-ImageNet.

Figure E1 : Mean query precision and test accuracy under mismatch ratio 30% on CIFAR-10/100 and 15% on Tiny-ImageNet.

Figure E2 : Cross-round query precision under varying mismatch ratios on three benchmarks. From top to bottom: results on CIFAR-10, CIFAR-100, and Tiny-ImageNet; from left to right: mismatch ratios of 20%, 30%, and 40%, respectively.

Dataset	CIFAR-10			CIFAR-100			Tiny-ImageNet
Mismatch ratio	20%	30%	40%	20%	30%	40%	10%	15%	20%
Random	94.48	91.11	87.18	64.45	58.48	55.48	47.93	46.00	43.32
Uncertainty [ 18 ]	95.70	89.77	83.61	62.25	53.52	50.83	45.83	43.40	35.43
Coreset [ 28 ]	94.20	89.56	86.38	63.53	56.62	55.00	50.60	47.33	45.35
BADGE [ 2 ]	94.95	90.91	87.12	64.00	56.49	50.20	49.70	48.16	46.23
MSP [ 11 ]	94.15	91.51	87.21	65.33	58.69	56.68	51.43	47.78	46.57

Appendix

Table F1 : Final-round test accuracy (%) of all methods under varying mismatch ratios on CIFAR-10, CIFAR-100, and Tiny-ImageNet. “Ours*” denotes a variant of our method where the target classifier is trained independently without leveraging labeled unknowns . The best result in each setting is highlighted in bold, while the second best is underlined. Due to the poor performance and high training cost of MQNet, we do not include it on Tiny-ImageNet, and thus mark it with “–”.

Feature source	CIFAR-10	CIFAR-100	Tiny-ImageNet
CLIP	97.52	72.10	64.02
MoCo	97.44	72.31	63.87

Appendix

Table G1: Final-round test accuracy (%) under fixed mismatch ratios (30% for CIFAR-10/100 and 15% for Tiny-ImageNet). “CLIP” and “MoCo” respectively refer to adaptive class estimation performed using features extracted from a CLIP model and from a self-supervised MoCo pretrained model.

Figure H1 : Ablation results for unknown class estimation on CIFAR-100 under mismatch ratios of 20%, 30%, and 40% (from left to right). Bar charts (bottom x-axis and left y-axis) show the total number of samples labeled per class in the final round. Green bars represent known classes, and blue bars represent unknown classes. Line plots (top x-axis and right y-axis) illustrate the evolution of the estimated total number of classes (

k+\hat{u}

), where the ground-truth total is 100.

Active Learning

Active learning selects the examples, labels, features, or experiments most useful for improving a model under limited budgets.

See all 288 papers

Energy-Based Open-Set Active Learning for Object Classification
Apr 22, 2026Zongyao Lyu, William J. BeksiActive Learning Open-Set Recognition
The University of Texas at Arlington, Arlington TX 76019, USA
Active Learning with Imperfect Labels: Optimal Labeler Assignment and Sample Selection
Dec 14, 2025Pouya Ahadi, Blair Winograd, Camille Zaug +3Human-in-the-Loop Annotation Active Learning
School of Industrial and Systems Engineering Georgia Institute of Technology Atlanta, GA 30332, USA · Ford Motor Company
SmoothOperator: Enhancing Representations for Fine-grained Open-set Recognition via Modulated Label Smoothing
Oct 1, 2026Thiru Thillai Nadarasar Bahavan, Yu Xia, Sachith Seneviratne +1Soft-Label Learning Open-Set Recognition
The University of Melbourne Parkville, VIC, Australia

Apr 22, 2026J/K move · Enter open · S save

Energy-Based Open-Set Active Learning for Object Classification

Zongyao Lyu, William J. Beksi

The University of Texas at Arlington, Arlington TX 76019, USA

Active Learning Open-Set Recognition Energy-Based Models Image Classification

Open paper AskPDF arXiv

Active learning (AL) has emerged as a crucial methodology for minimizing labeling costs in deep learning by selecting the most valuable samples from a pool of unlabeled data for annotation. Traditional AL operates under a closed-set assumption, where all classes in the dataset are known and consistent. However, real-world scenarios often present open-set conditions in which unlabeled data contains both known and unknown classes. In such environments, standard AL techniques struggle. They can mistakenly query samples from unknown categories, leading to inefficient use of annotation budgets. In this paper, we propose a novel dual-stage energy-based framework for open-set AL. Our method employs two specialized energy-based models (EBMs). The first, an energy-based known/unknown separator, filters out samples likely to belong to unknown classes. The second, an energy-based sample scorer, assesses the informativeness of the filtered known samples. Using the energy landscape, our models distinguish between data points from known and unknown classes in the unlabeled pool by assigning lower energy to known samples and higher energy to unknown samples, ensuring that only samples from classes of interest are selected for labeling. By integrating these components, our approach ensures efficient and targeted sample selection, maximizing learning impact in each iteration. Experiments on 2D (CIFAR-10, CIFAR-100, TinyImageNet) and 3D (ModelNet40) object classification benchmarks demonstrates that our framework outperforms existing approaches, achieving superior annotation efficiency and classification performance in open-set environments.

Abstract

Active Learning

Explore similar work

Energy-Based Open-Set Active Learning for Object Classification

Active Learning with Imperfect Labels: Optimal Labeler Assignment and Sample Selection

SmoothOperator: Enhancing Representations for Fine-grained Open-set Recognition via Modulated Label Smoothing