Learning through Creation: A Hash-Free Framework for On-the-Fly Category Discovery

Bohan Zhang; Weidong Tang; Zhixiang Chi; Yi Jin; Zhenbo Li; Yang Wang; Yanan Wu

cs.CVMar 14, 2026

Learning through Creation: A Hash-Free Framework for On-the-Fly Category Discovery

Authors: Bohan Zhang, Weidong Tang, Zhixiang Chi, Yi Jin, Zhenbo Li, Yang Wang, Yanan Wu

Organizations: College of Information and Electrical Engineering, China Agricultural University, China · Department of Electrical and Computer Engineering, University of Toronto, Canada · School of Computer and Information Technology, Beijing Jiaotong University, China · Department of Computer Science and Software Engineering, Concordia University, Canada

Abstract

On-the-Fly Category Discovery (OCD) aims to recognize known classes while simultaneously discovering emerging novel categories during inference, using supervision only from known classes during offline training. Existing approaches rely either on fixed label supervision or on diffusion-based augmentations to enhance the backbone, yet none of them explicitly train the model to perform the discovery task required at test time. It is fundamentally unreasonable to expect a model optimized on limited labeled data to carry out a qualitatively different discovery objective during inference. This mismatch creates a clear optimization misalignment between the offline learning stage and the online discovery stage. In addition, prior methods often depend on hash-based encodings or severe feature compression, which further limits representational capacity. To address these issues, we propose Learning through Creation (LTC), a fully feature-based and hash-free framework that injects novel-category awareness directly into offline learning. At its core is a lightweight, online pseudo-unknown generator driven by kernel-energy minimization and entropy maximization (MKEE). Unlike previous methods that generate synthetic samples once before training, our generator evolves jointly with the model dynamics and synthesizes pseudo-novel instances on the fly at negligible cost. These samples are incorporated through a dual max-margin objective with adaptive thresholding, strengthening the model's ability to delineate and detect unknown regions through explicit creation. Extensive experiments across seven benchmarks show that LTC consistently outperforms prior work, achieving improvements ranging from 1.5 percent to 13.1 percent in all-class accuracy. The code is available at https://github.com/brandinzhang/LTC

Figures & tables

Figure 1: Comparison of category discovery settings. Unlike NCD/GCD, which rely on a pre-defined query set during training, OCD enables new category discovery directly from streaming data without prior access to test samples.

Figure 2: Overview of the proposed LTC framework. During training, we extract feature representations from known-class samples and generate pseudo-unknowns via mixup and MKEE perturbation. A dual max-margin loss enforces separation between known and unknown regions, guided by an adaptive threshold. At inference, dynamic prototype matching enables online discovery of novel categories.

Panel of Figure 3 — Figure 3: MKEE generates semantically meaningful pseudo-unknowns. (a) Visualization of the feature space on the Oxford Pets dataset, illustrating known-class prototypes and the perturbation trajectories from $x_{\text{mix}}$ to $x_{\text{pus}}$ . (b) t-SNE visualization on the CIFAR-10 dataset, where five colored clusters correspond to the five known classes (1–5), and pseudo-unknowns (black triangles) clearly diverge from all known-class regions.

	Method	CIFAR-10 (%)			CIFAR-100 (%)			ImageNet-100 (%)			CUB-200-2011 (%)			Stanford Cars (%)			Oxford Pets (%)			Food101 (%)
	Method	All	Old	New	All	Old	New	All	Old	New	All	Old	New	All	Old	New	All	Old	New	All	Old	New
Greedy–Hungarian	SLC [ 15 ]	65.9	96.5	50.9	46.9	62.1	16.6	34.2	86.6	7.1	31.3	48.5	22.7	24.0	45.8	13.6	–	–	–	–	–	–
	MLDG [ 26 ]	71.6	97.5	58.6	58.4	69.0	37.3	33.6	74.4	13.1	34.2	57.9	22.4	28.0	49.1	17.7	–	–	–	–	–	–
	RankStat [ 13 ]	56.5	81.1	44.2	36.9	45.7	19.3	33.1	74.2	12.4	27.6	46.2	18.3	18.6	36.9	9.7	–	–	–	–	–	–
	WTA [ 21 ]	65.4	88.0	54.1	44.1	55.5	21.2	33.1	75.8	11.7	26.5	45.0	17.3	20.0	38.8	10.6	–	–	–	–	–	–
	SMILE [ 11 ]	78.2	99.3	67.6	61.3	70.7	42.5	39.9	87.1	16.2	41.1	67.6	27.8	33.4	58.4	21.3	54.1	66.1	47.8	34.4	64.0	19.4

Table 1: Comparison with other SOTA methods. “DiffGRE-P” and “DiffGRE-S” represent the integration of DiffGRE with SMILE and PHE, respectively.

Method	CUB ( $C{=}200$ )				SCars ( $C{=}196$ )
Method	#Cls	All	Old	New	#Cls	All	Old	New
SMILE-16bit [ 11 ]	924	31.9	52.7	21.5	896	27.5	52.5	15.4
SMILE-32bit [ 11 ]	2146	27.3	52.0	15.0	2953	21.9	46.8	9.9
PHE-16bit [ 63 ]	318	37.6	57.4	27.6	709	31.8	65.4	15.6
PHE-32bit [ 63 ]	474	38.5	59.9	27.8	762	31.5	64.0	15.8
LTC (ours)	210	57.8	83.9	44.8	389	56.6	90.3	40.4

Table 2: Comparison of estimated category numbers and accuracies on the CUB-200-2011 and Stanford Cars datasets.

Method	Pets (%)			SCars (%)
Method	All	Old	New	All	Old	New
$w/o$ $\mathcal{L}_{\text{ce}}$	71.2	94.0	59.2	55.4	90.2	38.6
$w/o$ $\mathcal{L}_{\text{sup}}$	67.6	89.5	56.2	51.9	84.4	36.2
$w/o$ $\mathcal{L}_{\text{mm}}$	68.5	94.4	54.9	55.8	90.1	39.3
$w/o$ MKEE	68.8	95.4	54.8	55.2	87.7	39.4
LTC (ours)	73.0	92.6	62.7	56.6	90.3	40.4

Table 3: Ablation study of key components in LTC. Results on Oxford Pets and Stanford Cars under the Greedy–Hungarian protocol.

\mathcal{L}_{\text{ce}}

\mathcal{L}_{\text{sup}}

, and

\mathcal{L}_{\text{mm}}

correspond to the cross-entropy, supervised contrastive, and dual max-margin losses, respectively, while “

w/o

MKEE” removes the pseudo-unknown generation module.

Method	All	Old	New	Time $\downarrow$
Mixup [ 59 ]	71.5	96.2	58.5	$<$ 1s
CutMix [ 57 ]	69.4	94.2	56.3	$<$ 1s
DiffGRE [ 29 ]	69.9	94.6	57.0	283.96s
MKEE w/o Entropy	72.3	95.8	59.9	$<$ 1s
MKEE w/o KDE	72.2	94.2	60.6	$<$ 1s
MKEE (ours)	73.0	92.6	62.7	$<$ 1s

Table 4: Comparison of different pseudo-unknown generation methods. Results are reported on the Pets dataset under the Greedy–Hungarian protocol. Synthesis time denotes the cost of generating 128 samples. MKEE achieves the highest all and novel-class accuracy with negligible computational overhead (

<1

s).

Panel of Figure 4 — Figure 4: Hyperparameter sensitivity on the CUB and SCars datasets. $\alpha$ and $\gamma_{mm}$ control the relative contributions of the contrastive and margin-based terms in the total loss $\mathcal{L}_{\text{total}}$ .

Panel of Figure 5 — Figure 5: Adaptive thresholding improves stability under different initializations. We study the effect of varying the initial similarity threshold $\tau_{\text{init}}$ and the update rate $\beta$ on overall accuracy. Results on CUB and SCars show that adaptive updates ( $\beta>0$ ) maintain stable performance across a wide range of $\tau_{\text{init}}$ values, while fixed thresholds ( $\beta=0$ ) are more sensitive to initialization.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

	Method	Backbone	CIFAR10 (%)			CIFAR100 (%)			ImageNet-100 (%)			CUB-200-2011 (%)			Stanford Cars (%)			Oxford Pets (%)			Food101 (%)
			All	Old	New	All	Old	New	All	Old	New	All	Old	New	All	Old	New	All	Old	New	All	Old	New
Greedy–Hungarian	SMILE	DINO	78.2	99.3	67.6	61.3	70.7	42.5	39.9	87.1	16.2	41.1	67.6	27.8	33.4	58.4	21.3	54.1	66.1	47.8	34.4	64.0	19.4
	SMILE	CLIP	82.4	97.4	74.9	56.4	64.6	40.0	47.5	71.0	35.7	43.7	69.7	30.8	36.7	57.2	26.8	58.2	77.5	48.1	40.5	70.4	25.2
	PHE	DINO	83.0	98.0	75.5	64.8	78.8	36.9	53.1	83.5	38.1	46.9	76.0	32.4	46.3	78.3	30.8	63.3	91.3	48.6	50.0	89.3	30.0
	PHE	CLIP	79.3	97.0	70.4	66.1	80.3	37.5	52.9	87.8	35.5	44.2	70.3	31.1	46.4	78.1	31.1	64.1	86.2	52.4	47.8	88.4	27.0
	LTC	DINO	80.5	98.1	71.7	66.8	81.2	37.8	54.0	90.5	35.7	51.9	82.9	36.3	42.3	79.5	24.4	68.6	92.5	56.0	43.0	82.2	23.0

Appendix

Table 5: Comparison of variants.

	CUB	Scars	Pets	Food	CIFAR10	CIFAR100	ImageNet100
$\|Y_{Q}\|$	200	196	38	101	10	100	100
$\|Y_{S}\|$	100	98	19	51	5	50	50
$\|\mathcal{D}_{S}\|$	1.5K	2.0K	0.9K	19.1K	12.5K	12.5K	32.5K
$\|\mathcal{D}_{Q}\|$	4.5K	6.1K	2.7K	56.6K	37.5K	37.5K	97.5K

Appendix

Table 6: Statistics of datasets used in our experiments.

Figure 6: Performance comparison of AutoTau against fixed $\tau_{\text{init}}$ values using the Strict Hungarian and Greedy Hungarian metrics. AutoTau improves model performance by mitigating the impact of suboptimal initial thresholds.

Figure 7: Impact of AutoTau on performance with different $\tau_{\text{init}}$ values. The improvement is more significant in the Strict Hungarian metric, reaching up to 5% for suboptimal $\tau_{\text{init}}$ values.

Figure 8: Parameter sensitivity analysis for

\alpha

and

\gamma_{mm}

. The results show the performance variations with different values of these hyperparameters.

Generalized Category Discovery

Generalized category discovery learns known classes from labeled examples while grouping unlabeled data into known and previously unseen categories.

See all 26 papers

TALON: Test-time Adaptive Learning for On-the-Fly Category Discovery
Mar 9, 2026Yanan Wu, Yuhan Yan, Tailai Chen +5Test-Time Adaptation Prototype-Based Classification
College of Information and Electrical Engineering, China Agricultural University, China · College of Science, China Agricultural University, China · Woof RoboT Co., Ltd., China +4
PACO: Proxy-Task Alignment and Online Calibration for On-the-Fly Category Discovery
Apr 13, 2026Weidong Tang, Bohan Zhang, Zhixiang Chi +3Generalized Category Discovery Model Calibration
China Agricultural University, College of Information and Electrical Engineering, Beijing, China · Concordia University, Department of Computer Science and Software Engineering, Montreal, Canada
DP-BOA: Dirichlet-Process Birth-or-Assign for On-the-Fly Category Discovery
Jul 15, 2026Peiyan Gu, Zixin Teng, Xuming HeDirichlet Process Mixture Models Bayesian Mixture Models
ShanghaiTech University, Shanghai, China · Shanghai Engineering Research Center of Intelligent Vision and Imaging

Mar 9, 2026J/K move · Enter open · S save

TALON: Test-time Adaptive Learning for On-the-Fly Category Discovery

Yanan Wu, Yuhan Yan, Tailai Chen, Zhixiang Chi+4

College of Information and Electrical Engineering, China Agricultural University, China · College of Science, China Agricultural University, China · Woof RoboT Co., Ltd., China+4

Test-Time Adaptation Prototype-Based Classification Online Continual Learning

Open paper AskPDF arXiv

On-the-fly category discovery (OCD) aims to recognize known categories while simultaneously discovering novel ones from an unlabeled online stream, using a model trained only on labeled data. Existing approaches freeze the feature extractor trained offline and employ a hash-based framework that quantizes features into binary codes as class prototypes. However, discovering novel categories with a fixed knowledge base is counterintuitive, as the learning potential of incoming data is entirely neglected. In addition, feature quantization introduces information loss, diminishes representational expressiveness, and amplifies intra-class variance. It often results in category explosion, where a single class is fragmented into multiple pseudo-classes. To overcome these limitations, we propose a test-time adaptation framework that enables learning through discovery. It incorporates two complementary strategies: a semantic-aware prototype update and a stable test-time encoder update. The former dynamically refines class prototypes to enhance classification, whereas the latter integrates new information directly into the parameter space. Together, these components allow the model to continuously expand its knowledge base with newly encountered samples. Furthermore, we introduce a margin-aware logit calibration in the offline stage to enlarge inter-class margins and improve intra-class compactness, thereby reserving embedding space for future class discovery. Experiments on standard OCD benchmarks demonstrate that our method substantially outperforms existing hash-based state-of-the-art approaches, yielding notable improvements in novel-class accuracy and effectively mitigating category explosion. The code is publicly available at \textcolor{blue}{https://github.com/ynanwu/TALON}.

Abstract

Generalized Category Discovery

Explore similar work

TALON: Test-time Adaptive Learning for On-the-Fly Category Discovery

PACO: Proxy-Task Alignment and Online Calibration for On-the-Fly Category Discovery

DP-BOA: Dirichlet-Process Birth-or-Assign for On-the-Fly Category Discovery