Organizations: China Agricultural University, College of Information and Electrical Engineering, Beijing, China · Concordia University, Department of Computer Science and Software Engineering, Montreal, Canada
On-the-Fly Category Discovery (OCD) requires a model, trained on an offline support set, to recognize known classes while discovering new ones from an online streaming sequence. Existing methods focus heavily on offline training. They aim to learn discriminative representations on the support set so that novel classes can be separated at test time. However, their discovery mechanism at inference is typically reduced to a single threshold. We argue that this paradigm is fundamentally flawed as OCD is not a static classification problem, but a dynamic process. The model must continuously decide 1) whether a sample belongs to a known class, 2) matches an existing novel category, or 3) should initiate a new one. Moreover, prior methods treat the support set as fixed knowledge. They do not update their decision boundaries as new evidence arrives during inference. This leads to unstable and inconsistent category formation. Our experiments confirm these issues. With properly calibrated and adaptive thresholds, substantial improvements can be achieved, even without changing the representation. Motivated by this, we propose PACO, a support-set-calibrated, tree-structured online decision framework. The framework models inference as a sequence of hierarchical decisions, including known-class routing, birth-aware novel assignment, and attach-versus-create operations over a dynamic prototype memory. Furthermore, we simulate the proxy discovery process to initialize the thresholds during offline training to align with inference. Thresholds are continuously updated during inference using mature novel prototypes. Importantly, PACO requires no heavy training and no dataset-specific tuning. It can be directly integrated into existing OCD pipelines as an inference-time module. Extensive experiments show significant improvements over SOTA baselines across seven benchmarks.
Figures & tables
Figure 1. Overview of the proposed framework. The model first learns a spherical representation from the support set. At test time, each streaming sample is routed by base-class evidence, checked by a support-calibrated birth boundary, and then either assigned to an existing prototype or used to create a new one. All key thresholds are estimated from support-only proxy tasks before online inference starts. Framework overview with three parts: spherical fine-tuning on the support set, routing regions for old-only, global, and new-only decisions on the sphere, and online novel-cluster expansion with support-calibrated thresholds.
Method
CIFAR-10 (%)
CIFAR-100 (%)
ImageNet-100 (%)
CUB-200-2011 (%)
Stanford Cars (%)
Oxford Pets (%)
Food101 (%)
Average (%)
All
Old
New
All
Old
New
All
Old
New
All
Old
New
All
Old
New
All
Old
New
All
Old
New
All
Old
New
Greedy–Hungarian
SLC ( Hartigan, 1975 )
65.9
96.5
50.9
46.9
62.1
16.6
34.2
86.6
7.1
31.3
48.5
22.7
24.0
45.8
13.6
–
–
–
–
–
–
–
–
–
MLDG ( Li et al., 2018 )
71.6
97.5
58.6
58.4
69.0
37.3
33.6
74.4
13.1
34.2
57.9
22.4
28.0
49.1
17.7
–
–
–
–
–
–
–
–
–
RankStat ( Han et al., 2021 )
56.5
81.1
44.2
36.9
45.7
19.3
33.1
74.2
12.4
27.6
46.2
18.3
18.6
36.9
9.7
–
–
–
–
–
–
–
–
–
WTA ( Jia et al., 2021 )
65.4
88.0
54.1
44.1
55.5
21.2
33.1
75.8
11.7
26.5
45.0
17.3
20.0
38.8
10.6
–
–
–
–
–
–
–
–
–
SMILE ( Du et al., 2023 )
78.2
99.3
67.6
61.3
70.7
42.5
39.9
87.1
16.2
41.1
67.6
27.8
33.4
58.4
21.3
54.1
66.1
47.8
34.4
64.0
19.4
48.9
73.3
34.7
Table 1. Comparison with other SOTA methods under the Greedy–Hungarian and Strict–Hungarian protocols. “DiffGRE-S” integrates DiffGRE with SMILE, while “DiffGRE-P” integrates DiffGRE with PHE. Average denotes the mean across all seven datasets; missing entries are marked with “–”. The best results are highlighted in red, and the second-best results in blue. Underlined numbers indicate results not reported in the original papers but reproduced using the official code.
Method
CUB ( C=200 )
SCars ( C=196 )
#Cls
All
Old
New
#Cls
All
Old
New
SMILE-16bit ( Du et al., 2023 )
924
31.9
52.7
21.5
896
27.5
52.5
15.4
SMILE-32bit ( Du et al., 2023 )
2146
27.3
52.0
15.0
2953
21.9
46.8
9.9
PHE-16bit ( Zheng et al., 2024a )
318
37.6
57.4
27.6
709
31.8
65.4
15.6
PHE-32bit ( Zheng et al., 2024a )
474
38.5
59.9
27.8
762
31.5
64.0
15.8
SynC ( Banerjee and Biswas, 2025 )
444
45.3
54.3
40.9
484
24.6
34.8
19.6
Table 2. Comparison of estimated category numbers and accuracies. Prior OCD methods exhibit "category explosion" with SMILE-32bit predicting 2953 clusters for Stanford Cars ( C=196 ). PACO provides more realistic estimates including 194 for CUB ( C=200 ). This stability results from dynamic birth control and threshold calibration which effectively suppress redundant prototype instantiation and ensure high fidelity to the true class distribution.
Variant
CUB (%)
SCars (%)
All
Old
New
All
Old
New
w/o StatReg
47.0
66.9
37.0
47.2
69.7
36.4
w/o AngMargin
23.7
6.3
32.4
14.6
5.3
19.0
w/o OldRefSel
48.0
63.9
40.1
47.3
72.2
35.3
w/o EvidenceRouting
46.6
66.7
36.5
47.1
70.4
35.9
w/o DynBirthTight
47.1
66.1
37.6
47.4
69.5
36.7
Table 3. Ablation study on CUB and Stanford Cars. The results validate the contribution of each module, with the angular-margin objective being most critical for establishing spherical geometry. Statistical regularization and OldRefSel stabilize Old-class accuracy, while EvidenceRouting and DynBirthTight optimize New-class discovery and mitigate redundant prototype instantiation.
Figure 2. Hyperparameter sensitivity analysis of m,λstat,α , and β . PACO exhibits robust performance across a wide range of settings, with default configurations (dashed lines) situated in stable regions for all metrics. While SCars New accuracy is sensitive to m>0.6 , the model remains remarkably stable across variations in λstat,α , and β . These results validate the reliability and generalizability of our default parameter selections. A two-row, four-column hyperparameter analysis figure. The top row is CUB and the bottom row is SCars. The four columns correspond to m, lambda stat, alpha, and beta. Each panel compares All, Old, New, and the default setting.
Figure 3. Visualization of support-set calibration on Stanford Cars. (a) Routing Regions: The pair (τhi,τlo) partitions the stream into Known-Only, Novel Exploration, and Full Competition zones, reserving an uncertainty buffer for ambiguous samples. (b) Attach/Create Regions: Calibrated thresholds categorize samples into Direct Predict, Attach, and Create regions. The distribution shows that "Novel First Occurrences" (red dots) are correctly isolated for creation, while "Novel Non-First Occurrences" (green dots) are absorbed, effectively suppressing redundant prototype births. A single-column figure with two side-by-side plots on Stanford Cars. The left plot shows routing regions induced by support-calibrated known-class evidence thresholds. The right plot shows direct-prediction, attach, and create regions induced by support-calibrated birth and creation thresholds.
Method
CUB (%)
SCars (%)
All
Old
New
All
Old
New
AGE ( Li et al., 2025 )
46.3
59.8
39.4
34.8
62.7
21.3
AGE+SSC
46.7
60.8
39.7
36.1
64.4
21.9
Δ
\mathbf{\uparrow{\color[rgb]{1,0,0}0.4}}
\mathbf{\uparrow{\color[rgb]{1,0,0}1.0}}
\mathbf{\uparrow{\color[rgb]{1,0,0}0.3}}
\mathbf{\uparrow{\color[rgb]{1,0,0}1.3}}
\mathbf{\uparrow{\color[rgb]{1,0,0}1.7}}
\mathbf{\uparrow{\color[rgb]{1,0,0}0.6}}
PHE-12bit ( Zheng et al., 2024a )
36.4
55.8
27.0
31.3
61.9
16.8
PHE-12bit+SSC
39.1
60.6
28.3
34.2
66.7
17.9
Table 4. Plug-and-play generalization of our support-set calibration (SSC) across existing OCD baselines. We evaluate the transferability of SSC by replacing the original threshold-fusion rule in AGE and the heuristic Hamming-radius rule in PHE-12bit with our calibrated logic under the Strict-Hungarian protocol. Δ indicates the absolute performance gain over the corresponding original methods.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
∣YQ∣
∣YS∣
∣Ynovel∣
∣DS∣
∣DQ∣
CIFAR-10
10
5
5
12,500
37,500
CIFAR-100
100
80
20
20,000
30,000
ImageNet-100
100
50
50
31,860
95,255
CUB-200-2011
200
100
100
1,498
4,496
Stanford Cars
196
98
98
2,000
6,144
Oxford-IIIT Pet
37
19
18
942
2,738
Appendix
Table 5. Dataset statistics under the OCD protocol used in this paper.
Dataset
Strict–Hungarian (%)
Greedy–Hungarian (%)
All
Old
New
All
Old
New
CIFAR-10
58.07 ± 0.57
57.60 ± 0.86
58.30 ± 0.94
87.13 ± 0.20
97.40 ± 0.09
82.00 ± 0.29
CIFAR-100
67.03 ± 0.48
78.80 ± 0.67
43.50 ± 1.24
71.10 ± 0.35
83.60 ± 0.50
46.10 ± 0.98
ImageNet-100
67.81 ± 0.34
92.70 ± 0.29
55.30 ± 0.81
68.87 ± 0.27
92.70 ± 0.23
56.90 ± 0.69
CUB-200-2011
49.70 ± 1.14
67.10 ± 1.47
41.00 ± 1.89
53.83 ± 0.79
79.70 ± 1.02
40.90 ± 1.58
Stanford Cars
49.87 ± 1.06
72.40 ± 1.28
39.00 ± 2.05
53.07 ± 0.71
79.10 ± 0.91
40.50 ± 1.43
Appendix
Table 6. Mean and standard deviation of PACO over five independent random seeds under the Strict–Hungarian and Greedy–Hungarian protocols. Residual variation is concentrated mainly on New accuracy for the fine-grained benchmarks.
Figure 4. Strict–Hungarian All/Old/New accuracy under different support ratios Three side-by-side plots showing Strict-Hungarian All, Old, and New accuracy against the support ratio for CUB-200-2011, Stanford Cars, and Oxford-IIIT Pet.
Figure 5. Stream-level traces of the adaptive birth update on Stanford Cars under the default setting. (a) The estimated category count rises rapidly early in the stream and then flattens, indicating that most prototype creation occurs before the novel bank stabilizes. (b) The mixing coefficient ηt increases as reliable novel structure accumulates, so the adaptive term in Eqs. ( 63 ) becomes more influential later in the stream. Markers denote 10% stream increments. Two side-by-side process plots on Stanford Cars. The first panel shows estimated category count over stream progress. The second panel shows the mixing coefficient eta over stream progress, with markers placed at ten-percent increments.
Dataset
Total (s)
Support feature extraction (s)
Support-only calibration (s)
CIFAR-10
55.06 ± 4.10
27.84 ± 0.07
27.23 ± 4.07
CIFAR-100
118.07 ± 19.88
44.03 ± 0.02
74.04 ± 19.87
ImageNet-100
186.00 ± 18.19
70.42 ± 0.32
115.58 ± 18.15
CUB-200-2011
8.48 ± 0.58
4.17 ± 0.07
4.31 ± 0.63
Stanford Cars
13.06 ± 3.05
7.39 ± 1.12
5.67 ± 2.06
Oxford-IIIT Pet
4.31 ± 0.07
2.93 ± 0.04
1.38 ± 0.06
Appendix
Table 7. Wall-clock time of the one-time support-set calibration stage that initializes PACO before online inference. For each dataset, we report the mean and standard deviation over five runs. The total cost is decomposed into support feature extraction and support-only calibration, and excludes both offline training and online stream processing.
Figure 6. Per-sample latency of the inference-time decision layer of PACO on Stanford Cars. Measurements use cached projected stream features, excluding backbone and projection-head extraction while retaining whitening and the downstream decision-stage operations. The curve is estimated from 4% stream-progress bins, with markers placed at 10% progress intervals. The latency remains below 1 ms for most of the stream and does not exhibit sustained growth as the active prototype bank evolves. A wide line plot showing decision-layer latency per sample against stream progress on Stanford Cars. The curve remains in a narrow band for most of the stream, with no sustained growth over time.
On-the-fly category discovery (OCD) aims to recognize known categories while simultaneously discovering novel ones from an unlabeled online stream, using a model trained only on labeled data. Existing approaches freeze the feature extractor trained offline and employ a hash-based framework that quantizes features into binary codes as class prototypes. However, discovering novel categories with a fixed knowledge base is counterintuitive, as the learning potential of incoming data is entirely neglected. In addition, feature quantization introduces information loss, diminishes representational expressiveness, and amplifies intra-class variance. It often results in category explosion, where a single class is fragmented into multiple pseudo-classes. To overcome these limitations, we propose a test-time adaptation framework that enables learning through discovery. It incorporates two complementary strategies: a semantic-aware prototype update and a stable test-time encoder update. The former dynamically refines class prototypes to enhance classification, whereas the latter integrates new information directly into the parameter space. Together, these components allow the model to continuously expand its knowledge base with newly encountered samples. Furthermore, we introduce a margin-aware logit calibration in the offline stage to enlarge inter-class margins and improve intra-class compactness, thereby reserving embedding space for future class discovery. Experiments on standard OCD benchmarks demonstrate that our method substantially outperforms existing hash-based state-of-the-art approaches, yielding notable improvements in novel-class accuracy and effectively mitigating category explosion. The code is publicly available at \textcolor{blue}{https://github.com/ynanwu/TALON}.
Yanan Wu, Yuhan Yan, Tailai Chen +5
College of Information and Electrical Engineering, China Agricultural University, China · College of Science, China Agricultural University, China · Woof RoboT Co., Ltd., China +4
On-the-fly category discovery requires deciding for each incoming test sample whether to assign it to an existing category or spawn a new one. Existing methods typically implement this decision through matching-based heuristics, such as radius- or hash-based rules. While effective in practice, these methods usually treat category birth implicitly as a fallback when no existing category matches confidently, rather than as an explicit alternative supported by its own statistical evidence. To address this, we propose DP-BOA, a posterior-predictive decision framework based on an online Dirichlet-process Gaussian mixture model with a Normal-Inverse-Wishart prior. During training, we use labeled data to calibrate a shared NIW prior over category Gaussians and warm-start the known-category posteriors. At test time, for each incoming sample, DP-BOA compares the posterior predictive evidence for assignment to existing categories against the evidence for spawning a new category induced by the DP prior, and then updates category statistics online after the decision. The method captures anisotropic category geometry and naturally adapts decision confidence as evidence accumulates. Across standard OCD benchmarks, DP-BOA consistently outperforms strong baselines and delivers particularly strong novel-class discovery performance while maintaining competitive known-class accuracy.
Peiyan Gu, Zixin Teng, Xuming He
ShanghaiTech University, Shanghai, China · Shanghai Engineering Research Center of Intelligent Vision and Imaging
On-the-Fly Category Discovery (OCD) aims to recognize known classes while simultaneously discovering emerging novel categories during inference, using supervision only from known classes during offline training. Existing approaches rely either on fixed label supervision or on diffusion-based augmentations to enhance the backbone, yet none of them explicitly train the model to perform the discovery task required at test time. It is fundamentally unreasonable to expect a model optimized on limited labeled data to carry out a qualitatively different discovery objective during inference. This mismatch creates a clear optimization misalignment between the offline learning stage and the online discovery stage. In addition, prior methods often depend on hash-based encodings or severe feature compression, which further limits representational capacity. To address these issues, we propose Learning through Creation (LTC), a fully feature-based and hash-free framework that injects novel-category awareness directly into offline learning. At its core is a lightweight, online pseudo-unknown generator driven by kernel-energy minimization and entropy maximization (MKEE). Unlike previous methods that generate synthetic samples once before training, our generator evolves jointly with the model dynamics and synthesizes pseudo-novel instances on the fly at negligible cost. These samples are incorporated through a dual max-margin objective with adaptive thresholding, strengthening the model's ability to delineate and detect unknown regions through explicit creation. Extensive experiments across seven benchmarks show that LTC consistently outperforms prior work, achieving improvements ranging from 1.5 percent to 13.1 percent in all-class accuracy. The code is available at https://github.com/brandinzhang/LTC
Bohan Zhang, Weidong Tang, Zhixiang Chi +4
College of Information and Electrical Engineering, China Agricultural University, China · Department of Electrical and Computer Engineering, University of Toronto, Canada · School of Computer and Information Technology, Beijing Jiaotong University, China +1