PACO: Proxy-Task Alignment and Online Calibration for On-the-Fly Category Discovery

Weidong Tang; Bohan Zhang; Zhixiang Chi; ZiZhang Wu; Yang Wang; Yanan Wu

cs.CVApr 13, 2026

PACO: Proxy-Task Alignment and Online Calibration for On-the-Fly Category Discovery

Authors: Weidong Tang, Bohan Zhang, Zhixiang Chi, ZiZhang Wu, Yang Wang, Yanan Wu

Organizations: China Agricultural University, College of Information and Electrical Engineering, Beijing, China · Concordia University, Department of Computer Science and Software Engineering, Montreal, Canada

Abstract

On-the-Fly Category Discovery (OCD) requires a model, trained on an offline support set, to recognize known classes while discovering new ones from an online streaming sequence. Existing methods focus heavily on offline training. They aim to learn discriminative representations on the support set so that novel classes can be separated at test time. However, their discovery mechanism at inference is typically reduced to a single threshold. We argue that this paradigm is fundamentally flawed as OCD is not a static classification problem, but a dynamic process. The model must continuously decide 1) whether a sample belongs to a known class, 2) matches an existing novel category, or 3) should initiate a new one. Moreover, prior methods treat the support set as fixed knowledge. They do not update their decision boundaries as new evidence arrives during inference. This leads to unstable and inconsistent category formation. Our experiments confirm these issues. With properly calibrated and adaptive thresholds, substantial improvements can be achieved, even without changing the representation. Motivated by this, we propose PACO, a support-set-calibrated, tree-structured online decision framework. The framework models inference as a sequence of hierarchical decisions, including known-class routing, birth-aware novel assignment, and attach-versus-create operations over a dynamic prototype memory. Furthermore, we simulate the proxy discovery process to initialize the thresholds during offline training to align with inference. Thresholds are continuously updated during inference using mature novel prototypes. Importantly, PACO requires no heavy training and no dataset-specific tuning. It can be directly integrated into existing OCD pipelines as an inference-time module. Extensive experiments show significant improvements over SOTA baselines across seven benchmarks.

Figures & tables

Figure 1. Overview of the proposed framework. The model first learns a spherical representation from the support set. At test time, each streaming sample is routed by base-class evidence, checked by a support-calibrated birth boundary, and then either assigned to an existing prototype or used to create a new one. All key thresholds are estimated from support-only proxy tasks before online inference starts. Framework overview with three parts: spherical fine-tuning on the support set, routing regions for old-only, global, and new-only decisions on the sphere, and online novel-cluster expansion with support-calibrated thresholds.

	Method	CIFAR-10 (%)			CIFAR-100 (%)			ImageNet-100 (%)			CUB-200-2011 (%)			Stanford Cars (%)			Oxford Pets (%)			Food101 (%)			Average (%)
	Method	All	Old	New	All	Old	New	All	Old	New	All	Old	New	All	Old	New	All	Old	New	All	Old	New	All	Old	New
Greedy–Hungarian	SLC ( Hartigan, 1975 )	65.9	96.5	50.9	46.9	62.1	16.6	34.2	86.6	7.1	31.3	48.5	22.7	24.0	45.8	13.6	–	–	–	–	–	–	–	–	–
	MLDG ( Li et al., 2018 )	71.6	97.5	58.6	58.4	69.0	37.3	33.6	74.4	13.1	34.2	57.9	22.4	28.0	49.1	17.7	–	–	–	–	–	–	–	–	–
	RankStat ( Han et al., 2021 )	56.5	81.1	44.2	36.9	45.7	19.3	33.1	74.2	12.4	27.6	46.2	18.3	18.6	36.9	9.7	–	–	–	–	–	–	–	–	–
	WTA ( Jia et al., 2021 )	65.4	88.0	54.1	44.1	55.5	21.2	33.1	75.8	11.7	26.5	45.0	17.3	20.0	38.8	10.6	–	–	–	–	–	–	–	–	–
	SMILE ( Du et al., 2023 )	78.2	99.3	67.6	61.3	70.7	42.5	39.9	87.1	16.2	41.1	67.6	27.8	33.4	58.4	21.3	54.1	66.1	47.8	34.4	64.0	19.4	48.9	73.3	34.7

Table 1. Comparison with other SOTA methods under the Greedy–Hungarian and Strict–Hungarian protocols. “DiffGRE-S” integrates DiffGRE with SMILE, while “DiffGRE-P” integrates DiffGRE with PHE. Average denotes the mean across all seven datasets; missing entries are marked with “–”. The best results are highlighted in red, and the second-best results in blue. Underlined numbers indicate results not reported in the original papers but reproduced using the official code.

Method	CUB ( $C{=}200$ )				SCars ( $C{=}196$ )
Method	#Cls	All	Old	New	#Cls	All	Old	New
SMILE-16bit ( Du et al., 2023 )	924	31.9	52.7	21.5	896	27.5	52.5	15.4
SMILE-32bit ( Du et al., 2023 )	2146	27.3	52.0	15.0	2953	21.9	46.8	9.9
PHE-16bit ( Zheng et al., 2024a )	318	37.6	57.4	27.6	709	31.8	65.4	15.6
PHE-32bit ( Zheng et al., 2024a )	474	38.5	59.9	27.8	762	31.5	64.0	15.8
SynC ( Banerjee and Biswas, 2025 )	444	45.3	54.3	40.9	484	24.6	34.8	19.6

Table 2. Comparison of estimated category numbers and accuracies. Prior OCD methods exhibit "category explosion" with SMILE-32bit predicting 2953 clusters for Stanford Cars (

C=196

). PACO provides more realistic estimates including 194 for CUB (

C=200

). This stability results from dynamic birth control and threshold calibration which effectively suppress redundant prototype instantiation and ensure high fidelity to the true class distribution.

Variant	CUB (%)			SCars (%)
Variant	All	Old	New	All	Old	New
w/o StatReg	47.0	66.9	37.0	47.2	69.7	36.4
w/o AngMargin	23.7	6.3	32.4	14.6	5.3	19.0
w/o OldRefSel	48.0	63.9	40.1	47.3	72.2	35.3
w/o EvidenceRouting	46.6	66.7	36.5	47.1	70.4	35.9
w/o DynBirthTight	47.1	66.1	37.6	47.4	69.5	36.7

Table 3. Ablation study on CUB and Stanford Cars. The results validate the contribution of each module, with the angular-margin objective being most critical for establishing spherical geometry. Statistical regularization and OldRefSel stabilize Old-class accuracy, while EvidenceRouting and DynBirthTight optimize New-class discovery and mitigate redundant prototype instantiation.

Figure 2. Hyperparameter sensitivity analysis of

m,\lambda_{\text{stat}},\alpha

, and

\beta

. PACO exhibits robust performance across a wide range of settings, with default configurations (dashed lines) situated in stable regions for all metrics. While SCars New accuracy is sensitive to

m>0.6

, the model remains remarkably stable across variations in

\lambda_{\text{stat}},\alpha

, and

\beta

. These results validate the reliability and generalizability of our default parameter selections. A two-row, four-column hyperparameter analysis figure. The top row is CUB and the bottom row is SCars. The four columns correspond to m, lambda stat, alpha, and beta. Each panel compares All, Old, New, and the default setting.

Panel of Figure 3 — Figure 3. Visualization of support-set calibration on Stanford Cars. (a) Routing Regions: The pair $(\tau_{\text{hi}},\tau_{\text{lo}})$ partitions the stream into Known-Only, Novel Exploration, and Full Competition zones, reserving an uncertainty buffer for ambiguous samples. (b) Attach/Create Regions: Calibrated thresholds categorize samples into Direct Predict, Attach, and Create regions. The distribution shows that "Novel First Occurrences" (red dots) are correctly isolated for creation, while "Novel Non-First Occurrences" (green dots) are absorbed, effectively suppressing redundant prototype births. A single-column figure with two side-by-side plots on Stanford Cars. The left plot shows routing regions induced by support-calibrated known-class evidence thresholds. The right plot shows direct-prediction, attach, and create regions induced by support-calibrated birth and creation thresholds.

Method	CUB (%)			SCars (%)
Method	All	Old	New	All	Old	New
AGE ( Li et al., 2025 )	46.3	59.8	39.4	34.8	62.7	21.3
AGE+SSC	46.7	60.8	39.7	36.1	64.4	21.9
$\Delta$	$\mathbf{\uparrow{\color[rgb]{1,0,0}0.4}}$	$\mathbf{\uparrow{\color[rgb]{1,0,0}1.0}}$	$\mathbf{\uparrow{\color[rgb]{1,0,0}0.3}}$	$\mathbf{\uparrow{\color[rgb]{1,0,0}1.3}}$	$\mathbf{\uparrow{\color[rgb]{1,0,0}1.7}}$	$\mathbf{\uparrow{\color[rgb]{1,0,0}0.6}}$
PHE-12bit ( Zheng et al., 2024a )	36.4	55.8	27.0	31.3	61.9	16.8
PHE-12bit+SSC	39.1	60.6	28.3	34.2	66.7	17.9

Table 4. Plug-and-play generalization of our support-set calibration (SSC) across existing OCD baselines. We evaluate the transferability of SSC by replacing the original threshold-fusion rule in AGE and the heuristic Hamming-radius rule in PHE-12bit with our calibrated logic under the Strict-Hungarian protocol.

\Delta

indicates the absolute performance gain over the corresponding original methods.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	$\|\mathcal{Y}_{Q}\|$	$\|\mathcal{Y}_{S}\|$	$\|\mathcal{Y}_{novel}\|$	$\|\mathcal{D}_{S}\|$	$\|\mathcal{D}_{Q}\|$
CIFAR-10	10	5	5	12,500	37,500
CIFAR-100	100	80	20	20,000	30,000
ImageNet-100	100	50	50	31,860	95,255
CUB-200-2011	200	100	100	1,498	4,496
Stanford Cars	196	98	98	2,000	6,144
Oxford-IIIT Pet	37	19	18	942	2,738

Appendix

Table 5. Dataset statistics under the OCD protocol used in this paper.

Dataset	Strict–Hungarian (%)			Greedy–Hungarian (%)
Dataset	All	Old	New	All	Old	New
CIFAR-10	58.07 $\pm$ 0.57	57.60 $\pm$ 0.86	58.30 $\pm$ 0.94	87.13 $\pm$ 0.20	97.40 $\pm$ 0.09	82.00 $\pm$ 0.29
CIFAR-100	67.03 $\pm$ 0.48	78.80 $\pm$ 0.67	43.50 $\pm$ 1.24	71.10 $\pm$ 0.35	83.60 $\pm$ 0.50	46.10 $\pm$ 0.98
ImageNet-100	67.81 $\pm$ 0.34	92.70 $\pm$ 0.29	55.30 $\pm$ 0.81	68.87 $\pm$ 0.27	92.70 $\pm$ 0.23	56.90 $\pm$ 0.69
CUB-200-2011	49.70 $\pm$ 1.14	67.10 $\pm$ 1.47	41.00 $\pm$ 1.89	53.83 $\pm$ 0.79	79.70 $\pm$ 1.02	40.90 $\pm$ 1.58
Stanford Cars	49.87 $\pm$ 1.06	72.40 $\pm$ 1.28	39.00 $\pm$ 2.05	53.07 $\pm$ 0.71	79.10 $\pm$ 0.91	40.50 $\pm$ 1.43

Appendix

Table 6. Mean and standard deviation of PACO over five independent random seeds under the Strict–Hungarian and Greedy–Hungarian protocols. Residual variation is concentrated mainly on New accuracy for the fine-grained benchmarks.

Panel of Figure 4 — Figure 4. Strict–Hungarian All/Old/New accuracy under different support ratios Three side-by-side plots showing Strict-Hungarian All, Old, and New accuracy against the support ratio for CUB-200-2011, Stanford Cars, and Oxford-IIIT Pet.

Panel of Figure 5 — Figure 5. Stream-level traces of the adaptive birth update on Stanford Cars under the default setting. (a) The estimated category count rises rapidly early in the stream and then flattens, indicating that most prototype creation occurs before the novel bank stabilizes. (b) The mixing coefficient $\eta_{t}$ increases as reliable novel structure accumulates, so the adaptive term in Eqs. ( 63 ) becomes more influential later in the stream. Markers denote $10\%$ stream increments. Two side-by-side process plots on Stanford Cars. The first panel shows estimated category count over stream progress. The second panel shows the mixing coefficient eta over stream progress, with markers placed at ten-percent increments.

Dataset	Total (s)	Support feature extraction (s)	Support-only calibration (s)
CIFAR-10	55.06 $\pm$ 4.10	27.84 $\pm$ 0.07	27.23 $\pm$ 4.07
CIFAR-100	118.07 $\pm$ 19.88	44.03 $\pm$ 0.02	74.04 $\pm$ 19.87
ImageNet-100	186.00 $\pm$ 18.19	70.42 $\pm$ 0.32	115.58 $\pm$ 18.15
CUB-200-2011	8.48 $\pm$ 0.58	4.17 $\pm$ 0.07	4.31 $\pm$ 0.63
Stanford Cars	13.06 $\pm$ 3.05	7.39 $\pm$ 1.12	5.67 $\pm$ 2.06
Oxford-IIIT Pet	4.31 $\pm$ 0.07	2.93 $\pm$ 0.04	1.38 $\pm$ 0.06

Appendix

Table 7. Wall-clock time of the one-time support-set calibration stage that initializes PACO before online inference. For each dataset, we report the mean and standard deviation over five runs. The total cost is decomposed into support feature extraction and support-only calibration, and excludes both offline training and online stream processing.

Figure 6. Per-sample latency of the inference-time decision layer of PACO on Stanford Cars. Measurements use cached projected stream features, excluding backbone and projection-head extraction while retaining whitening and the downstream decision-stage operations. The curve is estimated from

4\%

stream-progress bins, with markers placed at

10\%

progress intervals. The latency remains below

1

ms for most of the stream and does not exhibit sustained growth as the active prototype bank evolves. A wide line plot showing decision-layer latency per sample against stream progress on Stanford Cars. The curve remains in a narrow band for most of the stream, with no sustained growth over time.

Generalized Category Discovery

Generalized category discovery learns known classes from labeled examples while grouping unlabeled data into known and previously unseen categories.

See all 25 papers

TALON: Test-time Adaptive Learning for On-the-Fly Category Discovery
Mar 9, 2026Yanan Wu, Yuhan Yan, Tailai Chen +5Test-Time Adaptation Prototype-Based Classification
College of Information and Electrical Engineering, China Agricultural University, China · College of Science, China Agricultural University, China · Woof RoboT Co., Ltd., China +4
DP-BOA: Dirichlet-Process Birth-or-Assign for On-the-Fly Category Discovery
Jul 15, 2026Peiyan Gu, Zixin Teng, Xuming HeDirichlet Process Mixture Models Bayesian Mixture Models
ShanghaiTech University, Shanghai, China · Shanghai Engineering Research Center of Intelligent Vision and Imaging
Learning through Creation: A Hash-Free Framework for On-the-Fly Category Discovery
Mar 14, 2026Bohan Zhang, Weidong Tang, Zhixiang Chi +4Generalized Category Discovery
College of Information and Electrical Engineering, China Agricultural University, China · Department of Electrical and Computer Engineering, University of Toronto, Canada · School of Computer and Information Technology, Beijing Jiaotong University, China +1

Mar 9, 2026J/K move · Enter open · S save

TALON: Test-time Adaptive Learning for On-the-Fly Category Discovery

Yanan Wu, Yuhan Yan, Tailai Chen, Zhixiang Chi+4

College of Information and Electrical Engineering, China Agricultural University, China · College of Science, China Agricultural University, China · Woof RoboT Co., Ltd., China+4

Test-Time Adaptation Prototype-Based Classification Online Continual Learning

Open paper AskPDF arXiv

On-the-fly category discovery (OCD) aims to recognize known categories while simultaneously discovering novel ones from an unlabeled online stream, using a model trained only on labeled data. Existing approaches freeze the feature extractor trained offline and employ a hash-based framework that quantizes features into binary codes as class prototypes. However, discovering novel categories with a fixed knowledge base is counterintuitive, as the learning potential of incoming data is entirely neglected. In addition, feature quantization introduces information loss, diminishes representational expressiveness, and amplifies intra-class variance. It often results in category explosion, where a single class is fragmented into multiple pseudo-classes. To overcome these limitations, we propose a test-time adaptation framework that enables learning through discovery. It incorporates two complementary strategies: a semantic-aware prototype update and a stable test-time encoder update. The former dynamically refines class prototypes to enhance classification, whereas the latter integrates new information directly into the parameter space. Together, these components allow the model to continuously expand its knowledge base with newly encountered samples. Furthermore, we introduce a margin-aware logit calibration in the offline stage to enlarge inter-class margins and improve intra-class compactness, thereby reserving embedding space for future class discovery. Experiments on standard OCD benchmarks demonstrate that our method substantially outperforms existing hash-based state-of-the-art approaches, yielding notable improvements in novel-class accuracy and effectively mitigating category explosion. The code is publicly available at \textcolor{blue}{https://github.com/ynanwu/TALON}.

Abstract

Generalized Category Discovery

Explore similar work

TALON: Test-time Adaptive Learning for On-the-Fly Category Discovery

DP-BOA: Dirichlet-Process Birth-or-Assign for On-the-Fly Category Discovery

Learning through Creation: A Hash-Free Framework for On-the-Fly Category Discovery