cs.CVSep 27, 2026

Test-Time Generalized Category Discovery

Authors: Shambhavi Mishra, Omprakash Chakraborty, Julio Silva-Rodriguez, Ismail Ben Ayed, Marco Pedersoli, Jose Dolz

Organizations: ÉTS Montréal, Canada

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Test-Time Adaptation (TTA) and Generalized Category Discovery (GCD) are traditionally treated as disjoint problems: the former adapts models to domain shift assuming all test classes are known, while the latter discovers novel categories assuming labeled training data for known classes. However, real-world deployment rarely fits either setting. Motivated by this gap, we introduce Test-Time Generalized Category Discovery (TT-GCD), a unified and more realistic scenario where a vision-language model must adapt to distribution shifts, classify known categories using only textual supervision, and discover novel categories, all during test time and without access to labeled data. To address this challenging scenario, we propose PACT (Prototype Assignment for Category discovery at Test time), a fully unsupervised framework that casts known-class recognition and novel-class discovery via prototype assignment. PACT first re-aligns shifted visual features with the text-derived class representations of the VLM using confident zero-shot predictions. Known and novel categories are then both represented by prototypes in the visual embedding space, estimated from the unlabeled test stream, and each test image is assigned to the category whose prototype is most similar to its visual feature. Extensive experiments across corruption and domain-shift benchmarks demonstrate that PACT outperforms adapted state-of-the-art TTA and GCD methods, effectively bridging the gap between adaptation and discovery.

Figures & tables

Figure 1: The TT-GCD setting. A VLM trained on New York streets, with no labeled source data [ Left ], is deployed on an unlabeled Alberta stream [ Middle ] that combines domain shift (snow), known classes ( car ) and novel classes ( moose ). [ Right ] TTA misclassifies the novel class as a known class, Open-Set TTA discards it as an outlier, GCD needs labeled target-domain data, and TT-GCD adapts to the shift, classifies the known and discovers the novel.
Figure 2: Mean performance ( Known vs. novel ) over CIFAR-10-C, CIFAR-100-C and ImageNet-C. Dashed lines mark the best baseline on each axis.
SettingDomain ShiftNovel ClassesNovel DiscoveryNo Labeled Train Data
TTA ( Wang et al., 2021 ; Shu et al., 2022 ; Maharana et al., 2025 ; Mishra et al., 2026 )✓✗✗✓
Open-Set TTA ( Lee et al., 2023 ; Gao et al., 2024 )✓✓✗✓
GCD ( Vaze et al., 2022 ; Chiaroni et al., 2023 ; Wang et al., 2024 )✗✓✓✗
GCD w/ Shift ( Wang et al., 2025 )✓✓✓✗
TT-GCD (Ours)✓✓✓✓
Table 1: Comparison of TT-GCD with related test-time and category-discovery settings. TT-GCD uniquely combines distribution shift, novel categories, novel-category discovery, and fully unlabeled test-time adaptation.
 CIFAR-10-CCIFAR-100-CImageNet-C
MethodAllKnownNovelAllKnownNovelAllKnownNovel
Test Time Adaptation
TENT++33.41_{\text{\pm.10}}60.06_{\text{\pm.12}}6.75_{\text{\pm.25}}15.78_{\text{\pm.01}}17.16_{\text{\pm.02}}14.41_{\text{\pm.04}}13.14_{\text{\pm.07}}19.58_{\text{\pm.12}}6.69_{\text{\pm.03}}
BATCLIP++29.24_{\text{\pm.08}}46.88_{\text{\pm.01}}11.59_{\text{\pm.16}}15.57_{\text{\pm.01}}15.31_{\text{\pm.07}}15.83_{\text{\pm.09}}15.74_{\text{\pm.06}}18.83_{\text{\pm.05}}12.64_{\text{\pm.07}}
SAT++23.43_{\text{\pm.07}}34.48_{\text{\pm.31}}12.39_{\text{\pm.29}}13.25_{\text{\pm.07}}13.69_{\text{\pm.18}}12.81_{\text{\pm.12}}10.55_{\text{\pm.03}}12.31_{\text{\pm.04}}8.80_{\text{\pm.03}}
Open-Set Test Time Adaptation
Table 2: TT-GCD on corruption benchmarks (severity 5, mean over 15 corruption types). Clustering accuracy (%), mean over 3 seeds with standard deviation. Best in bold , second best underlined .
MethodClipartInfographPaintingQuickdrawRealSketchAverage
TENT++28.33_{\text{\pm.19}}17.42_{\text{\pm.09}}26.95_{\text{\pm.04}}0.52_{\text{\pm.00}}\underline{31.69}_{\text{\pm.03}}23.58_{\text{\pm.14}}21.41
BATCLIP++\underline{29.09}_{\text{\pm.15}}16.64_{\text{\pm.14}}\underline{28.10}_{\text{\pm.31}}\underline{6.04}_{\text{\pm.09}}30.63_{\text{\pm.39}}\underline{24.85}_{\text{\pm.40}}22.56
SAT++14.17_{\text{\pm.28}}14.28_{\text{\pm.41}}9.81_{\text{\pm.48}}2.20_{\text{\pm.01}}7.93_{\text{\pm.14}}10.90_{\text{\pm.09}}9.88
OSTTA++26.82_{\text{\pm.34}}19.19_{\text{\pm.18}}25.69_{\text{\pm.40}}5.71_{\text{\pm.21}}27.21_{\text{\pm.67}}23.92_{\text{\pm.12}}21.42
UniEnt++28.01_{\text{\pm.09}}\underline{19.70}_{\text{\pm.14}}26.81_{\text{\pm.17}}5.50_{\text{\pm.25}}30.24_{\text{\pm.34}}24.80_{\text{\pm.37}}22.51
SimGCD++17.10_{\text{\pm 15.65}}10.53_{\text{\pm 4.56}}15.35_{\text{\pm 16.43}}5.23_{\text{\pm.02}}17.97_{\text{\pm 14.91}}8.43_{\text{\pm 2.19}}12.44
Table 3: TT-GCD under natural shift. DomainNet, per domain, K=172K{=}172 . Clustering accuracy ( All , %), mean over 3 seeds with the standard deviation. Best in bold , second best underlined . Known and Novel per domain are reported in App. D.1 .
MethodB/16L/14BioCLIP-2
TENT++9.5415.0131.97
BATCLIP++11.5715.5228.63
SAT++9.1110.4226.22
OSTTA++8.8614.0028.82
UniEnt++7.5713.9620.58
SimGCD++13.7417.8524.76
Table 4: Fine-grained shift on CUB-C. All (%), severity 5.
  CIFAR-100-CDomainNet
Re-alignKnown protoAllKnownNovelAllKnownNovel
–Text15.1113.8316.3925.2224.4525.91
–Image23.0828.3817.7742.1351.7332.62
✓Text22.1422.0422.2334.1132.9035.25
✓Image28.9933.7424.2546.6453.7039.68
Table 5: Component ablation. Clustering accuracy (%)
Figure 3: (a) Time per batch of 128 (log scale) against mean All on DomainNet. (b) All (%) as a function of the buffer size WW .
Figure 4: TT-GCD under estimated LL . All accuracy (%) when the novel budget is estimated once per stream and shared by every method (App. D.3 ).
Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

MethodAdapted parametersNovelTest-time objective
Test Time Adaptation
TENT++ [ Wang et al., 2021 ]visual LN✗entropy minimization on known samples
BATCLIP++ [ Maharana et al., 2025 ]visual + text LN✗entropy + image-to-text alignment + diversity
SAT++ [ Mishra et al., 2026 ]visual LN✗Sinkhorn optimal-transport self-labeling
Open-Set Test Time Adaptation
OSTTA++ [ Lee et al., 2023 ]visual LN↑\uparrowknown entropy min. −- novel entropy + diversity
Appendix
Table 6: TT-GCD adaptation of each baseline, and PACT. LN denotes LayerNorm affine parameters. The Novel column indicates how the test-time objective treats novel samples: ✗ not at all, ↑\uparrow entropy maximization only, ✓ modeled as categories with their own parameters.
Method#Swept valuesBest All
TENT++1LR {\{ baseline, 10−3.510^{-3.5} , 10−2.5}10^{-2.5}\}24.86
BATCLIP++2LR {\{ baseline, 10−3.510^{-3.5} , 10−2.5}10^{-2.5}\} ; weight decay {\{ baseline, 0.01}0.01\}17.86
SAT++5LR {\{ baseline, 10−4.510^{-4.5} , 10−3.510^{-3.5} , 10−2.5}10^{-2.5}\} ; temperature {\{ baseline, 0.0050.005 , 0.05}0.05\} ; ε\varepsilon {\{ baseline, 0.40.4 , 1.0}1.0\} ; Sinkhorn iterations {\{ baseline, 11 , 5}5\} ; multi-template {\{ yes, no }\}26.09
OSTTA++4LR {\{ baseline, 10−3.510^{-3.5} , 10−2.5}10^{-2.5}\} ; αdiv\alpha_{\mathrm{div}} {\{ baseline, 0.10.1 , 0.5}0.5\} ; αood\alpha_{\mathrm{ood}} {\{ baseline, 0.050.05 , 0.2}0.2\} ; steps {\{ baseline, 2}2\}27.25
UniEnt++4LR {\{ baseline, 10−3.510^{-3.5} , 10−2.5}10^{-2.5}\} ; αdiv\alpha_{\mathrm{div}} {\{ baseline, 0.050.05 , 0.2}0.2\} ; αood\alpha_{\mathrm{ood}} {\{ baseline, 0.0250.025 , 0.1}0.1\} ; steps {\{ baseline, 2}2\}22.75
SimGCD++5LR {\{ baseline, 10−2.510^{-2.5} , 10−1.5}10^{-1.5}\} ; EMA {\{ baseline, 0.99}0.99\} ; wconsw_{\mathrm{cons}} {\{ baseline, 0.250.25 , 1.0}1.0\} ; wcontrw_{\mathrm{contr}} {\{ baseline, 0.750.75 , 3.0}3.0\} ; wdivw_{\mathrm{div}} {\{ baseline, 0.0050.005 , 0.05}0.05\}19.68
Appendix
Table 7: Baseline hyperparameter sweeps on the four held-out CIFAR-100-C corruptions. baseline denotes the value of the original implementation. Best All (%) is on the held-out corruptions and is not comparable to Table 2 , which reports the 15 evaluation corruptions.
DatasetStreams∣C∣|\mathcal{C}|KKLL
Coarse-grained corruptions
CIFAR-10-C151055
CIFAR-100-C151005050
ImageNet-C151000500500
Fine-grained corruptions
CUB-C7200100100
Appendix
Table 8: Benchmarks. Each corruption type forms one test stream.
ParameterMeaningValue
Re-alignment
WWbuffer size1024
EEepochs30
LRlearning rate10−210^{-2}
Streaming
emin⁡e_{\min}known activation threshold1.5
Appendix
Table 9: PACT configuration, same for every benchmark.
 CIFAR-100-CDomainNet
Methodms/batchpeak GBms/batchpeak GB
TENT++11068.7105010.3
BATCLIP++10668.4118110.5
SAT++10145.6108510.7
OSTTA++177118.0167318.1
UniEnt++175818.0167518.1
Appendix
Table 10: Cost per batch. Steady-state ms per batch of 128 and peak GPU memory (GB) on one H100 3g.40gb slice.
MethodClipartInfographPaintingQuickdrawRealSketch
TENT++\underline{49.29}_{\text{\pm.43}}29.16_{\text{\pm.21}}\underline{46.94}_{\text{\pm.06}}0.82_{\text{\pm.30}}\underline{57.02}_{\text{\pm.14}}41.77_{\text{\pm.09}}
BATCLIP++37.94_{\text{\pm.21}}21.84_{\text{\pm.07}}36.56_{\text{\pm.22}}7.29_{\text{\pm.11}}37.75_{\text{\pm 1.06}}33.21_{\text{\pm.28}}
SAT++16.79_{\text{\pm.14}}13.51_{\text{\pm.52}}11.53_{\text{\pm 1.37}}2.57_{\text{\pm.28}}9.32_{\text{\pm.40}}13.59_{\text{\pm.89}}
OSTTA++46.31_{\text{\pm.54}}\underline{31.50}_{\text{\pm.74}}43.63_{\text{\pm.93}}\underline{8.68}_{\text{\pm.35}}45.09_{\text{\pm.86}}40.93_{\text{\pm.15}}
UniEnt++48.79_{\text{\pm.19}}31.41_{\text{\pm.54}}45.98_{\text{\pm.36}}7.98_{\text{\pm.40}}51.87_{\text{\pm.91}}\underline{42.35}_{\text{\pm.86}}
SimGCD++23.70_{\text{\pm 23.75}}10.65_{\text{\pm 4.88}}22.43_{\text{\pm 27.54}}7.30_{\text{\pm.54}}25.42_{\text{\pm 23.25}}10.95_{\text{\pm 3.69}}
Appendix
Table 11: DomainNet, Known accuracy (%) per domain, mean over 3 seeds with the standard deviation. Best in bold , second best underlined .
MethodClipartInfographPaintingQuickdrawRealSketch
TENT++6.65_{\text{\pm.07}}5.93_{\text{\pm.13}}7.18_{\text{\pm.11}}0.23_{\text{\pm.30}}6.22_{\text{\pm.08}}7.22_{\text{\pm.29}}
BATCLIP++\underline{19.95}_{\text{\pm.12}}11.55_{\text{\pm.22}}\underline{19.74}_{\text{\pm.41}}\underline{4.80}_{\text{\pm.13}}\underline{23.48}_{\text{\pm.93}}\underline{17.34}_{\text{\pm.53}}
SAT++11.46_{\text{\pm.57}}\underline{15.04}_{\text{\pm.36}}8.11_{\text{\pm.39}}1.84_{\text{\pm.29}}6.53_{\text{\pm.50}}8.49_{\text{\pm.63}}
OSTTA++6.68_{\text{\pm.18}}7.13_{\text{\pm.61}}7.95_{\text{\pm.17}}2.77_{\text{\pm.19}}9.25_{\text{\pm.49}}8.62_{\text{\pm.11}}
UniEnt++6.53_{\text{\pm.06}}8.24_{\text{\pm.76}}7.86_{\text{\pm.68}}3.05_{\text{\pm.43}}8.50_{\text{\pm.47}}9.01_{\text{\pm 1.13}}
SimGCD++10.28_{\text{\pm 7.31}}10.42_{\text{\pm 4.26}}8.35_{\text{\pm 5.43}}3.17_{\text{\pm.51}}10.49_{\text{\pm 6.55}}6.15_{\text{\pm 1.02}}
Appendix
Table 12: DomainNet, Novel accuracy (%) per domain, mean over 3 seeds with the standard deviation. Best in bold , second best underlined .
 CLIP B/16CLIP L/14BioCLIP-2 L/14
MethodAllKnownNovelAllKnownNovelAllKnownNovel
TENT++9.5412.037.0415.0121.818.2131.9759.264.73
BATCLIP++11.5711.3711.7715.5217.0314.0128.6345.7011.60
SAT++9.119.059.1810.4211.379.4826.2240.0812.40
OSTTA++8.8611.586.1514.0020.927.1128.8253.154.53
UniEnt++7.579.725.4213.9618.609.3320.5834.336.85
Appendix
Table 13: Backbone study on CUB-C (7 corruption types). Clustering accuracy (%). All is in Table 4 .
 CIFAR-10-CCIFAR-100-CDomainNetImageNet-C
 L^=5\hat{L}{=}5 , L=5L{=}5L^=27\hat{L}{=}27 , L=50L{=}50L^=135\hat{L}{=}135 , L=173L{=}173L^=113\hat{L}{=}113 , L=500L{=}500
MethodAllKnownNovelAllKnownNovelAllKnownNovelAllKnownNovel
Test Time Adaptation
TENT++33.2059.896.5015.4717.0213.9221.5037.485.7714.2520.298.21
BATCLIP++28.9046.7311.0815.5415.3415.7422.5328.9816.1916.0718.8313.30
SAT++23.2734.2012.3313.1913.7112.679.8511.158.5711.4813.179.80
Appendix
Table 14: TT-GCD under estimated LL . Clustering accuracy (%). Best in bold , second best underlined .
 CIFAR-100-CDomainNet
RateAllKnownNovelAllKnownNovel
Known rate η\eta
0.0528.9933.7424.2546.6453.7039.68
0.1029.0734.6223.5146.3954.8738.01
0.3028.2335.4720.9943.6556.7430.70
0.6027.2236.0918.3540.2657.2723.40
Appendix
Table 15: Prototype update rates, varied one at a time around the fixed value. All, Known and Novel (%), mean over CIFAR-100-C corruptions and DomainNet domains.
Figure 5: Activation threshold emin⁡e_{\min} on CIFAR-100-C and DomainNet.

Explore similar work

CardsList

  1. Generalized Category Discovery under Domain Shifts: From Vision to Vision-Language Models

    Apr 29, 2026Hongjun Wang, Po Hu, Kai HanGeneralized Category DiscoveryVision-Language Model Adaptation

    School of Computing and Data Science, The University of Hong Kong

  2. Sparsity Hurts: Simple Linear Adapter Can Boost Generalized Category Discovery

    May 5, 2026Bo Ye, Kai Gan, Tong Wei +1Generalized Category DiscoveryPretraining

    School of Computer Science and Engineering, Southeast University, Nanjing 210096, China, and the Key Laboratory of Computer Network and Information Integration (Southeast University), Ministry of Education, China.

  3. Prototype-Based Test-Time Adaptation of Vision-Language Models

    Apr 23, 2026Zhaohong Huang, Yuxin Zhang, Wenjing Liu +2Vision-Language Model AdaptationTest-Time Adaptation

    Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China.