Uncertainty-Aware Prototype Learning for Open-Set Test-Time Adaptation
Authors: Jinwu Hu, Langyu Bian, Jian Sun, Dongjin Yang, Chao Shuai, Yufeng Wang, Kai Zhou, Bin Xiao, Mingkui Tan, Hongmin Cai, Wei Luo
Organizations: School of Software Engineering, South China University of Technology, Guangzhou, China · Zhejiang University, Hangzhou, China · School of Future Technology, South China University of Technology, Guangzhou, China · Chongqing University of Posts and Telecommunications, Chongqing, China · School of Software Engineering, South China University of Technology, and also with the Key Laboratory of Big Data and Intelligent Robot, Ministry of Education, Guangzhou, China · South China Agricultural University
Open-set test-time adaptation (TTA) aims to adapt source-trained models to unlabeled test streams where covariate-shifted in-distribution (csID) samples and covariate-shifted out-of-distribution (csOOD) samples may coexist. Existing TTA methods are typically designed under a closed-set assumption and therefore lack explicit mechanisms for reliably distinguishing csID from csOOD samples. Consequently, csOOD samples may be mistakenly included in confidence-enhancing adaptation, producing harmful gradients that degrade known-class performance. To address this, we propose Uncertainty-Aware Prototype Learning for Open-Set Test-Time Adaptation (UAPA), a plug-and-play method for reliable open-set adaptation. Specifically, we first introduce Uncertainty-Driven ID/OOD Separation, which combines a classifier-prototype score with a mini-batch-normalized evidence-derived uncertainty score to distinguish csID from csOOD samples. We then propose a Dynamic Prototype Refinement strategy that periodically refines class-specific target prototypes using buffered csID features to progressively follow the estimated class directions. Lastly, we introduce Contrastive Prototype Entropy Adaptation, which applies entropy minimization to csID samples and entropy maximization to csOOD samples, while prototype-level contrastive learning aligns csID features with their corresponding target prototypes and reduces the maximum similarity between csOOD features and known-class prototypes. Empirically, UAPA outperforms the strongest baselines on the CIFAR benchmarks and Tiny-ImageNet-C, improving average OSCR by 2.08 and 1.28 percentage points, respectively.
Figures & tables
Fig. 1 : OSCR comparison on CIFAR-100-C before and after incorporating UAPA into representative TTA methods [ 18 , 19 , 20 , 21 ] . Values above the arrows denote absolute OSCR gains in percentage points.
Fig. 2 : Overview of UAPA. For each incoming batch Bt , UAPA combines the classifier-prototype score Sproto with the mini-batch-normalized evidence-derived uncertainty score SEU to compute a separation score S(xt) , which partitions samples into csID and csOOD subsets using threshold τ . Contrastive Prototype Entropy Adaptation applies entropy minimization to csID samples, entropy maximization to csOOD samples, and prototype-level contrastive learning to regularize feature representations. Dynamic Prototype Refinement periodically updates class-specific target prototypes using buffered csID features.
Method
CIFAR-10-C
CIFAR-100-C
Average
Acc ↑
AUROC ↑
FPR@95TPR ↓
OSCR ↑
Acc ↑
AUROC ↑
FPR@95TPR ↓
OSCR ↑
Acc ↑
AUROC ↑
FPR@95TPR ↓
OSCR ↑
Source
81.73
77.89
79.47
68.44
53.25
60.55
94.97
39.87
67.49
69.22
87.22
54.16
BN Adapt [ 57 ]
84.20
80.40
76.85
72.13
57.16
72.45
84.29
47.10
70.68
76.43
80.57
59.62
CoTTA [ 58 ]
85.74
85.81
72.77
77.17
56.50
77.06
80.98
49.00
71.12
81.44
76.88
63.09
TENT [ 18 ]
79.12
67.79
95.76
58.27
50.19
59.42
93.54
36.25
64.66
63.61
94.65
47.26
EATA [ 19 ]
76.17
55.43
92.88
46.66
61.34
87.05
56.06
57.27
68.76
71.24
74.47
51.97
TABLE I : Performance comparison on CIFAR-10-C and CIFAR-100-C using a WideResNet-40-2 backbone. ↑ indicates higher is better, ↓ lower is better. Best in bold and second best in underlined . ‘ Ours ’ denotes TENT with our UAPA.
Method
ResNet-50
ViT-Base
Average
Acc ↑
AUROC ↑
FPR@95TPR ↓
OSCR ↑
Acc ↑
AUROC ↑
FPR@95TPR ↓
OSCR ↑
Acc ↑
AUROC ↑
FPR@95TPR ↓
OSCR ↑
Source
22.36
48.11
90.80
15.81
22.09
48.45
94.85
16.29
22.23
48.28
92.83
16.05
BN Adapt [ 57 ]
33.79
56.43
87.80
25.60
-
-
-
-
-
-
-
-
CoTTA [ 58 ]
32.68
47.60
92.27
22.45
16.17
40.99
95.38
10.30
24.43
44.30
93.83
16.38
TENT [ 18 ]
25.45
52.22
94.29
17.48
26.45
52.70
95.13
19.65
25.95
52.46
94.71
18.57
EATA [ 19 ]
34.46
55.86
92.52
25.92
27.95
52.61
95.14
21.31
31.21
54.24
93.83
23.62
TABLE II : Performance comparison on Tiny-ImageNet-C across different backbones (ResNet-50 and ViT-Base). ‘-’ indicates the experiment was not applicable. Best in bold and second best in underlined . ‘ Ours ’ is TENT with our UAPA.
Method
CIFAR-10-C
CIFAR-100-C
Average
Acc ↑
AUROC ↑
FPR@95TPR ↓
OSCR ↑
Acc ↑
AUROC ↑
FPR@95TPR ↓
OSCR ↑
Acc ↑
AUROC ↑
FPR@95TPR ↓
OSCR ↑
TENT
79.12
67.79
95.76
58.27
50.19
59.42
93.54
36.25
64.66
63.61
94.65
47.26
w/o PS
78.46
85.40
36.21
69.22
53.60
90.14
27.18
50.16
66.03
87.77
31.70
59.69
w/o UE
84.87
89.88
45.13
79.10
56.96
94.33
26.07
55.51
70.92
92.11
35.60
67.31
w/o DPR
84.35
93.57
28.10
80.95
59.64
86.75
51.70
55.32
72.00
90.16
39.90
68.14
w/o CL
84.38
93.28
26.05
80.66
60.03
89.58
39.40
56.59
72.21
91.43
32.73
68.63
TABLE III : Ablation experiments on CIFAR-10-C and CIFAR-100-C using a WideResNet-40-2 backbone. ‘PS’ denotes Prototype-based Similarity, ‘UE’ denotes Uncertainty Estimation, ‘DPR’ denotes Dynamic Prototype Refinement strategy, and ‘CL’ denotes Contrastive Learning Optimization.
Method
Acc ↑
AUROC ↑
FPR@95TPR ↓
OSCR ↑
Source
66.81
46.75
93.99
36.13
BN Adapt [ 57 ]
72.70
62.06
91.30
52.15
CoTTA [ 58 ]
72.74
61.65
89.96
51.42
TENT [ 18 ]
73.68
61.16
94.77
51.55
EATA [ 19 ]
73.16
61.34
87.49
50.52
SAR [ 22 ]
73.53
62.01
91.86
52.63
TABLE IV : Results under the small-gap classes setting, where CIFAR-10/CIFAR-100 classes are split into 8:2 (ID: OOD). This setting poses a more challenging scenario.
Method
CIFAR-10-C
CIFAR-100-C
Average
Acc ↑
AUROC ↑
FPR@95TPR ↓
OSCR ↑
Acc ↑
AUROC ↑
FPR@95TPR ↓
OSCR ↑
Acc ↑
AUROC ↑
FPR@95TPR ↓
OSCR ↑
Source
81.73
77.89
79.47
68.44
53.25
60.55
94.97
39.87
67.49
69.22
87.22
54.16
TENT [ 18 ]
79.12
67.79
95.76
58.27
50.19
59.42
93.54
36.25
64.66
63.61
94.65
47.26
Ours (w/ GMM [ 15 ] )
84.45
95.67
18.16
82.12
59.20
92.30
33.64
56.72
71.83
93.99
25.90
69.42
UAPA (Ours)
84.53
96.18
13.57
82.27
60.14
93.26
27.59
58.08
72.34
94.72
20.58
70.18
TABLE V : Performance comparison of different ID/OOD separation strategies on CIFAR-10-C and CIFAR-100-C using a WideResNet-40-2 backbone. “Ours (w/ GMM)” replaces the proposed Uncertainty-Driven ID/OOD Separation with a UniEnt-style GMM gate, while keeping all other components unchanged.
Fig. 3 : t-SNE visualization of CIFAR-10-C and SVHN-C features, where SVHN-C samples are treated as csOOD. Yellow denotes csOOD samples, while the remaining colors denote different csID classes.
Fig. 4 : OSCR comparison on CIFAR-10-C under varying csOOD ratios. Values above UAPA denote its gains in percentage points over the strongest baseline at each ratio.
Method
CIFAR-10-C
CIFAR-100-C
Average
Acc ↑
AUROC ↑
FPR@95TPR ↓
OSCR ↑
Acc ↑
AUROC ↑
FPR@95TPR ↓
OSCR ↑
Acc ↑
AUROC ↑
FPR@95TPR ↓
OSCR ↑
TENT
79.12
67.79
95.76
58.27
50.19
59.42
93.54
36.25
64.66
63.61
94.65
47.26
+ UAPA
84.53
96.18
13.57
82.27
60.14
93.26
27.59
58.08
72.34
94.72
20.58
70.18
EATA
76.17
55.43
92.88
46.66
61.34
87.05
56.06
57.27
68.76
71.24
74.47
51.97
+ UAPA
84.69
92.75
27.24
80.23
61.51
92.16
34.30
59.07
73.10
92.46
30.77
69.65
COME
85.31
75.34
84.01
69.29
58.09
72.28
79.24
47.49
71.70
73.81
81.63
58.39
TABLE VI : Plug-and-play performance of UAPA on CIFAR-10-C, CIFAR-100-C, and their average results.
Method
Acc ↑
AUROC ↑
FPR@95TPR ↓
OSCR ↑
Time (ms) ↓
TENT [ 18 ]
25.45
52.22
94.29
17.48
-
+ UAPA
33.85
67.47
81.06
28.10
+0.22
EATA [ 19 ]
34.46
55.86
92.52
25.92
-
+ UAPA
35.47
56.94
88.72
27.43
+0.26
OSTTA [ 20 ]
34.75
53.00
95.29
25.54
-
+ UAPA
34.87
53.34
92.95
25.68
+0.31
TABLE VII : Results of different methods on Tiny-ImageNet-C with ResNet-50 backbone. We report Accuracy (Acc), AUROC, FPR@95TPR, OSCR, and the per-image additional inference time introduced by UAPA (in milliseconds).
Method
Acc ↑
AUROC ↑
FPR@95TPR ↓
OSCR ↑
Source
22.09
48.45
94.85
16.29
BN Adapt [ 57 ]
22.09
51.96
94.52
16.54
CoTTA [ 58 ]
15.13
42.04
95.45
9.40
TENT [ 18 ]
14.86
47.92
96.05
10.07
EATA [ 19 ]
22.26
51.98
94.64
16.64
SAR [ 22 ]
22.24
51.76
94.58
16.50
TABLE VIII : Results on Tiny-ImageNet-C with ViT-Base under the single-class-per-batch setting.
Fig. 5 : Effect of key hyperparameters on OSCR on Tiny-ImageNet-C with ResNet-50 and ViT-Base backbones. We vary one hyperparameter at a time while keeping the others fixed at their selected values.