Test-time adaptation (TTA) aims to mitigate performance degradation under distribution shifts by updating model parameters during inference. Existing approaches have primarily framed adaptation around affine modulation, focusing on recalibrating normalization layers. This perspective, while effective, overlooks another influential component in representation dynamics: the activation function. We revisit this overlooked space and propose AcTTA, an activation-aware framework that reinterprets conventional activation functions from a learnable perspective and updates them adaptively at test time. AcTTA reformulates conventional activation functions (e.g., ReLU, GELU) into parameterized forms that shift their response threshold and modulate gradient sensitivity, enabling the network to adjust activation behavior under domain shifts. This functional reparameterization enables continuous adjustment of activation behavior without modifying network weights or requiring source data. Despite its simplicity, AcTTA achieves robust and stable adaptation across diverse corruptions. Across CIFAR10-C, CIFAR100-C, and ImageNet-C, AcTTA consistently surpasses normalization-based TTA methods. Our findings highlight activation adaptation as a compact and effective route toward domain-shift-robust test-time learning, broadening the prevailing affine-centric view of adaptation.
Figures & tables
Figure 1: Visualization of the adaptive activation function. (a) The base activation functions ϕ(x) . (b) The AcTTA activation output g(x) ; each parameter set [⋅,⋅,⋅] denotes [λ+,λ−,c] . (c) The first derivative g′(x) with varying (λ+,λ−) (d) The effect of the shift parameter c , which avoids strict zero-centering during adaptation.
Model
∼ 10%
∼ 25%
∼ 50%
∼ 75%
∼ 100%
WRN-40
36.17
35.14
34.05
35.13
44.97
ResNet50
67.67
67.02
65.89
63.45
63.47
ViT-B/16
56.29
51.40
52.47
57.81
65.84
Table 4: Effect of learnable depth ratio in AcTTA across different architectures (WRN on CIFAR100-C; ResNet50/ViT on ImageNet-C). Sky-blue cells denote ranges showing performance improvement.
Method
CIFAR10-C
CIFAR100-C
ImageNet-C
WRN28
WRN40
ViT-B/16
Source
43.52
46.75
62.10
TENT [ 28 ]
30.36
57.35
89.92
AcTTATENT
29.56
55.03
62.07
ETA [ 24 ]
30.09
57.49
98.37
AcTTAETA
29.11
57.04
72.70
Table 6: Comparison with state-of-the-art TTA methods on small batch size regime (batch size =4). Reported are average classification errors (%), averaged over 3 runs. Bests are in bold.
Figure 2: Visualizations of adapted activations during AcTTA. AcTTA-ReLU results are obtained from WRN-28 on CIFAR10-C after completing adaptation on the Gaussian noise corruption. AcTTA-GELU results are from ViT-B/16 on ImageNet-C at the same adaptation endpoint. The learned activation exhibits flexible, channel-specific shaping behavior, adapting its form according to the input distribution. The three legend values denote the adapted parameter values of AcTTA, [λ+,λ−,c] , corresponding to positive/negative slope modulation and shift.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
TENT
ETA
DeYO
Method
Base
AcTTA
Base
AcTTA
Base
AcTTA
ImageNet-A
66.13
59.31
63.52
61.33
61.27
57.29
ImageNet-R
48.96
47.18
44.55
42.01
41.68
41.07
PACS
16.75
15.14
18.20
18.09
16.58
13.09
Appendix
Table 10: Additional results on ImageNet-A, ImageNet-R, and PACS.
CIFAR10-C
CIFAR100-C
Method
TENT
ETA
DEYO
TENT
ETA
DEYO
Base
18.55
18.33
18.44
31.68
31.24
30.41
PReLU
31.73
31.56
31.61
60.97
66.53
62.70
PAU
37.50
36.38
34.81
95.29
91.13
88.56
ACON
18.06
17.92
18.05
30.97
31.51
30.61
AcTTA
17.11
17.00
17.29
30.64
30.99
30.03
Appendix
Table 11: Comparison with alternative activation functions. Error rate (%). Lower is better.
Conventional test-time adaptation (TTA) approaches typically adapt the model using only a small fraction of test samples, often those with low-entropy predictions, thereby failing to fully leverage the available information in the test distribution. This paper introduces DualTTA, a novel framework that improves performance under distribution shifts by utilizing a larger and more diverse set of test samples. DualTTA identifies two distinct groups: one where the model's predictions are likely consistent with the underlying semantics, and another where predictions are likely incorrect. For the first group, it minimizes prediction entropy to reinforce reliable decisions; for the second, it maximizes entropy to suppress overconfident errors and unlearn spurious behavior. These groups are adaptively selected using a new reliability criterion that measures prediction stability under both semantic-preserving and semantic-altering transformations, addressing the limitations of purely entropy-based selection. We further provide theoretical analysis and empirical justification showing that our approach enables a tighter separation between reliable and unreliable samples, in the context of their suitability for adaptation, leading to provably more effective model updates.
Nam Nguyen Phuong, Duc Nguyen The Minh, Phi Le Nguyen +2
Institute for AI Innovation and Societal Impact, Hanoi University of Science and Technology · Monash University · Adelaide University
Active Test-Time Adaptation (ATTA) improves model robustness under domain shift by selectively querying human annotations at deployment, but existing methods use heuristic uncertainty measures and suffer from low data selection efficiency, wasting human annotation budget. We propose Conformal Prediction Active TTA (CPATTA), which first brings principled, conformal uncertainty with coverage-aware online calibration into ATTA. CPATTA employs smoothed conformal scores with a top-K certainty measure, an online weight-update algorithm driven by pseudo coverage, a domain-shift detector that adapts human supervision, and a staged update scheme that balances human-labeled and model-labeled data. Extensive experiments demonstrate that CPATTA consistently outperforms the state-of-the-art ATTA methods by around 5% in accuracy.
Tingyu Shi, Fan Lyu, Haihua Zhu +2
Computer Science and Engineering, University of California San Diego · New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences · College of Computer Science and Electronic Engineering, Hunan University
Test-time adaptation (TTA) addresses distribution shift using only unlabeled test data. Existing methods typically adapt pretrained models by updating their parameters, limiting both what is adapted and where adaptation can occur within the network. We instead keep the pretrained network frozen and steer its intermediate representations. We introduce SPeaR (Steering Primitive for Realigning Representations), which inserts lightweight learnable modules at stage boundaries and optimizes them directly from the test stream, requiring neither source data nor supervised warm-up. Each primitive is optimized using a gated objective that reduces uncertainty only when adaptation is beneficial, along with a diversity regularizer to prevent collapse, and a multi-depth anchor to stabilize adaptation. We show that steering early representations is the most effective strategy, and that the same primitive transfers across convolutional and Transformer architectures. Across CIFAR-10-C, CIFAR-100-C, and ImageNet-C, SPeaR consistently matches or outperforms methods that adapt orders of magnitude more parameters, remains robust across a wide range of batch sizes, and preserves source-domain performance during continual adaptation.
Muhammad Sudipto Siam Dip, Ali Etemad
Department of Electrical and Computer Engineering Queen’s University, Canada