AcTTA: Rethinking Test-Time Adaptation via Dynamic Activation

Hyeongyu Kim; Geonhui Han; Dosik Hwang

cs.LGMar 27, 2026

AcTTA: Rethinking Test-Time Adaptation via Dynamic Activation

Authors: Hyeongyu Kim, Geonhui Han, Dosik Hwang

Organizations: Yonsei University · Korea Institute of Science and Technology

Abstract

Test-time adaptation (TTA) aims to mitigate performance degradation under distribution shifts by updating model parameters during inference. Existing approaches have primarily framed adaptation around affine modulation, focusing on recalibrating normalization layers. This perspective, while effective, overlooks another influential component in representation dynamics: the activation function. We revisit this overlooked space and propose AcTTA, an activation-aware framework that reinterprets conventional activation functions from a learnable perspective and updates them adaptively at test time. AcTTA reformulates conventional activation functions (e.g., ReLU, GELU) into parameterized forms that shift their response threshold and modulate gradient sensitivity, enabling the network to adjust activation behavior under domain shifts. This functional reparameterization enables continuous adjustment of activation behavior without modifying network weights or requiring source data. Despite its simplicity, AcTTA achieves robust and stable adaptation across diverse corruptions. Across CIFAR10-C, CIFAR100-C, and ImageNet-C, AcTTA consistently surpasses normalization-based TTA methods. Our findings highlight activation adaptation as a compact and effective route toward domain-shift-robust test-time learning, broadening the prevailing affine-centric view of adaptation.

Figures & tables

Figure 1: Visualization of the adaptive activation function. (a) The base activation functions

\phi(x)

. (b) The AcTTA activation output

g(x)

; each parameter set

[\,\cdot,\cdot,\cdot\,]

denotes

[\lambda^{+},\,\lambda^{-},\,c]

. (c) The first derivative

g^{\prime}(x)

with varying

(\lambda^{+},\lambda^{-})

(d) The effect of the shift parameter

c

, which avoids strict zero-centering during adaptation.

Model	$\sim$ 10%	$\sim$ 25%	$\sim$ 50%	$\sim$ 75%	$\sim$ 100%
WRN-40	36.17	35.14	34.05	35.13	44.97
ResNet50	67.67	67.02	65.89	63.45	63.47
ViT-B/16	56.29	51.40	52.47	57.81	65.84

Table 4: Effect of learnable depth ratio in AcTTA across different architectures (WRN on CIFAR100-C; ResNet50/ViT on ImageNet-C). Sky-blue cells denote ranges showing performance improvement.

Method	CIFAR10-C	CIFAR100-C	ImageNet-C
Method	WRN28	WRN40	ViT-B/16
Source	43.52	46.75	62.10
TENT [ 28 ]	30.36	57.35	89.92
$\text{AcTTA}_{{\text{TENT}}}$	29.56	55.03	62.07
ETA [ 24 ]	30.09	57.49	98.37
$\text{AcTTA}_{{\text{ETA}}}$	29.11	57.04	72.70

Table 6: Comparison with state-of-the-art TTA methods on small batch size regime (batch size =4). Reported are average classification errors (%), averaged over 3 runs. Bests are in bold.

Figure 2: Visualizations of adapted activations during AcTTA. AcTTA-ReLU results are obtained from WRN-28 on CIFAR10-C after completing adaptation on the Gaussian noise corruption. AcTTA-GELU results are from ViT-B/16 on ImageNet-C at the same adaptation endpoint. The learned activation exhibits flexible, channel-specific shaping behavior, adapting its form according to the input distribution. The three legend values denote the adapted parameter values of AcTTA,

[\lambda^{+},\,\lambda^{-},\,c]

, corresponding to positive/negative slope modulation and shift.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

	TENT		ETA		DeYO
Method	Base	AcTTA	Base	AcTTA	Base	AcTTA
ImageNet-A	66.13	59.31	63.52	61.33	61.27	57.29
ImageNet-R	48.96	47.18	44.55	42.01	41.68	41.07
PACS	16.75	15.14	18.20	18.09	16.58	13.09

Appendix

Table 10: Additional results on ImageNet-A, ImageNet-R, and PACS.

	CIFAR10-C			CIFAR100-C
Method	TENT	ETA	DEYO	TENT	ETA	DEYO
Base	18.55	18.33	18.44	31.68	31.24	30.41
PReLU	31.73	31.56	31.61	60.97	66.53	62.70
PAU	37.50	36.38	34.81	95.29	91.13	88.56
ACON	18.06	17.92	18.05	30.97	31.51	30.61
AcTTA	17.11	17.00	17.29	30.64	30.99	30.03

Appendix

Table 11: Comparison with alternative activation functions. Error rate (%). Lower is better.

Neural Network Activation Functions

Neural network activation functions shape how models represent nonlinear patterns, learn, remain stable, and map efficiently to hardware.

See all 78 papers

Dual Strategies for Test-Time Adaptation
Apr 19, 2026Nam Nguyen Phuong, Duc Nguyen The Minh, Phi Le Nguyen +2Distribution Shift Robustness Test-Time Adaptation
Institute for AI Innovation and Societal Impact, Hanoi University of Science and Technology · Monash University · Adelaide University
CPATTA: Conformal Supervision Allocation For Active Test-Time Adaptation
Sep 30, 2025Tingyu Shi, Fan Lyu, Haihua Zhu +2Distribution Shift Robustness Test-Time Adaptation
Computer Science and Engineering, University of California San Diego · New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences · College of Computer Science and Electronic Engineering, Hunan University
SPeaR: Test-Time Adaptation with Steering Primitives for Realigning Representations
Sep 21, 2026Muhammad Sudipto Siam Dip, Ali EtemadTest-Time Adaptation Continual Test-Time Adaptation
Department of Electrical and Computer Engineering Queen’s University, Canada

Apr 19, 2026J/K move · Enter open · S save

Dual Strategies for Test-Time Adaptation

Nam Nguyen Phuong, Duc Nguyen The Minh, Phi Le Nguyen, Ehsan Abbasnejad+1

Institute for AI Innovation and Societal Impact, Hanoi University of Science and Technology · Monash University · Adelaide University

Distribution Shift Robustness Test-Time Adaptation

Open paper AskPDF arXiv

Conventional test-time adaptation (TTA) approaches typically adapt the model using only a small fraction of test samples, often those with low-entropy predictions, thereby failing to fully leverage the available information in the test distribution. This paper introduces DualTTA, a novel framework that improves performance under distribution shifts by utilizing a larger and more diverse set of test samples. DualTTA identifies two distinct groups: one where the model's predictions are likely consistent with the underlying semantics, and another where predictions are likely incorrect. For the first group, it minimizes prediction entropy to reinforce reliable decisions; for the second, it maximizes entropy to suppress overconfident errors and unlearn spurious behavior. These groups are adaptively selected using a new reliability criterion that measures prediction stability under both semantic-preserving and semantic-altering transformations, addressing the limitations of purely entropy-based selection. We further provide theoretical analysis and empirical justification showing that our approach enables a tighter separation between reliable and unreliable samples, in the context of their suitability for adaptation, leading to provably more effective model updates.

Abstract

Neural Network Activation Functions

Explore similar work

Dual Strategies for Test-Time Adaptation

CPATTA: Conformal Supervision Allocation For Active Test-Time Adaptation

SPeaR: Test-Time Adaptation with Steering Primitives for Realigning Representations