Test-time adaptation (TTA) aims to improve model robustness under distribution shift by adapting a source model using unlabeled test data. Although methods such as TENT and EATA have demonstrated gains on corrupted data, aggregate accuracy can obscure the conditions under which adaptation fails or provides little benefit. We present a controlled comparison of three TTA strategies---BatchNorm-statistics adaptation (BN-Adapt), entropy-minimization adaptation (TENT), and reliability-filtered adaptation (a scoped re-implementation of EATA)---against an unadapted source model on the full CIFAR-10-C benchmark, covering 15 corruption types and 5 severity levels. All three methods improve mean accuracy over the source model by 12.2--13.3 percentage points (Wilcoxon signed-rank p<10−12). However, each method underperforms the source model on 8.0--9.3% of conditions, with failures concentrated in low-severity corruptions where the source model already performs near ceiling, particularly brightness, fog, contrast, and defocus blur. We further find that EATA closely tracks the gradient-free BN-Adapt baseline, with a mean absolute difference of 0.09 percentage points, compared with 1.08 percentage points relative to TENT. This suggests that reliability filtering can substantially restrict effective adaptation, causing EATA to behave more like a BatchNorm-statistics baseline than an entropy-minimization method. These results show that aggregate accuracy alone can mask systematic TTA failure modes and motivate condition-level evaluation of when adaptation helps, harms, or becomes effectively inactive.
Figures & tables
Severity
Source
BN-adapt
TENT
EATA
1
86.90
90.37
90.87
90.41
2
81.34
88.33
89.12
88.39
3
74.92
86.50
87.55
86.55
4
67.64
83.38
84.79
83.48
5
56.49
79.56
81.42
79.69
Table 1. Mean classification accuracy by corruption severity, averaged across all 15 corruption types (75 conditions total, 10,000 images per condition).
Figure 1. Mean accuracy versus corruption severity for the source model and each adaptation method. All three adaptation methods track closely and diverge from the source model as severity increases. Line chart of mean accuracy against severity levels one through five for Source, BN-adapt, TENT, and EATA.
Method
Mean Δ vs. Source (pp)
95% CI (bootstrap)
Wilcoxon p -value
BN-adapt
12.17
[9.40, 15.10]
5.3×10−13
TENT
13.29
[10.28, 16.29]
3.8×10−13
EATA
12.25
[9.41, 15.21]
4.9×10−13
Table 2. Mean accuracy improvement over source, bootstrap 95% confidence interval, and paired Wilcoxon signed-rank test p -value, computed over the 75 corruption–severity conditions.
Category
BN-adapt
TENT
EATA
Noise
25.94
27.95
26.06
Blur
14.40
15.42
14.46
Digital
7.26
8.37
7.33
Weather
4.55
5.10
4.59
Table 3. Mean accuracy improvement over source by corruption category, averaged across constituent corruption types and all five severities.
Severity bucket
BN-adapt
TENT
EATA
Low (1–2)
5.23
5.87
5.28
Mid (3)
11.58
12.63
11.63
High (4–5)
19.41
21.04
19.52
Table 4. Mean accuracy improvement over source by severity bucket, averaged across all 15 corruption types.
Figure 2. Mean accuracy gain over source by corruption category. Noise corruptions benefit most; weather corruptions benefit least and contain most conditions where adaptation underperforms source. Grouped bar chart of accuracy gains for BN-adapt, TENT, and EATA across blur, digital, noise, and weather corruption categories.
Method
Harm rate
Conditions harmed
BN-adapt
9.3%
7/75
TENT
8.0%
6/75
EATA
9.3%
7/75
Table 5. Share of corruption–severity conditions where each method’s accuracy falls below the unadapted source model.
Figure 3. Per-corruption, per-severity accuracy delta versus source (percentage points) for BN-adapt (left), TENT (center), and EATA (right). Dark green indicates large gains; orange or yellow near the color-scale midpoint indicates conditions near zero or negative gain, concentrated in the brightness, fog, and contrast rows at low severity. Three heatmaps showing accuracy differences from the source model by corruption type and severity for BN-adapt, TENT, and EATA.
Model or method
Accuracy (%)
Source
27.08
BN-adapt
63.73
TENT
66.96
EATA
63.96
Table 6. Accuracy on the single hardest corruption–severity condition in the benchmark (impulse noise, severity 5).
Batch size
Source
BN-adapt
TENT
EATA
32
68.51
82.27
83.82
82.28
128
68.51
83.35
85.06
83.39
512
68.51
83.62
84.77
83.80
Table 7. Mean accuracy by test batch size, averaged across six corruptions (Gaussian noise, motion blur, fog, contrast, JPEG compression, and snow) at severities 3 and 5.
Figure 4. Mean accuracy versus test batch size (base-2 logarithmic scale). BN-adapt and EATA rise monotonically; TENT peaks at batch size 128 and dips slightly at 512. Line chart of mean accuracy against batch sizes 32, 128, and 512 for the source model and three adaptation methods.
Corruption
Method
First occurrence (%)
Second occurrence (%)
Δ (pp)
Gaussian noise
BN-adapt
72.46
72.46
0.00
Gaussian noise
TENT
74.37
76.37
+2.00
Gaussian noise
EATA
72.46
72.46
0.00
Motion blur
BN-adapt
85.94
85.94
0.00
Motion blur
TENT
86.52
86.67
+0.15
Motion blur
EATA
86.04
86.04
0.00
Table 8. Mean batch accuracy on the first versus second occurrence of the same corruption within the 256-batch continual stream, separated by 160 intervening batches of adaptation on other corruptions.
Figure 5. Rolling-mean accuracy across the 256-batch continual adaptation stream with no state reset. Sharp swings track corruption-block boundaries rather than a sustained drift; no method shows a persistent downward trend. Line chart of rolling accuracy across 256 batches for BN-adapt, TENT, and EATA.
The University of Texas at Dallas, USA · LIVIA ETS Montreal, ILLS International Laboratory on Learning Systems (ILLS), McGill - ÉTS - MILA - CNRS - Université Paris-Saclay - CentraleSupélec · Tulane University, USA +2