Organizations: School of Electrical and Electronic Engineering, Yonsei University · Hyundai Motor Company · Department of Radiology and Center for Clinical Imaging Data Science, College of Medicine, Yonsei University · Artificial Intelligence and Robotics Institute, Korea Institute of Science and Technology
Test-time adaptation (TTA) updates a deployed model using only unlabeled target data, but standard entropy-based methods often rely on deterministic confidence and can reinforce overconfident errors under distribution shift. We propose MIRA, Mutual Information guided calibration for Reliable Test-Time Adaptation, a source-free online adaptation method that uses lightweight last-layer Gaussian perturbations to probe prediction reliability. For each target batch, MIRA reuses a single feature extraction pass and samples perturbed classifiers around the source classifier, forming an efficient local predictive distribution. Rather than fitting a Bayesian posterior, MIRA treats the perturbation scale as a local probing radius and selects a batch-specific scale using mutual information (MI) and prediction agreement. The resulting perturbation-sensitivity signal identifies predictions that are confident and locally stable under classifier perturbations, while suppressing brittle predictions with high stochastic disagreement. MIRA then performs adaptation with an MI-confidence weighted entropy loss, which down-weights perturbation-sensitive or low-confidence samples before they drive entropy minimization. This weighted loss reduces the influence of brittle predictions that can otherwise cause harmful adaptation under distribution shift. The method requires no source data, retraining, model ensembles, or offline posterior fitting, and can be applied as a plug-and-play modification to entropy-minimization TTA. Experiments on distribution-shift benchmarks show improvements in accuracy, calibration, and negative log-likelihood, suggesting that mutual-information-guided perturbation probing and weighted adaptation yield more robust online TTA than deterministic confidence.
Figures & tables
Figure 1: MIRA overview. (a) Cached features support Gaussian classifier probes with shared noise across candidate radii. At the selected σ∗ , fresh Monte Carlo samples supply detached MI–confidence weights and a stochastic entropy update through one backward pass. (b) Fixed-scale annotations report measured results; MI/agreement curves and eligibility bars schematically illustrate radius selection. HE and EH denote entropy of the mean prediction and mean predictive entropy.
Figure 2: Online accuracy and confidence during TENT and TENT+MIRA on ImageNet-C fog (severity 5, EMA coefficient 0.01). MIRA narrows the gap in both (a) the overconfident ResNet-34 and (b) the underconfident EfficientNet-B0. Shading and bars show the absolute accuracy–confidence gap, rather than ECE.
Figure 3: Confidence–accuracy plane; the diagonal marks confidence = accuracy. (a) Architectures, (b) MIRA on base TTA methods, (c) γ variants of MIRA TENT on ImageNet-C / ResNet-50.
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
Objective
Acc. (%) ↑
MMC
ECE ↓
NLL ↓
CIFAR-10-C / WRN; Adam, four corruptions, K=100
TENT
87.79\kern 0.2pt\hbox{\pm 0.05}
0.9412\kern 0.2pt\hbox{\pm 0.0005}
0.0635\kern 0.2pt\hbox{\pm 0.0006}
0.4462\kern 0.2pt\hbox{\pm 0.0025}
HE+EH
88.14\kern 0.2pt\hbox{\pm 0.05}
0.9221\kern 0.2pt\hbox{\pm 0.0008}
0.0409\kern 0.2pt\hbox{\pm 0.0011}
0.3905\kern 0.2pt\hbox{\pm 0.0032}
+ MI gate
88.34\kern 0.2pt\hbox{\pm 0.10}
0.9236\kern 0.2pt\hbox{\pm 0.0007}
0.0403\kern 0.2pt\hbox{\pm 0.0015}
0.3838\kern 0.2pt\hbox{\pm 0.0027}
+ confidence gate
88.29\kern 0.2pt\hbox{\pm 0.09}
0.9229\kern 0.2pt\hbox{\pm 0.0008}
0.0402\kern 0.2pt\hbox{\pm 0.0016}
0.3848\kern 0.2pt\hbox{\pm 0.0025}
Full MIRA TENT
\mathbf{88.42}\kern 0.2pt\hbox{\pm 0.07}
0.9235\kern 0.2pt\hbox{\pm 0.0007}
\mathbf{0.0396}\kern 0.2pt\hbox{\pm 0.0011}
\mathbf{0.3808}\kern 0.2pt\hbox{\pm 0.0023}
Appendix
Table 6: Gradient-matched loss and gating controls on CIFAR-10-C and CIFAR-100-C: global adaptation-gradient L2 norm 1, mean ± sample SD over three seeds. Panels use distinct protocols and are separate from ordinary adaptation. Bold marks best accuracy, ECE, and NLL within each panel.
Objective
Acc. (%) ↑
MMC
ECE ↓
NLL ↓
TENT
87.77\kern 0.2pt\hbox{\pm 0.06}
0.9407\kern 0.2pt\hbox{\pm 0.0005}
0.0634\kern 0.2pt\hbox{\pm 0.0008}
0.4449\kern 0.2pt\hbox{\pm 0.0029}
HE+EH
88.15\kern 0.2pt\hbox{\pm 0.06}
0.9218\kern 0.2pt\hbox{\pm 0.0007}
0.0405\kern 0.2pt\hbox{\pm 0.0011}
0.3902\kern 0.2pt\hbox{\pm 0.0038}
+ MI gate
88.32\kern 0.2pt\hbox{\pm 0.04}
0.9231\kern 0.2pt\hbox{\pm 0.0008}
0.0401\kern 0.2pt\hbox{\pm 0.0011}
0.3834\kern 0.2pt\hbox{\pm 0.0026}
+ confidence gate
88.30\kern 0.2pt\hbox{\pm 0.10}
0.9225\kern 0.2pt\hbox{\pm 0.0008}
0.0398\kern 0.2pt\hbox{\pm 0.0016}
0.3843\kern 0.2pt\hbox{\pm 0.0023}
Full MIRA TENT
\mathbf{88.39}\kern 0.2pt\hbox{\pm 0.02}
0.9232\kern 0.2pt\hbox{\pm 0.0007}
\mathbf{0.0396}\kern 0.2pt\hbox{\pm 0.0004}
\mathbf{0.3800}\kern 0.2pt\hbox{\pm 0.0029}
Appendix
Table 7: Unnormalized CIFAR-10-C objective and gating controls, mean ± sample SD over three seeds. The Adam/four-corruption/ K=100 protocol matches the corresponding normalized panel. Bold marks best accuracy, ECE, and NLL.
Construction
Acc. (%) ↑
MMC
ECE ↓
NLL ↓
TENT
63.63\kern 0.2pt\hbox{\pm 0.08}
0.7184\kern 0.2pt\hbox{\pm 0.0005}
0.0824\kern 0.2pt\hbox{\pm 0.0008}
1.4078\kern 0.2pt\hbox{\pm 0.0044}
Feature-wise
\mathbf{64.33}\kern 0.2pt\hbox{\pm 0.10}
0.6787\kern 0.2pt\hbox{\pm 0.0001}
0.0387\kern 0.2pt\hbox{\pm 0.0009}
1.3426\kern 0.2pt\hbox{\pm 0.0061}
Scalar
64.29\kern 0.2pt\hbox{\pm 0.15}
0.6589\kern 0.2pt\hbox{\pm 0.0006}
\mathbf{0.0296}\kern 0.2pt\hbox{\pm 0.0007}
1.3414\kern 0.2pt\hbox{\pm 0.0062}
Class-wise
64.28\kern 0.2pt\hbox{\pm 0.15}
0.6590\kern 0.2pt\hbox{\pm 0.0007}
0.0297\kern 0.2pt\hbox{\pm 0.0008}
\mathbf{1.3410}\kern 0.2pt\hbox{\pm 0.0062}
Appendix
Table 8: Ablation on the perturbation-scale parameterization on CIFAR-100-C. Mean ± sample SD over seeds 1, 2, and 3; WRN AugMix, K=100 , SGD 10−3 . Bold marks best accuracy, ECE, and NLL.
Construction
Acc. (%) ↑
MMC
ECE ↓
NLL ↓
TENT
63.63\kern 0.2pt\hbox{\pm 0.08}
0.7184\kern 0.2pt\hbox{\pm 0.0005}
0.0824\kern 0.2pt\hbox{\pm 0.0008}
1.4078\kern 0.2pt\hbox{\pm 0.0044}
He-in
\mathbf{64.33}\kern 0.2pt\hbox{\pm 0.10}
0.6787\kern 0.2pt\hbox{\pm 0.0001}
0.0387\kern 0.2pt\hbox{\pm 0.0009}
1.3426\kern 0.2pt\hbox{\pm 0.0061}
Fan-in
64.27\kern 0.2pt\hbox{\pm 0.12}
0.6599\kern 0.2pt\hbox{\pm 0.0001}
0.0206\kern 0.2pt\hbox{\pm 0.0015}
\mathbf{1.3326}\kern 0.2pt\hbox{\pm 0.0052}
Absolute
64.19\kern 0.2pt\hbox{\pm 0.15}
0.6440\kern 0.2pt\hbox{\pm 0.0001}
\mathbf{0.0152}\kern 0.2pt\hbox{\pm 0.0006}
1.3332\kern 0.2pt\hbox{\pm 0.0050}
Weight magnitude
64.23\kern 0.2pt\hbox{\pm 0.13}
0.6559\kern 0.2pt\hbox{\pm 0.0003}
0.0246\kern 0.2pt\hbox{\pm 0.0009}
1.3443\kern 0.2pt\hbox{\pm 0.0047}
Appendix
Table 9: Ablation on the base perturbation scale rule on CIFAR-100-C. Mean ± sample SD over seeds 1, 2, and 3; WRN AugMix, K=100 , SGD 10−3 . Bold marks best accuracy, ECE, and NLL.
Scope
Acc. (%) ↑
MMC
ECE ↓
NLL ↓
Time (s) ↓
VRAM (MiB) ↓
Params
Classifier only
37.04
0.4013
0.0458
3.2690
379.9
8180.4
513,000
Last 2 layers
37.15
0.3576
0.0519
3.2372
14,818.9
8212.3
2,872,296
Last 3 layers
37.31
0.3592
0.0649
3.2332
21,572.5
8212.3
5,231,592
Appendix
Table 10: Effect of perturbation scope on ImageNet-C / ResNet-34 (batch size 64), retained single-run results. Bold marks the best values in directional columns.
K
Acc. (%) ↑
ECE ↓
NLL ↓
Time (s) ↓
Peak VRAM (MiB) ↓
100
29.55
0.0410
3.6454
60
5,489
200
29.55
0.0408
3.6407
66
7,175
300
29.39
0.0420
3.6374
74
8,671
500
29.29
0.0443
3.6511
88
11,939
1000
29.76
0.0381
3.6387
116
22,953
Appendix
Table 11: Effect of the Monte Carlo sampling budget K on ImageNet-C / ResNet-50, retained single-run results. Bold marks the best values in directional columns.
Setting
Acc. (%) ↑
MMC
ECE ↓
NLL ↓
CIFAR-100-C / ResNeXt
α=0.01,ρ=0.85
68.58\kern 0.2pt\hbox{\pm 0.11}
0.7524\kern 0.2pt\hbox{\pm 0.0009}
0.0675\kern 0.2pt\hbox{\pm 0.0010}
1.2197\kern 0.2pt\hbox{\pm 0.0073}
α=0.01,ρ=0.90
68.58\kern 0.2pt\hbox{\pm 0.12}
0.7522\kern 0.2pt\hbox{\pm 0.0009}
0.0675\kern 0.2pt\hbox{\pm 0.0012}
1.2196\kern 0.2pt\hbox{\pm 0.0074}
α=0.01,ρ=0.95
68.59\kern 0.2pt\hbox{\pm 0.13}
0.7499\kern 0.2pt\hbox{\pm 0.0009}
0.0651\kern 0.2pt\hbox{\pm 0.0010}
1.2159\kern 0.2pt\hbox{\pm 0.0084}
α=0.03,ρ=0.85
68.64\kern 0.2pt\hbox{\pm 0.13}
0.7396\kern 0.2pt\hbox{\pm 0.0011}
0.0542\kern 0.2pt\hbox{\pm 0.0009}
1.2032\kern 0.2pt\hbox{\pm 0.0083}
α=0.03,ρ=0.90
68.69\kern 0.2pt\hbox{\pm 0.15}
0.7305\kern 0.2pt\hbox{\pm 0.0012}
0.0452\kern 0.2pt\hbox{\pm 0.0007}
1.1940\kern 0.2pt\hbox{\pm 0.0111}
Appendix
Table 12: Joint α – ρ sensitivity, mean ± sample SD over three seeds. The common default is (0.03,0.90) ; test labels are used only for evaluation. Bold marks best accuracy, ECE, and NLL within each dataset; displayed ties are included.
Setting
Acc. (%) ↑
MMC
ECE ↓
NLL ↓
CIFAR-100-C / ResNeXt
γ=0
67.91\kern 0.2pt\hbox{\pm 0.11}
0.7144\kern 0.2pt\hbox{\pm 0.0006}
\mathbf{0.0384}\kern 0.2pt\hbox{\pm 0.0003}
1.2209\kern 0.2pt\hbox{\pm 0.0064}
γ=0.25
68.22\kern 0.2pt\hbox{\pm 0.11}
0.7198\kern 0.2pt\hbox{\pm 0.0008}
0.0402\kern 0.2pt\hbox{\pm 0.0007}
1.2094\kern 0.2pt\hbox{\pm 0.0069}
γ=0.5
68.45\kern 0.2pt\hbox{\pm 0.12}
0.7241\kern 0.2pt\hbox{\pm 0.0010}
0.0420\kern 0.2pt\hbox{\pm 0.0006}
1.2022\kern 0.2pt\hbox{\pm 0.0083}
γ=1
68.69\kern 0.2pt\hbox{\pm 0.15}
0.7305\kern 0.2pt\hbox{\pm 0.0012}
0.0452\kern 0.2pt\hbox{\pm 0.0007}
\mathbf{1.1940}\kern 0.2pt\hbox{\pm 0.0111}
γ=2
\mathbf{68.87}\kern 0.2pt\hbox{\pm 0.11}
0.7391\kern 0.2pt\hbox{\pm 0.0011}
0.0517\kern 0.2pt\hbox{\pm 0.0009}
1.1984\kern 0.2pt\hbox{\pm 0.0064}
Appendix
Table 13: Entropy-coefficient sensitivity, mean ± sample SD over three seeds. The coefficient of EH changes independently of confidence weighting. Bold marks best accuracy, ECE, and NLL within each dataset.
Grid
Acc. (%) ↑
MMC
ECE ↓
NLL ↓
CIFAR-100-C / ResNeXt
Default (16)
\mathbf{68.69}\kern 0.2pt\hbox{\pm 0.15}
0.7305\kern 0.2pt\hbox{\pm 0.0012}
0.0452\kern 0.2pt\hbox{\pm 0.0007}
1.1940\kern 0.2pt\hbox{\pm 0.0111}
Coarse (9)
68.54\kern 0.2pt\hbox{\pm 0.15}
0.6570\kern 0.2pt\hbox{\pm 0.0011}
0.0299\kern 0.2pt\hbox{\pm 0.0004}
1.1859\kern 0.2pt\hbox{\pm 0.0091}
Dense (31)
68.68\kern 0.2pt\hbox{\pm 0.16}
0.7317\kern 0.2pt\hbox{\pm 0.0011}
0.0463\kern 0.2pt\hbox{\pm 0.0011}
1.1938\kern 0.2pt\hbox{\pm 0.0115}
Extended (20)
68.56\kern 0.2pt\hbox{\pm 0.17}
0.6603\kern 0.2pt\hbox{\pm 0.0014}
\mathbf{0.0273}\kern 0.2pt\hbox{\pm 0.0010}
\mathbf{1.1838}\kern 0.2pt\hbox{\pm 0.0093}
ImageNet-C / ResNet-50
Appendix
Table 14: Scale-grid sensitivity, mean ± sample SD over three seeds. Full MIRA and the headline dataset-specific TENT protocols are fixed. Bold marks best accuracy, ECE, and NLL within each dataset.
Backbone
Method
α
ρ
Acc. (%)
MMC
ECE
NLL
VGG-19-BN
Source
–
–
13.24
0.3653
0.2329
5.5392
VGG-19-BN
TENT
–
–
34.60
0.4578
0.1118
3.4309
VGG-19-BN
MIRA
0.10
0.90
34.69
0.3359
0.0281
3.3540
ResNet-50
Source
–
–
16.95
0.3945
0.2250
5.2586
ResNet-50
TENT
–
–
39.55
0.4350
0.0395
2.9735
ResNet-50
MIRA
0.01
0.90
39.67
0.4231
0.0266
2.9709
Appendix
Table 15: Numerical values and selected settings for the cross-architecture diagnostic; single-run ImageNet-C snow diagnostics. Bold marks best accuracy, ECE, and NLL per backbone.
Protocol
Method
Acc. (%) ↑
MMC
ECE ↓
NLL ↓
Continual
CoTTA
64.16
0.6170
0.0529
1.4173
MIRA CoTTA
65.20
0.6787
0.0342
1.3459
Dynamic
TENT
14.75
0.9675
0.8201
17.6335
MIRA TENT
15.16
0.9650
0.8134
17.0462
Dynamic
RoTTA
74.89
0.8331
0.0842
0.8933
MIRA RoTTA
75.67
0.8241
0.0674
0.8622
Appendix
Table 16: Continual and dynamic adaptation. Bold compares each base/MIRA pair.
Dataset
Method
Realistic
Acc. (%) ↑
MMC
ECE ↓
NLL ↓
ImageNet-C
TENT
No
42.71
0.5203
0.0935
2.9071
MIRA TENT
No
45.02
0.5007
0.0573
2.7948
TENT
Yes
38.97
0.4444
0.0572
3.0884
MIRA TENT
Yes
39.92
0.4412
0.0484
3.0072
ImageNet-R
TENT
No
41.84
0.5834
0.1649
3.2015
MIRA TENT
No
44.10
0.5771
0.1361
2.9875
Appendix
Table 17: Evaluation under conventional ( η=0 ) and computationally realistic ( η=1 ) TTA protocols. Bold compares each dataset and realistic regime.
Method
Acc. (%) ↑
MMC
ECE ↓
NLL ↓
Time (s) ↓
VRAM (MiB) ↓
TENT
67.10
0.7474
0.0779
1.2873
125
2338
CoTTA
64.16
0.6170
0.0529
1.4173
1142
3062
PETAL
62.30
0.5816
0.0582
1.5059
1586
3930
MIRA TENT
67.75
0.7310
0.0556
1.2401
157
2494
Appendix
Table 18: Single-run accuracy–cost comparison on CIFAR-100-C / ResNeXt (batch size 64). Bold marks the best values in directional columns.
Method
Acc. (%) ↑
MMC
ECE ↓
NLL ↓
Source
23.84\kern 0.2pt\hbox{\pm 0.35}
0.4623\kern 0.2pt\hbox{\pm 0.0013}
0.2240\kern 0.2pt\hbox{\pm 0.0029}
5.0889\kern 0.2pt\hbox{\pm 0.0089}
TENT
28.22\kern 0.2pt\hbox{\pm 0.42}
0.4507\kern 0.2pt\hbox{\pm 0.0031}
0.1685\kern 0.2pt\hbox{\pm 0.0014}
4.1317\kern 0.2pt\hbox{\pm 0.0163}
MIRA TENT
\mathbf{29.57}\kern 0.2pt\hbox{\pm 0.29}
0.4204\kern 0.2pt\hbox{\pm 0.0025}
\mathbf{0.1247}\kern 0.2pt\hbox{\pm 0.0007}
\mathbf{3.9175}\kern 0.2pt\hbox{\pm 0.0264}
Appendix
Table 19: ImageNet-Sketch / ResNet-50, mean ± sample SD over seeds 0, 1, and 2. Bold marks best accuracy, ECE, and NLL.
Method
Acc. (%) ↑
MMC
ECE ↓
NLL ↓
CIFAR-10-C
TENT
52.74
0.9443
0.4170
4.6682
MIRA TENT
54.13
0.9387
0.3977
4.5456
CIFAR-100-C
TENT
27.83
0.8382
0.5603
8.9705
MIRA TENT
32.71
0.8047
0.4784
7.5339
Appendix
Table 20: CLIP ViT-B/16 TENT-component comparison: one update per batch with a shared 10× released learning rate. Bold compares each dataset pair.
Figure 4: Predictive entropy and perturbation-induced MI across CIFAR-10-C, CIFAR-100-C, and ImageNet-C (left to right); color denotes batch accuracy. MI measures local classifier disagreement, rather than calibrated Bayesian uncertainty.
Figure 5: Batch accuracy at matched predictive-entropy bin centers, pooling corruption types. Low-MI batches tend to be more accurate than high-MI batches.
Test-time adaptation (TTA) aims to improve model robustness under distribution shift by adapting a source model using unlabeled test data. Although methods such as TENT and EATA have demonstrated gains on corrupted data, aggregate accuracy can obscure the conditions under which adaptation fails or provides little benefit. We present a controlled comparison of three TTA strategies---BatchNorm-statistics adaptation (BN-Adapt), entropy-minimization adaptation (TENT), and reliability-filtered adaptation (a scoped re-implementation of EATA)---against an unadapted source model on the full CIFAR-10-C benchmark, covering 15 corruption types and 5 severity levels. All three methods improve mean accuracy over the source model by 12.2--13.3 percentage points (Wilcoxon signed-rank p<10−12). However, each method underperforms the source model on 8.0--9.3% of conditions, with failures concentrated in low-severity corruptions where the source model already performs near ceiling, particularly brightness, fog, contrast, and defocus blur. We further find that EATA closely tracks the gradient-free BN-Adapt baseline, with a mean absolute difference of 0.09 percentage points, compared with 1.08 percentage points relative to TENT. This suggests that reliability filtering can substantially restrict effective adaptation, causing EATA to behave more like a BatchNorm-statistics baseline than an entropy-minimization method. These results show that aggregate accuracy alone can mask systematic TTA failure modes and motivate condition-level evaluation of when adaptation helps, harms, or becomes effectively inactive.
Sreeja Guha Majumdar, Aratrika Saha
Heritage Institute of Technology Kolkata, West Bengal, India
Conventional test-time adaptation (TTA) approaches typically adapt the model using only a small fraction of test samples, often those with low-entropy predictions, thereby failing to fully leverage the available information in the test distribution. This paper introduces DualTTA, a novel framework that improves performance under distribution shifts by utilizing a larger and more diverse set of test samples. DualTTA identifies two distinct groups: one where the model's predictions are likely consistent with the underlying semantics, and another where predictions are likely incorrect. For the first group, it minimizes prediction entropy to reinforce reliable decisions; for the second, it maximizes entropy to suppress overconfident errors and unlearn spurious behavior. These groups are adaptively selected using a new reliability criterion that measures prediction stability under both semantic-preserving and semantic-altering transformations, addressing the limitations of purely entropy-based selection. We further provide theoretical analysis and empirical justification showing that our approach enables a tighter separation between reliable and unreliable samples, in the context of their suitability for adaptation, leading to provably more effective model updates.
Nam Nguyen Phuong, Duc Nguyen The Minh, Phi Le Nguyen +2
Institute for AI Innovation and Societal Impact, Hanoi University of Science and Technology · Monash University · Adelaide University
Test-time adaptation (TTA) enhances the zero-shot robustness under distribution shifts by leveraging unlabeled test data during inference. Despite notable advances, several challenges still limit its broader applicability. First, most methods rely on backpropagation or iterative optimization, which limits scalability and hinders real-time deployment. Second, they lack explicit modeling of class-conditional feature distributions. This modeling is crucial for producing reliable decision boundaries and calibrated predictions, but it remains underexplored due to the lack of both source data and supervision at test time. In this paper, we propose ADAPT, an Advanced Distribution-Aware and backPropagation-free Test-time adaptation method. We reframe TTA as a Gaussian probabilistic inference task by modeling class-conditional likelihoods using gradually updated class means and a shared covariance matrix. This enables closed-form, training-free inference. To correct potential likelihood bias, we introduce lightweight regularization guided by CLIP priors and a historical knowledge bank. ADAPT requires no source data, no gradient updates, and no full access to target data, supporting both online and transductive settings. Extensive experiments across diverse benchmarks demonstrate that our method achieves state-of-the-art performance under a wide range of distribution shifts with superior scalability and robustness.