StableTTA: Improving Vision Model Performance by Training-free Test-Time Adaptation Methods

Zheng Li; Jerry Cheng; Huanying Helen Gu

cs.CVApr 6, 2026

StableTTA: Improving Vision Model Performance by Training-free Test-Time Adaptation Methods

Authors: Zheng Li, Jerry Cheng, Huanying Helen Gu

Organizations: Department of Computer Science New York Institute of Technology

Abstract

Ensemble methods improve predictive performance but often incur high memory and computational costs. We identify an aggregation instability induced by nonlinear projection and voting operations. To address both efficiency challenges and this inconsistency, we propose StableTTA, a training-free test-time adaptation method with two variants. StableTTA-I targets coherent-batch inference settings, where temporally or semantically adjacent observations are likely to belong to the same class. Examples include burst photography, video streams, robotics perception, and industrial inspection. Under coherent-batch inference, StableTTA-I substantially improves prediction consistency and accuracy through variance-aware logit aggregation. StableTTA-II establishes feature-level cropping, enabling efficient logit aggregation with a single forward pass on a single model backbone. Experiments on ImageNet-1K across 71 models demonstrate that StableTTA-I consistently improves prediction accuracy under coherent-batch inference, while StableTTA-II provides lightweight and architecture-agnostic accuracy improvements with minimal computational overhead. These results suggest that inference-time semantic coherence and aggregation stability provide useful perspectives for improving practical test-time adaptation systems.

Figures & tables

Regime	Assumption	Example Applications
IID inference	Independent samples	Standard image classification
Coherent-batch inference	Neighboring samples correlated	Burst imaging, inspection
Multi-view inference	Multiple views of same object	3D recognition
Sequential inference	Temporal continuity	Video and robotics

Table 1: Representative inference regimes in visual recognition systems.

Figure 1: (a) Aggregation Inconsistency: Given branch logits

\{\boldsymbol{z}^{(i)}\mid i=1,2,3\}

, probabilities

\boldsymbol{p}^{(i)}=\operatorname{softmax}(\boldsymbol{z}^{(i)})

, and predictions

\hat{y}^{(i)}=\operatorname{argmax}\boldsymbol{p}^{(i)}

, different aggregation strategies may produce inconsistent results. Here, logit averaging predicts

\hat{y}_{\text{logit}}=1

, soft voting predicts

\hat{y}_{\text{soft}}=2

, and hard voting predicts

\hat{y}_{\text{hard}}=3

. (a) Explanation: When

({\boldsymbol{z}^{(1)},\boldsymbol{z}^{(2)},\dots})

are sparsely distributed, inconsistency

\hat{y}_{\text{logit}}\neq\hat{y}_{\text{soft}}\neq\hat{y}_{\text{hard}}

are more likely due to the nonlinearity and non-bijectivity of softmax. In contrast, when the logits are more densely clustered, such inconsistencies are less likely.

Figure 2: Illustration of StableTTA-II. Given an input image

\boldsymbol{x}

, the backbone produces a feature representation

\boldsymbol{h}

. Multiple deterministic crops are sampled at the feature level and processed by a shared head to obtain multiple logits

\{\boldsymbol{z}^{(i)}\}

, which are aggregated to form the final prediction.

Figure 3: Comparison under coherent-batch inference. StableTTA-I improves model performance across a broad range of ImageNet-1K architectures under coherent-batch inference, highlighting the potential benefits of exploiting deployment-time semantic coherence during test-time aggregation.

Figure 4: Superior Efficiency and Accuracy of StableTTA-I. Comparison of the baseline (blue) and StableTTA-I (red) across the number of model parameters (left), peak GFLOPs in sequential aggregation mode (middle), and total GFLOPs in parallel aggregation mode (right).

Panel of Figure 5 — Figure 5: (a) TTA (with our sampler and augmentation) vs. StableTTA-I. (b) StableTTA-I is robust to $K$ , but disabling logit processing $(K=C)$ will significantly reduce accuracy.

		Baseline (%)		StableTTA-II (%)
Weight	Params	Acc@1	Acc@5	Acc@1	Acc@5
ConvNeXt_Tiny [ 18 , 36 ]	28.6M	82.520	96.146	82.570	96.218
ConvNeXt_Small [ 18 , 36 ]	50.2M	83.616	96.65	83.692	96.778
ConvNeXt_Base [ 18 , 36 ]	88.6M	84.062	96.87	84.208	96.952
ConvNeXt_Large [ 18 , 36 ]	197.8M	84.414	96.976	84.524	97.128
DenseNet121 [ 10 , 36 ]	8.0M	74.434	91.972	75.980	93.062

Table 2: Comparison of base models and the StableTTA-II-enhanced models on the ImageNet-1K.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: (a) Baseline: a single model with one forward pass. (b) Multi-model ensemble: multiple models with multiple forward passes. (c) TTA: a single model with multiple forward passes using augmented inputs. Both the multi-model ensemble and TTA aggregate outputs across forward passes.

Weight	Data Augmentation	TTA Acc	Source paper
VGG(v5)	10 $\times$ Crop	71.93	[ 7 ] He et al.
PReLU-net	10 $\times$ Crop	75.73	[ 7 ] He et al.
plain-34	10 $\times$ Crop	71.46	[ 7 ] He et al.
ResNet50	10 $\times$ Crop	77.15	[ 7 ] He et al.
ResNet101	10 $\times$ Crop	78.25	[ 7 ] He et al.
ResNet152	10 $\times$ Crop	78.57	[ 7 ] He et al.

Appendix

Table 3: Summary of experimental results reported in prior studies, evaluated using top-1 accuracy on the ImageNet-1K validation set. Both baseline and test-time augmentation (TTA) results are taken directly from the original papers, with each value corresponding to its respective reference.

Figure 7: Empirical cumulative distribution functions (ECDFs) of

p

-values from Jarque–Bera tests across models on ImageNet-1K validation set. Each model outputs 50,000

\times

1,000 groups from augmented images, with one test per group. The x-axis shows the significance level

\alpha

, and the y-axis shows the proportion of

p

-values

\leq\alpha

. The red bar marks the case of

\alpha=0.05

. A lower proportion (e.g., 22% of the tested groups under AlexNet) indicates that the majority of groups fail to reject the null hypothesis, suggesting stronger consistency with a normal distribution.

Figure 8: Monte Carlo simulation. The inconsistency probability increases as

\operatorname{Var}(\boldsymbol{z})

grows. In this simulation, we consider distributions:

\{\boldsymbol{z}\sim\mathcal{N}(\boldsymbol{\mu},\sigma I)\mid\mu\in\{(1,0.9),(1,0.7),(1,0.5)\},\sigma\in[0.05,0.25]\}

. The solid curves show Monte Carlo estimates of the relationship between

\sigma

and

\operatorname{P}(\hat{y}_{\text{logit}}\neq\hat{y}_{\text{hard}})

, while the dashed curves correspond to the theoretical (asymptotic) predictions. The empirical and theoretical results are closely matched.

Figure 9: Varying Effects of Data Augmentation Methods on the Variance of Logits. Under Hölder continuity, larger changes in the input lead to larger changes in the logits. Left: Translation preserves the semantic content of the image but introduces large pixel-wise differences, resulting in a higher variations in the logits. Right: Random erasing applies stronger modifications to the image, but the overall pixel-wise differences are smaller, leading to more concentrated logits that are better suited to stable aggregation.

	Random Data	Hard-vot. (%)		Soft-vot. (%)		Logit-avg. (%)
Model	Augmentation	Acc@1	Acc@5	Acc@1	Acc@5	Acc@1	Acc@5
AlexNet (# Params = 61.1 M) (7.1 GFLOPS) ( $N=10$ )	baseline	56.522	79.066	56.522	79.066	56.522	79.066
	flip	56.630	56.760	57.306	79.674	57.322	79.714
	crop	55.016	55.158	55.460	78.240	55.002	78.082
	affine	51.176	51.342	51.966	75.262	51.116	74.564
	erasing	56.552	56.688	56.574	78.914	56.508	79.070

Appendix

Table 4: Comparison of aggregation strategies (hard voting, soft voting, and logit averaging) under random data augmentation across different models. All results are derived from our aforementioned experiments using TorchVision pretrained models.

		Peak	Baseline (%)		StableTTA-I (%)
Weight	Params	GFLOPS	Acc@1	Acc@5	Acc@1	Acc@5
AlexNet [ 14 , 36 ]	61.1M	0.71	56.522	79.066	89.332	98.028
ConvNeXt_Tiny [ 18 , 36 ]	28.6M	4.46	82.52	96.146	95.958	99.884
ConvNeXt_Small [ 18 , 36 ]	50.2M	8.68	83.616	96.65	95.932	99.914
ConvNeXt_Base [ 18 , 36 ]	88.6M	15.36	84.062	96.87	95.964	99.906
ConvNeXt_Large [ 18 , 36 ]	197.8M	34.36	84.414	96.976	95.998	99.932

Appendix

Table 5: Comparison of base models and the StableTTA-I-enhanced models on the ImageNet-1K dataset: model size, computational cost, and top-1 and top-5 validation accuracy.

			TTA (%)		StableTTA-I (%)
Model	$N$	GFLOPS	Acc@1	Acc@5	Acc@1	Acc@5
AlexNet (# Params = 61.1M)	4	2.84	65.590	87.022	78.750	93.434
	8	5.68	68.604	89.432	84.810	96.216
	16	11.36	70.496	90.510	87.864	97.470
	32	22.72	71.258	90.938	89.532	98.030
ResNet50 † (# Params = 25.6M)	4	16.36	88.556	98.858	91.276	99.108

Appendix

Table 6: Ablation studies of StableTTA-I versus TTA under varying numbers of experts (

N

) across multiple architectures. StableTTA-I consistently outperforms TTA under mixup and CutMix.

		$N=4$		$N=8$		$N=16$		$N=32$
Model	$K$	Acc@1	Acc@5	Acc@1	Acc@5	Acc@1	Acc@5	Acc@1	Acc@5
AlexNet (61.1M) (0.71GFLOPS)	1	78.412	93.182	84.638	96.134	87.942	97.530	89.576	98.052
	2	78.444	93.262	84.982	96.312	87.946	97.528	89.496	98.052
	5	78.804	93.328	84.666	96.220	87.984	97.412	89.484	98.108
	10	78.750	93.434	84.810	96.216	87.864	97.470	89.532	98.030
	20	78.540	93.320	84.778	96.160	88.072	97.540	89.530	98.076

Appendix

Table 7: Sensitivity analysis of StableTTA-I over

K

and

N

. Results show robustness to

K

, consistent gains with larger

N

, and significant drops when logit processing is disabled (

K=C

AlexNet (%)		ResNet50 † (%)		ViT_B_16 (%)
Acc@1	Acc@5	Acc@1	Acc@5	Acc@1	Acc@5
89.492	98.034	95.596	99.862	95.888	99.868
89.558	98.056	95.622	99.854	95.830	99.898
89.642	98.020	95.620	99.868	95.916	99.896
89.620	98.070	95.684	99.842	95.848	99.882
89.588	98.096	95.666	99.858	95.876	99.878

Appendix

Table 8: ImageNet-1K validation accuracy (Top-1/Top-5). Mean and standard deviation are reported over five independent runs for both heavyweight and lightweight models.

Model	Head	Crops
ConvNeXt_Tiny	AvgPool+Norm+Linear(768,1000)	$\boldsymbol{h},\boldsymbol{h}_{1:-1,:},\boldsymbol{h}_{:-1,:},\boldsymbol{h}_{1:,:},\boldsymbol{h}_{:,1:-1}$
ConvNeXt_Small	AvgPool+Norm+Linear(768,1000)	$\boldsymbol{h},\boldsymbol{h}_{1:-1,:},\boldsymbol{h}_{:-1,:},\boldsymbol{h}_{1:,:},\boldsymbol{h}_{:,1:-1}$
ConvNeXt_Base	AvgPool+Norm+Linear(1024,1000)	$\boldsymbol{h},\boldsymbol{h}_{1:-1,:},\boldsymbol{h}_{:-1,:},\boldsymbol{h}_{1:,:},\boldsymbol{h}_{:,1:-1}$
ConvNeXt_Large	AvgPool+Norm+Linear(1536,1000)	$\boldsymbol{h},\boldsymbol{h}_{1:-1,:},\boldsymbol{h}_{:-1,:},\boldsymbol{h}_{1:,:},\boldsymbol{h}_{:,1:-1}$
DenseNet121	ReLU+AvgPool+Linear(1024,1000)	$\boldsymbol{h},\boldsymbol{h}_{1:-1,:},\boldsymbol{h}_{:,1:-1}$
DenseNet161	ReLU+AvgPool+Linear(2208,1000)	$\boldsymbol{h},\boldsymbol{h}_{1:-1,:},\boldsymbol{h}_{:,1:-1}$

Appendix

Table 9: Feature-level cropping configurations used in StableTTA-II across different architectures.

Figure 10: Sensitivity analysis of cropping strategies and random erasing. Left: Comparison of deterministic multi-crop methods (TripleCrop, FiveCrop, SevenCrop, NineCrop) and RandomCrop with varying probabilities

p

. Right: Effect of Random Erasing probability p on top-1 validation accuracy for different numbers of experts

N\in\{2,4,8\}

. Overall, moderate cropping diversity improves performance, while excessive randomness provides degraded returns.

Test-Time Adaptation

Test-time adaptation updates or recalibrates pretrained models at inference to handle distribution shifts, often without target labels.

See all 322 papers

Efficient Test-Time Adaptation through Latent Subspace Coefficients Search
Oct 13, 2025Xinyu Luo, Jie Liu, Kecheng Chen +6Test-Time Adaptation Continual Test-Time Adaptation
Department of Electrical Engineering, City University of Hong Kong, Hong Kong SAR, China
Adapting in the Dark: Efficient and Stable Test-Time Adaptation for Black-Box Models
Apr 17, 2026Yunbei Zhang, Shuaicheng Niu, Chengyi Cai +2Test-Time Adaptation Image Classification
1Tulane University · 2Nanyang Technological Univer · 3The University of Melbourne.
Continual Test-Time Adaptation in Computer Vision: Methods, Benchmarks, and Future Directions
Jul 9, 2026Sarthak Kumar Maharana, Shambhavi Mishra, Yunbei Zhang +6Source-Free Domain Adaptation Distribution Shift Robustness
The University of Texas at Dallas, USA · LIVIA ETS Montreal, ILLS International Laboratory on Learning Systems (ILLS), McGill - ÉTS - MILA - CNRS - Université Paris-Saclay - CentraleSupélec · Tulane University, USA +2

Oct 13, 2025J/K move · Enter open · S save

Efficient Test-Time Adaptation through Latent Subspace Coefficients Search

Xinyu Luo, Jie Liu, Kecheng Chen, Yihua Shao+5

Department of Electrical Engineering, City University of Hong Kong, Hong Kong SAR, China

Test-Time Adaptation Continual Test-Time Adaptation

Open paper AskPDF arXiv

Real-world deployment often exposes vision models to distribution shifts, making test-time adaptation (TTA) important for reliable prediction. Existing TTA methods can be difficult to deploy on edge devices because they require backpropagation, activation storage, or test-time batches, which increase computation, memory, and state. We propose ELaTTA (Efficient Latent Test-Time Adaptation), a forward-only, gradient-free framework for single-instance TTA. ELaTTA freezes the pretrained model and adapts each test input by searching a low-dimensional coefficient vector in a principal latent subspace computed offline from a source-domain cache. The basis is stored once and source data are discarded before deployment. At test time, CMA-ES performs covariance-adaptive Gaussian search, ranks candidate corrections by prediction entropy, and selects the best candidate within a fixed evaluation budget. On five visual benchmarks spanning corruption, natural, rendition, sketch, and cross-domain shifts, ELaTTA provides strong accuracy under strict and continual single-instance protocols. In the strict single-instance comparison, MEMO requires \emph{64.7

\times

} the computation and \emph{12.4

\times

} the peak memory of ELaTTA. A complementary keyword-spotting experiment shows that the same coefficient-search design transfers beyond vision. We further demonstrate ViT-Base adaptation on a Raspberry Pi 5, confirming the feasibility of deploying ELaTTA on an edge device.

Abstract

Test-Time Adaptation

Explore similar work

Efficient Test-Time Adaptation through Latent Subspace Coefficients Search

Adapting in the Dark: Efficient and Stable Test-Time Adaptation for Black-Box Models

Continual Test-Time Adaptation in Computer Vision: Methods, Benchmarks, and Future Directions