EmbodiTTA: Resource-Efficient Test-Time Adaptation for Embodied Visual Systems

Real-world deployment often exposes vision models to distribution shifts, making test-time adaptation (TTA) important for reliable prediction. Existing TTA methods can be difficult to deploy on edge devices because they require backpropagation, activation storage, or test-time batches, which increase computation, memory, and state. We propose ELaTTA (Efficient Latent Test-Time Adaptation), a forward-only, gradient-free framework for single-instance TTA. ELaTTA freezes the pretrained model and adapts each test input by searching a low-dimensional coefficient vector in a principal latent subspace computed offline from a source-domain cache. The basis is stored once and source data are discarded before deployment. At test time, CMA-ES performs covariance-adaptive Gaussian search, ranks candidate corrections by prediction entropy, and selects the best candidate within a fixed evaluation budget. On five visual benchmarks spanning corruption, natural, rendition, sketch, and cross-domain shifts, ELaTTA provides strong accuracy under strict and continual single-instance protocols. In the strict single-instance comparison, MEMO requires \emph{64.7$\times$} the computation and \emph{12.4$\times$} the peak memory of ELaTTA. A complementary keyword-spotting experiment shows that the same coefficient-search design transfers beyond vision. We further demonstrate ViT-Base adaptation on a Raspberry Pi 5, confirming the feasibility of deploying ELaTTA on an edge device.

Test-time adaptation (TTA) aims to improve model robustness under distribution shifts by adapting to unlabeled test data, but most existing methods rely on backpropagation (BP), which is computationally costly and incompatible with non-differentiable models such as quantized models, limiting practical deployment on numerous edge devices. Recent BP-free approaches alleviate overhead but remain either architecture-specific or limited in optimization capacity to handle high-dimensional models. We propose ZOTTA, a fully BP-free TTA framework that performs efficient adaptation using only forward passes via Zeroth-Order Optimization (ZOO). While ZOO is theoretically appealing, naive application leads to slow convergence under high-dimensional parameter spaces and unstable optimization due to the lack of labels. ZOTTA overcomes these challenges through 1) Distribution-Robust Layer Selection, which automatically identifies and freezes layers that already extract distribution-invariant features, updating only domain-sensitive layers to reduce the optimization dimensionality and accelerate convergence; 2) Spatial Feature Aggregation Alignment, which stabilizes ZOO by aligning globally aggregated spatial features between source and target to reduce gradient variance. Together, these components enable architecture-agnostic and stable BP-free adaptation. Extensive experiments on ImageNet-C/R/Sketch/A show that ZOTTA outperforms or matches BP-based methods, e.g., it reduces memory usage by 84% and improves accuracy by 3.9% over SAR on ImageNet-C. Code:https://github.com/Zhang-Ronghao/zotta.

Test-time adaptation (TTA) mitigates domain shifts by using incoming test data to update a model on the fly. The majority of TTA methods require resource-intensive backpropagation (BP) for model updates, particularly demanding large memory sizes, which makes it infeasible to deploy them on resource-limited devices (e.g., edge devices). To address this issue, we integrate two different techniques, zeroth-order optimization (ZOO) and model merging, under the recently established cross-device collaborative TTA (CDC-TTA) framework, where the system is composed of a mixture of resource-abundant and resource-limited devices, and the model information (e.g., model weights obtained on each device) is shared across the devices. Our method is executable on resource-limited devices by introducing ZOO, which requires only forward processing and bypasses the resource-intensive BP optimization. Concurrently, to mitigate the high-dimensional optimization difficulty caused by the side effect of ZOO, we incorporate model merging of the shared multiple models and set the merge coefficients as the optimization objective, which successfully reduces the optimization dimension. In addition, to enhance the synergistic combination of ZOO and model merging, we propose a unique preprocessing strategy that trims intra-model non-influential weights and reduces the inter-model information redundancy. We empirically confirmed the effectiveness of our method using common corruption and style-transferred image benchmarks.