Context is All You Need

Jean Erik Delanois; Shruti Joshi; Ryan Golden; Teresa Nick; Maxim Bazhenov

cs.LGApr 6, 2026

Context is All You Need

Authors: Jean Erik Delanois, Shruti Joshi, Ryan Golden, Teresa Nick, Maxim Bazhenov

Organizations: Department of Computer Science & Engineering, University of California, San Diego, La Jolla, California, USA · Department of Medicine, University of California, San Diego, La Jolla, California, USA · Microsoft Corporation, Redmond, WA, USA

Abstract

Artificial Neural Networks (ANNs) are increasingly deployed across diverse real-world settings, where they must operate under data distributions that differ from those seen during training. This challenge is central to Domain Generalization (DG), which trains models to generalize to unseen domains without target data, and Test-Time Adaptation (TTA), which improves robustness by adapting to unlabeled test data at deployment. Existing approaches to address these challenges are often complex, resource-intensive, and difficult to scale. We introduce CONTXT (Contextual augmentatiOn for Neural feaTure X Transforms), a simple and intuitive method for contextual adaptation. CONTXT modulates internal representations using simple additive and multiplicative feature transforms. Within a TTA setting, it yields consistent gains across discriminative tasks (e.g., ANN/CNN classification) and generative models (e.g., LLMs). The method is lightweight, easy to integrate, and incurs minimal overhead, enabling robust performance under domain shift without added complexity. More broadly, CONTXT provides a compact way to steer information flow and neural processing without retraining.

Figures & tables

Panel of Figure 1 — Figure 1: CONTXT : Contextual augmentation via feature transforms. (a) At a chosen layer, compare the current feature vector $\mathbf{h}$ to a precomputed contextual feature representation $\mathbf{c}$ to form an ”index” (their difference) $\mathbf{d}=\mathbf{c}-\mathbf{h}$ . (b) Add a scaled version of this index, $\alpha\mathbf{d}$ , to the features; $\alpha>0$ injects the context while $\alpha<0$ removes it. (c) Mix multiple contexts by linearly combining indices with separate scalars, e.g. $\alpha_{i}\mathbf{d}_{i}+\alpha_{j}\mathbf{d}_{j}$ .

Panel of Figure 2 — Figure 2: ”Cow on a beach” example. (a) A representative input image shown alongside contextual examples. (b/c) The vertical axis reports the model’s maximum softmax confidence, while the horizontal axis sweeps the strength of the farm / city index injection; each subplot corresponds to a different fixed level of beach context removal (strength annotated above each panel). For both injection and removal, $\alpha=0$ indicates that no context is applied. Text above each curve denotes the top-1 predicted class at that setting. Correct application of CONTXT yields proper classification.

Panel of Figure 3 — Figure 3: Baseline accuracy for the CCT (a) and PACS (b) models. Models were trained on a single domain (Location 38 / Photo), performance on the training domain is highest while accuracy quickly degrades when tested in other domains.

Panel of Figure 4 — Figure 4: Accuracy heatmaps for CCT (a) and PACS (b). Vertical axis: out-of-domain removal strength; horizontal axis: in-domain injection strength. Color encodes change in mean test accuracy averaged across all domains (trained and untrained) compared to baseline. CONTXT can improve performance by about 10%.

Panel of Figure 5 — Figure 5: Domain-wise change in accuracy on CCT (a) and PACS (b). Source domains show zero shift - Photo in PACS and Location 38 in CCT - while most unseen target domains exhibit substantial improvements.

Panel of Figure 6 — Figure 6: Flip rate (percentage of reviews whose predicted sentiment changes after rewriting) versus Self-BLEU between rewritten and original reviews for Llama 8B (a) and 70B (b). When asked to rephrase a review, the baseline (no CONTXT ) preserves sentiment, whereas models provided with opposing sentiment CONTXT flip the classification while maintaining fluency. Points along each trajectory correspond to increasing context-shift magnitude (lowest at the bottom of the plot).

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 8

Neural Network Robustness

Neural network robustness studies how models withstand adversarial inputs, data shifts, noise, and hardware faults while maintaining reliable predictions.

See all 237 papers

TestDG: Test-time Domain Generalization for Continual Test-time Adaptation
Apr 7, 2025Sohyun Lee, Nayeong Kim, Juwon Kang +2Continual Test-Time Adaptation Cross-Domain Generalization
POSTECH · GenGenAI · KAIST AI
DOME: Learning Transferable Domain Variables from Sparse Supervision for Test-Time Adaptation
Jun 2, 2026Xiaoran Xu, Yifan Xu, Yupeng Wu +2Representation Learning Test-Time Adaptation
MAIS, IACAS · Beijing, China
CPCANet: Deep Unfolding Common Principal Component Analysis for Domain Generalization
May 6, 2026Yu-Hsi Chen, Abd-Krim SeghouanePrincipal Component Analysis Domain Generalization
The University of Melbourne

Apr 7, 2025J/K move · Enter open · S save

TestDG: Test-time Domain Generalization for Continual Test-time Adaptation

Sohyun Lee, Nayeong Kim, Juwon Kang, Seong Joon Oh+1

POSTECH · GenGenAI · KAIST AI

Continual Test-Time Adaptation Cross-Domain Generalization Invariant Representation Learning

Open paper AskPDF arXiv

This paper studies continual test-time adaptation (CTTA), the task of adapting a model to constantly changing unseen domains in testing while preserving previously learned knowledge. Existing CTTA methods mostly focus on adaptation to the current test domain only, overlooking generalization to arbitrary test domains a model may face in the future. To tackle this limitation, we present a novel online test-time domain generalization framework for CTTA, dubbed TestDG. TestDG aims to learn features invariant to both current and previous test domains on the fly during testing, improving the potential for effective generalization to future domains. To this end, we propose a new model architecture and a test-time adaptation strategy dedicated to learning domain-invariant features, along with prototype selection and update mechanisms for effectively managing information from previous test domains. TestDG achieved state of the art on three of four public CTTA benchmarks and performed on par with the best prior method on ImageNet-C. Moreover, it showed superior generalization to unseen test domains.

Abstract

Neural Network Robustness

Explore similar work

TestDG: Test-time Domain Generalization for Continual Test-time Adaptation

DOME: Learning Transferable Domain Variables from Sparse Supervision for Test-Time Adaptation

CPCANet: Deep Unfolding Common Principal Component Analysis for Domain Generalization