Test-Time Adaptation of VLMs

VLM: Vision-Language Model

Overview

Test-time adaptation adjusts a vision-language model to the data it encounters at deployment, aiming to preserve performance when images, modalities, or tasks differ from pretraining. Common approaches include prompt or adapter tuning, entropy and consistency objectives, pseudo-labeling, and reinforcement learning, alongside training-free methods that use caches, prototypes, retrieval, or feature calibration. Work spans CLIP classification and robustness benchmarks as well as detection, segmentation, medical imaging, and multimodal question answering. A central challenge is obtaining reliable adaptation signals without amplifying errors, causing drift or miscalibration, or spending more computation than the gains justify.

Latest papers

CardsList

90 papers12 papers in the last four weeks, down 14% on the four weeks before. 0.1% of all new papers.

  1. PRISM: Primitive-Integrated Separable Modulation with Priority Visual Memory Queue for Compositional Zero-Shot Learning

    Nov 15, 2026Jianhua Wang, Zhihang Zheng, Yang Liu +1Zero-Shot LearningTest-Time Adaptation of VLMs

  2. When to Trust Adaptation: Contextual Fusion for Transferable Vision-Language Prompting

    Nov 9, 2026Zhenyuan Chen, Wenshuo Tao, Zhuo Huang +3vision-language model ensemblingVLM Adaptation

  3. Batch-Conditioned Semantic Anchors for Robust Transductive Adaptation of Vision--Language Models

    Nov 4, 2026Mohammed Rahman Sherif Khan Mohammad, Ardhendu Behera, Sandip Pradhan +2VLM AdaptationTest-Time Adaptation of VLMs

  4. Lookahead Prediction-Guided Adaptive Visual Refocusing for Interleaved Multimodal Reasoning

    Nov 2, 2026Mengzhao Wang, Yanli Ji, Peng YeMultimodal CoT ReasoningTest-Time Adaptation of VLMs

  5. Label-Free Localization and Test-Time Suppression of CLIP's Typographic Circuits

    Nov 1, 2026Bohan Liu, Wenqian Ye, Guangzhi Xiong +3VLM RobustnessAdversarial Attacks on VLMs

  6. Timbre: Reliability-aware harmonic consensus for label-free multimodal test-time rexinforcement learning

    Oct 29, 2026Haotian Lyu, Haoyu Wang, Ru Wang +4Test-Time Adaptation of VLMsTest-Time Adaptation

  7. Test-Time Prompt-Agnostic Decomposition

    Oct 26, 2026Junze Wang, Lei Fan, Dezheng Zhang +4Test-Time AdaptationTest-Time Prompt Tuning

  8. Zero-Shot Domain Generalization via Domain Prompt Learning

    Oct 22, 2026Shiming Chen, Shuhuang Chen, Salman Khan +1Domain GeneralizationMultimodal Domain Generalization

  9. In PRAISE of Attributes: Grounding Retrieval Geometry with Query-Conditioned Subspaces for Person Re-Identification

    Oct 20, 2026Rajarshi Bhattacharya, Aryan Shukla, Shakeeb Murtaza +4Person Re-IdentificationVision-Language Retrieval

  10. Memory Internalization via Test-Time Training for Long-Video Understanding

    Oct 17, 2026Haonan Zhao, Jiaxuan Chen, Qingyang Liu +4Memory-Augmented Video UnderstandingLong-Video QA

  11. Hidden Trajectories Don't Lie: Correcting Corruption Drift for Test-Time Adaptation

    Oct 16, 2026Amirhossein Sabour, Amir Khasahmadi, Mengliu ZhaoTest-Time AdaptationTest-Time Adaptation of VLMs

  12. FedRAPP: Federated Test-Time Prompt Tuning for Vision-Language Models with Reliability-Aware Prompt Personalization

    Oct 16, 2026Syed Md. Ahnaf Hasan, Md Akil Raihan Iftee, Sajib Mistry +2Prompt Learning for VLMsTest-Time Prompt Tuning

  13. On the Limits of Representation-Based Test-Time Adaptation for Multimodal LLMs

    Oct 14, 2026Mikołaj Piórczyński, Wojciech Masarczyk, Damian Sójka +2Test-Time AdaptationTest-Time Adaptation of VLMs

  14. CVIF: A Criticality-Driven Visual Intervention Framework for Geometric Diagram Understanding in MLLMs

    Oct 5, 2026Jiahui Kang, Bifan Wei, Lingling Zhang +4Test-Time Adaptation of VLMsVisual Reasoning

  15. RADC: Risk-Aware Dual Caching for Vision-Language Test-Time Adaptation

    Oct 3, 2026Siyu Huang, Yueyong Chen, Xuejiao Li +1Test-Time Adaptation of VLMsTest-Time Adaptation

  16. Technical note on: Zero-Training Feature-Space Alignment via Information Geometry

    Sep 29, 2026Behraj Khan, Tahir Qasim Syed, Syed Ahmad Chan BukhariDistribution Shift RobustnessInformation Geometry

  17. Position Aware Layer Queries for Test Time Training in Vision Language Models

    Sep 27, 2026Rajat Modi, Priyank Pathak, Xin Liang +1Test-Time Adaptation of VLMsVLM Distillation

  18. Test-Time Generalized Category Discovery

    Sep 27, 2026Shambhavi Mishra, Omprakash Chakraborty, Julio Silva-Rodriguez +3Test-Time Adaptation of VLMsGeneralized Category Discovery

  19. SphMind: Towards Robust, Training-Free VLM-based Spatial Reasoning with a 360 Camera

    Sep 27, 2026Shriram Damodaran, Soumyaratna Debnath, Cheston Tan +1Test-Time Adaptation of VLMs3D Spatial Reasoning

  20. TTRSD: Test-Time Reinforcement Learning with Self-Distillation for Vision-Language Models

    Sep 27, 2026Shuning Wang, Zhiheng Wu, Xun Zhou +8Test-Time Adaptation of VLMsVLM Distillation

  21. Domain Recentering and Confidence-Weighted Prior Calibration for Vision-Language Models

    Sep 24, 2026Youngeun Seol, Jimin Shin, Heeseo Yoon +1Source-Free Domain AdaptationUnsupervised Domain Adaptation

  22. Retrieval Geometry Shapes Cache-Based Clip Adaptation

    Sep 20, 2026Mahir Shahriar Tamim, Md. Samiul Alim, Azmine Toushik Wasi +5Memory-Augmented VLMsTest-Time Adaptation

  23. Balanced Prompt Adaptation against Entropy-Induced Collapse for Test-Time Binary Segmentation

    Sep 18, 2026Zhengshan Wang, Joshua Charles Webster-Ford, Yifei Tian +3Test-Time AdaptationImage Segmentation

  24. Label-free steering: Compressing test-time reinforcement learning into bias-only subspaces

    Sep 16, 2026Naveen Vakada, Mingyuan Li, Shaoxiong JiFine-TuningTest-Time Adaptation of VLMs

  25. Bridging the Confidence Gap: Temperature Scaling for Calibrating Test-Time Prompt Tuning

    Sep 15, 2026Yuwei Liang, Jian Liang, Dapeng Hu +2Post-Hoc CalibrationTest-Time Adaptation of VLMs