VLM Robustness

VLM: Vision-Language Model

Ask about this topic

Overview

Vision-language model robustness asks whether a model’s answers stay grounded and dependable when images or prompts are corrupted, altered, unfamiliar, or in conflict. Researchers test these failures with adversarial attacks, corruption and out-of-distribution benchmarks, prompt and context variations, and measures of grounding, consistency, calibration, and safety. Common defenses include adversarial training, test-time adaptation, visual evidence selection, and interventions on model representations or attention. A central challenge is that clean accuracy and even stable answers can hide reliance on shortcuts, while improving robustness can cost accuracy, utility, or efficiency.

Latest papers

CardsList

455 papers48 papers in the last four weeks, up 26% on the four weeks before. 0.3% of all new papers.

  1. Exploring Prompt Alignment with Clinical Factors in Zero-Shot Segmentation VLMs for NSCLC Tumor Segmentation

    May 2, 2026Suraj Pai, Thibault Heintz, Cosmin Ciausu +3VLM RobustnessImage Segmentation

    Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, USA

  2. Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs

    May 1, 2026Siyuan Huang, Xiaoye Qu, Yafu Li +6Memory-Augmented VLMsVLM Robustness

    Shanghai AI Laboratory · Shanghai Jiao Tong University · The Chinese University of Hong Kong +3

  3. Intrinsic Gradient Suppression for Label-Noise Prompt Tuning in Vision-Language Models

    May 1, 2026Jiayu Li, Jiaxin Qi, Sheng Zhou +2VLM RobustnessVLM Adaptation

  4. Embodied Interpretability: Linking Causal Understanding to Generalization in Vision-Language-Action Models

    May 1, 2026Hanxin Zhang, Mingshuo Xu, Abdulqader Dhafer +3VLM RobustnessVLM Interpretability

    1DANiLab, University of Leicester, UK · School of Computing and Mathematical Sciences, University of Leicester, UK · School of Metallurgy and Materials, University of Birmingham, UK

  5. The Effects of Visual Priming on Cooperative Behavior in Vision-Language Models

    Apr 30, 2026Kenneth J. K. OngVLM EvaluationVLM Robustness

    1ST Engineering, Singapore.

  6. Delineating Knowledge Boundaries for Honest Large Vision-Language Models

    Apr 29, 2026Junru Song, Yimeng Hu, Yijing Chen +4VLM RobustnessAbstention in VLMs

    Shandong University

  7. Beyond Shortcuts: Mitigating Visual Illusions in Frozen VLMs via Qualitative Reasoning

    Apr 29, 2026Hao Guo, Fei Wang, Junjie Chen +4VLM RobustnessVisual Reasoning

    Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, China · Anhui Polytechnic University, Wuhu, China · Hefei University of Technology, Hefei, China +2

  8. SIEVES: Selective Prediction Generalizes through Visual Evidence Scoring

    Apr 28, 2026Hector G. Rodriguez, Marcus RohrbachVisual Question AnsweringVLM Robustness

    TU Darmstadt & hessian.AI, Germany

  9. FCMBench-Video: Benchmarking Document Video Intelligence

    Apr 28, 2026Runze Cui, Fangxin Shang, Yehui Yang +3VLM EvaluationDocument Understanding

    AI Lab, Qifu Technology, Beijing, China · School of Future Technology, South China University of Technology, Guangzhou, China · Pazhou Lab, Guangzhou, China +1

  10. One Perturbation, Two Failure Modes: Probing VLM Safety via Embedding-Guided Typographic Perturbations

    Apr 28, 2026Ravikumar Balakrishnan, Sanket MendaparaVLM EvaluationVLM Robustness

    Cisco Systems

  11. Majorization-Guided Test-Time Adaptation for Vision-Language Models under Modality-Specific Shift

    Apr 27, 2026Lixian Chen, Yanhui Chen, Junyi LinVLM RobustnessTest-Time Adaptation

    Guangdong University of Technology · Guangzhou, China

  12. Robust Grounding with MLLMs Against Occlusion and Small Objects via Language-Guided Semantic Cues

    Apr 27, 2026Beomchan Park, Seongho Kim, Hyunjun Kim +2VLM RobustnessLLM Grounding

    Integrated Vision Language Lab., KAIST, Republic of Korea

  13. From Global to Local: Rethinking CLIP Feature Aggregation for Person Re-Identification

    Apr 24, 2026Aotian Zheng, Winston Sun, Bahaa Alattar +2VLM RobustnessPerson Re-Identification

    Department of Electrical and Computer Engineering, University of Washington, Seattle, WA 98195 · Applied Physics Laboratory, University of Washington, Seattle, WA 98195

  14. Ramen: Robust Test-Time Adaptation of Vision-Language Models with Active Sample Selection

    Apr 23, 2026Wenxuan Bao, Yanjun Zhao, Xiyuan Yang +1VLM RobustnessTest-Time Adaptation

    University of Illinois Urbana-Champaign

  15. Seeing Isn't Believing: Uncovering Blind Spots in Evaluator Vision-Language Models

    Apr 23, 2026Mohammed Safi Ur Rahman Khan, Sanjay Suryanarayanan, Tushar Anand +1VLM EvaluationVLM Robustness

    ♠Nilekani Centre at AI4Bharat ♢Indian Institute of Technology Madras · ♠Nilekani Centre at AI4Bharat · ♣BITS Pilani, Hyderabad

  16. Latent Denoising Improves Visual Alignment in Large Multimodal Models

    Apr 23, 2026Dhruv Parikh, Jacob Fein-Ashley, Rajgopal Kannan +1VLM RobustnessMultimodal Large Language Models

    University of Southern California Los Angeles, CA, USA · DEVCOM ARL Army Research Office USA

  17. How VLAs (Really) Work In Open-World Environments

    Apr 23, 2026Amir Rasouli, Yangzheng Wu, Zhiyuan Li +4VLM RobustnessLong-Horizon Robotic Manipulation

    Noahs Ark Laboratory, Huawei Technologies Canada. · while at Huawei Canada.

  18. If you're waiting for a sign... that might not be it! Mitigating Trust Boundary Confusion from Visual Injections on Vision-Language Agentic Systems

    Apr 21, 2026Jiamin Chang, Minhui Xue, Ruoxi Sun +3VLM RobustnessAdversarial Attacks on VLMs

    University of New South Wales, Sydney, NSW 2052, Australia · CSIRO’s Data61, Marsfield, NSW 2122, Australia · Macquarie University, NSW 2109, Australia

  19. Hierarchically Robust Zero-shot Vision-language Models

    Apr 20, 2026Junhao Dong, Yifei Zhang, Hao Zhu +2Hierarchical ClassificationVLM Robustness

    1Nanyang Technological University · 2CFAR, IHPC, A*STAR · 3Northwest Polytechnical University +2

  20. DUALVISION: RGB-Infrared Multimodal Large Language Models for Robust Visual Reasoning

    Apr 20, 2026Abrar Majeedi, Zhiyuan Ruan, Ziyi Zhao +3Cross-Modal LearningVisual Question Answering

    University of Wisconsin-Madison · Amazon · Northeastern University

  21. ProtoCLIP: Prototype-Aligned Latent Refinement for Robust Zero-Shot Chest X-Ray Classification

    Apr 20, 2026Florian Kittler, Sheethal Bhat, Andreas MaierVLM RobustnessChest X-Ray Classification

    Friedrich-Alexander University Erlangen-Nuremberg, Erlangen 91054, Germany

  22. Towards Robust Text-to-Image Person Retrieval: Multi-View Reformulation for Semantic Compensation

    Apr 20, 2026Chao Yuan, Yujian Zhao, Haoxuan Xu +1VLM RobustnessVision-Language Retrieval

    Beihang University

  23. OFlow: Injecting Object-Aware Temporal Flow Matching for Robust Robotic Manipulation

    Apr 20, 2026Kuanning Wang, Ke Fan, Chenhao Qiu +5VLM RobustnessFlow Matching

    Fudan University Shanghai, China · University of Southern California Los Angeles, CA, USA · KAUST Saudi Arabia

  24. Spatiotemporal Sycophancy: Negation-Based Gaslighting in Video Large Language Models

    Apr 20, 2026Ziyao Tang, Pengkun Jiao, Bin Zhu +3VLM EvaluationVLM Robustness

    Institute of Trustworthy Embodied AI, Fudan University · Shanghai Key Laboratory of Multimodal Embodied AI · Singapore Management University

  25. When Background Matters: Breaking Medical Vision Language Models by Transferable Attack

    Apr 19, 2026Akash Ghosh, Subhadip Baidya, Sriparna Saha +1VLM RobustnessAdversarial Attacks on VLMs

    Indian Institute of Technology Patna1 · Work done as Visiting Researcher at MBZUAI · MBZUAI3 +1

  26. RemoteShield: Enable Robust Multimodal Large Language Models for Earth Observation

    Apr 19, 2026Rui Min, Liang Yao, Shiyu Miao +5Remote Sensing Image UnderstandingVLM Robustness

    1Hohai University · 2Nanjing University · 3Southeast University

  27. Cross-Modal Attention Analysis and Optimization in Vision-Language Models: A Study on Visual Reliability

    Apr 19, 2026Lijie ZhouCross-Modal LearningVLM Robustness

    School of Computer Science University of Nottingham Ningbo China Ningbo, China

  28. Prompt Sensitivity in Vision-Language Grounding: How Small Changes in Wording Affect Object Detection

    Apr 18, 2026Dawar Jyoti Deka, Amit Sethi, Syed Mohammad AliOpen-Vocabulary Object DetectionVLM Robustness

    Department of Aerospace Engineering, Indian Institute of Technology Bombay, Mumbai, India · Department of Electrical Engineering, Indian Institute of Technology Bombay, Mumbai, India

  29. SIF: Semantically In-Distribution Fingerprints for Large Vision-Language Models

    Apr 18, 2026Yifei Zhao, Qian Lou, Mengxin ZhengVLM RobustnessLanguage Model Fingerprinting

    University of Central Florida

  30. Information Router for Mitigating Modality Dominance in Vision-Language Models

    Apr 17, 2026Seulgi Kim, Mohit Prabhushankar, Ghassan AlRegibVLM RobustnessVision-Language Models

    OLIVES at the Center for Signal and Information Processing CSIP, School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, USA