VLM Robustness

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

26 papers in the last four weeks, up 117% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 295

All topics

Detecting Adversarial Images through Response Profiles of Vision-Language Models
Oct 7, 2026Arash Vashagh, Roozbeh Razavi-FarAdversarial Example Detection VLM Robustness
Trustworthy and Secure AI (TSAI) Lab, Faculty of Computer Science, University of New Brunswick, Canada
Purifying Backdoored Large Vision-Language Models by Removing Hijacked Directions
Oct 7, 2026Bojun Yang, Haochen Zhou, Zhifang Zhang +3VLM Robustness Large Vision-Language Models
Southeast University · The University of Queensland · Zhejiang University
Understanding and Mitigating Token-Pruning-Induced Vulnerabilities in VLMs
Oct 7, 2026Shuailong Wang, Xinyu Lyu, Shengming Yuan +3Visual Token Pruning VLM Robustness
University of Electronic Science and Technology of China, Chengdu, China · Southwestern University of Finance and Economics, Chengdu, China · Tongji University, Shanghai, China +1
It Is Not Seeing the Hazard: A Frozen Vision-Language Safety Score Measures Its Caption Bank
Oct 7, 2026Samuel Tetteh, Cody FlemingVLM Evaluation VLM Robustness
Iowa State University Ames, Iowa, USA
Anti-Persona: Disrupting Unauthorized Identity Binding and Recognition in Personalized Vision--Language Models
Oct 1, 2026Abhishek Basu, Fahad Shamshad, Karthik NandakumarVLM Robustness Vision-Language Models
MBZUAI, UAE · Michigan State University, USA
Do MLLM Judges Judge the Edit? Auditing Bias in Image Editing Evaluation with Verified Quality Preservation
Oct 1, 2026Yuan Huang, Zirui Song, Xiuying ChenVLM Robustness LLM-as-a-Judge
Northeastern University · Mohamed bin Zayed University of Artificial Intelligence
Towards Reliable Vision-Language Models for Autonomous Driving
Oct 1, 2026Manasa Mariam Mammen, Priyanka Mary Mammen, Zafer Kayatas +1VLM Robustness Image Corruption Robustness
Technical University of Munich, Heilbronn, Germany · Mercedes Benz AG, Sindelfingen, Germany · University of Massachusetts, Amherst, USA
When Masking Helps or Hurts Robustness in Compressed CLIP: A Pre-Deployment Diagnostic
Sep 30, 2026Muhammad Zawish, Steven DavyVLM Robustness Efficient VLM Inference
Technological University Dublin, Ireland
Typographic Attack Against VLM-based AI-generated Image Detection
Sep 30, 2026Eunmin Lee, Jungwoo Kim, Jong-Seok LeeVLM Evaluation VLM Robustness
School of Integrated Technology, Yonsei University
Feature-Aware Token Attack for Compression-Triggered Stealthy Failures in Large Vision-Language Models
Sep 30, 2026Shilinlu Yan, Bowen Chen, Yuechen Zhang +3VLM Robustness Adversarial Attacks on VLMs
Beijing University of Posts and Telecommunications · Jiangnan University · Nanyang Technological University +1
SceneJail: Exploiting Video Scenario Context to Jailbreak Multimodal LLMs
Sep 30, 2026Wenyu Chen, Li Wang, Chuanchao Zang +5VLM Robustness Jailbreak Attacks
Shandong University
Uncovering Uncontrolled Repetition through Residual Stream Dynamics
Sep 30, 2026Yuanhe Zhang, Xinyao Zhou, Haoran Gao +5VLM Robustness
Beijing University of Posts and Telecommunications · JIUTIAN Research · Nanyang Technological University +1
It Takes Little to Rewrite Perception: Targeted Semantic Substitution in Vision-Language Models at $ε\leq 4/255$
Sep 29, 2026Binchi Zhang, Atrisha Sarkar, Apurva NarayanVLM Robustness Adversarial Attacks on VLMs
University of Western Ontario
It's Not What the Image Shows: Irrelevant Context Destabilises VLM Judges Without Informing Them
Sep 29, 2026Nagham Omar, Mahmoud Jabarin, Kinan Ibraheem +1VLM Evaluation Figurative Language Understanding
Faculty of Data and Decision Sciences, Technion
Can Vision-Language Models Stay Helpful When Facing Implicit Risks? Intent-Privilege OPSD for Efficient Safety-Helpfulness Alignment
Sep 29, 2026Haotian Deng, Wenbin Xing, Gang Xu +5VLM Robustness Vision-Language Models
Southern University of Science and Technology · Sun Yat-sen University · Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) +5
Selective Channel Restoration for Backdoored Vision-Language Models
Sep 29, 2026Shuming Liu, Zhifang Zhang, Suqin Yuan +3VLM Robustness Large Vision-Language Models
Southeast University · The University of Queensland · University of Sydney +1
DSPO: Diversity-aware Subjective Policy Optimization for Robust Emotional Reasoning
Sep 29, 2026Cheng Ye, Weidong Chen, Bingyan Xu +1VLM Robustness Multimodal Reasoning
University of Science and Technology of China, Hefei
Still There, No Longer Seen: Exposing Compression-Induced Risk in Large Vision-Language Models
Sep 28, 2026Qiankun Li, Yuechen Zhang, Bowen Chen +4VLM Robustness Adversarial Attacks on VLMs
Nanyang Technological University · Beijing University of Posts and Telecommunications
Backdoor as Probe: Test-Time Adversarial Defense for CLIP
Sep 28, 2026Zhongqi Wang, Jie Zhang, Nie Sen +3VLM Robustness Backdoor Defense
Key Laboratory of AI Safety of CAS, Institute of Computing Technology, Chinese Academy of Sciences (CAS), Beijing, China · University of Chinese Academy of Sciences, Beijing, China · Xuzhou University of Technology, China
MoGround: Measuring and Mitigating Modality Distraction in Vision-Language Models
Sep 27, 2026Luca Zhou, Bo Zhao, Rose Yu +2VLM Evaluation Multimodal Grounding
Sapienza University of Rome · Harvard University · UC San Diego +2
Looks the Same, Answers Differently: Flip-Direction Steering for Robust Vision-Language Reasoning
Sep 23, 2026Yeonsung Jung, Joonhyun Jeong, Hoang Pham +4VLM Robustness Counterfactual Evaluation of VLMs
Graduate School of AI, KAIST · NAVER Cloud · Department of Computer Science and Engineering, The Ohio State University +3
PRISM-VLM: A Multi-Axis Discriminative Benchmark for Compact Vision-Language Models
Sep 23, 2026Sanghee Park, Kee-Eung KimVLM Evaluation VLM Robustness
NAVER Cloud AI · KAIST AI
Cross-Modal Attention Acts as a Frequency Filter: Why Verbose Prompts Improve Robustness in Vision-Language Models
Sep 17, 2026Farooq Ahmad Wani, Maria Sofia Bucarelli, Mujtaba Hussain Mirza +5VLM Robustness Prompt Sensitivity
Sapienza University of Rome · Université Côte d’Azur · CNRS +3
Decodable but Misrouted: Sparse Features Uncover a Readout Gap in Vision-Language Models for Harmful Meme Detection
Sep 16, 2026Girish A. Koushik, Diptesh Kanojia, Helen TreharneVLM Evaluation VLM Robustness
Nature-Inspired Computing & Engineering Department of Computer Science University of Surrey Guildford, United Kingdom · Surrey Centre for Cyber Security Department of Computer Science University of Surrey Guildford, United Kingdom
EDCT-Bench: Uncovering Faithfulness Gaps in VLMs via Explanation-Driven Counterfactual Testing
Sep 16, 2026Sihao Ding, Santosh Vasa, Aditi Ramadwar +1VLM Evaluation VLM Robustness
Mercedes-Benz Research & Development North America
Beyond Accuracy: Robustness, Cost, and Governance Trade-offs for Vision-Language Models in Templated Document Extraction
Sep 14, 2026Kushal Patel, Pushkal Shrivastava, Mackenzie Lees +4VLM Evaluation VLM Robustness
John Hancock, 200 Berkeley St, Boston, MA 02116, USA
Can Edge-Deployable Vision-Language Models Identify Species?
Sep 12, 2026William Zhou, Mayukha Siripuram, Xiao Yan +2VLM Evaluation VLM Robustness
Plano West Senior High School
Isotropic Embedding Perturbations for Robust Vision Language Encoders
Sep 9, 2026Hyesong Choi, Daeun Kim, Song Park +5VLM Robustness Data Augmentation
Soongsil Univ. · NAVER AI Lab · Ewha W. Univ.
Do Input-Level Defenses Transfer to Observation-Level Attacks on VideoLLMs?
Sep 8, 2026Bangshuo Zhu, Wei Song, Yuxin Cao +4VLM Robustness Adversarial Attacks on VLMs
School of Computer Science and Engineering, University of New South Wales, New South Wales 2052, Australia · School of Information and Communication Technology, Griffith University, Queensland 4111, Australia · School of Computing, National University of Singapore, Singapore 117417, Singapore +2
CS-CLIP: Compositional Scene Graph-guided CLIP for Robust Compositional Reasoning
Sep 8, 2026SeongJun Jeong, Minjoon Jung, Woo Suk Choi +2VLM Robustness Contrastive Learning
Seoul National University

Oct 7, 2026, cs.CVJ/K move · Enter open · S save

Detecting Adversarial Images through Response Profiles of Vision-Language Models

Arash Vashagh, Roozbeh Razavi-Far

Trustworthy and Secure AI (TSAI) Lab, Faculty of Computer Science, University of New Brunswick, Canada

Adversarial Example Detection VLM Robustness Adversarial Attacks on VLMs

Open paper AskPDF arXiv

Adversarial perturbations can alter the predictions of frozen vision-language models (VLMs) while leaving their confidence and image--text similarity patterns seemingly plausible. We investigate whether we can identify adversarial inputs based on the broader way an image interacts with a collection of general semantic prompts. Our detector summarizes these responses using category-level statistics, relationships among prompts, deviations from clean reference distributions, and stability under weak image transformations, producing a compact response profile that is classified by a lightweight model while the VLM remains fixed. We evaluate the approach on multiple public image datasets, several CLIP-style visual backbones, and a range of gradient-based, optimization-based, automated, and spatial attacks. The detector achieves strong discrimination in attack-specific settings and retains substantial performance when evaluated on attacks not seen during training. Under a controlled detector-specific protocol, the response-profile representation outperforms the evaluated embedding-geometry baselines. Additional analyses show that the feature groups provide complementary information and that the method remains effective under variations in the prompt configuration. We also examine inference cost and performance against detector-aware adaptive attacks. Overall, the results indicate that response patterns across semantic prompts provide a useful complementary signal for adversarial image detection in frozen VLMs.

Momentum

Related topics

Latest papers 295