Adversarial Attacks on VLMs

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

8 papers in the last four weeks, up 14% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 106

All topics

Understanding Adversarial Transferability in Vision-Language Models for Autonomous Driving: A Cross-Architecture Analysis
Apr 30, 2026David Fernandez, Pedram MohajerAnsari, Amir Salarpour +1Adversarial Attacks on VLMs Vision-Language Models
School of Computing, Clemson University, USA
One Perturbation, Two Failure Modes: Probing VLM Safety via Embedding-Guided Typographic Perturbations
Apr 28, 2026Ravikumar Balakrishnan, Sanket MendaparaVLM Evaluation VLM Robustness
Cisco Systems
If you're waiting for a sign... that might not be it! Mitigating Trust Boundary Confusion from Visual Injections on Vision-Language Agentic Systems
Apr 21, 2026Jiamin Chang, Minhui Xue, Ruoxi Sun +3VLM Robustness Adversarial Attacks on VLMs
University of New South Wales, Sydney, NSW 2052, Australia · CSIRO’s Data61, Marsfield, NSW 2122, Australia · Macquarie University, NSW 2109, Australia
When Background Matters: Breaking Medical Vision Language Models by Transferable Attack
Apr 19, 2026Akash Ghosh, Subhadip Baidya, Sriparna Saha +1VLM Robustness Adversarial Attacks on VLMs
Indian Institute of Technology Patna1 · Work done as Visiting Researcher at MBZUAI · MBZUAI3 +1
Cross-Modal Attention Analysis and Optimization in Vision-Language Models: A Study on Visual Reliability
Apr 19, 2026Lijie ZhouCross-Modal Learning VLM Robustness
School of Computer Science University of Nottingham Ningbo China Ningbo, China
Robustness of Vision Language Models Against Split-Image Harmful Input Attacks
Feb 8, 2026Md Rafi Ur Rashid, MD Sadik Hossain Shanto, Vishnu Asutosh Dasu +1VLM Robustness Adversarial Attacks on VLMs
Pennsylvania State University · Bangladesh University of Engineering and Technology
Stage-wise Attention-Guided Region Sequencing for Adversarial Attacks on Large Vision-Language Models
Feb 4, 2026Jaehyun Kwak, Nam Cao, Boryeong Cho +3Adversarial Attacks on VLMs Adversarial Attacks
KAIST · KENTECH
ChartAttack: Testing the Vulnerability of LLMs to Malicious Prompting in Chart Generation
Jan 19, 2026Jesus-German Ortiz-Barajas, Jonathan Tonglet, Vivek Gupta +1Data Visualization Adversarial Attacks on VLMs
INSAIT · KU Leuven · Arizona State University +2
High-Entropy Tokens as Multimodal Failure Points in Vision-Language Models
Dec 26, 2025Mengqi He, Xinyu Tian, Xin Shen +4VLM Robustness Adversarial Attacks on VLMs
The Australia National University Canberra, Australia · The University of Queensland Brisbane, Queensland, Australia · GE research Niskayuna, New York, USA
Counterfeit Answers: Adversarial Forgery against OCR-Free Document Visual Question Answering
Dec 4, 2025Marco Pintore, Maura Pintor, Dimosthenis Karatzas +1Visual Question Answering VLM Robustness
University of Cagliari, Italy · CINI, Italy · Computer Vision Center, UAB, Spain
BackdoorVLM: A Benchmark for Backdoor Attacks and Defenses on Vision-Language Models
Nov 24, 2025Juncheng Li, Yige Li, Hanxun Huang +5VLM Evaluation VLM Robustness
Fudan University · Singapore Management University · The University of Melbourne
FoCLIP: A Feature-Space Misalignment Framework for CLIP-Based Image Manipulation and Detection
Nov 10, 2025Yulin Chen, Zeyuan Wang, Tianyuan Yu +2VLM Robustness Adversarial Attacks on VLMs
Laboratory for Big Data and Decision, National University of Defense Technology, Changsha 410073, China
TokenSwap: Backdoor Attack on the Compositional Understanding of Large Vision-Language Models
Sep 29, 2025Zhifang Zhang, Qiqi Tao, Jiaqi Lv +3VLM Robustness Adversarial Attacks on VLMs
School of Computer Science and Engineering, Southeast University, China · School of Electrical Engineering and Computer Science, University of Queensland, Australia · Information Systems Technology and Design Pillar, Singapore University of Technology and Design, Singapore +2
VISOR: Visual Input-based Steering for Output Redirection in Vision-Language Models
Aug 11, 2025Mansi Phute, Ravikumar BalakrishnanVLM Robustness Adversarial Attacks on VLMs
Georgia Institute of Technology · HiddenLayer, Inc
Exploiting Vision Encoder Vulnerabilities for Universal Adversarial Perturbations on Large Vision-Language Models
Dec 11, 2024Hee-Seon Kim, Minbeom Kim, Seokil Ham +1VLM Robustness Adversarial Attacks on VLMs
School of Electrical Engineering, KAIST, Daejeon 34141, Republic of Korea
InstructTA: Instruction-Tuned Targeted Attack for Large Vision-Language Models
Dec 4, 2023Xunguang Wang, Pingchuan Ma, Zhenlan Ji +3Adversarial Attacks on VLMs Large Vision-Language Models
The Hong Kong University of Science and Technology, Hong Kong, China

Apr 30, 2026, cs.CVJ/K move · Enter open · S save

Understanding Adversarial Transferability in Vision-Language Models for Autonomous Driving: A Cross-Architecture Analysis

David Fernandez, Pedram MohajerAnsari, Amir Salarpour, Mert D. Pese

School of Computing, Clemson University, USA

Adversarial Attacks on VLMs Vision-Language Models VLMs for Autonomous Driving Autonomous Driving Transfer-Based Adversarial Attacks Adversarial Transferability+1

Open paper AskPDF arXiv

Vision-language models (VLMs) are increasingly used in autonomous driving because they combine visual perception with language-based reasoning, supporting more interpretable decision-making, yet their robustness to physical adversarial attacks, especially whether such attacks transfer across different VLM architectures, is not well understood and poses a practical risk when attackers do not know which model a vehicle uses. We address this gap with a systematic cross-architecture study of adversarial transferability in VLM-based driving, evaluating three representative architectures (Dolphins, OmniDrive, and LeapVAD) using physically realizable patches placed on roadside infrastructure in both crosswalk and highway scenarios. Our transfer-matrix evaluation shows high cross-architecture effectiveness, with transfer rates of 73-91% (mean TR = 0.815 for crosswalk and 0.833 for highway) and sustained frame-level manipulation over 64.7-79.4% of the critical decision window even when patches are not optimized for the target model.

Momentum

Related topics

Latest papers 106