Adversarial Attacks on VLMs

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

8 papers in the last four weeks, up 14% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 106

All topics

JECA^2: Judgment-Explanation Consistent Adversarial Attack against Forensic Vision-Language Models
May 27, 2026Jiachen QianVLM Robustness Adversarial Attacks on VLMs
City University of Hong Kong
KSAFE-MM: A Multimodal Safety Benchmark via Localized Contextualization for Korean Cultural Risks
May 27, 2026Yongwoo Kim, Sojung An, Yunjin Park +8Adversarial Attacks on VLMs Language Model Safety Evaluation
Korea University · KT Corporation
Structure-Guided Visual Perturbation Neutralization for LVLMs
May 27, 2026Yuanhe Zhang, Xueting Wang, YanBin Ren +6VLM Robustness Adversarial Attacks on VLMs
1Beijing University of Posts and Telecommunications · University of Science and Technology of China · 3JIUTIAN Research +2
Unveiling the Fragility of Vision-Language Models: Multi-Modal Adversarial Synergy via Texture-Constrained Perturbations and Cross-Modal Optimization
May 26, 2026Xiang Fang, Wanlong Fang, Changshuo WangVLM Robustness Adversarial Attacks on VLMs
School of Software Engineering, Huazhong University of Science and Technology · Nanyang Technological University, Singapore · University College London
StructBreak: Structural Cognitive Overload-Induced Safety Failures in MLLMs
May 25, 2026Yang Luo, Xinran Liu, Tiantian Ji +3Adversarial Attacks on VLMs Language Model Safety Evaluation
Key Laboratory of Trustworthy Distributed Computing and Service (MoE), Beijing University of Posts and Telecommunications · Institute of Computing Technology, Chinese Academy of Sciences
Localization then Neutralization: Gradient-guided Token Suppression against Visual Prompt Injection Attack
May 24, 2026Dongpeng Zhang, Ke Ma, Yangbangyan Jiang +4Gradient-Based Attribution VLM Robustness
School of Advanced Interdisciplinary Sciences, UCAS · School of Electronic, Electrical and Communication Engineering, UCAS · School of Computer Science and Technology, UCAS +4
Furina: Fragmented Uncertainty-Driven Refusal Instability Attack
May 24, 2026Tongxi Wu, Jian Zhang, Yang GaoAdversarial Attacks on VLMs LLM Safety
School of Intelligence Science and Technology, Nanjing University, Suzhou, China · State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China
Med-R2: An Adversarial Benchmark for Evidence-Grounded Reasoning in Medical VLMs
May 23, 2026Wen Ma, Fucheng Niu, Zhiting Fan +3Adversarial Attacks on VLMs Healthcare
Zhejiang University, Zhejiang, China · Guangdong Institute of Intelligence Science and Technology
Same Model, Different Weakness: How Language and Modality Reshape the Jailbreak Attack Surface in Frontier MLLMs
May 22, 2026Casey Ford, Madison Van Doren, Sicheng Jin +1Multilingual Language Model Evaluation Adversarial Attacks on VLMs
Appen
Exposing Vulnerabilities in Visible-Infrared VLMs: A Unified Geometric Adversarial Framework with Cross-Task Transferability
May 21, 2026Xiang Chen, Yuxian Dong, Chao Li +6VLM Robustness Adversarial Patch Attacks
Frequency-Domain Regularized Adversarial Alignment for Transferable Attacks against Closed-Source MLLMs
May 20, 2026Leitao Yuan, Qinghua Mao, Daizong Liu +5Adversarial Attacks on VLMs Multimodal Large Language Models
Shanghai Artificial Intelligence Laboratory · Zhejiang University · Shanghai Jiao Tong University +3
Towards Fine-Grained Robustness: Attention-Guided Test-Time Prompt Tuning for Vision-Language Models
May 19, 2026Jia-Wei Hai, Yijun Wang, Xiu-Shen WeiVLM Robustness Adversarial Attacks on VLMs
School of Computer Science and Engineering, and Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications, Southeast University, China. · School of Intelligence Science and Engineering, Southeast University, China.
Surviving the Unseen: Predictive Defense for Novel Multi-Turn Multimodal Attacks
May 18, 2026Doohee YouAdversarial Attacks on VLMs Multimodal Robustness
Trust and Safety, Google
Not What You Asked For: Typographic Attacks in Household Robot Manipulation
May 18, 2026Ali Iranmanesh, Peng LiuAdversarial Attacks on VLMs Robotic Manipulation
Cyber Security Lab The Pennsylvania State University State College, USA
TAME: Test-Time Adversarial Prompt Tuning via Mixture-of-Experts for Vision-Language Models
May 17, 2026Xin Wang, Yixu Wang, Jiaming Zhang +6Adversarial Attacks on VLMs Test-Time Adaptation
Attention Hijacking: Response Manipulation Across Queries in Vision-Language Models
May 17, 2026Zhiqiang Wang, Dongrui Liu, Yan Li +4Adversarial Attacks on VLMs Adversarial Attacks
Hong Kong University of Science and Technology · Shanghai Jiao Tong University · Beihang University
Right Predictions, Misleading Explanations: On the Vulnerability of Vision-Language Model Explanations
May 15, 2026Narges Babadi, Hadis KarimipourVLM Robustness Adversarial Attacks on VLMs
Department of Electrical and Software Engineering, University of Calgary, Calgary, Canada.
A Cross-Modal Prompt Injection Attack against Large Vision-Language Models with Image-Only Perturbation
May 15, 2026Hao Yang, Zhuo Ma, Yang Liu +3VLM Robustness Adversarial Attacks on VLMs
Xidian University Xi’an, Shaanxi, China
DarkLLM: Learning Language-Driven Adversarial Attacks with Large Language Models
May 15, 2026Ye Sun, Xin Wang, Jiaming Zhang +7Adversarial Attacks on VLMs Adversarial Perturbations
Fudan University · Nanyang Technological University · Tongji University
SafeSteer: A Decoding-level Defense Mechanism for Multimodal Large Language Models
May 12, 2026Xinyi Zeng, Xue Yang, Jingyuan Zhang +5Adversarial Attacks on VLMs Multimodal Robustness
Tsinghua University, Beijing, China · Shanghai Jiao Tong University · Kuaishou Technology, Beijing, China +4
Break the Brake, Not the Wheel: Untargeted Jailbreak via Entropy Maximization
May 11, 2026Mengqi He, Xinyu Tian, Xin Shen +6Adversarial Attacks on VLMs Adversarial Attacks
Australian National University, ACT, Australia · The University Of Queensland, Brisbane, Queensland, Australia · GE research, Niskayuna, New York, USA +2
Knowledge Poisoning Attacks on Medical Multi-Modal Retrieval-Augmented Generation
May 11, 2026Peiru Yang, Haoran Zheng, Tong Ju +6Multimodal RAG Adversarial Attacks on VLMs
1Tsinghua University · 2Beijing University of Posts and Telecommunications · 3Northwestern Polytechnical University +1
Adversarial Attacks Against MLLMs via Progressive Resolution Processing and Adaptive Feature Alignment
May 11, 2026Haobo Wang, Xiaorong Ma, Weiqi Luo +2Adversarial Attacks on VLMs Multimodal Large Language Models
Sun Yat-sen University, China · Nanyang Technological University, Singapore · Shenzhen MSU-BIT University, China
Sparse Autoencoders as Plug-and-Play Firewalls for Adversarial Attack Detection in VLMs
May 8, 2026Hao Wang, Yiqun Sun, Pengfei Wei +2VLM Robustness Adversarial Attacks on VLMs
2Waseda University · 1Magellan Technology Research Institute (MTRI)
GPO-V: Jailbreak Diffusion Vision Language Model by Global Probability Optimization
May 8, 2026Yu Pan, Andi Zhang, Yi Wang +2Diffusion Model Alignment Adversarial Attacks on VLMs
ShanghaiTech University · University of Warwick · SUN YAT-SEN UNIVERSITY
Hard to Read, Easy to Jailbreak: How Visual Degradation Bypasses MLLM Safety Alignment
May 8, 2026Zhixue Song, Boyan Han, Yiwei Wang +1VLM Robustness Adversarial Attacks on VLMs
1AGI Lab, Westlake University, China · University of California, Merced, USA
Laundering AI Authority with Adversarial Examples
May 5, 2026Jie Zhang, Pura Peetathawatchai, Florian Tramèr +1VLM Robustness Adversarial Attacks on VLMs
ETH Zurich · Switzerland
CBV: Clean-label Backdoor Attacks on Vision Language Models via Diffusion Models
May 4, 2026Ji Guo, Xiaolong Qin, Cencen Liu +3VLM Robustness Adversarial Attacks on VLMs
Laboratory of Intelligent Collaborative Computing, University of Electronic Science and Technology of China, China · School of Software Engineering, Chengdu University of Information Technology, China · The Hong Kong University of Science and Technology, China
VisInject: Disruption != Injection -- A Dual-Dimension Evaluation of Universal Adversarial Attacks on Vision-Language Models
May 2, 2026Pang Liu, Yingjie LaoVLM Evaluation VLM Robustness
Department of Electrical and Computer Engineering, Tufts University
Jailbreaking Vision-Language Models Through the Visual Modality
May 1, 2026Aharon Azulay, Jan Dubiński, Zhuoyun Li +2Adversarial Attacks on VLMs Vision-Language Models
Independent · NASK National Research Institute, Warsaw, Poland · Warsaw University of Technology, Warsaw, Poland +3

May 27, 2026, cs.CVJ/K move · Enter open · S save

JECA^2: Judgment-Explanation Consistent Adversarial Attack against Forensic Vision-Language Models

Jiachen Qian

City University of Hong Kong

VLM Robustness Adversarial Attacks on VLMs Digital Image Forensics

Open paper AskPDF arXiv

Forensic vision-language models (VLMs) have recently been developed to detect image tampering and provide natural-language explanations. However, their robustness against adversarial manipulation remains underexplored. Existing adversarial attacks typically aim to flip the model's binary judgment, while the accompanying explanation may still reveal forensic cues and contradict the attacked judgment. In this paper, we study judgment-explanation consistent adversarial attacks against forensic VLMs and propose JECA^2, a controlled white-box red-team diagnostic that jointly redirects visual attribution and aligns textual explanations with the target judgment. On the visual side, JECA^2 uses Grad-CAM-guided perturbations to divert attribution from tampered regions toward benign regions. On the textual side, it optimizes prompt embeddings toward authenticity-affirming semantics under a token-proximity constraint. Experiments on forensic VLM benchmarks show that JECA^2 achieves higher attack success and automated judgment-explanation consistency than implemented baselines under white-box threat settings, while transfer to closed-source VLMs remains measurable but limited. Our results highlight a consistency failure mode in explanation-based forensic VLMs and motivate future robustness evaluation beyond binary detection accuracy.

Momentum

Related topics

Latest papers 106