Backdoor Attacks on VLMs

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

3 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 18

All topics

Purifying Backdoored Large Vision-Language Models by Removing Hijacked Directions
Oct 7, 2026Bojun Yang, Haochen Zhou, Zhifang Zhang +3VLM Robustness Large Vision-Language Models
Southeast University · The University of Queensland · Zhejiang University
Visual Memory Attacks Can Persist Through The KV Cache
Oct 6, 2026David Dobre, Leo Schwinn, Gauthier Gidel +3Adversarial Attacks on VLMs Backdoor Attacks on VLMs
ServiceNow AI Research · Université de Montréal · Mila +2
The Poisoned Conversation: Privacy-Leaking Watermarks in Unified Multimodal Models
Oct 4, 2026Tobias Braun, Jonas Henry Grebe, Emil Sivic +4Unified Multimodal Models Multimodal Generation
TU Darmstadt, Germany · hessian.AI, Germany · Zuse School
Selective Channel Restoration for Backdoored Vision-Language Models
Sep 29, 2026Shuming Liu, Zhifang Zhang, Suqin Yuan +3VLM Robustness Large Vision-Language Models
Southeast University · The University of Queensland · University of Sydney +1
TraceGuard: Adaptive Multimodal Poison Filtering through Cross-Feature Rank Agreement
Sep 24, 2026Haoyang Li, Yaxin Xiao, Linyan Dai +5Multimodal Robustness Backdoor Attacks on VLMs
The Hong Kong Polytechnic University · Commonwealth Scientific and Industrial Research Organisation (CSIRO)
Once Poisoned, Arbitrarily Controlled: A Programmable Backdoor in VLMs
Aug 11, 2026Tao Lin, Gaojie Jin, Zongxin Liu +2Adversarial Attacks on VLMs Vision-Language Models
Key Laboratory of System Software (Chinese Academy of Sciences), Beijing, China · Institute of Software, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China +2
Architectural Backdoors in Vision-Language Model Supply Chains via Representation Steering
Jul 28, 2026Maria Rosaria Briglia, Igor Maljkovic, Antonio Emanuele Cinà +3Adversarial Attacks on VLMs Representation Engineering
Sapienza University of Rome, Italy · University of Genoa, Italy · University of Trieste, Italy +1
TrustVLA: Mechanism-Guided Inference-Time Defense Against Vision-Language-Action Backdoors
Jul 14, 2026Pinhan Fu, Xianda Guo, Xuetao Li +5Adversarial Attacks on VLA Models Vision-Language-Action Models
School of Computer Science, Wuhan University · Institute of Automation, Chinese Academy of Sciences · 3HUST +1
!Imperio, smolVLA: The Implications of Data Poisoning on Open Source Robotics
Jul 5, 2026Stefan Bühler, Mark SchuteraNeural Network Robustness Adversarial Attacks on VLA Models
Independent Researcher · Duale Hochschule Baden-Württemberg, Ravensburg
Beyond Native Success: Auditing Deployment-Interface Exposure of CLIP Backdoors
Jun 16, 2026Kunlan Xiang, Haomiao Yang, Wenbo JiangVLM Robustness Adversarial Attacks on VLMs
University of Electronic Science and Technology of China
BYORn: Bootstrap Your Own Responses to Defend Large Vision-Language Models Against Backdoor Attacks
Jun 1, 2026Ivan Sabolić, Marin Oršić, Josip Šarić +1Supervised Fine-Tuning VLM Robustness
Faculty of Electrical Engineering and Computing, University of Zagreb, Croatia.
EntropyScan: Towards Model-level Backdoor Detection in LVLMs via Visual Attention Entropy
May 15, 2026Xuanyu Ge, Zhongqi Wang, Jie Zhang +2Large Vision-Language Models Backdoor Detection
China University of Geosciences · University of the Chinese Academy of Sciences · Institute of Computing Technology, Chinese Academy of Sciences
ATAAT: Adaptive Threat-Aware Adversarial Tuning Framework against Backdoor Attacks on Vision-Language-Action Models
May 9, 2026Kewei Chen, Yayu Long, Shuai Li +1Adversarial Attacks Adversarial Attacks on VLA Models
Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Sciences · Chongqing School, University of Chinese Academy of Sciences · Faculty of Information Technology and Electrical Engineering, University of Oulu, Finland
CBV: Clean-label Backdoor Attacks on Vision Language Models via Diffusion Models
May 4, 2026Ji Guo, Xiaolong Qin, Cencen Liu +3VLM Robustness Adversarial Attacks on VLMs
Laboratory of Intelligent Collaborative Computing, University of Electronic Science and Technology of China, China · School of Software Engineering, Chengdu University of Information Technology, China · The Hong Kong University of Science and Technology, China
ProjLens: Unveiling the Role of Projectors in Multimodal Model Safety
Apr 21, 2026Kun Wang, Cheng Qian, Miao Yu +6LLM Backdoor Attacks Multimodal Large Language Models
3Nanyang Technological University · 2Beijing University of Aeronautics and Astronautics · University of Science and Technology of China +2
BackdoorVLM: A Benchmark for Backdoor Attacks and Defenses on Vision-Language Models
Nov 24, 2025Juncheng Li, Yige Li, Hanxun Huang +5VLM Evaluation VLM Robustness
Fudan University · Singapore Management University · The University of Melbourne
TokenSwap: Backdoor Attack on the Compositional Understanding of Large Vision-Language Models
Sep 29, 2025Zhifang Zhang, Qiqi Tao, Jiaqi Lv +3VLM Robustness Adversarial Attacks on VLMs
School of Computer Science and Engineering, Southeast University, China · School of Electrical Engineering and Computer Science, University of Queensland, Australia · Information Systems Technology and Design Pillar, Singapore University of Technology and Design, Singapore +2
DropVLA: An Action-Level Backdoor Attack on Vision-Language-Action Models
Date pendingZonghuan Xu, Jiayu Li, Yunhan Zhao +3Adversarial Attacks on VLA Models Robotic Manipulation
Institute of Trustworthy Embodied AI, Fudan University, Shanghai, China. · Shanghai Key Laboratory of Multimodal Embodied AI, Shanghai, China. · City University of Hong Kong, Hong Kong SAR, China.

Oct 7, 2026, cs.CVJ/K move · Enter open · S save

Purifying Backdoored Large Vision-Language Models by Removing Hijacked Directions

Bojun Yang, Haochen Zhou, Zhifang Zhang, Haobo Wang+2

Southeast University · The University of Queensland · Zhejiang University

VLM Robustness Large Vision-Language Models Backdoor Attacks on VLMs

Open paper AskPDF arXiv

Large vision-language models (LVLMs) are increasingly deployed in safety-critical applications, yet they remain vulnerable to backdoor attacks. Defending against such attacks remains costly, as existing methods require either extensive retraining on clean data or per-query intervention at inference time. To address this limitation, we propose OrthoPurify, a more efficient method to purify backdoored model weights via one-step orthogonal projection. Specifically, through structural analysis of backdoor weight updates, we find that the backdoor is encoded by diverting a small number of weight update directions from task adaptation to backdoor shortcut encoding, a phenomenon we term direction hijacking. However, identifying these hijacked directions requires a benign reference model, which is typically inaccessible to the defender. We show that a pseudo-benign model, obtained by fine-tuning the pretrained weights on only a small set of clean samples, provides a sufficient approximation, as the dominant update directions stabilize within the first few gradient steps. OrthoPurify uses this pseudo-benign reference to isolate the hijacked directions and removes them through a single projection on the weight update. Extensive experiments show that OrthoPurify reduces the attack success rate to near zero while preserving the original performance across diverse benchmarks, without retraining the backdoored model or introducing inference-time overhead. Our code is publicly available at https://github.com/womeimingzi/OrthoPurify.

Momentum

Related topics

Latest papers 18