Backdoor Attacks on VLMs

VLM: Vision-Language Model

Overview

Backdoor attacks on vision-language models study how poisoned training data or altered model components can make an image, phrase, or semantic pattern trigger an attacker-chosen response. Attacks use methods such as text or visual triggers, clean-label poisoning, and manipulation of multimodal representations, targeting tasks from image retrieval and question answering to embodied actions. Defenses include filtering suspicious training examples, detecting anomalies in attention or representations, and repairing models through fine-tuning, channel restoration, or feature purification. A central challenge is that stealthy attacks can persist across fine-tuning, interfaces, or long conversations, while defenses must reduce attack success without harming ordinary performance or missing unfamiliar triggers.

Latest papers

CardsList

40 papers3 papers in the last four weeks, down 67% on the four weeks before. 0.0% of all new papers.

  1. As Student Rollouts Continue, Visual Backdoors Keep Growing

    Nov 17, 2026Zane Fu, Yanzhao Zheng, Jinze Liu +7Backdoor Attacks on VLMsOn-Policy Distillation

  2. HalluProbe: Exploiting VLM Hallucinations for Backdoor Detection

    Nov 15, 2026Ji Guo, Xiaolong Qin, Jin Jiang +4Backdoor Attacks on VLMs

  3. Context-Shielded Backdoor Attack against Large Vision-Language Models

    Oct 31, 2026Zhifang Zhang, Qiang Huang, Chenpeng Wang +6Backdoor Attacks on VLMs

  4. ShadowFPT: Backdooring Federated Prompt Tuning via Shadow Triggers

    Oct 16, 2026Kun Zhai, Teng Li, Yunhao Feng +1Backdoor Attacks in Federated LearningBackdoor Attacks on VLMs

  5. Clean Data Can Still Carry Backdoors: Support-Persistent Backdoors for Model Reuse

    Oct 15, 2026Junhoo Lee, Baekseung Kim, Seungyeon Kim +1Backdoor AttacksBackdoor Attacks on VLMs

  6. EvolvAttack: Backdooring self Evolving VLM Agents

    Oct 11, 2026Chuan Zou, Yang Xiang, Zexin Li +2Backdoor AttacksBackdoor Attacks on VLMs

  7. Purifying Backdoored Large Vision-Language Models by Removing Hijacked Directions

    Oct 7, 2026Bojun Yang, Haochen Zhou, Zhifang Zhang +3VLM RobustnessLarge Vision-Language Models

  8. Visual Memory Attacks Can Persist Through The KV Cache

    Oct 6, 2026David Dobre, Leo Schwinn, Gauthier Gidel +3Adversarial Attacks on VLMsBackdoor Attacks on VLMs

  9. The Poisoned Conversation: Privacy-Leaking Watermarks in Unified Multimodal Models

    Oct 4, 2026Tobias Braun, Jonas Henry Grebe, Emil Sivic +4Unified Multimodal ModelsMultimodal Generation

  10. Selective Channel Restoration for Backdoored Vision-Language Models

    Sep 29, 2026Shuming Liu, Zhifang Zhang, Suqin Yuan +3VLM RobustnessLarge Vision-Language Models

  11. Region-Level Black-Box Defense Against Stealthy Embedding-Space Backdoors in CLIP

    Sep 25, 2026Ahmed Abdelnaby, Mohamed ElmahallawyBackdoor DefenseVLM Robustness

  12. TraceGuard: Adaptive Multimodal Poison Filtering through Cross-Feature Rank Agreement

    Sep 24, 2026Haoyang Li, Yaxin Xiao, Linyan Dai +5Multimodal RobustnessBackdoor Attacks on VLMs

  13. DEFUSE: Generalizable Backdoor Defense for Self-Supervised Encoders with Generative Priors

    Aug 26, 2026Tuo Chen, Jie Gui, Minjing Dong +4Backdoor DefenseBackdoor Attacks on VLMs

  14. Not All Tokens Are Equal: Region-Aware Consistency Repair of Backdoors in MLLMs

    Aug 25, 2026Jiali Wei, Ming Fan, Mingkun Zhang +6Backdoor Attacks on VLMsMultimodal Large Language Models

  15. Once Poisoned, Arbitrarily Controlled: A Programmable Backdoor in VLMs

    Aug 11, 2026Tao Lin, Gaojie Jin, Zongxin Liu +2Adversarial Attacks on VLMsVision-Language Models

  16. StepTrigger: Contact-State-Triggered Backdoor Attacks on VLM-Powered Legged Robots

    Aug 7, 2026Jiageng Zhang, Doniyorkhon Obidov, Kaichen YangBackdoor AttacksBackdoor Attacks on VLMs

  17. Architectural Backdoors in Vision-Language Model Supply Chains via Representation Steering

    Jul 28, 2026Maria Rosaria Briglia, Igor Maljkovic, Antonio Emanuele Cinà +3Adversarial Attacks on VLMsRepresentation Engineering

  18. TrustVLA: Mechanism-Guided Inference-Time Defense Against Vision-Language-Action Backdoors

    Jul 14, 2026Pinhan Fu, Xianda Guo, Xuetao Li +5Adversarial Attacks on VLA ModelsVision-Language-Action Models

  19. !Imperio, smolVLA: The Implications of Data Poisoning on Open Source Robotics

    Jul 5, 2026Stefan Bühler, Mark SchuteraNeural Network RobustnessAdversarial Attacks on VLA Models

  20. Beyond Native Success: Auditing Deployment-Interface Exposure of CLIP Backdoors

    Jun 16, 2026Kunlan Xiang, Haomiao Yang, Wenbo JiangVLM RobustnessAdversarial Attacks on VLMs

  21. BYORn: Bootstrap Your Own Responses to Defend Large Vision-Language Models Against Backdoor Attacks

    Jun 1, 2026Ivan Sabolić, Marin Oršić, Josip Šarić +1Supervised Fine-TuningVLM Robustness

  22. EntropyScan: Towards Model-level Backdoor Detection in LVLMs via Visual Attention Entropy

    May 15, 2026Xuanyu Ge, Zhongqi Wang, Jie Zhang +2Large Vision-Language ModelsBackdoor Detection

  23. ATAAT: Adaptive Threat-Aware Adversarial Tuning Framework against Backdoor Attacks on Vision-Language-Action Models

    May 9, 2026Kewei Chen, Yayu Long, Shuai Li +1Adversarial AttacksAdversarial Attacks on VLA Models

  24. CBV: Clean-label Backdoor Attacks on Vision Language Models via Diffusion Models

    May 4, 2026Ji Guo, Xiaolong Qin, Cencen Liu +3VLM RobustnessAdversarial Attacks on VLMs

  25. ProjLens: Unveiling the Role of Projectors in Multimodal Model Safety

    Apr 21, 2026Kun Wang, Cheng Qian, Miao Yu +6LLM Backdoor AttacksMultimodal Large Language Models

  26. CLIP-Inspector: Model-Level Backdoor Detection for Prompt-Tuned CLIP via OOD Trigger Inversion

    Apr 10, 2026Akshit Jindal, Saket Anand, Chetan Arora +1Backdoor Attacks on VLMs

  27. Phantasia: Context-Adaptive Backdoors in Vision Language Models

    Apr 9, 2026Nam Duong Tran, Phi Le NguyenBackdoor Attacks on VLA ModelsBackdoor Attacks on VLMs