cs.CVOct 31, 2026

Context-Shielded Backdoor Attack against Large Vision-Language Models

Authors: Zhifang Zhang, Qiang Huang, Chenpeng Wang, Weitong Chen, Olaf Maennel, Wei Emma Zhang, Lei Feng, Lei Zhang, 

Organizations: The University of Queensland · Fudan University · YiXin-AILab · Adelaide University · 

Abstract

Large vision-language models (LVLMs) routinely use inference-time visual-token compression to prune hundreds of patch tokens down to a compact budget. Existing LVLM backdoors fire whenever a triggered image appears, whereas prior deployment-triggered attacks activate only when model weights are quantized or pruned. However, conventional LVLM backdoors exhibit nearly 100% attack success rate (ASR) under full-token inference, immediately exposing compromised checkpoints during pre-deployment auditing. We reveal context stripping: surrounding clean visual tokens suppress an embedded trigger at full length, whereas token pruning strips this context away and activates the backdoor. Motivated by this finding, we propose StripWake, a projector-only backdoor that remains dormant under full-token auditing yet activates across unseen compressors. Specifically, StripWake pairs clean-supervised full-token views with target-supervised randomized compressed views, and aligns early visual attention with a frozen benign teacher to preserve clean utility. On LLaVA-1.5-7B across nine unseen compressors, StripWake stays dormant at 0.4% full-token ASR yet achieves 94.0%-99.7% compressed ASR across five primary compressors without CIDEr loss.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. TokenSwap: Backdoor Attack on the Compositional Understanding of Large Vision-Language Models

    Sep 29, 2025Zhifang Zhang, Qiqi Tao, Jiaqi Lv +3VLM RobustnessAdversarial Attacks on VLMs

  2. Test-Time Attention Purification for Backdoored Large Vision Language Models

    Mar 13, 2026Zhifang Zhang, Bojun Yang, Shuo He +5Backdoor Attacks on VLMsAdversarial Attacks on VLMs

  3. Feature-Aware Token Attack for Compression-Triggered Stealthy Failures in Large Vision-Language Models

    Sep 30, 2026Shilinlu Yan, Bowen Chen, Yuechen Zhang +3VLM RobustnessAdversarial Attacks on VLMs