cs.CRNov 15, 2026

HalluProbe: Exploiting VLM Hallucinations for Backdoor Detection

Authors: Ji Guo, Xiaolong Qin, Jin Jiang, Yijing Liu, Pei Ke, Aiguo Chen, Wenbo Jiang

Organizations: Laboratory of Intelligent Collaborative Computing, University of Electronic Science and Technology of China · School of Software Engineering, Chengdu University of Information Technology · National Key Laboratory of Wireless Communications, University of Electronic Science and Technology of China · School of Computer Science and Engineering, University of Electronic Science and Technology of China

Abstract

Vision-Language Models (VLMs) are vulnerable to backdoor attacks introduced through data poisoning, making effective backdoor detection a critical security challenge. In this paper, we uncover a novel connection between VLM hallucinations and backdoor behaviors and exploit this connection for backdoor detection. Our key insight is that poisoned training data induces strong language priors toward the backdoor target token, while such priors are also a major source of hallucinations. This suggests that enhancing hallucinations can expose hidden backdoor-related priors. Specifically, backdoored VLMs tend to hallucinate backdoor target tokens more frequently across diverse inputs, while the hallucinated tokens of clean VLMs are largely random and exhibit no stable target-specific bias. Based on this observation, we propose HalluProbe, which enhances hallucinations during output decoding and measures the distributional consistency of hallucinated tokens to identify backdoored VLMs. We evaluate HalluProbe against six representative VLM backdoor attacks across three models. Experimental results demonstrate that HalluProbe can effectively detect backdoored VLMs with an accuracy exceeding 90%, outperforming existing backdoor detection methods.

Figures & tables

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Phantasia: Context-Adaptive Backdoors in Vision Language Models

    Apr 9, 2026Nam Duong Tran, Phi Le NguyenBackdoor Attacks on VLA ModelsBackdoor Attacks on VLMs

  2. EntropyScan: Towards Model-level Backdoor Detection in LVLMs via Visual Attention Entropy

    May 15, 2026Xuanyu Ge, Zhongqi Wang, Jie Zhang +2Large Vision-Language ModelsBackdoor Detection

  3. BackdoorVLM: A Benchmark for Backdoor Attacks and Defenses on Vision-Language Models

    Nov 24, 2025Juncheng Li, Yige Li, Hanxun Huang +5VLM EvaluationVLM Robustness