Beyond Static Cues: Unified Dynamic Detection of MLLM Backdoors via Dual-Flow Trajectory Analysis

Yinzhen Huang; Jiahe Xie; Yitong Sun; Ruochen Zhang; Xingxing Wei

cs.CVOct 14, 2026

Beyond Static Cues: Unified Dynamic Detection of MLLM Backdoors via Dual-Flow Trajectory Analysis

Authors: Yinzhen Huang, Jiahe Xie, Yitong Sun, Ruochen Zhang, Xingxing Wei

Organizations: Beihang University, Beijing, Beijing, China

Abstract

Multimodal Large Language Models (MLLMs) are increasingly deployed in safety-critical domains, yet their susceptibility to backdoor attacks remains a formidable challenge. While existing defenses have evolved from visual-centric patch cues to unsupervised internal attention statistics, they are fundamentally constrained by a reliance on static snapshots and attack-specific heuristics. Given the intricate interplay between dual modalities, the vast design space for multimodal triggers allows diverse threats, such as textual injections, sub-perceptual noises, and coordinated cross-modal triggers, to easily circumvent such rigid priors, particularly when attack types are unknown in real-world applications. In this paper, we rethink backdoor detection by shifting the diagnostic paradigm from static observations to dynamic output evolution. We observe that a successful attack inevitably hijacks the model's decoding trajectory, skewing internal modal contributions during generation while maintaining an abnormal output invariance despite input perturbations. To capture this behavior, we propose EVO-Detect, a unified detection framework that monitors the dynamic evolution of internal signals during early decoding steps through two complementary channels: Attention Flow and Logits Flow. The attention branch measures how image-text attention allocation evolves during early generation, whereas the logits branch quantifies the sensitivity of the output distribution to modal perturbations through interventional ablations. We then convert these trajectories into compact sample-level metrics and normalize them against negative-control signatures so that natural perturbations are less likely to be mistaken for backdoor evidence. Extensive evaluations show that our approach substantially outperforms state-of-the-art baselines such as BYE and TCAP across diverse attack families.

Figures & tables

Figure 1. An illustration of our unified detection framework’s successful interception based on analysis of decoding evolution across various trigger forms: visual attacks, textual injections, and challenging dual-modal triggers, while preserving benign behavior on clean inputs.

Figure 2. Visualization of poisoned samples across different attack families. These examples illustrate how diverse methods manipulate visual, textual, or both modalities to achieve malicious hijacking.

Figure 3. Qualitative views of the proposed sample-filtering signals. The left column illustrates the Attention Map Similarity (

M_{t}

), while the right column shows the Contribution Gap (

G_{t}

). Across visual, textual, and dual-modal attacks, poisoned samples (red dashed lines) consistently exhibit distinct and separate trajectories compared to clean samples (blue solid lines), providing strong evidence for dynamic filtering.

Figure 4. Overview of the proposed EVO-Detect framework for securing downstream MLLM fine-tuning. The pipeline consists of four stages. First, candidate samples undergo Trajectory Analysis over early decoding steps (

K=5

) via dual streams: Attention Flow and Logits Flow. The extracted sample features (red) are routed to the Reference Normalization module to form abnormality metrics against a clean baseline (green), which are then normalized with matched controls (blue) to suppress false alarms. Finally, a unified suspiciousness score

S(x)

is thresholded to retain clean data for downstream fine-tuning. A placeholder overview figure for the proposed control-aware dual-flow filtering framework.

Model	Attack	Vanilla FT		SentiNet		ZIP		BYE		TCAP		Ours
Model	Attack	CP ( $\uparrow$ )	ASR ( $\downarrow$ )	CP ( $\uparrow$ )	ASR ( $\downarrow$ )	CP ( $\uparrow$ )	ASR ( $\downarrow$ )	CP ( $\uparrow$ )	ASR ( $\downarrow$ )	CP ( $\uparrow$ )	ASR ( $\downarrow$ )	CP ( $\uparrow$ )	ASR ( $\downarrow$ )
	BadNet	89.5	100.0	88.9	44.4	88.9	100.0	88.9	41.8	83.6	13.8	90.0	2.0
	Blended	89.5	97.0	40.0	23.3	90.0	97.0	82.6	45.6	60.0	56.7	87.0	2.0
	WaNet	89.8	96.4	40.0	52.2	80.0	96.4	51.1	30.0	83.0	81.1	80.0	0.0
	Text Prefix	88.2	100.0	55.6	11.0	88.9	100.0	78.0	68.2	78.9	21.4	89.0	16.0
	HiddenKiller	89.1	100.0	70.0	0.0	90.0	100.0	80.1	74.2	80.0	97.8	85.0	17.0

Table 1. ScienceQA overview across attack modes and defense methods. Results are reported in percentage (%). Bold font indicates the best performance among all defense methods.

Figure 5. Distribution of the unified suspiciousness score

S(x)

across different attack families. Evaluated using Qwen3-VL on the COCO dataset. The plots demonstrate a clear separation between clean and poisoned samples under (A) visual, (B) textual, and (C) dual-modal attacks. The horizontal dashed lines represent the filtering thresholds

\tau

Method	ASR by Attack Type (%) $\downarrow$			Overall ASR $\downarrow$	Overall CP $\uparrow$
Method	Visual	Text	Dual	Overall ASR $\downarrow$	Overall CP $\uparrow$
Vanilla FT	97.7	99.7	99.8	98.7	74.9
SentiNet	63.1	14.3	53.1	45.2	67.4
ZIP	72.7	99.7	99.5	86.2	74.0
BYE	69.7	83.4	81.3	76.2	70.0
TCAP	36.0	43.4	71.8	44.4	70.7

Table 2. Average CP and ASR across all datasets and backbones. Results are in percentage (%).

Table 8

Backdoor Detection

Backdoor detection finds hidden triggers or poisoned behavior in trained models before they can secretly redirect predictions or actions.

See all 53 papers

BackdoorVLM: A Benchmark for Backdoor Attacks and Defenses on Vision-Language Models
Nov 24, 2025Juncheng Li, Yige Li, Hanxun Huang +5VLM Evaluation VLM Robustness
Fudan University · Singapore Management University · The University of Melbourne
ProjLens: Unveiling the Role of Projectors in Multimodal Model Safety
Apr 21, 2026Kun Wang, Cheng Qian, Miao Yu +6LLM Backdoor Attacks Multimodal Large Language Models
3Nanyang Technological University · 2Beijing University of Aeronautics and Astronautics · University of Science and Technology of China +2
A Patch-based Cross-view Regularized Framework for Backdoor Defense in Multimodal Large Language Models
Apr 6, 2026Tianmeng Fang, Yong Wang, Zetai Kong +4Backdoor Defense in LLMs LLM Security
School of Computing and Information Systems, Singapore Management University, Singapore, 178902, , Singapore. · School of Artificial Intelligence, China University of Mining and Technology, Beijing, 102206, Beijing, PR China. · Faculty of Arts, The University of Melbourne, Melbourne, Carlton VIC 3053, Australia. +3

Nov 24, 2025J/K move · Enter open · S save

BackdoorVLM: A Benchmark for Backdoor Attacks and Defenses on Vision-Language Models

Juncheng Li, Yige Li, Hanxun Huang, Yunhao Chen+4

Fudan University · Singapore Management University · The University of Melbourne

VLM Evaluation VLM Robustness Adversarial Attacks on VLMs Backdoor Defense Backdoor Attacks on VLMs

Open paper AskPDF arXiv

Backdoor attacks undermine the reliability and trustworthiness of machine learning systems by injecting hidden behaviors that can be maliciously activated at inference time. While such threats have been extensively studied in unimodal settings, their impact on multimodal foundation models, particularly vision-language models (VLMs), remains largely underexplored. In this work, we introduce BackdoorVLM, the first comprehensive benchmark for evaluating backdoor attacks and defenses on VLMs across a broad range of settings. It adopts a unified perspective that injects and analyzes backdoors across core vision-language tasks, including image captioning and visual question answering. BackdoorVLM organizes multimodal backdoor threats into 5 representative categories: targeted refusal, malicious injection, jailbreak, concept substitution, and perceptual hijack. Each category captures a distinct pathway through which an adversary can manipulate a model's behavior. We evaluate these threats using 12 representative attack methods spanning text, image, and bimodal triggers, tested on 2 open-source VLMs and 3 multimodal datasets along with 5 representative defense methods. Our analysis reveals that VLMs exhibit strong sensitivity to textual instructions, and in bimodal backdoors the text trigger typically overwhelms the image trigger when forming the backdoor mapping. Notably, backdoors involving the textual modality remain highly potent, with poisoning rates as low as 1% yielding over 90% success across most tasks. Moreover, existing defenses lack generalization across trigger types and backdoor scenarios, leading to unreliable protection against diverse triggers across modalities and forms. These findings highlight significant, underexplored vulnerabilities in current VLMs. We hope that BackdoorVLM can serve as a useful benchmark for analyzing and mitigating multimodal backdoor threats.

Abstract

Backdoor Detection

Explore similar work

BackdoorVLM: A Benchmark for Backdoor Attacks and Defenses on Vision-Language Models

ProjLens: Unveiling the Role of Projectors in Multimodal Model Safety

A Patch-based Cross-view Regularized Framework for Backdoor Defense in Multimodal Large Language Models