Text-to-image (T2I) diffusion models have achieved remarkable success in image synthesis, but their reliance on large-scale data and open ecosystems introduces serious backdoor security risks. Existing defenses, particularly input-level methods, are more practical for deployment but often rely on observable anomalies that become unreliable under stealthy, semantics-preserving trigger designs. As modern backdoor attacks increasingly embed triggers into natural inputs, these methods degrade substantially, raising a critical question: can more stable, implicit, and trigger-agnostic differences between benign and backdoor inputs be exploited for detection? In this work, we address this challenge from an active probing perspective. We introduce controlled scaling perturbations on cross-attention and uncover a novel phenomenon termed Cross-Attention Scaling Response Divergence (CSRD), where benign and backdoor inputs exhibit systematically different response evolution patterns across denoising steps. Building on this insight, we propose SET, an input-level backdoor detection framework that constructs response-offset features under multi-scale perturbations and learns a compact benign response space from a small set of clean samples. Detection is then performed by measuring deviations from this learned space, without requiring prior knowledge of the attack or access to model training. Extensive experiments demonstrate that SET consistently outperforms existing baselines across diverse attack methods, trigger types, and model settings, with particularly strong gains under stealthy implicit-trigger scenarios. Overall, SET improves AUROC by 9.1% and ACC by 6.5% over the best baseline, highlighting its effectiveness and robustness for practical deployment.
Figures & tables
Figure 1. Robustness to stealthy and explicit triggers. Dark bars denote the AUROC under the stealthy IBA attack, while light bars represent the deviation from the mean AUROC across four explicit-trigger attacks.
Figure 2. Cross-Attention Scaling Response Divergence. We measure the response difference of an input along the conditional injection pathway by computing the mean squared error between the scaled response and the reference response in four representative backdoored T2I models: RickBKD, VillanBKD, IBA, and BadT2I. As shown in the figure, after scaling, benign and backdoor inputs exhibit stable and separable response shifts.
Figure 3. Overview of SET. We apply controlled scaling to selected self- and cross-attention modules in the U-Net to extract attention-response shifts and aggregate them into a response-shift vector f(x) . Benign vectors are normalized and encoded into low-dimensional embeddings to build a compact benign space by estimating a robust center c and optimizing decision radius R with a one-class objective. At test time, backdoor inputs are identified by their distance to the benign center.
Method
RickBKD
VillanBKD
BadT2I
EvilEdit
IBA
Avg.
T2IShield FTT
99.9
73.3
72.3
36.3
40.9
64.5
T2IShield CDA
99.9
96.1
99.8
45.0
5.5
69.3
DAA-I
84.0
74.8
87.6
79.1
72.1
79.5
DAA-S
96.0
82.0
97.3
76.6
78.3
86.0
NaviT2I
99.9
99.5
99.9
70.8
54.9
85.0
UFID
37.4
99.9
28.7
49.0
13.7
45.7
Table 1. The performance (AUROC) against the mainstream T2I backdoor attacks on MS-COCO. We mark the best results in bold and the second-best results with underlines for comparison.
Method
RickBKD
VillanBKD
BadT2I
EvilEdit
IBA
Avg.
T2IShield FTT
85.7
67.6
64.4
42.4
40.1
60.0
T2IShield CDA
98.4
86.8
97.0
49.7
49.2
76.2
DAA-I
87.9
74.1
92.1
73.1
46.8
74.8
DAA-S
94.3
85.5
98.0
65.0
48.6
78.3
NaviT2I
88.2
93.2
93.1
61.2
52.1
77.6
UFID
50.1
99.8
50.1
51.8
50.0
60.4
Table 2. Detection accuracy (ACC) against mainstream T2I backdoor attacks on MS-COCO. Key results are highlighted as in Tab. 1 . For this binary classification task, even “random guessing” yields 50.0%.
Metric
SET (Ours)
Layer selection
Self-attn scaling
Scaling factors
Benign training samples
Scaling position
Denoising step
all
up
mid
down
No scaling
F1
F2
600
800
1200
1500
innoV
outV
outnoV
1
3
7
9
AUROC ↑
95.1
64.7
56.0
53.9
57.9
33.8
34.3
77.6
37.8
61.8
44.5
70.1
37.5
45.7
42.6
14.3
42.0
68.3
30.8
ACC ↑
84.8
63.3
59.4
50.0
68.8
51.8
59.5
75.3
54.9
63.7
62.1
74.7
57.6
53.3
58.4
49.5
57.2
71.6
49.5
Table 3. Ablation study of SET under different configurations. The default configuration of SET (ours) uses the up and down layers, self-attention scaling, scaling factors {0.2,0.3,7,10,20} , 1,000 benign training samples, scaling at inV , and the first five denoising steps. Each variant changes only one factor relative to this default setting.
Institute of Information Engineering, CAS · School of Cyber Security, University of Chinese Academy of Sciences · State Key Laboratory of AI Safety, Institute of Computing Technology, CAS +4