Backdoor attacks have emerged as a critical security threat in machine learning, where adversaries embed triggers within poisoned training samples to manipulate a model's predictions on targeted inputs while leaving its general behavior unaffected. This stealthy nature makes backdoor attacks difficult to detect, as the model functions normally until activated by a specific malicious trigger. Current purification-based defenses use image transformations (e.g., blurring) to disrupt trigger patterns but often degrade image quality. Even with advanced restoration techniques like diffusion models, these approaches can still alter visual or semantic information, reducing utility for downstream tasks. To address this, we propose the first semantics-aware data purification framework for black-box backdoor defense, SemPurify, which explicitly incorporates multi-level semantics (i.e., low-level semantic embeddings and high-level image tags) within a text-to-image (T2I) framework to guide image restoration during purification. Using this semantic information, the T2I model subsequently refines the transformed samples, helping preserve visual details and semantic information during restoration. Extensive experiments spanning multiple backdoor scenarios, i.e., classification models, CLIP-style encoders, and generative models, demonstrate that SemPurify reliably removes backdoors while effectively preserving semantic information for downstream tasks.
Figures & tables
Figure 1: Illustration of semantic descriptions extracted by the tagging model RAM [ 60 ] from clean and backdoored inputs. BadNet [ 8 ] is applied to cause the image labeled as “church” to be misclassified as “dog”. While two tags (highlighted) are introduced, they remain contextually related to the original image, with none associated with the target label “dog”.
Figure 2: The overview of SemPurify: In Stage 1, we use an image tagging model to extract multi-level semantics. In Stage 2, we use a linear transformation such as blurring to destruct the trigger patterns in the poisoned image xp . In Stage 3, we use a conditional text-to-image model to restore the semantic information. This process ensures that x0 retains the original semantic information from xp while effectively neutralizing any backdoor trigger patterns.
No Defense
ShrinkPad
BDMAE
ZIP
DataElixir
SampDetox
SemPurify
Dataset
Attack
CA ↑
ASR ↓
PA ↑
CA ↑
ASR ↓
PA ↑
CA ↑
ASR ↓
PA ↑
CA ↑
ASR ↓
PA ↑
CA ↑
ASR ↓
PA ↑
CA ↑
ASR ↓
PA ↑
CA ↑
ASR ↓
PA ↑
No Attack
88.48
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
BadNet
88.41
100.00
10.00
57.99
2.85
57.85
87.00
0.66
86.17
82.55
0.90
82.13
84.40
0.96
71.30
77.54
1.04
76.11
86.66
0.90
86.18
PhysicalBA
94.59
100.00
10.00
91.45
91.14
17.50
94.38
16.14
77.92
87.00
0.80
86.71
94.57
1.02
72.93
81.48
1.06
79.92
90.80
0.11
90.76
Blended
87.96
100.00
10.00
59.14
0.56
36.85
86.69
30.51
30.11
72.65
1.60
58.14
88.03
0.97
64.84
77.19
0.72
69.17
86.47
1.17
82.84
Semantic Blended
88.30
99.62
10.31
59.51
1.57
51.72
86.96
97.00
2.80
77.33
1.23
75.82
88.30
0.89
68.14
77.55
1.19
72.79
84.41
2.37
83.16
Table 1: Defense performance comparison against classification backdoor attacks.
Figure 3: Qualitative comparisons of images produced by different purification methods.
Dataset
Metric
No Defense
ShrinkPad
BDMAE
ZIP
DataElixir
SampDetox
SemPurify
Caltech101 [ 5 ]
CA ↑
92.00
91.70
86.24
90.70
90.39
90.61
91.30
ASR ↓
98.30
0.90
35.59
1.10
0.33
0.33
0.20
PA ↑
2.60
91.00
57.97
90.70
91.27
91.16
91.30
OxfordPets [ 36 ]
CA ↑
96.00
93.40
86.80
90.20
95.02
89.26
93.40
ASR ↓
99.30
47.90
49.33
5.70
6.26
4.08
3.20
PA ↑
5.50
51.30
48.99
90.40
91.44
92.11
93.30
Table 3: Defense performance comparison against BadCLIP.
No Defense
BDMAE
ZIP
DataElixir
SampDetox
SemPurify
CS ↑
PS ↓
CS ↑
PS ↓
CS ↑
PS ↓
CS ↑
PS ↓
CS ↑
PS ↓
CS ↑
PS ↓
0.89
0.86
0.54
0.24
0.69
0.38
0.61
0.21
0.66
0.27
0.78
0.20
Table 4: Defense performance comparison against generative backdoor attacks.
Figure 4: Comparison of the generated image using different purification methods.
w/ Se
w/ St
w/ Se+St
Attack
CA ↑
ASR ↓
PA ↑
CA ↑
ASR ↓
PA ↑
CA ↑
ASR ↓
PA ↑
BadNet
83.21
1.07
83.36
68.84
0.16
69.22
83.18
0.99
83.18
PhysicalBA
89.63
7.36
86.85
77.50
3.6
77.09
89.85
7.33
87.05
Blended
84.02
0.31
81.98
70.92
0.02
70.77
84.38
0.28
82.64
Semantic Blended
73.42
6.96
72.58
56.78
0.31
60.22
79.36
6.11
77.65
WaNet
82.39
1.80
82.70
65.75
0.39
65.37
76.94
1.55
76.91
Table 5: Impact of semantic embeddings and image tags.
Multimodal Diffusion Language Models (MDLMs) have recently emerged as a competitive alternative to their autoregressive counterparts. Yet their vulnerability to backdoor attacks remains largely unexplored. In this work, we show that well-established data-poisoning pipelines can successfully implant backdoors into MDLMs, enabling attackers to manipulate model behavior via specific triggers while maintaining normal performance on clean inputs. However, defense strategies effective to these models are yet to emerge. To bridge this gap, we introduce a backdoor defense framework for MDLMs named DiSP (Diffusion Self-Purification). DiSP is driven by a key observation: selectively masking certain vision tokens at inference time can neutralize a backdoored model's trigger-induced behaviors and restore normal functionality. Building on this, we purify the poisoned dataset using the compromised model itself, then fine-tune the model on the purified data to recover it to a clean one. Given such a specific design, DiSP can remove backdoors without requiring any auxiliary models or clean reference data. Extensive experiments demonstrate that our approach effectively mitigates backdoor effects, reducing the attack success rate (ASR) from over 90% to typically under 5%, while maintaining model performance on benign tasks.
Despite significant advances in computer vision, semantic segmentation models remain vulnerable to backdoor attacks, posing a security risk. These attacks redirect pixels of a chosen victim class to an attacker-specified target class when a trigger is present, while preserving clean predictions. Existing defenses are largely derived from image classification, while dedicated segmentation defenses remain scarce and typically require retraining or model modification. We introduce RePAIR, a post-hoc, label-free, attack-unaware defense based on the observation that backdoor-induced shifts can be stronger in the classifier head than in internal representations, leaving recoverable victim-class semantics. RePAIR infers the victim-target relation from an unlabeled, potentially contaminated image pool, constructs a safety-approved semantic reference, and localizes suspicious pixels by comparing head predictions with internal semantic evidence. It then applies feature-space and output corrections toward the inferred victim semantics, while conservative localization and output constraints limit changes to benign predictions. We model the attack as adversarial probability transport and repair as its approximate reversal, deriving bounds and sufficient conditions for localization, residual attack success, clean-side changes, defense-induced target predictions, and attack-unaware pair identification. Experiments across segmentation architectures and attack settings show that RePAIR substantially reduces attack success while largely preserving clean performance.
Traditional backdoor attacks inject artificial trigger patterns into training data, causing models to misclassify triggered inputs while maintaining normal behavior on clean samples. Since these artificial triggers are absent in clean datasets, standard knowledge distillation (KD) typically eliminates them, leading to the common belief that KD serves as an effective purification defense. In this paper, we propose DAR (Discover-and-Relabel), a support-persistent backdoor method that retains its behavior even after clean-data distillation. DAR identifies naturally abundant patterns in low-dimensional feature subspaces using subspace clustering and applies label-only poisoning to samples that inherently satisfy the discovered rule. At test time, the backdoor is activated by locally modifying the corresponding subspace of the target images. By instantiating spatial and frequency operators, DAR achieves attack accuracy comparable to state-of-the-art backdoor attacks in ImageNet classification and CLIP-based prompt tuning. Notably, DAR keeps attack success above 96% after clean KD and remains defense-resistant in practice. Project page: https://junhoo.me/backdoor