Some backdoor attacks never touch an image. FLIP only changes labels: on CIFAR-10, flipping 3% of them makes a ResNet-32 send 99.7% of triggered source-class images to the attacker's class. The usual defenses look for the poison inside a model trained on those same labels. So when they struggle, is the problem the test they run, or the space they run it in? We answer with a controlled swap: we keep the test, the labels and the metric fixed, change only the feature space, and judge each defense by retraining the victim on what it keeps. The space decides. On dog→cat the same nearest-neighbor check scores AUROC 0.546 on the poisoned model's features, 0.967 on frozen DINOv2 features that never saw a label, and 0.995 on the same network trained on clean labels, an oracle control rather than a defense. We reproduce SPECTRE, the defense FLIP's authors recommend, to 90.94 / 0.97 against its published 91.47 / 1.7, then price it: on truck→deer it deletes 20% of every class, while a prototype audit on the frozen features deletes 6% for higher clean accuracy and lower attack success. On three harder pairs that audit leaves about a quarter of the attack standing. A count read off encoder geometry and clean labels, before any attack is run, separates the pair where the audit removes the attack from those where it only partly does, though on a single zero-count pair. Three results warn against the easy story. A better detection score does not mean a safer model: on two of four pairs Confident Learning scores the higher AUROC, and on dog→cat leaves 2.6 times the attack success. The prototype family, our best static performer, is the first an adaptive attacker slips past. And past a point, catching more poisons stops reducing attack success. Check labels in a space that never learned them, and judge a defense by the retrained model, not by its detection score.
Figures & tables
Fig. 1: The pipeline, and where each claim acts. FLIP corrupts labels only, so the images reaching the defender are the ones that were collected. An audit then has three knobs: the space features live in, the statistic computed on them, and the rule turning scores into deletions. We turn one at a time, always retraining the victim rather than stopping at a ranking. The clean-label space ( ∗ ) is a control, not a defense: it holds the architecture fixed and changes only the labels the features were fitted to, separating “supervised features are poor here” from “features fitted to the poison are poor here”. Two dashed arrows leave the main flow: the defense-aware attacker feeds back into the attack, and the stealthy-flip pool reads the encoder and clean labels before any attack exists. Each claim sits under the knob it concerns. The two statistic families at knob 4 are defined in ( 4 ) and ( 5 ). Claim 1 is Section V , Claim 2 and the negative result are Section VI , and Claim 3 is Section VII . AUROC values are knn_disagreement on dog → cat (Section V ).
Fig. 2: The two triggers, applied by the same code the experiments use. The sinusoidal trigger perturbs almost the whole image faintly (951 of 1 024 pixels, peak change 6/255); the Turner patch changes 9 pixels hard (peak 199/255).
setting
source
CTA
PTA
clean labels ( m=0 )
published
92.38
0.1
ours
92.57
0.1
sinusoidal, m=1500
published
89.87
99.8
ours
89.47±0.35
99.73±0.15
Turner, m=1500
published
89.91
99.9
ours
89.94
32.4
TABLE I: Reproducing FLIP on our setup. Published values are FLIP’s Table 1, CIFAR-10 with a ResNet-32. Ours are the same configuration run here; the sinusoidal row is three seeds, the others are single runs. The clean and sinusoidal rows reproduce. The Turner row does not, and Section IV says why.
supervised
frozen DINOv2
detector
poisoned
clean
ViT-S/14
ViT-B/14
knn_disagreement
.26/.05
.97/.86
.61/.35
.74/.52
knn_sim_margin
.25/.05
.97/.88
.63/.42
.75/.59
prototype_margin
.22/.04
1.0/1.0
.97/.67
.99/.85
prototype_trimmed
.27/.04
1.0/1.0
.98/.73
.99/.87
TABLE II: AUPRC of every detector in every feature space, reported as truck → deer / dog → cat. Same labels, same detectors, same metric; only the space changes. Reading across a row isolates the effect of the space. The clean column is an oracle control, not a deployable defense.
defense
cut
CTA
PTA
truck → deer
none
0
89.47±0.35
99.73±0.15
SPECTRE, Q=0.20
10 000
90.94±0.23
0.97±0.46
prototype_trimmed
2 981
92.23±0.14
0.20±0.20
dog → cat
none
0
89.33±0.30
94.67±0.42
TABLE III: Victim-level comparison, mean ± sd over three seeds. “cut” is how many of the 50 000 training images the defense deletes. SPECTRE is given the per-class budget it is designed for (Section III ); FLIP report it at CTA 91.47 / PTA 1.7 on truck → deer [ 3 ] .
defense
removed
clean destroyed
truck → deer
SPECTRE, Q=0.20 [ 8 ]
10 000
8 555
activation clustering, Q=0.20 [ 6 ]
10 000
8 630
spectral signatures, Q=0.25 [ 5 ]
12 500
11 145
prototype_trimmed
1 419
69
dog → cat
TABLE IV: What each published defense deletes to catch 90% of the poisons, deployed per class as all three are designed to be. Detection only, no retraining, so these sit alongside Figure 3 .
Fig. 3: Protection against its price. At 90% recall on truck → deer a prototype audit on frozen features loses 69 clean images; SPECTRE, deployed per class as designed, loses 8 555.
detector
AUROC
AUPRC
CTA
PTA
truck → deer
knn_disagreement
.986
.614
91.99
8.6
knn_sim_margin
.987
.630
92.04
11.4 ↓
prototype_margin
.996
.968
92.34
0.4
prototype_trimmed
.996
.976
92.31
0.2
cleanlab [ 13 ]
.974
.584
91.48
51.3
TABLE V: Every detector at the same removal budget (2 981 and 2 728 images), so a difference in the model is about which samples were cut. Detection and outcome disagree repeatedly: the arrows mark pairs of rows where the better-ranking detector produces the worse model.
Fig. 4: Twenty victim trainings sweeping k on truck → deer. Detection peaks at k=200 while the trained model keeps improving past it. The lower panel is the reason the curves separate: at the auto threshold k also sets how much data is cut.
Fig. 5: The statistic that wins under a defense-unaware attacker is the one a defense-aware attacker dismantles first.
Fig. 6: Attack × defense, dog → cat, three seeds.
pair
pool
AUPRC
PTA none
PTA defended
truck → deer
0
.976
99.7±0.2
0.2±0.2
car → truck
72
.789
99.4±0.3
24.6±1.3
cat → dog
82
.786
94.4±0.7
24.4±2.7
dog → cat
265
.727
94.7±0.4
24.3±2.5
TABLE VI: The pool against what happened. Pool sizes come from encoder geometry alone; AUPRC is prototype_trimmed ; PTA columns are mean ± sd over three seeds at each pair’s matched budget. Detection tracks the pool; the trained outcome only separates zero from non-zero.
Fig. 7: What the pool means for the attacker. As the stealth budget τ tightens, dog → cat degrades smoothly while truck → deer collapses as soon as any constraint applies: with a pool of zero there is no flip that both installs the backdoor and survives the audit.
stage
seconds
vs. training
paid once per dataset
DINOv2 ViT-S/14 pass (22.1 M)
277.6
8.2%
DINOv2 ViT-B/14 pass (86.6 M)
1 190.0
35.3%
k -NN graph, k=20
36.4
1.1%
paid per audit
prototype_trimmed
0.09
<0.01 %
TABLE VII: Measured cost of each stage, against one victim training. Encoder timings are a single GPU run on a Tesla P6; the CPU rows and the victim median come from a separate run on the same cluster. The encoder is the only substantial new cost, and it is paid once per dataset.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Fig. 8: The AUPRC landscape over k for knn_disagreement , with what each oracle-free rule picks. The dashed line is the best AUPRC any k achieves, which a defender cannot find without the poison mask. Bimodality sits at k=1 on both pairs and gives up most of the curve. Stability and agreement trade places between the pairs and between budgets. Markers are nudged horizontally where rules pick the same k ; heights are exact.
Fig. 9: Recall of knn_disagreement under thresholds a defender can set without the poison mask. The dashed line marks 90% recall, the equal-protection cut used in Section V-A ; it is an oracle operating point, not a rule. Otsu is oracle-free and clears it on both pairs, but by deleting roughly twice as much; the trusted n rules assume a clean subset our threat model does not grant.
Fig. 10: The stealthy-flip pool across all 90 ordered CIFAR-10 pairs, ranked. The four pairs we trained victims for are highlighted. One pair dominates, three more form a cluster behind it, and the distribution then decays into a long tail of pairs with no stealthy flip available at all.
pair
budget
oracle k
rule
picks k
AUPRC gap
truck → deer
3%
200
stability
100
+0.0006
bimodality
1
+0.4190
agreement
50
+0.0083
5%, 10%
200
stability
50
+0.0083
bimodality
1
+0.4190
agreement
2000
+0.0167
Appendix
TABLE VIII: Choosing k without an oracle, for knn_disagreement on frozen DINOv2 features. For each pair and audit budget B we give the k that maximises AUPRC (the oracle a defender cannot consult) and what each oracle-free rule picks instead, with the AUPRC it gives up. AUPRC ranks the whole score vector and so does not depend on B ; the oracle column is therefore constant down a pair, and only the rules move with the budget. Stability takes the smallest k whose top- B selected set has Jaccard overlap at least 0.85 with the top- B set at 2k , which is why it is undefined where 2k is off the grid. Bimodality maximises Otsu separation of the score histogram. Agreement maximises top- B overlap with prototype_trimmed , so it is not independent of the prototype audit and we do not read it as evidence about prototypes.
pair
detector
rule
flagged
recall
F1
truck → deer
knn_disag.
fixed_0.5
2981
91.3%
0.611
otsu
5746
99.1%
0.410
trusted1000_fpr0.05
3367
93.5%
0.576
trusted1000_fpr0.02
1992
75.0%
0.644
proto_margin
fixed_0.5
61
4.1%
0.078
otsu
1405
88.3%
0.912
Appendix
TABLE IX: What a deployable threshold actually catches. fixed_0.5 and otsu are both oracle-free: the first is a fixed cut, the second thresholds the score histogram by maximising between-class variance and uses no labels or clean data of any kind. Only the trusted n rules assume a clean subset, of size n , at the stated false-positive rate; our threat model does not grant one (Section II ), so those rows are a check from outside it. Recall is of the true poison mask. The 90% recall used for the equal-protection comparison in Section V-A is an oracle operating point, not one of these.
pair
τ=0.5
τ=0.7
τ=0.9
dog → cat ⋆
265
527
1164
truck → car
83
164
425
cat → dog ⋆
82
314
1223
car → truck ⋆
72
143
402
bird → plane
43
68
165
deer → frog
42
79
150
Appendix
TABLE X: Stealthy-flip pool for all 90 ordered CIFAR-10 pairs, computed on frozen DINOv2 features in seconds and with no training. The pool counts source-class images whose neighbourhood already leans to the target strongly enough that a flip survives the audit at τ . The ten largest are shown; 29 of the 90 pairs have an empty pool at τ=0.5 . Pairs we trained victims for are marked ⋆ .
1Neurint, LLC, Baton Rouge, LA, USA. · 2U.S. Army Cyber Corps, U.S. Army Reserve, USA. · 3Northwestern State University of Louisiana, Natchitoches, LA, USA. +2
School of Computer Science and Information Engineering Hefei University of Technology Hefei, Anhui, China · School of Information Science and Technology University of Science and Technology of China Hefei, Anhui, China · Faculty of Business Data Science Kansai University Japan +3