RPP: A Certified Poisoned-Sample Detection Framework for Backdoor Attacks under Dataset Imbalance
Organizations: Old Dominion University · University of Texas at El Paso · University of Central Florida · Iowa State University ·
Abstract
Deep neural networks are highly susceptible to backdoor attacks, yet most defense methods to date rely on balanced data, overlooking the pervasive class imbalance in real-world scenarios that can amplify backdoor threats. This paper presents the first in-depth investigation of how the dataset imbalance amplifies backdoor vulnerability, showing that (i) the imbalance induces a majority-class bias that increases susceptibility and (ii) conventional defenses degrade significantly as the imbalance grows. To address this, we propose Randomized Probability Perturbation (RPP), a certified poisoned-sample detection framework that operates in a black-box setting using only model output probabilities. For any inspected sample, RPP determines whether the input has been backdoor-manipulated, while offering provable within-domain detectability guarantees and a probabilistic upper bound on the false positive rate. Extensive experiments on five benchmarks (MNIST, SVHN, CIFAR-10, TinyImageNet and ImageNet10) covering 10 backdoor attacks and 12 baseline defenses show that RPP achieves significantly higher detection accuracy than state-of-the-art defenses, particularly under dataset imbalance. RPP establishes a theoretical and practical foundation for defending against backdoor attacks in real-world environments with imbalanced data.
Figures & tables
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
| Layer | # of Channels | Filter Size | Stride | Activation |
|---|---|---|---|---|
| Conv | 32 | 1 | ReLU | |
| MaxPool | 32 | 2 | - | |
| Conv | 64 | 1 | ReLU | |
| MaxPool | 64 | 2 | - | |
| FC | 128 | - | - | ReLU |
| FC | 10 | - | - | Softmax |
| Attack | = 1 | = 10 | = 200 | |||
|---|---|---|---|---|---|---|
| AUC | ASR | AUC | ASR | AUC | ASR | |
| Badnets | 100.0 | 13.8 | 100.0 | 14.5 | 98.9 | 21.8 |
| Blend | 100.0 | 16.3 | 100.0 | 18.8 | 99.0 | 22.9 |
| Trojan | 100.0 | 29.4 | 100.0 | 30.6 | 99.0 | 35.6 |
| SIG | 100.0 | 29.0 | 100.0 | 33.7 | 99.0 | 38.2 |
| ISSBA | 99.8 | 30.2 | 99.8 | 47.6 | 98.8 | 53.3 |
| Attack | |||||
|---|---|---|---|---|---|
| Badnets | 100.0 | 100.0 | 100.0 | 100.0 | 99.8 |
| Blend | 100.0 | 100.0 | 100.0 | 100.0 | 99.6 |
| Trojan | 100.0 | 100.0 | 100.0 | 100.0 | 99.7 |
| Narcissus | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 |
| = 1 | 100.0 | 84.6 | 77.4 | 68.3 |
|---|---|---|---|---|
| = 2 | 99.9 | 83.7 | 77.7 | 70.3 |
| = 10 | 99.7 | 94.8 | 89.3 | 83.9 |
| = 100 | 100.0 | 98.6 | 94.4 | 90.9 |
| = 200 | 100.0 | 99.1 | 97.5 | 93.1 |
| Badnets | Blend | Trojan | SIG | ISSBA | WaNet | |
|---|---|---|---|---|---|---|
| TPR | 81.1 | 82.2 | 74.8 | 77.0 | 78.5 | 87.1 |
| FPR | 18.7 | 17.2 | 15.6 | 10.9 | 24.3 | 12.8 |
| TPR | FPR | TPR | FPR | TPR | FPR | TPR | FPR | TPR | FPR | |
|---|---|---|---|---|---|---|---|---|---|---|
| MNIST | 100.0 | 5.7 | 100.0 | 6.9 | 82.4 | 0.5 | 100.0 | 1.9 | 100.0 | 6.7 |
| SVHN | 99.7 | 11.6 | 98.8 | 3.6 | 100.0 | 5.9 | 98.9 | 2.9 | 89.7 | 4.6 |
| CIFAR10 | 98.0 | 5.8 | 93.1 | 3.0 | 97.4 | 7.0 | 86.1 | 3.8 | 87.0 | 12.4 |
| TinyImageNet | 100.0 | 1.7 | 100.0 | 2.9 | 98.0 | 2.6 | 85.7 | 4.7 | – | – |
| ImageNet10 | 97.8 | 7.3 | 94.5 | 8.1 | 90.2 | 10.9 | 88.5 | 12.0 | 82.6 | 13.7 |
| Checkpoint | AUC | Minority AUC | ASR | TPR | FPR | |
|---|---|---|---|---|---|---|
| 1 | Early | 0.892 | 0.892 | 0.829 | 0.881 | 0.098 |
| Middle | 0.925 | 0.925 | 0.937 | 0.960 | 0.063 | |
| Final | 0.996 | 0.996 | 0.988 | 0.985 | 0.045 | |
| 2 | Early | 0.904 | 0.903 | 0.839 | 0.868 | 0.089 |
| Middle | 0.952 | 0.951 | 0.958 | 0.958 | 0.061 | |
| Final | 0.992 | 0.992 | 0.999 | 0.968 | 0.056 |
| RAB | 41.5 | 40.2 | 38.8 | 21.0 | 15.7 |
|---|---|---|---|---|---|
| DPA | 56.2 | 46.1 | 26.6 | 22.7 | 14.8 |
| Minority class (9) | Intermediate class (4) | |||
|---|---|---|---|---|
| TPR | FPR | TPR | FPR | |
| = 2 | 99.4 | 7.3 | 93.9 | 3.1 |
| = 10 | 93.4 | 10.1 | 96.2 | 3.3 |
| = 100 | 99.9 | 2.9 | 99.3 | 1.5 |
| = 200 | 90.0 | 6.0 | 79.6 | 4.6 |
| ASR (Without LA) | ASR (With LA) | TPR | FPR | |
|---|---|---|---|---|
| = 2 | 100.0 | 99.9 | 99.2 | 22.2 |
| = 10 | 99.8 | 99.4 | 96.2 | 20.3 |
| = 100 | 100.0 | 97.4 | 97.8 | 26.1 |
| = 200 | 99.7 | 94.3 | 83.0 | 10.4 |
| No Defense | No Defense | No Defense | RPP | No Defense | RPP | No Defense | RPP | No Defense | RPP | |||||||||||
| ASR | AUC | ASR | AUC | ASR | AUC | TPR | FPR | ASR | AUC | TPR | FPR | ASR | AUC | TPR | FPR | ASR | AUC | TPR | FPR | |
| 14.3 | 100.0 | 33.7 | 100.0 | 35.6 | 100.0 | – | – | 39.2 | 100.0 | – | – | 52.3 | 100.0 | 73.1 | 4.1 | 90.5 | 100.0 | 97.2 | 8.0 | |
| 14.8 | 100.0 | 37.8 | 100.0 | 33.5 | 100.0 | – | – | 47.9 | 100.0 | – | – | 69.6 | 100.0 | 88.3 | 6.9 | 90.3 | 100.0 | 96.0 | 6.1 | |
| 15.4 | 100.0 | 30.5 | 100.0 | 38.3 | 100.0 | – | – | 49.4 | 100.0 | – | – | 84.8 | 100.0 | 90.1 | 9.6 | 99.8 | 100.0 | 97.9 | 7.7 | |
| 23.0 | 99.3 | 39.6 | 99.4 | 52.1 | 98.6 | 73.8 | 10.1 | 100.0 | 99.4 | 87.5 | 10.7 | 100.0 | 99.6 | 95.2 | 10.1 | 100.0 | 99.5 | 100.0 | 10.9 | |
| 4.0 | 5.3 | 5.1 | 4.4 | 10.7 | 9.9 | 9.1 | 8.8 | |
| 4.9 | 4.1 | 4.6 | 4.0 | 9.3 | 9.9 | 10.1 | 9.0 | |
| 5.5 | 4.9 | 4.6 | 4.2 | 10.7 | 9.8 | 9.5 | 9.1 | |
| 5.5 | 4.7 | 4.8 | 5.0 | 9.9 | 9.2 | 10.2 | 10.0 | |
| 5.1 | 4.9 | 4.8 | 4.7 | 10.4 | 10.0 | 9.8 | 10.1 | |
Backdoor Attacks
Backdoor attacks implant hidden, trigger-dependent behaviors in models while aiming to preserve normal performance.
See all 135 papers