RPP: A Certified Poisoned-Sample Detection Framework for Backdoor Attacks under Dataset Imbalance

Miao Lin; Feng Yu; Rui Ning; Lusi Li; Jiawei Chen; Qian Lou; Mengxin Zheng; Chunsheng Xin

cs.CRJan 30, 2026

RPP: A Certified Poisoned-Sample Detection Framework for Backdoor Attacks under Dataset Imbalance

Authors: Miao Lin, Feng Yu, Rui Ning, Lusi Li, Jiawei Chen, Qian Lou, Mengxin Zheng, Chunsheng Xin,

Organizations: Old Dominion University · University of Texas at El Paso · University of Central Florida · Iowa State University ·

Abstract

Deep neural networks are highly susceptible to backdoor attacks, yet most defense methods to date rely on balanced data, overlooking the pervasive class imbalance in real-world scenarios that can amplify backdoor threats. This paper presents the first in-depth investigation of how the dataset imbalance amplifies backdoor vulnerability, showing that (i) the imbalance induces a majority-class bias that increases susceptibility and (ii) conventional defenses degrade significantly as the imbalance grows. To address this, we propose Randomized Probability Perturbation (RPP), a certified poisoned-sample detection framework that operates in a black-box setting using only model output probabilities. For any inspected sample, RPP determines whether the input has been backdoor-manipulated, while offering provable within-domain detectability guarantees and a probabilistic upper bound on the false positive rate. Extensive experiments on five benchmarks (MNIST, SVHN, CIFAR-10, TinyImageNet and ImageNet10) covering 10 backdoor attacks and 12 baseline defenses show that RPP achieves significantly higher detection accuracy than state-of-the-art defenses, particularly under dataset imbalance. RPP establishes a theoretical and practical foundation for defending against backdoor attacks in real-world environments with imbalanced data.

Figures & tables

Figure 2: Relative frequency distribution of RPP ( $\tilde{\Delta}P$ ) score on the SVHN dataset with $\mu=0.9$ under imbalance ratios: (a) $\rho=2$ and (b) $\rho=100$ .

Figure 4: Performance of RPP against BadNets attacks with perturbation magnitude $\|\delta\|_{2}\geq 0.8$ , measured by TPR and FPR across balanced ( $\rho=1$ ) and imbalanced ( $\rho=2,10,100,200$ ) settings, with $n=100$ and varying $\alpha$ on SVHN, CIFAR-10, TinyImageNet and ImageNet10.

Figure 5: Performance of RPP against BadNets attacks with $\|\delta\|_{2}\geq 0.8$ , measured by TPR and FPR on SVHN, CIFAR-10, TinyImageNet and ImageNet10 under balanced ( $\rho=1$ ) and imbalanced ( $\rho=2,10,100,200$ ) settings for varying $\sigma$ , with $\alpha=0.05$ and $n=100$ .

Table 6

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Class proportion types on the MNIST, SVHN, CIFAR10, TinyImageNet and ImageNet datasets. (a) Long-Tailed Imbalance ( $\mu=1/(\rho+1)$ , $\rho$ = 2, 10, 100, 200). (b) Step Imbalance ( $\mu$ = 0.9, $\rho$ = 2, 10, 100, 200).

Figure 7: Examples of nine types of backdoor attacks on CIFAR10.

Layer	# of Channels	Filter Size	Stride	Activation
Conv	32	$3\times 3$	1	ReLU
MaxPool	32	$2\times 2$	2	-
Conv	64	$3\times 3$	1	ReLU
MaxPool	64	$2\times 2$	2	-
FC	128	-	-	ReLU
FC	10	-	-	Softmax

Appendix

Table 5: Model Architecture for MNIST.

Figure 8: The AUC and ASR of the Badnets backdoor attacks on balanced ( $\rho=1$ ) and imbalanced ( $\mu=0.9,\rho=200$ ) MNIST training datasets with the same poisoned ratios (p = 0.3%) or same number (r = 18) of backdoor samples.

Attack	$\rho$ = 1		$\rho$ = 10		$\rho$ = 200
Attack	AUC	ASR	AUC	ASR	AUC	ASR
Badnets	100.0	13.8	100.0	14.5	98.9	21.8
Blend	100.0	16.3	100.0	18.8	99.0	22.9
Trojan	100.0	29.4	100.0	30.6	99.0	35.6
SIG	100.0	29.0	100.0	33.7	99.0	38.2
ISSBA	99.8	30.2	99.8	47.6	98.8	53.3

Appendix

Table 6: The ASR and AUC under

\lVert\delta\rVert_{2}<0.8

on the balanced MNIST dataset (

\rho

= 1) and two types of imbalanced MNIST datasets (

\mu

= 0.9,

\rho

= 10, 200) with same poisoned ratio (p=0.3%).

Figure 9: Relative frequency distribution of $\tilde{\Delta}P$ for clean and poisoned samples on SVHN dataset with imbalance ratios: (a) $\rho=1$ , (b) $\mu$ = 0.9, $\rho=10$ and (c) $\mu$ = 0.9, $\rho=200$ .

Attack	$\rho=1$	$\rho=2$	$\rho=10$	$\rho=100$	$\rho=200$
Badnets	100.0	100.0	100.0	100.0	99.8
Blend	100.0	100.0	100.0	100.0	99.6
Trojan	100.0	100.0	100.0	100.0	99.7
Narcissus	100.0	100.0	100.0	100.0	100.0

Appendix

Table 7: Percentage of poisoned samples satisfying Assumption 1 under different imbalance ratios

\rho

with

\sigma=1.0

on CIFAR-10, evaluated on four backdoor attacks: Badnets, Blend, Trojan, and Narcissus.

	$\sigma=0$	$\sigma=0.5$	$\sigma=1.0$	$\sigma=1.5$
$\rho$ = 1	100.0	84.6	77.4	68.3
$\rho$ = 2	99.9	83.7	77.7	70.3
$\rho$ = 10	99.7	94.8	89.3	83.9
$\rho$ = 100	100.0	98.6	94.4	90.9
$\rho$ = 200	100.0	99.1	97.5	93.1

Appendix

Table 8: The ASR of Badnets backdoor attack was evaluated on the balanced CIFAR10 training dataset

(\rho=1)

and imbalanced CIFAR10 training datasets

(\rho=2,10,100,200)

with poisoning ratio (p = 0.3%) under different standard deviations of isotropic Gaussian noise (

\sigma=0,0.5,1.0,1.5

Figure 10: Performance of RPP against Badnets backdoor attack with perturbation magnitude $\|\delta\|_{2}\geq 0.8$ across balanced dataset ( $\rho=1$ ) and varying imbalance ratios ( $\mu$ = 0.9, $\rho=2,10,100,200$ ) under different J with $n=100$ , $\alpha=0.05$ on SVHN, CIFAR-10, TinyImageNet and ImageNet10 datasets.

Figure 11: Performance of RPP against Badnets backdoor attacks with perturbation magnitude $\|\delta\|_{2}\geq 0.8$ , measured by TPR and FPR across balanced dataset ( $\rho=1$ ), and varying imbalance ratios ( $\mu$ = 0.9, $\rho=2,10,100,200$ ) for a range of $n$ with $\alpha=0.05$ on SVHN, CIFAR-10, TinyImageNet and ImageNet10 datasets.

Figure 12: Performance of the RPP detection against Badnets backdoor attacks with perturbation magnitude $\|\delta\|_{2}\geq 0.8$ , measured by TPR and FPR across balanced dataset ( $\rho=1$ ) , and varying imbalance ratios ( $\mu$ = 0.9, $\rho=2,10,100$ ) on MNIST dataset. (a) For different $\alpha$ values with $\sigma=0.1$ and $n$ = 100. (b) For a range of $\sigma$ values with $\alpha=0.05$ and $n$ = 100. (c) For various calibration size $n$ with $\alpha=0.05$ and $\sigma=0.1$ .

	Badnets	Blend	Trojan	SIG	ISSBA	WaNet
TPR	81.1	82.2	74.8	77.0	78.5	87.1
FPR	18.7	17.2	15.6	10.9	24.3	12.8

Appendix

Table 10: Detection performance of RPP against different backdoor attack types with

n=100

\alpha=0.05

, and

\sigma=1.0

on the iNaturalist dataset.

	$\rho=1$		$\rho=2$		$\rho=10$		$\rho=100$		$\rho=200$
	TPR	FPR	TPR	FPR	TPR	FPR	TPR	FPR	TPR	FPR
MNIST	100.0	5.7	100.0	6.9	82.4	0.5	100.0	1.9	100.0	6.7
SVHN	99.7	11.6	98.8	3.6	100.0	5.9	98.9	2.9	89.7	4.6
CIFAR10	98.0	5.8	93.1	3.0	97.4	7.0	86.1	3.8	87.0	12.4
TinyImageNet	100.0	1.7	100.0	2.9	98.0	2.6	85.7	4.7	–	–
ImageNet10	97.8	7.3	94.5	8.1	90.2	10.9	88.5	12.0	82.6	13.7

Appendix

Table 11: Performance of the RPP detection against Badnets backdoor attacks with perturbation magnitude

\|\delta\|_{2}\geq 0.8

, measured by TPR and FPR across balanced dataset (

\rho=1

) and varying imbalance ratios (

\mu=0.9

\rho=2,10,100,200

) on MNIST, SVHN, CIFAR-10, TinyImageNet, and ImageNet10 datasets.

$\rho$	Checkpoint	AUC	Minority AUC	ASR	TPR	FPR
1	Early	0.892	0.892	0.829	0.881	0.098
	Middle	0.925	0.925	0.937	0.960	0.063
	Final	0.996	0.996	0.988	0.985	0.045
2	Early	0.904	0.903	0.839	0.868	0.089
	Middle	0.952	0.951	0.958	0.958	0.061
	Final	0.992	0.992	0.999	0.968	0.056

Appendix

Table 13: Impact of preliminary model quality on RPP detection under different CIFAR-10 imbalance ratios

\rho

with Badnets attacks. Early, middle, and final denote checkpoints at 20, 50, and 100 epochs, respectively.

\displaystyle\hat{q}=\inf\left\{q:\frac{|\{i:s_{i}\leq q\}|}{n}\geq\frac{\lceil\alpha(n+1)\rceil}{n}\right\}

Appendix

Algorithm 1 Identification of Backdoored Samples

	$\rho=1$	$\rho=2$	$\rho=10$	$\rho=100$	$\rho=200$
RAB	41.5	40.2	38.8	21.0	15.7
DPA	56.2	46.1	26.6	22.7	14.8

Appendix

Table 15: Certified accuracy (%) of RAB and DPA against Badnets attack on the balanced CIFAR-10 training dataset (

\rho=1

) and imbalanced CIFAR-10 training datasets (

\mu=0.9

\rho=2,10,100,200

	Minority class (9)		Intermediate class (4)
	TPR	FPR	TPR	FPR
$\rho$ = 2	99.4	7.3	93.9	3.1
$\rho$ = 10	93.4	10.1	96.2	3.3
$\rho$ = 100	99.9	2.9	99.3	1.5
$\rho$ = 200	90.0	6.0	79.6	4.6

Appendix

Table 16: Performance of RPP against the BadNets backdoor attack on imbalanced CIFAR-10 training datasets (

\rho=2,10,100,200

) with a

0.3\%

poisoning ratio, where the target class is either a minority class (class 9) or an intermediate-frequency class (class 4).

	ASR (Without LA)	ASR (With LA)	TPR	FPR
$\rho$ = 2	100.0	99.9	99.2	22.2
$\rho$ = 10	99.8	99.4	96.2	20.3
$\rho$ = 100	100.0	97.4	97.8	26.1
$\rho$ = 200	99.7	94.3	83.0	10.4

Appendix

Table 17: Performance of RPP against BadNets attack under logits adjustment on imbalanced CIFAR-10 datasets (

\rho=2,10,100,200

) with a poisoning ratio of 0.3.

	$\\|\delta\\|_{2}=0.8$		$\\|\delta\\|_{2}=2$		$\\|\delta\\|_{2}=3$				$\\|\delta\\|_{2}=4$				$\\|\delta\\|_{2}=6$				$\\|\delta\\|_{2}=8$
	No Defense		No Defense		No Defense		RPP		No Defense		RPP		No Defense		RPP		No Defense		RPP
	ASR	AUC	ASR	AUC	ASR	AUC	TPR	FPR	ASR	AUC	TPR	FPR	ASR	AUC	TPR	FPR	ASR	AUC	TPR	FPR
$\rho=1$	14.3	100.0	33.7	100.0	35.6	100.0	–	–	39.2	100.0	–	–	52.3	100.0	73.1	4.1	90.5	100.0	97.2	8.0
$\rho=2$	14.8	100.0	37.8	100.0	33.5	100.0	–	–	47.9	100.0	–	–	69.6	100.0	88.3	6.9	90.3	100.0	96.0	6.1
$\rho=10$	15.4	100.0	30.5	100.0	38.3	100.0	–	–	49.4	100.0	–	–	84.8	100.0	90.1	9.6	99.8	100.0	97.9	7.7
$\rho=100$	23.0	99.3	39.6	99.4	52.1	98.6	73.8	10.1	100.0	99.4	87.5	10.7	100.0	99.6	95.2	10.1	100.0	99.5	100.0	10.9

Appendix

Table 18: Performance of RPP under various trigger sizes with a

0.3\%

poisoning ratio on the balanced MNIST (

\rho=1

) and imbalanced MNIST datasets (

\mu

= 0.9,

\rho=2,10,100,200

) for

n=100

and

\alpha=0.05

	$\alpha=0.05$				$\alpha=0.1$
	$n=100$	$n=200$	$n=400$	$n=600$	$n=100$	$n=200$	$n=400$	$n=600$
$\rho=1$	4.0	5.3	5.1	4.4	10.7	9.9	9.1	8.8
$\rho=2$	4.9	4.1	4.6	4.0	9.3	9.9	10.1	9.0
$\rho=10$	5.5	4.9	4.6	4.2	10.7	9.8	9.5	9.1
$\rho=100$	5.5	4.7	4.8	5.0	9.9	9.2	10.2	10.0
$\rho=200$	5.1	4.9	4.8	4.7	10.4	10.0	9.8	10.1

Appendix

Table 19: The FPR (%) under varying imbalance ratio (

\rho

), isotropic Gaussian noise standard deviation (

\alpha

), and calibration set size (

n

) on CIFAR10 dataset.

Backdoor Attacks

Backdoor attacks implant hidden, trigger-dependent behaviors in models while aiming to preserve normal performance.

See all 135 papers

CSC: Turning the Adversary's Poison against Itself
Apr 23, 2026Yuchen Shi, Xin Guo, Huajie Chen +3Backdoor Attacks Backdoor Defense
Faculty of Data Science, City University of Macau, Taipa, Macau. · Faculty of Engineering and Information Technology, University of Technology Sydney, New South Wales, Australia.
Fast and Lightweight Backdoor Detection via Head Random Probing
May 17, 2026Yinbo Yu, Xueyu Yin, Jing Fang +4Backdoor Detection
College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu, 210016, China · School of Cybersecurity, Northwestern Polytechnical University, Xi’an 710072, China · Shenzhen Research Institute of Northwestern Polytechnical University, Shenzhen, Guangdong, 518057, China +1
Checkerboard: A Simple, Effective, Efficient and Learning-free Clean Label Backdoor Attack with Low Poisoning Budget
May 2, 2026Yi Yang, Jinyang Huang, Binbin Liu +5Backdoor Attacks Data Poisoning Attacks
School of Computer Science and Information Engineering Hefei University of Technology Hefei, Anhui, China · School of Information Science and Technology University of Science and Technology of China Hefei, Anhui, China · Faculty of Business Data Science Kansai University Japan +3

Apr 23, 2026J/K move · Enter open · S save

CSC: Turning the Adversary's Poison against Itself

Yuchen Shi, Xin Guo, Huajie Chen, Tianqing Zhu+2

Faculty of Data Science, City University of Macau, Taipa, Macau. · Faculty of Engineering and Information Technology, University of Technology Sydney, New South Wales, Australia.

Backdoor Attacks Backdoor Defense Data Poisoning Attacks

Open paper AskPDF arXiv

Poisoning-based backdoor attacks pose significant threats to deep neural networks by embedding triggers in training data, causing models to misclassify triggered inputs as adversary-specified labels while maintaining performance on clean data. Existing poison restraint-based defenses often suffer from inadequate detection against specific attack variants and compromise model utility through unlearning methods that lead to accuracy degradation. This paper conducts a comprehensive analysis of backdoor attack dynamics during model training, revealing that poisoned samples form isolated clusters in latent space early on, with triggers acting as dominant features distinct from benign ones. Leveraging these insights, we propose Cluster Segregation Concealment (CSC), a novel poison suppression defense. CSC first trains a deep neural network via standard supervised learning while segregating poisoned samples through feature extraction from early epochs, DBSCAN clustering, and identification of anomalous clusters based on class diversity and density metrics. In the concealment stage, identified poisoned samples are relabeled to a virtual class, and the model's classifier is fine-tuned using cross-entropy loss to replace the backdoor association with a benign virtual linkage, preserving overall accuracy. CSC was evaluated on four benchmark datasets against twelve poisoning-based attacks, CSC outperforms nine state-of-the-art defenses by reducing average attack success rates to near zero with minimal clean accuracy loss. Contributions include robust backdoor patterns identification, an effective concealment mechanism, and superior empirical validation, advancing trustworthy artificial intelligence.

Abstract

Backdoor Attacks

Explore similar work

CSC: Turning the Adversary's Poison against Itself

Fast and Lightweight Backdoor Detection via Head Random Probing

Checkerboard: A Simple, Effective, Efficient and Learning-free Clean Label Backdoor Attack with Low Poisoning Budget