Power Side-Channel Membership Inference Attack on Embedded Machine Learning
Organizations: Department of Computer & Information Science & Engineering, University of Florida, USA
Abstract
Membership inference attacks (MIAs) threaten the privacy of machine learning (ML) training data by determining whether a sample was used to train a target model. Existing MIAs rely on model outputs, ranging from prediction probabilities to predicted labels, an assumption that can be restrictive for on-device ML systems with limited or inaccessible outputs. However, suppressing model outputs does not eliminate the data-dependent computations that produce them, which may remain observable through physical side channels. We present PSCMIA, a power side-channel membership inference attack against embedded ML models that can infer membership directly from power traces without requiring prediction probabilities or even the predicted labels. We evaluate PSCMIA across multiple datasets (MNIST, FMNIST, CIFAR10, CINIC10), fully connected (FC) and convolutional neural network (CNN) architectures, and two embedded platforms (STM32F3, XMEGA). PSCMIA achieves ROC-AUC values of up to 0.907 on FC models. For CNN models, the ROC-AUC gap between PSCMIA and probability vector-based shadow MIA ranges from 0.006 to 0.116. Across the FC and CNN evaluations, PSCMIA outperforms label-only MIA in 11 of 16 model-dataset-hardware configurations, demonstrating that physical execution can expose membership information even when conventional model outputs are unavailable through unintended power side-channel leakage.
Figures & tables
| FC | CNN | ||||||
|---|---|---|---|---|---|---|---|
| Dataset | Train Size | Quant. | Train Acc. | Test Acc. | Quant. | Train Acc. | Test Acc. |
| MNIST | 2000 | 8-bit | 100.00 | 92.15 | 8/2/8-bit | 99.93 | 95.35 |
| FMNIST | 2000 | 8-bit | 99.85 | 83.23 | 8/2/8-bit | 99.86 | 81.74 |
| CIFAR10 | 2000 | 8-bit | 85.28 | 25.45 | 8/2/8-bit | 83.75 | 32.35 |
| CINIC10 | 2000 | 8-bit | 93.90 | 19.00 | 8/2/8-bit | 78.79 | 24.13 |
| MNIST | FMNIST | CIFAR10 | CINIC10 | |||||||||||||
| Metric | Shadow Model | Label Only | PSCMIA | Shadow Model | Label Only | PSCMIA | Shadow Model | Label Only | PSCMIA | Shadow Model | Label Only | PSCMIA | ||||
| STM32F3 | XMEGA | STM32F3 | XMEGA | STM32F3 | XMEGA | STM32F3 | XMEGA | |||||||||
| ROC-AUC | 0.6582 | 0.5281 | 0.6193 | 0.6186 | 0.6804 | 0.6016 | 0.6795 | 0.6477 | 0.8794 | 0.8137 | 0.7958 | 0.7952 | 0.9445 | 0.8960 | 0.9065 | 0.8925 |
| FPR | 0.7090 | 0.3185 | 0.1010 | 0.4885 | 0.7215 | 0.4715 | 0.4340 | 0.5330 | 0.1920 | 0.2355 | 0.2540 | 0.2435 | 0.1365 | 0.2000 | 0.1445 | 0.1760 |
| Recall | 0.9365 | 0.3545 | 0.1505 | 0.6505 | 0.9775 | 0.6260 | 0.6815 | 0.7445 | 0.8010 | 0.7795 | 0.6870 | 0.7215 | 0.8930 | 0.9035 | 0.8060 | 0.8415 |
| TPR@1%FPR | 0.0155 | 0.0305 | 0.0140 | 0.0300 | 0.0265 | 0.0515 | 0.0365 | 0.0145 | 0.1635 | 0.0905 | 0.0675 | 0.0885 | 0.3925 | 0.1730 | 0.1965 | 0.2180 |
| MNIST | FMNIST | CIFAR10 | CINIC10 | |||||||||||||
| Metric | Shadow Model | Label Only | PSCMIA | Shadow Model | Label Only | PSCMIA | Shadow Model | Label Only | PSCMIA | Shadow Model | Label Only | PSCMIA | ||||
| STM32F3 | XMEGA | STM32F3 | XMEGA | STM32F3 | XMEGA | STM32F3 | XMEGA | |||||||||
| ROC-AUC | 0.6089 | 0.5111 | 0.5821 | 0.5784 | 0.6945 | 0.5681 | 0.6514 | 0.6588 | 0.7726 | 0.7285 | 0.7464 | 0.7668 | 0.8572 | 0.7978 | 0.7417 | 0.7757 |
| FPR | 0.5040 | 0.1040 | 0.0560 | 0.5165 | 0.6880 | 0.2300 | 0.1370 | 0.4875 | 0.3170 | 0.2420 | 0.2360 | 0.3010 | 0.2290 | 0.2125 | 0.2350 | 0.2995 |
| Recall | 0.6420 | 0.1405 | 0.0680 | 0.6060 | 0.9535 | 0.3750 | 0.2460 | 0.7010 | 0.7190 | 0.6210 | 0.5350 | 0.7490 | 0.7800 | 0.7265 | 0.5770 | 0.7315 |
| TPR@1%FPR | 0.0120 | 0.0235 | 0.0140 | 0.0240 | 0.0300 | 0.0455 | 0.0360 | 0.0215 | 0.0780 | 0.0870 | 0.0540 | 0.0050 | 0.0880 | 0.1315 | 0.0405 | 0.0475 |
| Target property | Model | Classification | Base Acc. | Acc. | Base BAcc. | BAcc. | F1 | ROC-AUC | Top-2 Acc. |
|---|---|---|---|---|---|---|---|---|---|
| Probability-vector Shadow Model MIA | LR | Membership | 0.500 | 0.878 | 0.500 | 0.878 | 0.880 | 0.944 | – |
| Class prediction | LR | 10 classes | 0.111 | 0.522 | 0.100 | 0.520 | 0.517 | 0.929 | 0.805 |
| Class prediction | ET | 10 classes | 0.111 | 0.622 | 0.100 | 0.620 | 0.617 | 0.955 | 0.872 |
| Correctness prediction | LR | Binary | 0.565 | 0.593 | 0.500 | 0.594 | 0.591 | 0.637 | – |
| Correctness prediction | ET | Binary | 0.565 | 0.616 | 0.500 | 0.598 | 0.596 | 0.650 | – |
| Top-1 probability bin | LR | 8 classes | 0.194 | 0.193 | 0.125 | 0.283 | 0.180 | 0.698 | 0.428 |
| Train | Train | Test | Gen. | Shadow MIA | PSCMIA Pooled ROC-AUC | |
|---|---|---|---|---|---|---|
| Size | Acc. (%) | Acc. (%) | Gap | Pooled AUC | STM32F3 | XMEGA |
| 200 | 100.00 | 70.40 | 29.60 | 0.8974 | 0.7936 | 0.7918 |
| 500 | 99.96 | 83.68 | 16.28 | 0.8345 | 0.7095 | 0.7442 |
| 1000 | 99.90 | 89.70 | 10.20 | 0.7238 | 0.6699 | 0.6768 |
| 2000 | 100.00 | 92.15 | 7.85 | 0.6582 | 0.6193 | 0.6186 |
| 5000 | 100.00 | 94.94 | 5.06 | 0.5681 | 0.5680 | 0.5632 |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| Attack | Dataset | ROC-AUC | FPR | TPR@1%FPR | Acc. |
|---|---|---|---|---|---|
| Shadow MIA [ 1 ] | MNIST | 0.6582 | 0.7090 | 0.0155 | 0.6138 |
| FMNIST | 0.6804 | 0.7215 | 0.0265 | 0.6280 | |
| CIFAR10 | 0.8794 | 0.1920 | 0.1635 | 0.8045 | |
| CINIC10 | 0.9445 | 0.1365 | 0.3925 | 0.8782 | |
| Label-only [ 11 ] | MNIST | 0.5281 | 0.3185 | 0.0305 | 0.5180 |
| FMNIST | 0.6016 | 0.4715 | 0.0515 | 0.5773 |
| Attack | Dataset | ROC-AUC | FPR | TPR@1%FPR | Acc. |
|---|---|---|---|---|---|
| Shadow MIA [ 1 ] | MNIST | 0.6089 | 0.5040 | 0.0120 | 0.5690 |
| FMNIST | 0.6945 | 0.6880 | 0.0300 | 0.6328 | |
| CIFAR10 | 0.7726 | 0.3170 | 0.0780 | 0.7010 | |
| CINIC10 | 0.8572 | 0.2290 | 0.0880 | 0.7755 | |
| Label-only [ 11 ] | MNIST | 0.5111 | 0.1040 | 0.0235 | 0.5182 |
| FMNIST | 0.5681 | 0.2300 | 0.0455 | 0.5725 |
| Board | Model/Dataset | Attack | ROC-AUC | FPR | Recall | Acc. |
|---|---|---|---|---|---|---|
| STM32F3 | FC / MNIST | LR | 0.6113 | 0.1310 | 0.1925 | 0.5308 |
| ET | 0.6193 | 0.1010 | 0.1505 | 0.5248 | ||
| RF | 0.6165 | 0.1045 | 0.1475 | 0.5215 | ||
| STM32F3 | FC / FMNIST | LR | 0.6708 | 0.4860 | 0.7290 | 0.6215 |
| ET | 0.6795 | 0.4340 | 0.6815 | 0.6238 | ||
| RF | 0.6736 | 0.3945 | 0.6125 | 0.6090 |
| Defense | Model | ROC-AUC | FPR | Recall | TPR@1%FPR | Acc. |
|---|---|---|---|---|---|---|
| Without | MNIST-FC | 0.6193 | 0.1010 | 0.1505 | 0.0140 | 0.5248 |
| FMNIST-FC | 0.6795 | 0.4340 | 0.6815 | 0.0365 | 0.6238 | |
| CIFAR10-FC | 0.7958 | 0.2540 | 0.6870 | 0.0675 | 0.7165 | |
| CINIC10-FC | 0.9065 | 0.1445 | 0.8060 | 0.1965 | 0.8308 | |
| MNIST-CNN | 0.5821 | 0.0560 | 0.0680 | 0.0140 | 0.5060 | |
| FMNIST-CNN | 0.6514 | 0.1370 | 0.2460 | 0.0360 | 0.5545 |
| Board | Model | Dataset | Class-0 | Class-1 | Class-2 | Class-3 | Class-4 | Class-5 | Class-6 | Class-7 | Class-8 | Class-9 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| STM32F3 | FC | MNIST | 0.5776 | 0.5682 | 0.5877 | 0.7035 | 0.6838 | 0.6132 | 0.5749 | 0.6649 | 0.6832 | 0.6624 |
| STM32F3 | FC | FMNIST | 0.7140 | 0.5722 | 0.6939 | 0.5898 | 0.7181 | 0.6187 | 0.8078 | 0.6644 | 0.5871 | 0.5979 |
| STM32F3 | FC | CIFAR10 | 0.7631 | 0.8280 | 0.8538 | 0.8242 | 0.8500 | 0.8205 | 0.8307 | 0.8039 | 0.8031 | 0.7381 |
| STM32F3 | FC | CINIC10 | 0.8729 | 0.8940 | 0.9422 | 0.8915 | 0.9092 | 0.9079 | 0.9244 | 0.9083 | 0.8992 | 0.9011 |
| STM32F3 | CNN | MNIST | 0.5373 | 0.6042 | 0.6351 | 0.6460 | 0.6080 | 0.5289 | 0.5463 | 0.5694 | 0.6241 | 0.6348 |
| STM32F3 | CNN | FMNIST | 0.6965 | 0.5592 | 0.7555 | 0.6349 | 0.7527 | 0.5943 | 0.8637 | 0.5817 | 0.5631 | 0.5442 |