Noise-Aware Misclassification Attack Detection in Collaborative DNN Inference
Organizations: City University of New York
Abstract
Collaborative inference of object classification Deep neural Networks (DNNs) where resource-constrained end-devices offload partially processed data to remote edge servers to complete end-to-end processing, is becoming a key enabler of edge-AI. However, such edge-offloading is vulnerable to malicious data injections leading to stealthy misclassifications that are tricky to detect, especially in the presence of environmental noise. In this paper, we propose a semi-gray-box and noise- aware anomaly detection framework fueled by a variational autoencoder (VAE) to capture deviations caused by adversarial manipulation. The proposed framework incorporates a robust noise-aware feature that captures the characteristic behavior of environmental noise to improve detection accuracy while reducing false alarm rates. Our evaluation with popular object classification DNNs demonstrate the robustness of the proposed detection (up to 90% AUROC across DNN configurations) under realistic noisy conditions while revealing limitations caused by feature similarity and elevated noise levels.
Figures & tables
| Level | Acc(%) | RWCG | ||||
|---|---|---|---|---|---|---|
| No noise | – | – | – | 0.00 | 0.00 | 0.000 |
| Light | 1.8 | 0.01 | 0.005 | 0.15 | 0.00 | 0.000 |
| Moderate | 1.6 | 0.02 | 0.010 | 0.30 | -0.16 | +0.011 |
| Severe | 1.4 | 0.08 | 0.10 | 0.30 | -0.57 | -0.006 |
| Extreme | 1.2 | 0.12 | 0.15 | 0.50 | -8.29 | -0.054 |
| Method | Env. | Acc. | Prec. | Rec. | F1 | FPR |
|---|---|---|---|---|---|---|
| adVAE [ 31 ] + Radius | Clean | 0.969 | 0.996 | 0.945 | 0.970 | 0.004 |
| Noisy | 0.835 | 0.778 | 0.958 | 0.859 | 0.301 | |
| adVAE + OC-SVM | Clean | 0.85 | 1.00 | 0.70 | 0.301 | 0.880 |
| Noisy | 0.73 | 0.51 | 0.97 | 0.028 | 0.660 |
| Layer 3 % | Layer 6 % | Layer 8 % | Layer 10 % | |||||
|---|---|---|---|---|---|---|---|---|
| ASR | Conf | ASR | Conf | ASR | Conf | ASR | Conf | |
| 0.0 | 83.33 | 40.76 | 43.57 | 52.82 | 53.83 | 46.79 | 47.42 | 64.26 |
| 0.2 | 91.02 | 32.25 | 57.68 | 47.64 | 65.37 | 38.54 | 64.09 | 54.43 |
| 0.4 | 96.15 | 27.23 | 85.89 | 31.66 | 83.33 | 32.56 | 88.46 | 54.23 |
| 0.6 | 98.72 | 29.49 | 96.15 | 34.97 | 94.87 | 36.24 | 97.43 | 74.25 |
| 0.8 | 97.43 | 38.73 | 98.72 | 58.86 | 97.43 | 49.22 | 100.00 | 91.44 |
| Layer 12 % | Layer 16 % | Layer 18 % | Layer 20 % | |||||
|---|---|---|---|---|---|---|---|---|
| ASR | Conf | ASR | Conf | ASR | Conf | ASR | Conf | |
| 0.0 | 76.89 | 49.53 | 39.10 | 63.50 | 18.49 | 74.66 | 5.11 | 88.51 |
| 0.2 | 82.97 | 42.18 | 41.73 | 63.50 | 35.52 | 70.62 | 48.91 | 82.87 |
| 0.4 | 92.70 | 35.54 | 46.29 | 61.69 | 81.75 | 70.52 | 97.57 | 87.07 |
| 0.6 | 97.57 | 39.42 | 48.21 | 66.41 | 100.00 | 82.28 | 91.48 | 95.74 |
| 0.8 | 98.78 | 57.01 | 48.36 | 68.01 | 100.00 | 89.58 | 100.00 | 98.53 |
| Layer 20 % | Layer 40 % | Layer 50 % | Layer 63 % | |||||
|---|---|---|---|---|---|---|---|---|
| ASR | Conf | ASR | Conf | ASR | Conf | ASR | Conf | |
| 0.0 | 97.12 | 51.38 | 94.24 | 81.13 | 77.67 | 55.22 | 88.70 | 48.81 |
| 0.2 | 98.56 | 46.40 | 97.12 | 81.91 | 94.24 | 49.66 | 88.82 | 49.62 |
| 0.4 | 98.56 | 56.71 | 98.56 | 86.26 | 96.40 | 51.50 | 88.93 | 48.85 |
| 0.6 | 98.56 | 61.97 | 100.00 | 89.22 | 97.12 | 49.94 | 97.12 | 28.12 |
| 0.8 | 98.56 | 64.48 | 100.00 | 92.36 | 99.28 | 45.77 | 98.56 | 39.72 |
| Accuracy | Balanced accuracy | AUROC | ||||||
|---|---|---|---|---|---|---|---|---|
| NA | NU | NA | NU | NA | NU | NA | NU | |
| 0.2 | 0.835 | 0.476 | 0.834 | 0.105 | 0.837 | 0.479 | 0.824 | 0.348 |
| 0.4 | 0.868 | 0.513 | 0.858 | 0.185 | 0.814 | 0.548 | 0.819 | 0.370 |
| 0.6 | 0.875 | 0.456 | 0.870 | 0.093 | 0.881 | 0.481 | 0.834 | 0.685 |
| 0.8 | 0.852 | 0.489 | 0.877 | 0.108 | 0.845 | 0.511 | 0.815 | 0.443 |
| 1.0 | 0.867 | 0.800 | 0.890 | 0.833 | 0.855 | 0.790 | 0.857 | 0.730 |
| Accuracy | Balanced accuracy | AUROC | ||||||
|---|---|---|---|---|---|---|---|---|
| NA | NU | NA | NU | NA | NU | NA | NU | |
| 0.2 | 0.811 | 0.529 | 0.803 | 0.257 | 0.814 | 0.803 | 0.820 | 0.548 |
| 0.4 | 0.850 | 0.496 | 0.839 | 0.202 | 0.856 | 0.515 | 0.744 | 0.567 |
| 0.6 | 0.830 | 0.513 | 0.831 | 0.282 | 0.833 | 0.539 | 0.853 | 0.603 |
| 0.8 | 0.738 | 0.474 | 0.786 | 0.008 | 0.729 | 0.497 | 0.555 | 0.314 |
| 1 | 0.779 | 0.747 | 0.830 | 0.807 | 0.759 | 0.726 | 0.657 | 0.647 |
| Accuracy | Balanced accuracy | AUROC | ||||||
|---|---|---|---|---|---|---|---|---|
| NA | NU | NA | NU | NA | NU | NA | NU | |
| 0.2 | 0.637 | 0.637 | 0.742 | 0.740 | 0.618 | 0.618 | 0.601 | 0.566 |
| 0.4 | 0.630 | 0.621 | 0.506 | 0.475 | 0.646 | 0.639 | 0.685 | 0.639 |
| 0.6 | 0.647 | 0.606 | 0.523 | 0.433 | 0.665 | 0.626 | 0.689 | 0.641 |
| 0.8 | 0.557 | 0.478 | 0.672 | 0.033 | 0.542 | 0.501 | 0.465 | 0.434 |
| 1 | 0.547 | 0.457 | 0.355 | 0.181 | 0.575 | 0.488 | 0.594 | 0.510 |
| Layer | Accuracy | Balanced accuracy | AUROC | |||||
|---|---|---|---|---|---|---|---|---|
| NA | NU | NA | NU | NA | NU | NA | NU | |
| 12 | 0.991 | 0.749 | 0.991 | 0.688 | 0.992 | 0.742 | 1.000 | 0.794 |
| 16 | 0.984 | 0.669 | 0.984 | 0.561 | 0.984 | 0.669 | 1 | 0.750 |
| 18 | 0.990 | 0.498 | 0.990 | 0.000 | 0.990 | 0.480 | 0.994 | 0.662 |
| Noise | Layer | Accuracy | Balanced acc. | AUROC | |||||
|---|---|---|---|---|---|---|---|---|---|
| NA | NU | NA | NU | NA | NU | NA | NU | ||
| Moderate | 3 | 0.891 | 0.641 | 0.886 | 0.624 | 0.891 | 0.642 | 0.916 | 0.705 |
| 6 | 0.836 | 0.677 | 0.808 | 0.664 | 0.832 | 0.677 | 0.733 | 0.636 | |
| 8 | 0.847 | 0.769 | 0.836 | 0.782 | 0.853 | 0.768 | 0.808 | 0.815 | |
| 10 | 0.820 | 0.609 | 0.821 | 0.578 | 0.820 | 0.612 | 0.770 | 0.592 | |
| Severe | 3 | 0.897 | 0.510 | 0.902 | 0.417 | 0.897 | 0.512 | 0.953 | 0.534 |
| Noise | Layer | Accuracy | Balanced accuracy | AUROC | |||||
|---|---|---|---|---|---|---|---|---|---|
| NA | NU | NA | NU | NA | NU | NA | NU | ||
| Moderate | 20 | 0.831 | 0.520 | 0.811 | 0.211 | 0.831 | 0.521 | 0.838 | 0.481 |
| 40 | 0.829 | 0.793 | 0.844 | 0.817 | 0.834 | 0.799 | 0.830 | 0.817 | |
| 50 | 0.849 | 0.670 | 0.836 | 0.538 | 0.848 | 0.665 | 0.945 | 0.726 | |
| Severe | 20 | 0.745 | 0.533 | 0.663 | 0.243 | 0.746 | 0.534 | 0.819 | 0.542 |
| 40 | 0.927 | 0.855 | 0.928 | 0.843 | 0.929 | 0.853 | 0.981 | 0.918 | |
| Model | Cut Layer | Output Size | Latency (ms) |
|---|---|---|---|
| VGG19 | 12 | 28×28×512 | 46.84 |
| 16 | 14×14×512 | 13.25 | |
| 18 | 14×14×512 | 13.22 | |
| 20 | 14×14×512 | 13.09 | |
| AlexNet | 3 | 27×27×192 | 17.41 |
| 6 | 13×13×384 | 8.82 |
Adversarial Example Detection
Adversarial example detection identifies inputs crafted to exploit machine-learning models before they cause unreliable predictions.
See all 47 papers