Adversarial Robustness of Kernel Machines: A Comprehensive Analysis

Rahul Kumar Yadav; Tunir Ghosh; Sudhendu Ahir; Parthe Pandit; Arjun Bhagoji

model	hyperparameters	time ( $1\text{v}1$ ) †	time (multi-class) †
FCNN	width, depth, lr, weight decay	$\approx 40$ s	$\approx 300$ s
FCNN-adv	$\cdots$ + $\varepsilon$	$\approx 300$ s	$\approx 900$ s
LeNet	N.A.	$\approx 40$ s	$\approx 240$ s
adv-LeNet	N.A.	$\approx 120$ s	$\approx 720$ s
KRR	$\lambda$ , $\sigma$	$\approx 5$ s	$\approx 30$ s
RFM	$\cdots$ + #iterations	$\approx 25$ s	$\approx 100$ s

Appendix

Table 3A: Computational complexity of training on a single NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation GPU. ( † time in seconds(s) to train a single model on Fashion MNIST. Note that KRR and RFM have fewer hyperparameters to tune, hence the total time is smaller.)

Figure 1A: Comparison of the robust accuracy against

\ell_{2}

perturbations (

\uparrow

is better) for 10 models across 3 datasets. Results were obtained using autoattack , using apgd-ce.

Figure 2A: Gap in robust accuracy (ra) for

\ell_{2}

perturbation on 45

1\text{v}1

binary classification tasks from FashionMNIST comparing 2 neural networks (columns) and 3 kernel machines (rows) with the Laplacian kernel. Perturbation size increases as we go down each heatmap, and values correspond to ra(row model) - ra(col model). The teal colour indicates that the kernel machine is more robust, whereas the orange color indicates that the neural network is more robust. Each of the 6 heatmaps compares 45 pairs of models, each finetuned for best accuracy on clean data. For each heatmap, class pairs for binary classification are sorted according to values in the bottom row. PGD attack is used to attack the models.

Figure 3A: Gap in robust accuracy (ra) for

\ell_{2}

perturbation on 45

1\text{v}1

binary classification tasks from FashionMNIST comparing 2 neural networks (columns) and 2 RFCs with different number of features (rows) with the Laplacian kernel. Perturbation size increases as we go down each heatmap, and values correspond to ra(row model) - ra(col model). The teal colour indicates that the RFC is more robust, whereas the orange color indicates that the neural network is more robust. Each of the 4 heatmaps compares 45 pairs of models. For each heatmap, class pairs for binary classification are sorted according to values in the bottom row. (Top) RFC with 5000 random features (Bottom) RFC with 20000 random features. PGD attack is used to attack the models.

Figure 4A: Summary of results for the multiclass KMNIST dataset. We compare robust accuracy of 13 models (3 adversarially trained, 10 benign) on KMNIST (10 classes) attacked via PGD using

\ell_{\infty}

perturbations. Columns with yellow borders correspond to adversarially trained models with training adversary’s attack level highlighted by the red row. Black boxes mark the best robust accuracy across each row. Note the best models for high

\varepsilon

are kernel machines with logistic loss.

Figure 5A: Summary of results for the multiclass QMNIST dataset. We compare robust accuracy of 13 models (3 adversarially trained, 10 benign) on QMNIST (10 classes), attacked via PGD using

\ell_{\infty}

perturbations.. Columns with yellow borders correspond to adversarially trained models with training adversary’s attack level highlighted by the red row. Black boxes mark the best robust accuracy across each row. Note the best models for high

\varepsilon

are kernel machines with logistic loss.

Figure 6A: Test for robustness as a pointwise property. We plot the empirical CDF of the ratio (KLR/other models) of the smallest

\ell_{\infty}

perturbations

\widehat{\varepsilon}

needed to successfully attack 423 random samples from the test set of KMNIST. Note the x-axis is in log-scale. See Section 5.2 for more details. KRR is more robust than both FCNN and LeNet. For the median datum, KLR is nearly 2x more robust than Adv-FCNN and Adv-LeNet. The adversarially trained FCNN, LeNet and KLR were trained with an

\ell_{\infty}

(matched) adversary with a budget of

\varepsilon=\frac{16}{255}

. Adversarially trained KM remains most robust.

Figure 7A: Test for robustness as a pointwise property. We plot the empirical CDF of the ratio (KLR/other models) of the smallest

\ell_{\infty}

perturbations

\widehat{\varepsilon}

needed to successfully attack 479 random samples from the test set of QMNIST. Note the x-axis is in log-scale. See Section 5.2 for more details. For most samples, KRR is harder to attack than the FCNN. For the median datum, KLR is more robust than Adv-FCNN and Adv-LeNet. The adversarially trained FCNN, LeNet and KLR were trained with an

\ell_{\infty}

(matched) adversary with a budget of

\varepsilon=\frac{24}{255}

. Adversarially trained KM remains most robust.

Figure 8A: Gap in robust accuracy (ra) for

\ell_{\infty}

perturbation on 45

1\text{v}1

binary classification tasks from FashionMNIST comparing 2 neural networks (columns) and 3 kernel machines (rows) with the Laplacian kernel. Perturbation size increases as we go down each heatmap, and values correspond to ra(row model) - ra(col model). The teal colour indicates that the kernel machine is more robust, whereas the orange color indicates that the neural network is more robust. Each of the 6 heatmaps compares 45 pairs of models. For each heatmap, class pairs for binary classification are sorted according to values in the bottom row. (Top) Kernel machines are more robust than fully connected neural networks and convolutional networks. (Middle) Recursive Feature Machines retain much of the robustness of the base kernel machine (

M=I_{d}

). (Bottom) Laplacian SVM are more robust to both FCNN and LeNet. PGD attack is used to attack the models.

Figure 9A: Laplacian KLR versus FCNN:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 10A: Laplacian KLR versus LeNet:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 11A: Laplacian KRR versus FCNN:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 12A: Laplacian KRR versus LeNet:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 13A: Laplacian RFM versus FCNN:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 14A: Laplacian RFM versus LeNet:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 15A: Laplacian SVM versus FCNN:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 16A: Laplacian SVM versus LeNet:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 17A: Laplacian KLR vs Laplacian KRR:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 18A: Laplacian KLR vs Laplacian RFM:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 19A: Laplacian KLR vs Laplacian SVM:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 20A: Laplacian KRR vs Laplacian RFM:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 21A: Laplacian KRR vs Laplacian SVM:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 22A: Laplacian RFM vs Laplacian SVM:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 23A: Laplacian RFC (5k features) versus FCNN:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. Laplace RFC is trained using square loss. PGD attack is used to attack the models.

Figure 24A: Laplacian RFC (20k features) versus FCNN:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. Laplace RFC is trained using square loss. PGD attack is used to attack the models.

Figure 25A: Laplacian KLR vs Laplacian RFC (5k features):

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 26A: Laplacian KLR vs Laplacian RFC (20k features):

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 27A: Laplacian KRR vs Laplacian RFC (5k features):

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 28A: Laplacian KRR vs Laplacian RFC (20k features):

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 29A: Laplacian RFM vs Laplacian RFC (5k features):

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 30A: Laplacian RFM vs Laplacian RFC (20k features):

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 31A: Laplacian SVM vs Laplacian RFC (5k features):

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 32A: Laplacian SVM vs Laplacian RFC (20k features):

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 33A: Laplacian RFC (5k features) versus LeNet:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 34A: Laplacian RFC (20k features) versus LeNet:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 35A: Gaussian KLR versus FCNN:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 36A: Gaussian KRR versus FCNN:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 37A: Gaussian RFM versus FCNN:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 38A: Gaussian SVM versus FCNN:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 39A: Gaussian KLR versus Gaussian KRR:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 40A: Gaussian KLR versus Gaussian RFM:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 41A: Gaussian KLR versus Gaussian SVM:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 42A: Gaussian KRR versus Gaussian RFM:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 43A: Gaussian KRR versus Gaussian SVM:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 44A: Gaussian RFM versus Gaussian SVM:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 45A: Gap in robust accuracy (ra) for

\ell_{\infty}

perturbation on 45

1\text{v}1

binary classification tasks from FashionMNIST comparing 2 neural networks (columns) and 2 RFCs with different number of features (rows) with the Laplacian kernel. Perturbation size increases as we go down each heatmap, and values correspond to ra(row model) - ra(col model). The teal colour indicates that the RFC is more robust, whereas the orange color indicates that the neural network is more robust. Each of the 4 heatmaps compares 45 pairs of models. For each heatmap, class pairs for binary classification are sorted according to values in the bottom row. (Top) RFC with 5000 random features are a bit less robust compared to fully connected neural networks and convolutional neural networks. (Bottom) RFC with 20000 random features are competing in robustness with fully connected neural networks and convolutional neural networks. Although, these results shouldn’t be taken at face value as we see in Section D.7 that Laplace RFCs exhibit gradient masking. PGD attack is used to attack the models.

Figure 46A: Gaussian RFC (5k features) versus FCNN:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 47A: Gaussian RFC (20k features) versus FCNN:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 48A: Gaussian KLR versus Gaussian RFC (5k features):

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 49A: Gaussian KLR versus Gaussian RFC (20k features):

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 50A: Gaussian KRR versus Gaussian RFC (5k features):

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 51A: Gaussian KRR versus Gaussian RFC (20k features):

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 52A: Gaussian RFM versus Gaussian RFC (5k features):

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 53A: Gaussian RFM versus Gaussian RFC (20k features):

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 54A: Gaussian SVM versus Gaussian RFC (5k features):

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 55A: Gaussian SVM versus Gaussian RFC (20k features):

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 56A: Gaussian KLR versus LeNet:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 57A: Gaussian KRR versus LeNet:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 58A: Gaussian RFM versus LeNet:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 59A: Gaussian SVM versus LeNet:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 60A: Gaussian RFC (5k features) versus LeNet:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 61A: Gaussian RFC (20k features) versus LeNet:

\ell_{2}

and

\ell_{\infty}

perturbation for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD attack is used to attack the models.

Figure 62A:

\ell_{\infty}

and

\ell_{2}

attacks on SVHN . (Left) corresponds to

\ell_{\infty}

and (Right) corresponds to

\ell_{2}

autoattacks.

$\epsilon$ ( $L_{\infty}$ )	Gaussian Kernel ( $\%$ )	Gaussian RFM ( $\%$ )
$0$	62.56	82.46
$0.1/255$	62.11	81.48
$1/255$	58.22	71.10
$2/255$	53.86	56.59
$4/255$	46.01	36.08
$6/255$	39.48	26.30

Appendix

Table 4A: Robust accuracies for the Gaussian kernel and Gaussian RFM with bandwidth

0.05

on SVHN

Figure 63A: Gap in robust accuracy (ra) for

\ell_{\infty}

perturbation on 5

1\text{v}1

binary classification tasks from CIFAR-10 . (Top) row has Laplace Kernel, and (Bottom) row has Laplace RFM. (First) Column is FCNN, and (Second) column is LeNet. PGD is used to attack the models.

Figure 64A: Gap in robust accuracy (ra) for

\ell_{2}

perturbation on 5

1\text{v}1

binary classification tasks from CIFAR-10 . (Top) row has Laplace Kernel, and (Bottom) row has Laplace RFM. (First) column is FCNN, and (Second) column is LeNet. PGD is used to attack the models.

Figure 65A: Gap in robust accuracy (ra) for

\ell_{\infty}

perturbation on 5

1\text{v}1

binary classification tasks from CIFAR-10 . (Top) row has Gaussian Kernel, and (Bottom) row has Gaussian RFM. (First) column is FCNN, and (Second) column is LeNet. PGD is used to attack the models.

Figure 66A: Gap in robust accuracy (ra) for

\ell_{2}

perturbation on 5

1\text{v}1

binary classification tasks from CIFAR-10 . (Top) row has Gaussian Kernel, and (Bottom) row has Gaussian RFM. (First) column is FCNN, and (Second) column is LeNet. PGD is used to attack the models.

	FMNIST		KMNIST		QMNIST
$\epsilon$	WRN	LeNet	WRN	LeNet	WRN	LeNet
0	93.57	90.54	98.61	95.49	99.46	98.91
0.1/255	92.64	90.03	98.58	95.42	99.46	98.90
1/255	77.60	84.04	98.45	94.92	99.37	98.75
2/255	55.64	74.82	98.19	94.20	99.27	98.54
4/255	13.54	55.67	97.60	92.40	99.03	98.00

Appendix

Table 5A: PGD robust accuracy (%) of WRN-28-10 and LeNet-5 on FMNIST, KMNIST, and QMNIST. Bold marks the better model per dataset.

Figure 67A: Adv-KLR versus KLR for FashionMNIST and KMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (bottom) row corresponds to the KMNIST dataset. PGD is used to attack the models.

Figure 68A: Adv-KLR versus FCNN for FashionMNIST and KMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (bottom) row corresponds to the KMNIST dataset. PGD is used to attack the models.

Figure 69A: Adv-KLR versus Adv-FCNN trained at norm

\ell_{\infty}=\frac{12}{255}

: for FashionMNIST and KMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Bottom) row corresponds to the KMNIST dataset. PGD is used to attack the models.

Figure 70A: Laplacian KRR versus

\ell_{2}=1.0

norm and

\ell_{\infty}=\frac{6}{255}

adversarially trained FCNNs:

\ell_{2}

and

\ell_{\infty}

norm adversarially trained FCNNs were attacked at their respective norm adversarial perturbations for FashionMNIST, KMNIST and QMNIST. (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries. PGD is used to attack the models.

	FMNIST			KMNIST			QMNIST
$\varepsilon$	ra(SPSA)	ra(PGD)	diff	ra(SPSA)	ra(PGD)	diff	ra(SPSA)	ra(PGD)	diff
$0$	$90.26$	$90.26$	$0.00$	$92.62$	$92.62$	$0.00$	$98.23$	$98.23$	$0.00$
$\frac{1}{255}$	$89.28$	$88.57$	$0.71$	$92.23$	$91.95$	$0.28$	$98.12$	$98.05$	$0.07$
$\frac{2}{255}$	$88.16$	$86.14$	$2.02$	$91.92$	$91.33$	$0.59$	$98.02$	$97.91$	$0.11$
$\frac{4}{255}$	$85.25$	$81.19$	$4.06$	$91.3$	$89.95$	$1.35$	$97.86$	$97.51$	$0.35$
$\frac{6}{255}$	$82.34$	$74.02$	$8.32$	$90.47$	$88.29$	$2.18$	$97.62$	$97.06$	$0.56$

Appendix

Table 6A: Comparison of robust accuracies of Laplacian KRR in the multi-class setting for SPSA, PGD, and the difference of the two across datasets for

\ell_{\infty}

perturbations. The difference is computed as SPSA minus PGD.

	FMNIST			QMNIST
$\varepsilon$	ra(SPSA)	ra(PGD)	diff	ra(SPSA)	ra(PGD)	diff
$0$	$86.95$	$86.95$	$0.00$	$97.47$	$97.47$	$0.00$
$\tfrac{1}{255}$	$86.10$	$85.56$	$0.54$	$97.37$	$97.33$	$0.05$
$\tfrac{8}{255}$	$78.51$	$72.30$	$6.21$	$96.32$	$95.66$	$0.66$
$\tfrac{55}{255}$	$43.25$	$35.07$	$8.18$	$82.31$	$65.17$	$17.13$

Appendix

Table 7A: Comparison of robust accuracies of Gaussian KLR in the multi-class setting for SPSA, PGD, and the difference between the two across datasets for

\ell_{\infty}

perturbations. The difference is computed as SPSA minus PGD.

	FMNIST			KMNIST
$\varepsilon$	ra(SPSA)	ra(PGD)	diff	ra(SPSA)	ra(PGD)	diff
$0$	$88.18$	$88.18$	$0.00$	$92.26$	$92.26$	$0.00$
$\frac{1}{255}$	$87.10$	$86.64$	$0.46$	$91.82$	$91.56$	$0.26$
$\frac{2}{255}$	$85.86$	$84.99$	$0.87$	$91.39$	$90.92$	$0.47$
$\frac{4}{255}$	$83.56$	$81.44$	$2.12$	$90.29$	$89.31$	$0.98$
$\frac{8}{255}$	$77.93$	$73.03$	$4.90$	$88.00$	$85.08$	$2.92$

Appendix

Table 8A: Comparison of robust accuracies of Laplacian KLR in the multi-class setting under SPSA and PGD attacks across FMNIST and KMNIST for

\ell_{\infty}

perturbations. The difference is computed as SPSA minus PGD. All values are reported in percentages.

	KMNIST
$\varepsilon$	ra(SPSA)	ra(PGD)	diff
$0$	$88.40$	$88.40$	$0.00$
$\tfrac{1}{255}$	$88.35$	$88.24$	$0.11$
$\tfrac{8}{255}$	$87.26$	$86.27$	$0.99$
$\tfrac{20}{255}$	$85.24$	$82.40$	$2.84$
$\tfrac{32}{255}$	$82.78$	$77.63$	$5.15$

Appendix

Table 9A: Comparison of robust accuracies of adversarially trained Gaussian KLR in the multi-class setting under SPSA and PGD attacks for

\ell_{\infty}

perturbations on KMNIST. The difference is computed as

\mathrm{ra(SPSA)}-\mathrm{ra(PGD)}

.

	FMNIST			KMNIST
$\varepsilon$	ra(SPSA)	ra(PGD)	diff	ra(SPSA)	ra(PGD)	diff
$0$	$90.01$	$90.01$	$0.00$	$90.54$	$90.54$	$0.00$
$\tfrac{0.1}{255}$	$89.90$	$89.87$	$0.03$	$90.53$	$90.47$	$0.06$
$\tfrac{1}{255}$	$89.17$	$88.39$	$0.78$	$90.27$	$89.87$	$0.40$
$\tfrac{2}{255}$	$88.38$	$86.63$	$1.75$	$90.03$	$89.02$	$1.01$
$\tfrac{4}{255}$	$86.76$	$82.18$	$4.58$	$89.43$	$87.34$	$2.09$

Appendix

Table 10A: Comparison of robust accuracies of Laplacian SVM in the multi-class setting for SPSA, PGD, and the difference between the two across datasets for

\ell_{\infty}

perturbations.

	FMNIST			KMNIST
$\varepsilon$	ra(SPSA)	ra(PGD)	diff	ra(SPSA)	ra(PGD)	diff
$0$	$89.62$	$89.62$	$0.00$	$93.14$	$93.14$	$0.00$
$\tfrac{0.1}{255}$	$89.49$	$89.29$	$0.20$	$93.09$	$93.00$	$0.09$
$\tfrac{1}{255}$	$88.42$	$86.28$	$2.14$	$92.76$	$92.11$	$0.65$
$\tfrac{2}{255}$	$86.91$	$82.41$	$4.50$	$92.40$	$91.02$	$1.38$
$\tfrac{4}{255}$	$84.02$	$73.17$	$10.85$	$91.72$	$88.74$	$2.98$

Appendix

Table 11A: Comparison of robust accuracies of Gaussian SVM in the multi-class setting under SPSA and PGD attack, along with their differences across FMNIST and KMNIST datasets for

\ell_{\infty}

perturbations.

$\varepsilon$ ( $L_{\infty}$ )	PGD (%)	SPSA (%)
$0$	89.66	89.66
$0.1/255$	89.28	89.34
$1/255$	84.63	85.69
$2/255$	78.54	81.16
$4/255$	63.72	69.58
$6/255$	51.09	58.07

Appendix

Table 12A: Comparison of PGD and SPSA robust accuracy (%) for Gaussian Random Feature Classifiers (RFC) trained with cross-entropy loss on Fashion-MNIST. The model uses 50,000 random features and random seed 150.

$\varepsilon$ ( $L_{\infty}$ )	PGD (%)	SPSA (%)
$0$	89.43	89.43
$0.1/255$	88.46	87.57
$1/255$	87.22	77.11
$2/255$	86.66	69.48
$4/255$	85.84	56.25
$6/255$	85.20	42.82

Appendix

Table 13A: Comparison of PGD and SPSA robust accuracy (%) for Laplacian Random Feature Classifiers (RFC) trained with cross-entropy loss on KMNIST. The model uses 50,000 random features and random seed 150.

Figure 71A: SPSA attacked Laplacian KRR versus PGD attacked Laplacian KRR . (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries.

Figure 72A: SPSA attacked Laplacian RFM versus PGD attacked Laplacian RFM . (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries.

Figure 73A: SPSA attacked Gaussian KRR versus PGD attacked Gaussian KRR . (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries.

Figure 74A: SPSA attacked Gaussian RFM versus PGD attacked Gaussian RFM . (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries.

Figure 75A: SPSA attacked FCNN versus PGD attacked FCNN . (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries.

Figure 76A: SPSA attacked LeNet versus PGD attacked LeNet . (Top) row corresponds to the results for FashionMNIST dataset, (Middle) row corresponds to the KMNIST dataset, and (Bottom) row corresponds to the QMNIST dataset. Of these, (First) column corresponds to the robust accuracies for

\ell_{\infty}

adversaries while (Second) column portrays the robust accuracies for

\ell_{2}

adversaries.

$\varepsilon$	stage	FCNN	KRR	RFM	FCNN	KRR	RFM	FCNN	KRR	RFM
		Phishing			Bank Marketing			Adult
0.01	Benign	94.8	92.6	95.2	87.8	89.7	89.7	84.3	72.4	80.8
	Post CaFA	53.0	52.4	70.5	90.3	89.5	78.6	73.0	61.2	43.1
	Post projection	42.0	31.1	63.5	84.3	85.7	75.0	64.2	51.2	38.0
0.03	Benign	94.8	92.6	95.2	87.8	89.7	89.7	84.3	72.4	80.8
	Post CaFA	46.7	44.3	60.6	89.5	90.5	79.9	69.6	55.6	39.2

Appendix

Table 14A: CaFA attacks on FCNN, KRR and RFM on the Phishing, Bank Marketing and Adult datasets. Robust accuracy numbers are reported only on compliant samples after each stage. The accuracy values are rounded to the nearest tenth.

Figure 77A: Robust Accuracy variation with Matérn Kernel smoothness parameter

\nu

for SVHN, KMNIST, QMNIST, FMNIST.

Figure 78A: Robust Accuracy variation with iterations in Laplacian RFM for SVHN, KMNIST, QMNIST, FMNIST.

Figure 79A: Robust Accuracy variation with length scale (

\sigma

) in Laplacian Kernel for SVHN, KMNIST, QMNIST, FMNIST across different attack

\varepsilon

.

Figure 80A: Robust Accuracy variation with length scale (

\sigma

) in Gaussian Kernel for SVHN, KMNIST, QMNIST, FMNIST across different attack

\varepsilon

.

Figure 81A: Robust Accuracy variation with ridge (

\lambda

) in Laplacian KRR for SVHN, KMNIST, QMNIST, FMNIST across different attack

\varepsilon

.

Figure 82A: Robust Accuracy variation with Number of Features in Gaussian RFC (trained using Square Loss) for FMNIST across different seeds per attack. SPSA and PGD denote attacks on the RFC, each line shows the robust accuracy of a model at a given perturbation radius (

\varepsilon

).

Figure 83A: Robust Accuracy variation with Number of Features in Gaussian RFC (trained using Cross Entropy) for FMNIST across different seeds per attack. SPSA and PGD denote attacks on the RFC, each line shows the robust accuracy of a model at a given perturbation radius (

\varepsilon

).

Figure 84A: Robust Accuracy variation with Number of Features in Gaussian RFC (trained using Square Loss) for KMNIST across different seeds per attack. SPSA and PGD denote attacks on the RFC, each line shows the robust accuracy of a model at a given perturbation radius (

\varepsilon

).

Figure 85A: Robust Accuracy variation with Number of Features in Gaussian RFC (trained using Cross Entropy) for KMNIST across different seeds per attack. SPSA and PGD denote attacks on the RFC, each line shows the robust accuracy of a model at a given perturbation radius (

\varepsilon

).

Figure 86A: Robust Accuracy variation with Number of Features in Gaussian RFC (trained using Square Loss) for QMNIST across different seeds per attack. SPSA and PGD denote attacks on the RFC, each line shows the robust accuracy of a model at a given perturbation radius (

\varepsilon

).

Figure 87A: Robust Accuracy variation with Number of Features in Gaussian RFC (trained using Cross Entropy Loss) for QMNIST across different seeds per attack. SPSA and PGD denote attacks on the RFC, each line shows the robust accuracy of a model at a given perturbation radius (

\varepsilon

).

Figure 88A: Robust Accuracy variation with Number of Features in Laplace RFC (trained using Square Loss) for FMNIST across different seeds per attack. SPSA and PGD denote attacks on the RFC, each line shows the robust accuracy of a model at a given perturbation radius (

\varepsilon

).

Figure 89A: Robust Accuracy variation with Number of Features in Laplacian RFC (trained using Cross Entropy) for FMNIST across different seeds per attack. SPSA and PGD denote attacks on the RFC, each line shows the robust accuracy of a model at a given perturbation radius (

\varepsilon

).

Figure 90A: Robust Accuracy variation with Number of Features in Laplace RFC (trained using Square Loss) for KMNIST across different seeds per attack. SPSA and PGD denote attacks on the RFC, each line shows the robust accuracy of a model at a given perturbation radius (

\varepsilon

).

Figure 91A: Robust Accuracy variation with Number of Features in Laplacian RFC (trained using Cross Entropy) for KMNIST across different seeds per attack. SPSA and PGD denote attacks on the RFC, each line shows the robust accuracy of a model at a given perturbation radius (

\varepsilon

).

Figure 92A: Robust Accuracy variation with Number of Features in Laplace RFC (trained using Square Loss) for QMNIST across different seeds per attack. SPSA and PGD denote attacks on the RFC, each line shows the robust accuracy of a model at a given perturbation radius (

\varepsilon

).

Figure 93A: Robust Accuracy variation with Number of Features in Laplace RFC (trained using Cross Entropy Loss) for QMNIST across different seeds per attack. SPSA and PGD denote attacks on the RFC, each line shows the robust accuracy of a model at a given perturbation radius (

\varepsilon

).

advKLR	adversarially trained KLR
CAFA	Cost-aware Feasible Attack	[ 5 ]
CNN	convolutional neural network
FCN	fully connected neural network (a.k.a. MLP)
KM	kernel machine
KRR	kernel ridge regression
KSVM	kernel support vector machines

Adversarial Robustness of Kernel Machines: A Comprehensive Analysis

Abstract

Figures & tables

Appendix

Adversarial Robustness

KAN-Robust-Bench: A Benchmark for Evaluating the Robustness of Kolmogorov-Arnold Networks

Evasion Attacks: How Adversarial Noise Bypasses ML Classifiers

Useful nonrobust features are ubiquitous in biomedical images

Abstract

Adversarial Robustness

Explore similar work

KAN-Robust-Bench: A Benchmark for Evaluating the Robustness of Kolmogorov-Arnold Networks

Evasion Attacks: How Adversarial Noise Bypasses ML Classifiers

Useful nonrobust features are ubiquitous in biomedical images