Boosting Adversarial Robustness and Generalization with Dictionary Structure

Zhichao Hou; Weizhi Gao; Hamid Krim; Runze Li; Xiaorui Liu

cs.LGFeb 2, 2025

Boosting Adversarial Robustness and Generalization with Dictionary Structure

Authors: Zhichao Hou, Weizhi Gao, Hamid Krim, Runze Li, Xiaorui Liu

Organizations: Department of Computer Science, North Carolina State University, Raleigh, USA.

Abstract

This work investigates a novel approach to boost adversarial robustness and generalization by incorporating structural prior into the design of deep learning models. Specifically, our study surprisingly reveals that existing dictionary learning-inspired convolutional neural networks (CNNs) are robust against random noise but remain highly vulnerable to adversarial attacks. To address this, we propose Elastic Dictionary Learning Networks (EDLNets), a novel ResNet architecture that significantly enhances adversarial robustness and generalization. Extensive and reliable experiments demonstrate consistent improvements in adversarial robustness across multiple datasets, backbone architectures, and threat models. To the best of our knowledge, this is the first work to discover and validate that dictionary structure can reliably enhance deep learning robustness under strong adaptive attacks, unveiling a promising direction for future research.

Figures & tables

Figure 1: Overview of Elastic Dictionary Learning Networks (EDLNets). EDLNets are constructed by replacing the convolutional layers in conventional backbones (e.g., ResNets) with EDL layers obtained by unrolling the proposed efficient RISTA algorithm. Each EDL layer incorporates a dictionary-structure prior, assuming that the input signal ${\bm{x}}^{(l)}$ can be represented by a sparse code ${\bm{x}}^{(l+1)}$ using a small number of atoms from the dictionary $\mathbf{A}^{(l)}$ . During backward propagation, the task loss is propagated through all unrolled RISTA iterations to jointly optimize the layer-wise dictionaries $\{\mathbf{A}^{(l)}\}_{l=0}^{L-1}$ and balancing parameters $\{\beta^{(l)}\}_{l=0}^{L-1}$ , as well as the classifier parameters $\theta$ .

Model $\backslash$ Noise Level	L-1	L-2	L-3	L-4	L-5	PGD
Classic ConvNet (ResNet18)	81.44	57.23	48.32	32.49	16.98	0.00
Vanilla DL (Fixed $\lambda=0.1$ )	82.39	68.90	59.28	40.8	23.83	0.01
Vanilla DL (Tuned $\lambda$ )	82.39	68.90	59.28	43.71	33.43	0.13

Table 1: Preliminary study on CIFAR10 comparing standard ResNet18 with a Vanilla DL variant under varying levels of random noise and adaptive PGD attack (

\epsilon=\frac{8}{255}

). Vanilla DL exhibits strong robustness against weak random noise but is easily broken by the strong PGD attack.

	Natural Acc.			Robust Acc.
Method	Final	Best	Diff	Final	Best	Diff
Vanilla	78.98	79.90	0.92	44.90	48.01	3.11
$\ell_{1}$ Reg.	64.84	65.71	0.87	40.94	41.97	1.03
$\ell_{2}$ Reg.	78.88	79.39	0.51	42.73	48.26	5.53
$\ell_{2}$ + $\ell_{1}$ Reg.	66.86	67.62	0.76	42.53	43.33	0.80
Cutout	75.11	75.58	0.47	47.12	48.23	1.11

Table 2: Natural and robust performance of PGD-based adversarial training with different methods to mitigate the overfitting on CIFAR10. BEST represents the highest test accuracy achieved during training, while FINAL is the average accuracy over the last five epochs. DIFF, the difference between BEST and FINAL, measures the ability to mitigate overfitting. The best performance is highlighted in bold , while the second-best is underlined .

Panel of Figure 3 — Figure 3: Adversarial robustness under various settings. Our Elastic DL outperforms Vanilla DL across various datasets (CIFAR10 / CIFAR100 / Tiny-ImageNet), backbones (ResNet10 / ResNet18 / ResNet34 / ResNet50) and attacks (PGD / FGSM / CW / AA).

Figure 6: Hidden embedding visualization under clean and attacked scenarios. The difference between clean and attacked embeddings in Elastic DL is smaller compared to Vanilla DL, with this effect becoming more significant in deeper layers. Consequently, while an adversarial attack alters the Vanilla DL output from ” SHIP ” to ” FROG ”, Elastic DL successfully preserves the correct prediction.

Method	Train (h)	Params (M)	FLOPs (G)	Size (MB)
Vanilla DL	13.8	11.174	4.776	42.71
Elastic DL	15.2	11.174	4.776	42.71

Table 6: Computational cost comparison.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 10: Comparison of training-from-scratch and pretraining–finetuning strategies for Elastic DL. Pretraining–finetuning achieves more stable optimization and better final performance.

Figure 11: Performance of the Vanilla DL variant of ResNet18 under random impulse noise at different levels.

Figure 12: Adversarial training curve of our Elastic DL. During epochs 100–150, the model experiences a catastrophic robust overfitting problem. Introducing the Elastic DL structural prior at epoch 150 and fine-tuning effectively mitigates overfitting and yields significantly improved robustness and generalization.

Panel of Figure 13 — Figure 13: Training curves of baselines.

Panel of Figure 14 — Figure 14: Comparison of training curves across all methods.

Method	Natural	PGD	FGSM	C&W	AA
Vanilla DL + ResNet10	81.55	45.48	52.53	45.85	41.60
Elastic DL + ResNet10	82.69	49.54	64.52	57.37	46.30
Vanilla DL + ResNet18	83.28	45.64	53.88	41.22	43.70
Elastic DL + ResNet18	83.57	53.22	69.35	60.80	52.90
Vanilla DL + ResNet34	82.45	45.37	54.32	42.12	44.40
Elastic DL + ResNet34	82.95	55.88	70.19	61.74	53.80

Appendix

Table 7: Adversarial robustness on CIFAR10 with different backbones.

Method	Natural	PGD	FGSM	C&W	AA
Vanilla DL + ResNet10	55.94	22.45	26.57	18.90	21.00
Elastic DL + ResNet10	55.20	26.30	35.34	26.45	22.60
Vanilla DL + ResNet18	57.24	22.17	26.81	17.43	21.60
Elastic DL + ResNet18	57.70	27.27	37.62	28.87	26.30
Vanilla DL + ResNet34	56.18	21.77	26.14	16.38	20.80
Elastic DL + ResNet34	56.38	32.67	43.39	39.34	29.20

Appendix

Table 8: Adversarial robustness on CIFAR100 with different backbones.

Method	Natural	PGD	FGSM	C&W	AA
Vanilla DL + ResNet10	49.60	27.17	32.46	37.91	20.20
Elastic DL + ResNet10	50.12	32.93	39.64	40.10	24.90
Vanilla DL + ResNet18	50.22	31.45	36.46	39.02	30.90
Elastic DL + ResNet18	50.52	37.60	43.10	46.64	36.30
Vanilla DL + ResNet34	50.03	33.54	37.24	37.19	29.30
Elastic DL + ResNet34	50.40	34.80	41.75	44.72	34.60

Appendix

Table 9: Adversarial robustness on Tiny-ImageNet with different backbones.

Figure 15: Different adversarial training methods. Our Elastic DL is orthogonal to existing adversarial training methods and can be combined with them to further improve performance.

Figure 16: Different attack measurements. Our Elastic DL consistently outperforms Vanilla DL across attacks (PGD- $\ell_{\infty}$ , PGD- $\ell_{2}$ , SparseFool) evaluated under various metrics ( $\ell_{\infty},\ell_{2},\ell_{0}$ norms).

Num. of Layers	3	5	8	10	15	20	25	30	40	50
Relative Error	0.00066	0.00095	0.00152	0.00092	0.00118	0.00098	0.00112	0.00082	0.00060	0.00093

Appendix

Table 10: Zero-order gradient analysis: relative difference between autograd and zero-order gradients across network depths.

Budget	0	8/255	16/255	32/255	64/255
Elastic DL	83.57	53.29	34.13	23.84	12.45
Elastic DL + BPDA	83.57	53.20	34.08	23.51	12.44

Appendix

Table 11: Adversarial robustness under standard adaptive attack versus BPDA attack, showing comparable performance and ruling out gradient obfuscation.

Figure 17: Embedding difference across layers.

Figure 18: Hidden embedding visualization (Part 1).

Figure 19: Hidden embedding visualization (Part 2).

Error	$\\|\cdot\\|_{1}$	$\\|\cdot\\|_{2}$	$\\|\cdot\\|_{\infty}$
${\bm{\epsilon}}_{\text{random}}-\hat{{\bm{\epsilon}}}$	1294.75 $\pm$ 406.78	26.09 $\pm$ 7.04	0.901 $\pm$ 0.10
${\bm{\epsilon}}_{\text{resnet}}-\hat{{\bm{\epsilon}}}$	131.51 $\pm$ 10.53	2.93 $\pm$ 0.22	0.163 $\pm$ 0.01
${\bm{\epsilon}}_{\text{vanilla}}-\hat{{\bm{\epsilon}}}$	129.07 $\pm$ 13.22	2.85 $\pm$ 0.26	0.157 $\pm$ 0.01
${\bm{\epsilon}}_{\text{elastic}}-\hat{{\bm{\epsilon}}}$	122.62 $\pm$ 9.92	2.69 $\pm$ 0.22	0.149 $\pm$ 0.01

Appendix

Table 12: Reconstruction error between the recovered noise

\hat{{\bm{\epsilon}}}

and various input noises, including random noise (

{\bm{\epsilon}}_{\text{random}}

), transfer noise from ResNet (

{\bm{\epsilon}}_{\text{resnet}}

) and Vanilla DL (

{\bm{\epsilon}}_{\text{vanilla}}

), and adaptive noise from our Elastic DL (

{\bm{\epsilon}}_{\text{elastic}}

). Our Elastic DL achieves the smallest reconstruction error, indicating that our approach can adaptively recover and neutralize the input perturbation, thereby mitigating its impact.

Figure 21: Reconstruction process (ImageNet).

Figure 22: Reconstruction process (CIFAR10, Part 1).

Figure 23: Reconstruction process (CIFAR10, Part 2).

Explore similar work

Adversarial Rademacher Complexity of Deep Neural Networks
Nov 27, 2022Jiancong Xiao, Yanbo Fan, Ruoyu Sun +1Adversarial Training Generalization Bounds
The Chinese University of Hong Kong, Shenzhen Shenzhen, China · Nanjing University Suzhou, China
A combination of noise and bilateral filters achieve supralinear and scalable adversarial robustness in CNNs
Jun 1, 2026Nicolas Stalder, Benjamin F. Grewe, Matteo Saponati +1Convolutional Neural Networks Noise-Aware
Institute of Neuroinformatics
Improving Adversarial Robustness via Activation Amplification and Attenuation
Jun 26, 2026Taïga Gonçalves, Yongsong Huang, Tomo Miyazaki +1Convolutional Block Attention Module
Graduate School of Engineering, Tohoku University, Japan

Nov 27, 2022, cs.LGJ/K move · Enter open · S save

Adversarial Rademacher Complexity of Deep Neural Networks

Jiancong Xiao, Yanbo Fan, Ruoyu Sun, Zhi-Quan Luo

The Chinese University of Hong Kong, Shenzhen Shenzhen, China · Nanjing University Suzhou, China

Adversarial Training Generalization Bounds Adversarial Examples Neural Network Maximum Independent Set

Open paper AskPDF arXiv

Deep neural networks (DNNs) are highly vulnerable to adversarial attacks. Ideally, a robust model should perform well on both perturbed training data and unseen perturbed test data. While DNNs can fit perturbed training data, generalizing to perturbed test data remains a significant challenge. This motivates the study of generalization guarantees from a learning theory perspective. This paper focuses on adversarial Rademacher complexity (ARC), first introduced by Khim and Loh (2018) and Yin et al. (2019). Their work primarily addressed linear functions and highlighted the open question of how to bound ARC for neural networks. Since then, several attempts have been made, with the latest results applying ARC only to two-layer neural networks. The main challenge arises from the dynamic nature and unknown closed-form solution of adversarial examples. In this paper, we resolve this issue and provide the first bound on ARC for deep neural networks. Our bound is qualitatively comparable to Rademacher complexity bounds in similar settings. The key ingredient is a new concept we introduce, termed intermediate adversarial examples, along with a framework for calculating the covering number that is compatible with them. Finally, we present experiments to analyze poor robust generalization, demonstrating that the weight norm is a crucial factor influencing the robust generalization gap.

Abstract

Explore similar work

Adversarial Rademacher Complexity of Deep Neural Networks

A combination of noise and bilateral filters achieve supralinear and scalable adversarial robustness in CNNs

Improving Adversarial Robustness via Activation Amplification and Attenuation