Robust Overfitting

Robust overfitting in adversarial training describes the phenomenon where deep neural networks, while achieving near-perfect accuracy on adversarially perturbed training data, fail to generalize well to unseen, similarly perturbed test data. Current research focuses on mitigating this issue through techniques like data augmentation, label refinement, and novel regularization methods applied to various architectures including Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs). Understanding and overcoming robust overfitting is crucial for developing truly robust and reliable deep learning models, with significant implications for the security and trustworthiness of AI systems in real-world applications.

Papers

March 18, 2024

Robust Overfitting Does Matter: Test-Time Adversarial Purification With FGSM
Linyu Tang, Lei Zhang
Adversarial Attack Adversarial Robustness Artefact PARTICLE Adversarial Purification Robust Overfitting Fast Gradient Sign Method

March 15, 2024

Revisiting Adversarial Training under Long-Tailed Distributions
Xinli Yue, Ningping Mou, Qian Wang, Lingchen Zhao
Adversarial Attack Adversarial Training Long Tailed Distribution Robust Overfitting Balanced Softmax

March 14, 2024

Soften to Defend: Towards Adversarial Robustness via Self-Guided Label Refinement
Daiwei Yu, Zhuorong Li, Lina Wei, Canghong Jin, Yun Zhang, Sixian Chan
Adversarial Attack Adversarial Training Adversarial Robustness Robust Generalization Label Refinement Robust Overfitting

February 18, 2024

The Effectiveness of Random Forgetting for Robust Generalization
Vijaya Raghavan T Ramkumar, Bahram Zonooz, Elahe Arani
Adversarial Attack Adversarial Training Anti Forgetting Robust Generalization Adversarial Method Robust Overfitting

January 24, 2024

Can overfitted deep neural networks in adversarial training generalize? -- An approximation viewpoint
Zhongjie Shi, Fanghui Liu, Yuan Cao, Johan A. K. Suykens
Neural Network Deep Neural Network Adversarial Perturbation Robust Generalization Robust Overfitting

November 28, 2023

On robust overfitting: adversarial training induced distribution matters
Runzhi Tian, Yongyi Mao
Adversarial Training Loss Function Generalization Error Product Distribution Robust Overfitting

October 30, 2023

Balance, Imbalance, and Rebalance: Understanding Robust Overfitting from a Minimax Game Perspective
Yifei Wang, Liangchen Li, Jiansheng Yang, Zhouchen Lin, Yisen Wang
Adversarial Training Balancing Strategy Group Imbalance Robust Overfitting Minimax Game

October 9, 2023

Theoretical Analysis of Robust Overfitting for Wide DNNs: An NTK Approach
Shaopeng Fu, Di Wang
Neural Network Adversarial Training Neural Tangent Kernel Theoretical Analysis Wide Neural Network Robust Overfitting Infinite Width Neural Network

October 1, 2023

On the Onset of Robust Overfitting in Adversarial Training
Chaojian Yu, Xiaolong Shi, Jun Yu, Bo Han, Tongliang Liu
Adversarial Training Adversarial Robustness Adversarial Perturbation Robust Neural Network Robust Overfitting Early Onset

June 12, 2023

AROID: Improving Adversarial Robustness through Online Instance-wise Data Augmentation
Lin Li, Jianing Qiu, Michael Spratling
Data Augmentation Adversarial Example Adversarial Training Adversarial Robustness Robust Generalization Robust Overfitting Instance Augmentation

June 2, 2023

Towards Understanding Clean Generalization and Robust Overfitting in Adversarial Training
Binghui Li, Yuanzhi Li
Adversarial Training Adversarial Perturbation Robust Generalization Robust Overfitting Low Training Error

January 24, 2023

Data Augmentation Alone Can Improve Adversarial Training
Lin Li, Michael Spratling
Data Augmentation Adversarial Training Augmentation Method Robust Overfitting

December 9, 2022

Understanding and Combating Robust Overfitting via Input Loss Landscape Analysis and Regularization
Lin Li, Michael Spratling
Adversarial Attack Adversarial Training Human Understanding Regularization Effect Robust Overfitting Adversarial Direction

October 3, 2022

Stability Analysis and Generalization Bounds of Adversarial Training
Jiancong Xiao, Yanbo Fan, Ruoyu Sun, Jue Wang, Zhi-Quan Luo
Adversarial Example Adversarial Training Adversarial Perturbation Generalization Bound Adversarial Loss Stability Analysis Robust Overfitting

September 30, 2022

Improving Robustness with Adaptive Weight Decay
Amin Ghiasi, Ali Shafahi, Reza Ardekani
Native Robustness Adversarial Robustness Better Robustness Weight Decay Robust Overfitting

July 18, 2022

NeuForm: Adaptive Overfitting for Neural Shape Editing
Connor Z. Lin, Niloy J. Mitra, Gordon Wetzstein, Leonidas Guibas, Paul Guerrero
Neural Representation Neural Scene Representation Generalizable Representation Shape Space Robust Overfitting 3D Neural

June 17, 2022

Understanding Robust Overfitting of Adversarial Training and Beyond
Chaojian Yu, Bo Han, Li Shen, Jun Yu, Chen Gong, Mingming Gong, Tongliang Liu
Adversarial Training Adversarial Robustness Adversarial Data Robust Overfitting Large Loss

May 24, 2022

Alleviating Robust Overfitting of Adversarial Training With Consistency Regularization
Shudong Zhang, Haichang Gao, Tianwei Zhang, Yunyi Zhou, Zihui Wu
Adversarial Attack Adversarial Example Adversarial Training Consistency Regularization Consistency Loss Robust Overfitting

March 14, 2022

Adversarial amplitude swap towards robust image classifiers
Chun Yang Tan, Kazuhiko Kawamoto, Hiroshi Kera
Native Robustness Image Classification Adversarial Perturbation Adversarial Image Robust Overfitting General Robustness Adversarial Covariate Shift

November 24, 2021

Subspace Adversarial Training
Tao Li, Yingwen Wu, Sizhe Chen, Kun Fang, Xiaolin Huang
Adversarial Training Catastrophic Overfitting Robust Overfitting Subspace Training