Catastrophic Overfitting

Catastrophic overfitting (CO) is a significant challenge in fast adversarial training (FAT) of neural networks, where models rapidly lose robustness against stronger attacks despite initially showing improved performance. Current research focuses on understanding the underlying causes of CO, particularly within the context of single-step adversarial training methods applied to convolutional neural networks (CNNs) and vision transformers (ViTs), and developing techniques to mitigate it through loss function modifications, regularization strategies, and adaptive training procedures. Addressing CO is crucial for advancing the development of efficient and robust machine learning models, with implications for improving the security and reliability of AI systems in various applications.

Papers

July 22, 2024

Improving Fast Adversarial Training Paradigm: An Example Taxonomy Perspective
Jie Gui, Chengze Jiang, Minjing Dong, Kun Tong, Xinli Shi, Yuan Yan Tang, Dacheng Tao
Adversarial Example Adversarial Training Comprehensive Taxonomy Fast Adversarial Training Catastrophic Overfitting Projected Gradient Descent Attack

July 17, 2024

Preventing Catastrophic Overfitting in Fast Adversarial Training: A Bi-level Optimization Perspective
Zhaoxin Wang, Handing Wang, Cong Tian, Yaochu Jin
Adversarial Attack Adversarial Example Adversarial Training Bi Level Optimization Fast Algorithm Fast Adversarial Training Catastrophic Overfitting Step Attack

May 25, 2024

Layer-Aware Analysis of Catastrophic Overfitting: Revealing the Pseudo-Robust Shortcut Dependency
Runqi Lin, Chaojian Yu, Bo Han, Hang Su, Tongliang Liu
Layer Wise Multi Step Adversarial Attack Shortcut Pattern Avoidance Loss Catastrophic Overfitting Single Step Adversarial Training Adversarial Weight Perturbation

February 28, 2024

Catastrophic Overfitting: A Potential Blessing in Disguise
Mengnan Zhao, Lihe Zhang, Yuqiu Kong, Baocai Yin
Adversarial Example Adversarial Robustness Adversarial Data Digital STEALTH Metric Fast Adversarial Training Catastrophic Overfitting Blessing Generation

January 21, 2024

Efficient local linearity regularization to overcome catastrophic overfitting
Elias Abad Rocamora, Fanghui Liu, Grigorios G. Chrysos, Pablo M. Olmos, Volkan Cevher
Adversarial Perturbation Catastrophic Overfitting Similarity Regularization Single Step Adversarial Training Test Time Adversarial

October 29, 2023

Blacksmith: Fast Adversarial Training of Vision Transformers via a Mixture of Single-step and Multi-step Methods
Mahdi Salmani, Alireza Dehghanpour Farashah, Mohammad Azizmalayeri, Mahdi Amiri, Navid Eslami, Mohammad Taghi Manzuri, Mohammad Hossein Rohban
Vision Transformer Adversarial Training Adversarial Perturbation Multi Step Novel Training Fast Adversarial Training Catastrophic Overfitting Fast Gradient Sign Method

October 19, 2023

Learn from the Past: A Proxy Guided Adversarial Defense Framework with Self Distillation Regularization
Yaohua Liu, Jiaxin Gao, Xianghao Jiao, Zhu Liu, Xin Fan, Risheng Liu
Adversarial Training Model Robustness Catastrophic Overfitting Proxy Model Defense Framework

October 13, 2023

On the Over-Memorization During Natural, Robust and Catastrophic Overfitting
Runqi Lin, Chaojian Yu, Bo Han, Tongliang Liu
Robust Version Model Overfitting Adversarial Pattern Catastrophic Overfitting Unintended Memorization

August 24, 2023

Fast Adversarial Training with Smooth Convergence
Mengnan Zhao, Lihe Zhang, Yuqiu Kong, Baocai Yin
Adversarial Robustness Fast Adversarial Training Fast Convergence Catastrophic Overfitting

June 22, 2023

An Agnostic View on the Cost of Overfitting in (Kernel) Ridge Regression
Lijia Zhou, James B. Simon, Gal Vardi, Nathan Srebro
Model Overfitting Hidden CoST Ridge Regression Kernel Extension Agnostic Approach Catastrophic Overfitting Ridgeless Regression

February 23, 2023

Investigating Catastrophic Overfitting in Fast Adversarial Training: A Self-fitting Perspective
Zhengbao He, Tao Li, Sizhe Chen, Xiaolin Huang
Adversarial Example Adversarial Training Fast Adversarial Training Catastrophic Overfitting Self Fitting Method

October 31, 2022

FL Games: A Federated Learning Framework for Distribution Shifts
Sharut Gupta, Kartik Ahuja, Mohammad Havaei, Niladri Chatterjee, Yoshua Bengio
Federated Learning Distribution Shift Nash Equilibrium Game Theoretic Graph AutoEncoder Catastrophic Overfitting

October 11, 2022

Stable and Efficient Adversarial Training through Local Linearization
Zhuorong Li, Daiwei Yu
Adversarial Training Iterative Training Catastrophic Overfitting Efficient Adversarial Training Local Linearity Single Step Adversarial Training

September 6, 2022

Bag of Tricks for FGSM Adversarial Training
Zichao Li, Li Liu, Zeyu Wang, Yuyin Zhou, Cihang Xie
Adversarial Training Bag Prototype Unconventional Rabbit Hat Trick Fast Adversarial Training Catastrophic Overfitting Fast Gradient Sign Method

July 3, 2022

Stabilizing Off-Policy Deep Reinforcement Learning from Pixels
Edoardo Cetin, Philip J. Ball, Steve Roberts, Oya Celiktutan
Policy Reinforcement Learning Tetromino Pixel Convolutional Encoder DeepMind Control Suite Catastrophic Overfitting Policy Deep Reinforcement Learning Training Instability

June 16, 2022

Catastrophic overfitting can be induced with discriminative non-robust features
Guillermo Ortiz-Jiménez, Pau de Jorge, Amartya Sanyal, Adel Bibi, Puneet K. Dokania, Pascal Frossard, Gregory Rogéz, Philip H. S. Torr
Native Robustness Adversarial Training Robust Neural Network Catastrophic Overfitting Robustness Problem Non Robust Feature

June 6, 2022

Fast Adversarial Training with Adaptive Step Size
Zhichao Huang, Yanbo Fan, Chen Liu, Weizhong Zhang, Yong Zhang, Mathieu Salzmann, Sabine Süsstrunk, Jue Wang
Adversarial Attack Adversarial Training Step Size Fast Adversarial Training Catastrophic Overfitting Step Attack Multi Step Attack

April 1, 2022

FrequencyLowCut Pooling -- Plug & Play against Catastrophic Overfitting
Julia Grabinski, Steffen Jung, Janis Keuper, Margret Keuper
Adversarial Attack CNN Architecture Pooling Layer Play Style Catastrophic Overfitting Free Down Sampling Operation Regular Random Down Sampling

February 11, 2022

Fast Adversarial Training with Noise Augmentation: A Unified Perspective on RandStart and GradAlign
Axi Niu, Kang Zhang, Chaoning Zhang, Chenshuang Zhang, In So Kweon, Chang D. Yoo, Yanning Zhang
Unified View Gradient Alignment Explicit Regularization Fast Adversarial Training Catastrophic Overfitting Adversarial Training Algorithm Noise Augmentation

February 2, 2022

Make Some Noise: Reliable and Efficient Single-Step Adversarial Training
Pau de Jorge, Adel Bibi, Riccardo Volpi, Amartya Sanyal, Philip H. S. Torr, Grégory Rogez, Puneet K. Dokania
Adversarial Training Industrial Disturbing Noise Catastrophic Overfitting Single Step Adversarial Training