Catastrophic Overfitting, Entropy Gap and Participation Ratio: A Noiseless $l^p$ Norm Solution for Fast Adversarial Training

Fares B. Mehouachi; Saif Eddin Jabari

cs.LGMay 5, 2025

Catastrophic Overfitting, Entropy Gap and Participation Ratio: A Noiseless $l^p$ Norm Solution for Fast Adversarial Training

Authors: Fares B. Mehouachi, Saif Eddin Jabari

Organizations: New York University of Abu Dhabi, Saadiyat Island, Abu Dhabi, UAE · Department of Civil and Urban Engineering, NYU Tandon School of Engineering, New York, USA

Abstract

Adversarial training is a cornerstone of robust deep learning, but fast methods like the Fast Gradient Sign Method (FGSM) often suffer from Catastrophic Overfitting (CO), where models become robust to single-step attacks but fail against multi-step variants. While existing solutions rely on noise injection, regularization, or gradient clipping, we propose a novel solution that purely controls the $l^p$ training norm to mitigate CO. Our study is motivated by the empirical observation that CO is more prevalent under the $l^{\infty}$ norm than the $l^2$ norm. Leveraging this insight, we develop a framework for generalized $l^p$ attack as a fixed point problem and craft $l^p$ -FGSM attacks to understand the transition mechanics from $l^2$ to $l^{\infty}$ . This leads to our core insight: CO emerges when highly concentrated gradients, where information localizes in few dimensions, interact with aggressive norm constraints. By quantifying gradient concentration through Participation Ratio and entropy measures, we develop an adaptive $l^p$ -FGSM that automatically tunes the training norm based on gradient information. Extensive experiments demonstrate that this approach achieves competitive robustness without requiring additional regularization or noise injection, providing a novel and theoretically-principled pathway to mitigate the CO problem.

Figures & tables

Figure 1: CO phenomena on CIFAR-10 Krizhevsky (2009) using WideResNet-28-10 Zagoruyko and Komodakis (2016) : Upper:

l^{\infty}

training (

\epsilon=8/255

) shows accuracy collapse against PGD-50 (

\epsilon=8/255

) Madry et al. (2017) attacks, while

l^{2}

(

\epsilon=32/255

, both training and attack) remains stable. Lower: CO onset in

l^{\infty}

training correlates with gradient norm increase, absent in

l^{2}

training (norms normalized at epoch 1).

Figure 2: Impact of

l^{p}

norm choice on training dynamics and robustness for CIFAR-10 Krizhevsky (2009) with WideResNet-28-10 Zagoruyko and Komodakis (2016) . The choice of

p

reveals a key trade-off: higher values (

p\geq 32

) initially show better robustness but become vulnerable to Catastrophic Overfitting (CO) Kim et al. (2021) ; He et al. (2023) , evident in the

l^{\infty}

PGD-50 Madry et al. (2017) plot (second left). Lower

p

values prevent CO but with reduced adversarial robustness. Notably,

l^{2}

PGD-50 accuracy (rightmost) remains stable across different

p

values, suggesting

l^{2}

robustness is less sensitive to norm choice. Results shown for

\epsilon=8/255

over 30 epochs.

l^{p}

-FGSM used for these plots is detailed in Algorithm 1 .

Figure 3: Depiction of training effect on CIFAR-10’s loss landscape at different training points. The upper panels display the landscape after one epoch and the lower ones ten epochs with $l^{p}$ -FGSM (Alg. 1 ). Training points are positioned at $(0,0)$ , $\varepsilon_{1}$ and $\varepsilon_{2}$ are eigenvectors corresponding to the Hessian’s ( $\nabla^{2}_{x}\ell$ ) extreme eigenvalues for each sample. Training induces local convexity.

Figure 4: Illustration of the initial two ascents of the fixed-point algorithm ( 8 ) for optimal perturbation identification under the $l^{2}$ constraint.

Figure 5: Variation of the

l^{p}

transition function

\Upsilon_{p}

for different values of

p

. The high-pass filtering effect mirrors the thresholding behavior in ZeroGrad Golgooni et al. (2021) .

Figure 6: Evolution of Participation Ratios (

\mathtt{PR}

\mathtt{PR_{1}}

) and entropy gap during training. Sharp declines in these metrics align with the onset of Catastrophic Overfitting (CO), highlighting the link between gradient concentration and adversarial vulnerability. Shaded areas correspond to

\pm

standard deviations within the test dataset. Experiment conducted with CIFAR-10, WRN-28-10, ADAM with 0.001 learning rate and a batch size of 128.

Figure 7: Effect of the

l^{p}

norm on attack geometry and sensitivity to gradient noise. Left: An ideal scenario, where the angles between

\delta_{2}

\delta_{\infty}

, and any

\delta_{p}

are zero. Right: Under small gradient noise (common in ML),

l^{\infty}

shows high sensitivity with large angular separation, whereas

l^{p}

yields more stable attacks with better gradient alignment (higher cosine similarity).

Figure 8: Performance landscape of fixed

l^{p}

-FGSM across datasets (

\epsilon=8/255

). The results reveal a fundamental dilemma: lower

p

values (e.g.,

p=2,4

) ensure training stability but sacrifice adversarial robustness, achieving only 20-40% accuracy against PGD-50. Higher values (

p\geq 32

) enhance robustness initially but inevitably trigger Catastrophic Overfitting (CO). Dataset complexity critically impacts this threshold: CIFAR-10 tolerates

p\approx 16

32

before collapse, SVHN handles higher values, while CIFAR-100 exhibits heightened sensitivity with failure at

p\geq 16

. This dataset-dependent variability makes fixed norm selection impractical and motivates our adaptive approach. Training was done for 30 epochs with an ADAM Kingma and Ba (2014) optimizer (lr=0.001, weight decay=0.0005) and a batch size of 128.

Figure 9: Training dynamics of adaptive

l^{p}

-FGSM on CIFAR-10 with WideResNet-28-10 (

\beta=0.02

\epsilon=8/255

). ADAM Kingma and Ba (2014) optimizer was used with constant learning rate of 0.001, a weight decay of 0.0005, and a batch size of 128. Despite continuous norm adaptation (top panel showing median and quartiles), training remains stable. The norm distribution maintains controlled variation within predictable bounds rather than erratic fluctuations. This translates to smooth accuracy progression (middle panel) without the oscillations or sudden drops characteristic of CO. The training loss (bottom panel) decreases steadily. While robust overfitting gradually emerges in later epochs, a known phenomenon in adversarial training, no catastrophic collapse occurs, validating that adaptive norm selection enhances rather than disrupts training stability.

Figure 10: Performance comparison of adaptive

l^{p}

-FGSM against established fast adversarial training methods. Evaluation uses PGD-50-10 attacks (50 steps, 10 restarts) for robust assessment. All methods trained for 30 epochs with single backpropagation, maintaining computational efficiency. Training employs SGD optimizer with cosine learning rate schedule (0.001-0.2), weight decay

5\cdot 10^{-4}

, and dropout 0.1. Hyperparameter

\beta=0.01

for CIFAR-10 and

\beta=0.1

for CIFAR-100 (Eq. ( 26 )), representing the only tunable parameter in our method. Extended 200-epoch stability analysis in Supplementary Materials Section VIII confirms long-term reliability.

ImageNet-1k ResNet-50
Method	$\epsilon=2/255$	$\epsilon=4/255$	$\epsilon=6/255$
FGSM	54.72%	48.50%	48.55%
	38.21%	25.86%	0.08%
RS-FGSM	56.29%	50.81%	47.67%
	36.86%	25.12%	16.49%
N-FGSM	54.39%	47.56%	47.70%

Table 1: Robustness evaluation on ImageNet-1k against PGD-50-10 attacks. Baseline results from de Jorge Aranda et al. (2022) . All methods use ImageNet pre-trained weights with 15 epochs of adversarial fine-tuning. Results show clean accuracy (top) and robust accuracy (bottom) for each perturbation level.

CIFAR-10 WRN-28-10 AutoAttack
$\epsilon\cdot 255$	FGSM	RS-FGSM	N-FGSM	$l^{p}$ -FGSM
2	90.81% ±0.07	90.64% ±0.12	89.27% ±0.21	89.02% ±0.41
2	74.72% ±0.37	71.47% ±0.44	73.14% ±0.68	76.14% ±0.62
4	87.86% ±0.23	86.58% ±0.22	86.34% ±0.36	85.71% ±0.53
4	61.58% ±0.12	54.85% ±0.16	59.81% ±0.27	62.12% ±0.42
8	84.89% ±1.20	80.14% ±0.88	74.73% ±0.46	79.81% ±0.57

Table S1: CIFAR-10 (WRN-28-10) Clean and AutoAttack Accuracy Evaluation. Results are averaged over three seeds. Clean accuracy (top) and AutoAttack accuracy (bottom).

Figure S1: Comparative evaluation using AutoAttack on CIFAR-10 with WideResNet-28-10 across different perturbation magnitudes.

Figure S2: Extended training performance of

l^{p}

-FGSM on CIFAR-10 and CIFAR-100. While Catastrophic Overfitting (CO) was not observed, the experiment highlights the occurrence of robust overfitting over a prolonged training period.

SVHN PreAct-18 PGD-50-10
$\epsilon\cdot 255$	$l^{p}$ -FGSM	RS-FGSM	N-FGSM	GradAlign	ZeroGrad
2	94.20% ±0.52	96.16% ±0.13	96.04% ±0.24	96.01% ±0.25	96.08% ±0.22
	86.22% ±0.22	86.17% ±0.17	86.46% ±0.12	86.44% ±0.15	86.47% ±0.17
4	94.16% ±0.64	95.07% ±0.08	94.56% ±0.18	94.57% ±0.24	94.83% ±0.19
	77.86% ±0.75	71.25% ±0.43	72.54% ±0.21	72.18% ±0.22	71.64% ±0.24
6	92.26% ±0.65	95.16% ±0.48	92.27% ±0.36	92.55% ±0.26	93.52% ±0.24

Table S2: Comparative Analysis of Fast Adversarial Training Methods on SVHN Dataset

CIFAR-10 WRN-28-10 PGD-50-10
$\epsilon\cdot 255$	$l^{p}$ -FGSM	RS-FGSM	N-FGSM	GradAlign	ZeroGrad
2	91.12% ±0.52	92.86% ±0.14	92.49% ±0.14	92.54% ±0.13	92.62% ±0.16
	80.84% ±0.25	80.91% ±0.14	81.42% ±0.34	81.32% ±0.43	81.41% ±0.32
4	88.07% ±0.34	90.74% ±0.23	89.64% ±0.23	89.93% ±0.34	90.21% ±0.22
	69.62% ±0.84	68.24% ±0.19	69.10% ±0.27	69.80% ±0.48	69.21% ±0.21
6	83.23% ±0.46	88.25% ±0.22	85.74% ±0.32	86.94% ±0.16	86.11% ±0.45

Table S3: Comparative Analysis of Fast Adversarial Training Methods on CIFAR-10 Dataset

CIFAR-100 WRN-28-10 PGD-50-10
$\epsilon\cdot 255$	$l^{p}$ -FGSM	RS-FGSM	N-FGSM	GradAlign	ZeroGrad
2	66.42% ±0.15	72.62% ±0.24	71.52% ±0.14	71.61% ±0.23	71.64% ±0.22
	55.29% ±0.64	51.62% ±0.56	52.24% ±0.35	51.51% ±0.48	52.63% ±0.64
4	61.32% ±0.34	68.27% ±0.21	66.51% ±0.48	67.09% ±0.19	67.21% ±0.18
	45.73% ±0.46	39.56% ±0.14	39.96% ±0.31	39.81% ±0.48	39.61% ±0.32
6	58.79% ±0.45	65.62% ±0.66	61.42% ±0.63	62.86% ±0.10	63.65% ±0.12

Table S4: Comparative Analysis of Fast Adversarial Training Methods on CIFAR-100 Dataset

Figure S3: Analysis of

\varepsilon

-softening and noise effects on CIFAR-10 using WideResNet-28-10 against PGD-50 (

\epsilon=8/255

). Top: Effect of

\varepsilon

-softening on clean (dashed) and adversarial (solid) accuracy for various

p

values. Optimal

\varepsilon

enhances stability against CO. Bottom: Synergistic effects of noise injection showing improved robustness against CO and enhanced overall accuracy. In these runs, both components help prevent catastrophic overfitting while maintaining competitive performance.

Figure S4: Evolution of Participation Ratios (

\mathtt{PR_{1}}

) and entropy gap during training with and without

l^{p}

-FGSM. Sharp patterns in these metrics align with the onset of Catastrophic Overfitting (CO), highlighting the link between gradient concentration and adversarial vulnerability. Same experimental setting as in the main paper (CIFAR-10, WRN-28-10, ADAM, learning rate 0.001, batch size 128).

Method	Clean	PGD-10	PGD-50	AutoAttack	CR	Cost
PGD-AT-WA [ 19 , 70 ]	$82.36\pm 0.32$	$55.05\pm 0.34$	$54.17\pm 0.17$	$50.49\pm 0.15$	$50.51\pm 0.16$	3.52
NuAT [ 71 ]	$80.98\pm 0.91$	$55.33\pm 0.14$	$54.63\pm 0.25$	$50.10\pm 0.18$	$50.11\pm 0.19$	1.47
LIET [ 60 ]	$80.68\pm 0.51$	$56.69\pm 0.18$	$55.97\pm 0.19$	$49.99\pm 0.15$	$49.93\pm 0.15$	1.31
FGSM-PGI [ 35 ]	$79.72\pm 0.63$	$56.43\pm 0.18$	$55.82\pm 0.21$	$49.89\pm 0.08$	$49.89\pm 0.12$	1.33
FGSM-FUAP [ 69 ]	$80.64\pm 1.28$	$56.35\pm 0.31$	$55.75\pm 0.38$	$49.73\pm 0.12$	$49.71\pm 0.12$	1.47
FGSM-CUAP [ 69 ]	$80.76\pm 2.10$	$56.30\pm 0.02$	$55.58\pm 0.22$	$49.55\pm 0.22$	$49.54\pm 0.22$	1.36

Table S5: CIFAR-10 under the FastAT Benchmark protocol (ResNet-18,

\epsilon=8/255

). Accuracy in %, mean

\pm

standard deviation over seeds 0, 1 and 2. Rows above the line are the values published in [ 61 ] ; rows below are ours, trained with the benchmark code and protocol. Cost: training time relative to FGSM-AT (published times for the benchmark rows; same code and GPU for ours). Methods sorted by AutoAttack accuracy, also shown in Figure S6 .

Method	Clean	PGD-10	PGD-50	AutoAttack	CR	Cost
PGD-AT-WA [ 19 , 70 ]	$57.62\pm 0.34$	$32.31\pm 0.16$	$31.90\pm 0.14$	$26.90\pm 0.17$	$26.87\pm 0.17$	3.39
LIET [ 60 ]	$50.47\pm 0.69$	$32.64\pm 0.53$	$32.44\pm 0.51$	$26.64\pm 0.13$	$26.63\pm 0.14$	1.13
FGSM-PGI [ 35 ]	$56.61\pm 0.34$	$32.18\pm 0.39$	$31.85\pm 0.23$	$26.29\pm 0.16$	$26.31\pm 0.12$	1.28
FGSM-RS-CS [ 73 ]	$54.79\pm 1.07$	$32.42\pm 0.29$	$32.08\pm 0.29$	$26.20\pm 0.26$	$26.20\pm 0.26$	0.91
FGSM-FUAP [ 69 ]	$56.29\pm 1.13$	$31.76\pm 0.31$	$31.36\pm 0.46$	$25.95\pm 0.08$	$25.94\pm 0.05$	1.45
FGSM-CUAP [ 69 ]	$55.12\pm 1.80$	$31.65\pm 0.26$	$31.40\pm 0.31$	$25.85\pm 0.50$	$25.83\pm 0.49$	1.38

Table S6: CIFAR-100 under the FastAT Benchmark protocol (ResNet-18,

\epsilon=8/255

). Accuracy in %, mean

\pm

l^{p}

-FGSM alone with

\beta=0.04

overfits catastrophically late in training, after the checkpoint selected on the validation images; a larger

\beta

should be used on this dataset.

Figure S5: Validation PGD-10 accuracy during training on CIFAR-10, seed 0, under the FastAT Benchmark protocol. With the same noisy start and consistency term, the FGSM step collapses at epoch 37 on this seed, while the adaptive

l^{p}

step does not. The dashed line is

l^{p}

-FGSM without these two ingredients.

Figure S6: AutoAttack accuracy of every method of Tables S5 (CIFAR-10, left) and S6 (CIFAR-100, right), mean and standard deviation over three seeds. Gray: values published in [ 61 ] ; at the bottom, in color: ours. NS: noisy start; CT: consistency term.

Adversarial Training

Adversarial training improves model reliability by training on examples or conditions designed to expose weaknesses.

See all 183 papers

Apr 27, 2026

Mitigating Error Amplification in Fast Adversarial Training

Fast Adversarial Training (FAT) has proven effective in enhancing model robustness by encouraging networks to learn perturbation-invariant representations. However, FAT often suffers from catastrophic overfitting (CO), where the model overfits to the training attack and fails to generalize to unseen ones. Moreover, robustness oriented optimization typically leads to notable performance degradation on clean inputs, and such degradation becomes increasingly severe as the perturbation budget grows. In this work, we conduct a comprehensive analysis of how guidance strength affects model performance by modulating perturbation and supervision levels across distinct confidence groups. The findings reveal that low confidence samples are the primary contributors to CO and the robustness accuracy trade off. Building on this insight, we propose a Distribution-aware Dynamic Guidance (DDG) strategy that dynamically adjusts both the perturbation budget and supervision signal. Specifically, DDG scales the perturbation magnitude according to the sample confidence at the ground truth class, thereby guiding samples toward consistent decision boundaries while mitigating the influence of learning spurious correlations. Simultaneously, it dynamically adjusts the supervision signal based on the prediction state of each sample, preventing overemphasis on incorrect signals. To alleviate potential gradient instability arising from dynamic guidance, we further design a weighted regularization constraint. Extensive experiments on standard benchmarks demonstrate that DDG effectively alleviates both CO and the robustness accuracy trade off.

Mengnan Zhao, Lihe Zhang, Bo Wang +3

Anhui University, Anhui, China · Dalian University of Technology, Liaoning, China · Zhejiang University, Zhejiang, China +1

Adversarial Training Neural Network Generalization Adversarial Robustness

Ask PDF

Apr 27, 2026

Unveiling the Backdoor Mechanism Hidden Behind Catastrophic Overfitting in Fast Adversarial Training

Fast Adversarial Training (FAT) has attracted significant attention due to its efficiency in enhancing neural network robustness against adversarial attacks. However, FAT is prone to catastrophic overfitting (CO), wherein models overfit to the specific attack used during training and fail to generalize to others. While existing methods introduce diverse hypotheses and propose various strategies to mitigate CO, a systematic and intuitive explanation of CO remains absent. In this work, we innovatively interpret CO through the lens of backdoor. Through validations on pathway division, diverse feature predictions, and universal class distinguishable triggers in CO, we conceptualize CO as a weak trigger variant of unlearnable tasks, unifying CO, backdoor attacks, and unlearnable tasks under a common theoretical framework. Guided by this, we leverage several backdoor inspired strategies to mitigate CO: (i) Recalibrate CO affected model parameters using vanilla fine tuning, linear probing, or reinitialization-based techniques; (ii) Introduce a weight outlier suppression constraint to regulate abnormal deviations in model weights. Extensive experiments support our interpretation of CO and show the efficacy of the proposed mitigation strategies.

Mengnan Zhao, Lihe Zhang, Tianhang Zheng +2

School of Computer Science and Technology, Anhui University, Hefei 230601, China · School of Information and Communication Engineering, Dalian University of Technology (DUT), Dalian 116024, China · School of Computer Science and Technology, Zhejiang University, Hangzhou 310058, China +1

Adversarial Training Neural Network Generalization Adversarial Robustness

Ask PDF

May 30, 2026

SORA: Free Second-Order Attacks in Fast Adversarial Training

Adversarial Training (AT) is a leading defense against adversarial examples but often suffers from Catastrophic Overfitting (CO) in efficient single-step variants, where robustness to multi-step attacks collapses despite high single-step performance. We address this failure mode with two contributions. First, we formalize Epsilon Overfitting (EO), a perspective in which fixed perturbation magnitudes and directions exacerbate CO, and show that introducing perturbation variability significantly improves robust generalization across different architectures and datasets. Second, we propose PertAlign (Perturbation Alignment), a theoretically grounded, computationally negligible metric that predicts CO onset by measuring gradient alignment across attack stages. Leveraging these insights, we introduce SORA, an adaptive step-size AT method that dynamically adjusts perturbations based on loss surface geometry. SORA consistently prevents CO, achieves state-of-the-art robustness and clean accuracy, and generalizes across datasets and architectures using a single fixed set of hyperparameters, which is essential for applicability in fast AT. Extensive experiments on diverse datasets and architectures show that SORA matches or surpasses the robustness of prior methods while delivering higher clean accuracy and superior efficiency. Code is available at https://github.com/SecondOrderAT/SORA.

Mazdak Teymourian, Ramtin Moslemi, Farzan Rahmani +1

Department of Computer Engineering, Sharif University of Technology, Tehran, Iran.

Adversarial Training Neural Network Generalization Neural Network Robustness Adversarial Robustness

Ask PDF

Abstract

Adversarial Training

Explore similar work

Mitigating Error Amplification in Fast Adversarial Training

Unveiling the Backdoor Mechanism Hidden Behind Catastrophic Overfitting in Fast Adversarial Training

SORA: Free Second-Order Attacks in Fast Adversarial Training