Catastrophic Overfitting is Weight Drift the Training Loss Cannot See
Organizations: Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen 518055, China
Abstract
Fast adversarial training (FAT) makes robust training affordable by replacing the multi-step attack with a single-step one, but it suffers from catastrophic overfitting (CO): partway through training, robustness to multi-step attacks collapses to near zero while the single-step attack used for training keeps being repelled. We show that CO is a quiet escape of the weights. Across the event the network's clean behavior quietly improves, and moving from the weights just before CO to those just after it changes the training objective only marginally while destroying most of the robustness. The training objective is therefore nearly blind to the displacement it causes, so it exerts no force to undo it. The exponential moving average (EMA) of the weights, which FAT protocols already maintain, does not accumulate this drift and stays robust through the entire event. This observation suggests a nearly free fix: when validation robustness drops, reset the weights to their EMA. We build this guard on top of a coupling-based base method designed so that its average stays inside the robust region. The resulting training removes CO in every one of our runs and attains the strongest AutoAttack accuracy among fast methods on all three benchmark datasets, including a clear margin on CIFAR-100, where it also clears the multi-step reference at roughly half its cost. Code is available at https://github.com/fzjcdt/co-weight-drift.
Figures & tables
| CIFAR-10 | CIFAR-100 | Tiny-ImageNet | ||||
| Method | AA | CR | AA | CR | AA | CR |
| multi-step (reference) | ||||||
| PGD-AT [ 15 ] | 44.33 0.08 | 44.50 0.13 | 21.44 0.23 | 20.81 0.30 | 17.04 0.08 | 16.84 0.12 |
| PGD-AT-WA [ 10 ] | 50.49 0.15 | 50.51 0.16 | 26.90 0.17 | 26.87 0.17 | 19.77 0.16 | 19.75 0.13 |
| fast | ||||||
| NuAT [ 24 ] | 50.10 0.18 | 50.11 0.19 | 14.41 3.29 | 17.66 0.42 | 10.20 0.44 | 10.19 0.44 |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| Configuration | Per-seed AA (%) | Note |
|---|---|---|
| ours | ||
| Base + guard, | 50.19/50.40/50.61/50.60 | fires 0/0/4/0 |
| guard | 50.72/50.43/50.57/50.63 | no change |
| guard | 50.42/49.90/50.54 | 2/5/3 spurious |
| threshold and trigger controls | ||
| guard , same traj. | 7.94 | misses epoch 19 |
| Model | Seed | Clean | PGD-10 | PGD-50 | AA |
|---|---|---|---|---|---|
| Base + guard | 0 | 79.46 | 55.43 | 54.60 | 50.19 |
| 1 | 80.62 | 55.18 | 54.43 | 50.40 | |
| 2 | 79.35 | 55.20 | 54.52 | 50.61 | |
| 3 | 79.28 | 55.67 | 54.98 | 50.60 | |
| Base, no guard | 0 | 79.84 | — | 54.94 | survived |
| (seed status) | 3 | 79.28 | — | 54.98 | survived |
| Setting | Per-seed AA (%) | Mean |
|---|---|---|
| (one seed) | 11.20 | |
| (one seed) | 22.44 | |
| , | 25.53/26.41/26.23/26.24 | 26.10 0.39 |
| no nuclear term (3 seeds) | 26.36/25.82/24.93 | |
| 26.35/26.74/26.27/26.42 | 26.45 | |
| 26.61/26.35/26.66/26.64 | 26.57 0.14 |
| Setting | Per-seed AA (%) | Note |
|---|---|---|
| , (ours) | 20.49/20.57/20.36 | 20.47 0.11 |
| (one seed) | 7.80 | toxic |
| (one seed) | 20.69 | one guard fire |
| / (seed 0) | 20.42 / 20.40 | fire-free, level |
| Quantity | |
|---|---|
| Pairwise mean over 16 healthy models | 0.82 |
| PGD-20 (5 restarts), pairwise | 0.870 |
| PGD-20 vs. AutoAttack ordering, per model | 0.984 |
| Clean-difficulty baseline | 0.79 |
| Collapsed seed | 0.386 |
| Logit-surgery variant | 0.624 |
Adversarial Training
Adversarial training improves model reliability by training on examples or conditions designed to expose weaknesses.
See all 202 papers