Survival of the Fittest Perturbations: Uncertainty-Aware Evolution of Adversarial Distributions for Robust Generalization
Authors: Junhao Dong, Zicheng Yu, Xinghua Qu, Z. Jane Wang, Yew-Soon Ong
Organizations: College of Computing and Data Science, Nanyang Technological University, Singapore · Center for Frontier AI Research, Agency for Science, Technology and Research (A*STAR), Singapore · Bytedance, Singapore · Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC V6T 1Z4, Canada
Despite deep learning's broad successes across diverse scenarios, its vulnerability to adversarial examples impedes deployment, making adversarial training a leading approach to improve robustness. However, pointwise adversarial training maps each clean sample to a single adversary, overlooking surrounding adversarial variability and the associated shifts in feature statistics. To address these issues, we propose Evolutionary Uncertainty-Aware Distributional Adversarial Training (EvoUAD-AT). Specifically, we design an evolutionary adversarial uncertainty modeling module that maintains an instance-wise search distribution over adversarial feature statistics around an adversarially generated anchor. Candidates are evaluated under the current model, selected by their threat to a benign reference, and used to adapt the distribution; strategy parameters are retained across epochs. To avoid anchoring on misclassified clean samples, we refine the alignment reference and match benign and adversarial distributions. Introspective gradient matching further regularizes input gradients without external supervision. Our analysis characterizes the elite fitness surrogate and gives a conditional certificate for latent adversarial features. Extensive evaluations across multiple benchmarks demonstrate that our approach consistently outperforms existing state-of-the-art methods in both clean accuracy and robustness, with thorough analyses confirming the effectiveness of each proposed component.
Figures & tables
Fig. 1 : Motivation for evolutionary distributional training. (a) Point-wise AT trains on one PGD anchor; EvoAUM explores its feature-statistic neighborhood and selects threatening candidates (schematic). Under stronger PGD attacks, the standard PGD-AT model [ 2 ] exhibits larger increases in (b) feature variance and (c) input-gradient norm relative to clean inputs.
Fig. 2 : t-SNE visualization of legitimate feature and its 500 adversarial counterparts generated via the random start strategy on CIFAR-10 [ 22 ] . Each figure refers to a specific clean input.
Fig. 3 : Pipeline and components of our EvoUAD-AT. Evolutionary adversarial uncertainty modeling augments feature statistics, distributional alignment matches predictions, feature statistics across branches, and gradient alignment matches input gradients.
Fig. 4 : (a) Average robust accuracy, (b) feature variance, and (c) gradient norm under different attack strengths for an adversarially trained model (PGD-AT [ 2 ] ). Test samples are sorted by cross-entropy (ascending) and split into two equal halves. Negative perturbation radii denote benign refinement by flipping the attack gradient sign during adversary generation. The green line marks clean samples.
Method
CIFAR-10
CIFAR-100
SVHN
Clean
PGD
CW
AA
Clean
PGD
CW
AA
Clean
PGD
CW
AA
PGD-AT [ 2 ]
83.80
51.40
50.17
47.68
57.39
28.36
26.29
23.18
92.46
50.55
50.40
46.07
TRADES [ 4 ]
82.45
52.21
50.29
48.88
54.36
27.49
24.19
23.14
90.63
58.10
55.13
52.62
MART [ 11 ]
82.20
53.94
50.43
48.04
54.78
28.79
26.15
24.58
89.88
58.48
52.48
48.44
HAT [ 5 ]
84.86
52.04
50.33
48.85
58.73
27.92
24.60
23.34
92.06
57.35
54.77
52.06
APT [ 18 ]
83.47
54.24
51.92
50.06
58.27
31.05
27.49
26.02
92.32
57.80
53.69
50.26
TABLE I : Comparison of EvoUAD-AT using ResNet-18 with other adversarial training approaches on CIFAR-10, CIFAR-100, and SVHN. The ℓ∞ -norm perturbations are restricted in the radius ϵ=8/255 . We report both clean and robust accuracy (%).
Method
Clean
PGD
CW
AA
TRADES [ 4 ]
62.65
38.46
36.15
34.99
HAT [ 5 ]
64.23
38.98
36.53
35.24
UIAT [ 6 ]
64.90
40.27
37.39
36.08
ARoW [ 29 ]
64.72
40.18
37.70
36.25
DHAT [ 30 ]
65.53
40.54
37.95
36.69
RAAT [ 31 ]
64.38
39.37
37.41
35.33
TABLE II : Robust accuracy (%) on ImageNet using ResNet-50 under the ℓ∞ threat model with ϵ=4/255 .
Method
ϵ=10/255
ϵ=12/255
ϵ=16/255
Clean
Robust
Clean
Robust
Clean
Robust
TRADES [ 4 ]
82.28
38.55
79.37
31.84
74.89
18.70
HAT [ 5 ]
81.94
40.12
79.43
33.28
74.45
19.42
APT [ 18 ]
81.43
40.20
79.08
33.39
73.71
20.16
UIAT [ 6 ]
82.79
40.61
79.50
34.32
74.29
21.82
ARoW [ 29 ]
81.43
40.36
79.24
33.81
74.11
20.97
TABLE III : Adversarial robustness under different attack configurations using ResNet-18 on CIFAR-10. We present clean accuracy (%) and (Auto-Attack) robust accuracy (%).
Method
Clean
PGD
CW
AA
PGD-AT [ 2 ]
86.99
73.40
69.56
67.11
TRADES [ 4 ]
87.38
74.93
70.48
68.31
MART [ 11 ]
87.02
74.89
69.79
67.86
HAT [ 5 ]
87.95
75.10
70.83
68.92
APT [ 18 ]
87.79
75.04
70.62
68.59
UIAT [ 6 ]
88.14
75.28
70.91
68.89
TABLE IV : L2 robust acc. (%) on CIFAR-10 with ResNet-18.
Method
Clean
Corruption
PGD-AT [ 2 ]
91.45
79.37
TRADES [ 4 ]
92.35
80.54
MART [ 11 ]
91.74
80.49
HAT [ 5 ]
92.76
81.46
APT [ 18 ]
92.02
80.15
UIAT [ 6 ]
92.97
82.38
TABLE V : Robust acc. (%) on CIFAR-10-C with ResNet-18.
Architecture
Generative Model
Method
CIFAR-10
CIFAR-100
Clean
Robust
Clean
Robust
PRN-18
DDPM
Rebuffi et al. [ 36 ]
83.53
56.66
56.87
28.50
HAT [ 5 ]
86.86
57.09
61.50
28.88
APT [ 18 ]
85.21
56.33
59.17
28.24
UIAT [ 6 ]
87.34
58.46
62.20
29.40
ARoW [ 29 ]
86.92
58.11
61.25
29.07
TABLE VI : Comparison of adversarial training on CIFAR-10/CIFAR-100 with 1M auxiliary synthetic data. We report clean accuracy (%) and (Auto-Attack) robust accuracy (%).
Method
ϵ=4/255
ϵ=6/255
ϵ=8/255
ϵ=10/255
ϵ=12/255
Time(s)
Clean
Robust
Clean
Robust
Clean
Robust
Clean
Robust
Clean
Robust
N-FGSM [ 41 ]
87.24
66.69
84.66
56.36
80.29
48.24
75.59
41.54
71.01
36.92
48.4
N-FGSM + EvoUAD-AT
88.61
68.95
85.61
58.99
82.06
50.15
78.26
43.08
73.35
38.95
54.9
RS-FGSM [ 42 ]
88.18
66.64
86.72
55.28
84.07
46.15
86.32
0.00
85.55
0.00
32.4
RS-FGSM + EvoUAD-AT
88.79
67.03
87.78
56.34
85.21
46.82
88.13
0.00
87.34
0.00
36.7
GradAlign [ 43 ]
88.00
65.74
83.85
55.25
80.17
46.57
76.46
39.85
72.38
34.82
96.0
TABLE VII : Extension with single-step adversarial training methods on CIFAR-10. We conduct single-step adversarial training with various perturbation radii. We report the clean accuracy (%), robust accuracy (%) on PGD-based adversaries with 50 steps and 10 restarts, and the average training time (seconds) for an epoch.
TABLE VIII : Ablation study using ResNet-18 of three components of EvoUAD-AT for accuracy (%) on CIFAR-10.
Modeling Strategy
Clean
PGD
AA
Single Point (TRADES [ 4 ] )
82.45
52.21
48.88
Random Statistics ( N(0,I) )
81.97
49.83
43.29
Random Statistics ( N(μ(x),I) )
84.40
50.96
45.62
Random Statistics ( N(μ(x^),I) )
82.34
51.76
47.86
EvoAUM (w/ Intermediate Adversaries)
84.18
53.98
49.03
EvoAUM (w/ Historical Adversaries)
84.41
54.15
49.19
TABLE IX : Comparison (Acc. (%)) of diverse adversary modeling strategies using ResNet-18 on CIFAR-10.
Sample Set
Clean
PGD
AA
Time (m)
w/o Uncertainty
82.45
52.21
48.88
2.58
Random Start (3 Samples)
84.16
54.27
49.24
6.93
Random Start (5 Samples)
84.80
54.49
49.79
11.30
Random Start (10 Samples)
85.18
54.97
50.43
21.99
Random Start (15 Samples)
85.07
55.05
50.57
32.78
EvoAUM (Ours)
85.25
54.88
50.41
3.20
TABLE X : Comparison of uncertainty estimation on diverse sample sets using ResNet-18 on CIFAR-10. We report both the accuracy (%) and the average training time (min/epoch).
Position for Uncertainty Modeling
Clean
PGD
AA
1st Convolutional Block
86.04
54.92
50.83
2nd Convolutional Block
86.83
55.78
52.02
3rd Convolutional Block
85.41
55.49
51.30
4th Convolutional Block
86.17
55.28
51.21
Penultimate Layer
86.29
55.12
51.64
TABLE XI : Impact (Acc.) of the position for adversarial uncertainty modeling using ResNet-18 on CIFAR-10.
Method
Clean
PGD
AA
Time (m)
PGD-AT [ 2 ]
83.80
51.40
47.68
2.95
TRADES [ 4 ]
82.45
52.21
48.88
3.28
MART [ 11 ]
82.20
53.94
48.04
3.78
HAT [ 5 ]
84.86
52.04
48.85
4.16
APT [ 18 ]
83.47
54.24
50.06
4.35
UIAT [ 6 ]
85.01
54.63
49.11
3.13
TABLE XII : Comparison of benign refinement with diverse iterative steps using ResNet-18 on CIFAR-10. We report both the accuracy (%) and the average training time (min/epoch).
Fig. 5 : Accuracy-robustness trade-off of our EvoUAD-AT method by tuning the hyper-parameter β . We report both clean accuracy (%) and (Auto-Attack) robust accuracy (%).
Fig. 6 : t-SNE visualization of the legitimate feature, adversarial features, and augmented features on CIFAR-10 [ 22 ] . Each figure refers to a specific clean input.
Fig. 7 : Grad-CAM visualizations of adversaries against diverse models on CIFAR-10 using ResNet-18.
Fig. 8 : Learning curves of (a) PGD-AT [ 2 ] and (b) our method on CIFAR-10. We report both clean and (PGD-20) robust accuracies (%) on the training and test sets.
Fig. 9 : Hyper-parameter sensitivity of EvoUAD-AT on clean and robust accuracies on CIFAR-10. We report the results of tuning factors λ1 and λ2 in (a) and (b), respectively.
Fig. 10 : Increases from clean to adversarial inputs in (a) feature variance and (b) input-gradient norm across attack radii. The solid bars show TRADES, and the hatched bars show ours.
Fig. 11 : CIFAR-10 accuracy of EvoUAD-AT minus that of TRADES [ 4 ] , in percentage points. Both models were trained at ϵ=8/255 (dashed line). Radius zero gives clean accuracy. Positive radii give AutoAttack accuracy. The insets show the clean image and adversarial examples at 8/255 and 16/255 .
Safety-critical applications require classifiers that are both robust and reliable. Adversarial training is a widely adopted defense for improving robustness in deep neural networks; however, its effect on the reliability of predictive uncertainty remains underexplored. We investigate this gap through the lens of selective classification, which has rarely been systematically analyzed alongside adversarial robustness. We introduce a unified benchmark for the robustness-uncertainty trade-off. It standardizes architectures, augmentations, threat models, and evaluation metrics across clean, adversarial, and common-corruption settings. Across a wide range of state-of-the-art adversarial training methods, we uncover a recurring failure mode: several approaches improve robust accuracy while degrading uncertainty ranking, leading to poorer selective behavior. To address this, we propose Evidential Adversarial Training (EV-AT), which models uncertainty through a Dirichlet distribution and combines (i) an evidence-based loss promoting clean accuracy and reliable uncertainty with (ii) a robust evidence-alignment loss matching clean and adversarial predictions in log Dirichlet-parameter space. Extensive experiments show that EV-AT shifts the Pareto frontier of robustness-uncertainty trade-offs beyond prior state-of-the-art adversarial training methods. Our source code is publicly available at https://github.com/NicolasSournac/Robustness_Meets_Uncertainty.EV-AT.
Nicolas Sournac, Ahmed Baha Ben Jmaa, Bertrand Braeckeveldt
Multitel Research & Innovation Center, Artificial Intelligence Department, Belgium · TRAIL – Trusted AI Labs, Belgium
Building on a probabilistic perspective in which adversarial examples arise from the overlap between a distance-based distribution pdis and a victim-classifier-induced distribution pvic, we start from a simple intuition: adversarial examples become harder to generate when these two distributions are pushed apart, as their overlap becomes smaller, thereby increasing robustness. This intuition naturally motivates a KL-based robustness objective. We then prove that KL(pdis∥pvic)−logZvic is a lower bound on probabilistic robustness (PR), where Zvic denotes the normalizing constant of pvic. Since PR is generally intractable to compute directly, maximizing this KL-based lower bound provides a tractable surrogate objective for improving PR. We further show that this objective recovers a scaled form of adversarial training, offering a probabilistic interpretation of adversarial training and a principled route to robustness improvement. We call the resulting method probabilistic adversarial training. Experiments show that it consistently improves PR, and ablation studies demonstrate that the induced scaling factor can even enhance the PR of non-probabilistic adversarial training methods.
Nowadays, pretrained models are increasingly used as general-purpose backbones and adapted at test-time to downstream environments where target data are scarce and unlabeled. While this paradigm has proven effective for improving clean accuracy on the target domain, adversarial robustness has received far less attention, especially when the original pretrained model is not explicitly designed to be robust. This raises a practical question: can a pretrained, non-robust model be adapted at test-time to improve adversarial robustness on a target distribution? To face this question, this work studies how adversarial training strategies behave when integrated into adaptation schemes for the unsupervised test-time setting, where only a small set of unlabeled target samples is available. It first analyzes how classical adversarial training formulations can be extended to this scenario, showing that straightforward distillation-based adaptations remain unstable and highly sensitive to hyperparameter tuning, particularly when the teacher itself is non-robust. To address these limitations, the work proposes a label-free framework that uses the predictions of a non-robust teacher model as a semantic anchor for both the clean and adversarial objectives during adaptation. We further provide theoretical insights showing that our formulation yields a more stable alternative to the self-consistency-based regularization commonly used in classical adversarial training. Experiments evaluate the proposed approach on CIFAR-10 and ImageNet under induced photometric transformations. The results support the theoretical insights by showing that the proposed approach achieves improved optimization stability, lower sensitivity to parameter choices, and a better robustness-accuracy trade-off than existing baselines in this post-deployment test-time setting.
Stefano Bianchettin, Giulio Rossolini, Giorgio Buttazzo
Department of Excellence in Robotics and AI, Sant’Anna School of Advanced Study, Via Moruzzi, 1, Pisa, 56124, Tuscany, Italy