cs.LGNov 14, 2026

Probing SAM Perturbations Through Statistical Replay

Authors: Ting Han, Jens Kleesiek, Michael Kamp

Organizations: TU Dortmund, Germany · Lamarr Institute for Machine Learning and Artificial Intelligence, Germany · Institute for Artificial Intelligence in Medicine, UK Essen, Germany

Abstract

Sharpness Aware Minimization (SAM) updates model parameters using gradients evaluated at adversarially perturbed weights, but it remains unclear which aspects of this procedure are sufficient to reproduce its generalization behavior. In our ResNet 18 experiments on CIFAR 10, simple curvature regularizers based on the Hessian trace and largest eigenvalue remain below SAM in validation accuracy, motivating a direct examination of its perturbations. We introduce statistical replay, a diagnostic that summarizes perturbations from reference SAM trajectories using elementwise means and standard deviations over local windows and constructs new perturbations during independent SGD training. Pure replay uses only these summaries, whereas mixed replay adds isotropic Gaussian noise. Pure replay does not reproduce SAM's validation behavior, while mixed replay and pure isotropic perturbations can approach its validation performance at selected scales without reproducing its training dynamics. An equal radius random direction within SAM performs similarly to SGD, and source, temporal, and layer interventions further suggest that replay behavior depends on perturbation construction. Together, these preliminary results weaken several simple explanations of SAM while leaving an important part of its mechanism unexplained.

Figures & tables

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Fix the Loss, Not the Radius: Rethinking the Adversarial Perturbation of Sharpness-Aware Minimization

    May 11, 2026Jinping Wang, Qinhan Liu, Zhiwu Xie +1Neural Network GeneralizationSharpness-Aware Minimization

  2. One Size Does Not Fit All: Curvature-Aware Adaptive Perturbation for SAM

    Oct 14, 2026Alessia Rossi, Felipe Campelo, Gabriel L. OliveiraCurvature-Aware OptimizationSharpness-Aware Minimization