cs.LGNov 8, 2026

A Stability Analysis of AdamW: Unstable Equilibria and Non-Convergence

Authors: Baiyu Su, Lizhang Chen, Qiang Liu

Organizations: The University of Texas at Austin

Abstract

AdamW is ubiquitous in deep learning, yet its behavior remains poorly understood. We analyze its dynamics through the lens of dynamical systems and show that AdamW admits an implicit fixed-point objective: its fixed points coincide with the stationary points of a constrained and regularized optimization problem. However, not all of these fixed points are stable under AdamW's dynamics, and stability depends sensitively on curvature, weight decay, and momentum parameters. Even in one-dimensional settings, equilibria may be unstable, and trajectories can exhibit persistent oscillations rather than convergence to them. We further extend the analysis to higher dimensions, deriving sufficient conditions for local stability of the continuous-time dynamics, and propose a convergent modification that preserves AdamW's fixed points. These results clarify what optimization problem AdamW is associated with, when its convergence can be expected, and how its dynamics could inspire more reliable optimizers.

Figures & tables

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. On the Convergence of Continuous-Time AdamW

    Nov 16, 2026Bangjie Wang, Ning NingOptimization Convergence AnalysisAdaptive Gradient Methods

  2. Adam at the Edge of Stability: Adaptive Feedback, Provable Oscillation, and Gradient Reversal

    Aug 21, 2026Yiman Fong, Heng YangEdge of StabilityAdaptive Gradient Methods

  3. On the O(dK1/4)O(\frac{\sqrt{d}}{K^{1/4}}) Convergence Rate of AdamW Measured by ℓ1\ell_1 Norm

    May 17, 2025Huan Li, Yiming Dong, Zhouchen LinOptimization Convergence AnalysisStochastic Optimization Convergence