When Are Nesterov Trajectories Eventually Locally Minimizing?
Organizations: Department of Computer Sciences University of Wisconsin–Madison
Abstract
We revisit the variational characterization of Nesterov's celebrated method and study when Nesterov trajectories are eventually locally minimizing in a variational sense, a notion that we define precisely. We show that this question is equivalent to the nonnegativity of an associated Schrödinger operator acting on variations of the trajectory. For power objectives of the form
(where denotes the Euclidean norm), we give a complete characterization: Among all exponents , the Nesterov trajectory with non-zero initial position is eventually locally minimizing if and only if . We further interpret this exceptional exponent through the stability theory of semilinear elliptic equations.
Accelerated Gradient Methods
Accelerated gradient methods use momentum, curvature, or adaptive updates to reduce the iterations needed for optimization.
See all 44 papers