Accelerated Gradient Descent

Accelerated gradient descent (AGD) methods aim to improve the speed and efficiency of finding optimal solutions in various optimization problems, particularly prevalent in machine learning and related fields. Current research focuses on enhancing AGD's performance through techniques like preconditioning, adaptive learning rates, and momentum, as well as exploring its application in diverse contexts such as constrained optimization, distributed systems, and deep neural network training. These advancements are significant because they lead to faster training of complex models, improved generalization performance, and more efficient solutions for a wide range of scientific and engineering applications.

Papers

March 21, 2022

Image Classification on Accelerated Neural Networks
Ilkay Sikdokur, Inci Baytas, Arda Yurdakul
Convolutional Neural Network Image Classification Field Programmable Gate Array Convolution Layer Multi FPGA Accelerated Gradient Descent

March 3, 2022

Accelerated SGD for Non-Strongly-Convex Least Squares
Aditya Varre, Nicolas Flammarion
Least Square Stochastic Approximation Accelerated Gradient Descent Classical Algorithm

January 27, 2022

Restarted Nonconvex Accelerated Gradient Descent: No More Polylogarithmic Factor in the $O(\epsilon^{-7/4})$ Complexity
Huan Li, Zhouchen Lin
Gradient Descent Gradient Method Nonconvex Optimization General Nonconvex Accelerated Gradient Descent Accelerated Gradient Method

December 30, 2021

Adaptive Gaussian Process based Stochastic Trajectory Optimization for Motion Planning
Feng Yichang, Zhang Haiyun, Wang Jin, Lu Guodong
Gaussian Process Motion Planning Optimal Motion Planning Accelerated Gradient Descent Stochastic Trajectory Optimization

December 8, 2021

Reverse image filtering using total derivative approximation and accelerated gradient descent
Fernando J. Galetto, Guang Deng
Gradient Descent Accelerated Gradient Descent Image Filtering

Accelerated Gradient Descent

Papers

Image Classification on Accelerated Neural Networks

Accelerated SGD for Non-Strongly-Convex Least Squares

Restarted Nonconvex Accelerated Gradient Descent: No More Polylogarithmic Factor in the $O(\epsilon^{-7/4})$ Complexity

Adaptive Gaussian Process based Stochastic Trajectory Optimization for Motion Planning

Reverse image filtering using total derivative approximation and accelerated gradient descent