Gradient Descent

Ask about this topic

Recent momentum

-57%

9 papers in the last 28 days · 0.2% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Also in Gradient Descent

Neighbouring directions

Stochastic Gradient DescentGradientDp-SgdGradcamGradient-Based OptimizationSpectral DescentLarge Step Sizes

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-14

5 new papers

A weekly snapshot of new work published in Gradient Descent.

Period ending 2026-09-07

3 new papers

A weekly snapshot of new work published in Gradient Descent.

124 papers

Latest in Gradient Descent

Open your feed →
CardsList

  1. Geometry of learning dynamics: Gradient descent versus natural gradient on the ridge of optimization

    Sep 15, 2026Akira TamamoriGradient DescentGeometric Machine Learning

  2. AdamX: Cosine similarity meets gradient descent

    Sep 10, 2026Francisco Caldas, Ruben Belo, Cláudia SoaresAdamZeroth-Order Optimization

  3. Generalization Analysis of Distributed Kernel-based Robust Gradient Descent Algorithms

    Sep 10, 2026Jun-Yi Meng, Zheng-Chu Guo, Yuan MaoDistributional LearningGeneralization Bounds

  4. Exact-Form Regret for Gradient Descent, Mirror Descent and Follow-the-Regularized-Leader

    Sep 8, 2026Ashkan Soleymani, Gabriele Farina, Patrick JailletMirror DescentLinear Regret

  5. Why shared attention vectors fail: a case for outcome-indexed tuning

    Sep 8, 2026Lenard DomeVectorsGradient Descent

  6. A Theoretical Analysis of Generalization Dynamics in Neural Networks under Gradient Descent with Weight Decay

    Sep 7, 2026Yuqing Wang, Ioannis G. Kevrekidis, Mikhail BelkinGeneralization BoundsTwo-Layer Neural Networks

  7. The Multiple Timescales of Gradient Descent on the Edge of Stability: A Perturbative Derivation of the Central Flow

    Sep 1, 2026Raphaël BerthierGradient DescentTemporal Scales

  8. Generalization as a robust performance property of learning-enabled dynamical systems

    Aug 31, 2026Filippo FabianiGeneralization BoundsLyapunov Function

  9. Convergence rates for the RMSprop optimizer with full control of the hyperparameters

    Aug 31, 2026Steffen Dereich, Arnulf JentzenAdamGradient Descent

  10. Kähler landscapes for complex neural network descents and guarantees including a search and destroy of the Calabi-Yau manifold

    Aug 20, 2026Andrew GracykConvex LossManifolds

  11. Neural Quadratic Forms: A Unified Minimal Model for Sudden Learning and Scaling Laws

    Aug 13, 2026Liu Ziyin, Yizhou Xu, Tomaso Poggio +1Two-Layer Neural NetworksPower-Law Distribution

  12. Convergence Guarantees of Gradient Descent for Neural Networks via Generalized Lipschitz Smoothness

    Aug 11, 2026Siqiao Mu, Diego KlabjanGradient DescentLipschitz Constant

  13. A lower bound for stepsize-based acceleration of gradient descent

    Aug 11, 2026Jianhao Ma, Yuxin ChenLarge Step SizesGradient Descent

  14. Generalized Convexity and Smoothness via Conjugate Duality: Optimization Theory for Deep Neural Networks

    Aug 10, 2026Binchuan QiGradient DescentConvex Optimization

  15. Adaptive Hybrid Particle Swarm Optimization with Gradient Descent

    Aug 7, 2026Aryan GurudeoParticle Swarm OptimizationGradient Descent

  16. Minimax Optimal Early-Stopped Gradient Descent for Gaussian Mixture Classification

    Aug 6, 2026Alex Buna, Shirley Xiaoqi Liu, Patrick RebeschiniEmpirical Risk MinimizationGaussian Mixture Models

  17. Non-asymptotic implicit bias of logistic regression at early-stage gradient descent dynamics

    Aug 5, 2026Han BaoGradient DescentOptimization Convergence Analysis

  18. A Spectral Filtering Approach to Regret Analysis of Distributed Online Control for Linear Dynamical Systems

    Aug 3, 2026Ting-Jui ChangLinear Quadratic RegulatorGradient Descent

  19. Adaptivity via a Parallel Architecture for Stochastic Gradient Methods

    Jul 31, 2026Bin FuStochastic Gradient DescentGradient Descent

  20. Simultaneous Coverage and Efficiency Guarantee in Online Conformal Prediction

    Jul 29, 2026Rahul VazeConformal InferenceConformal Prediction

  21. Data-Dependent Regret and Polyak Corrections for Constrained Online Convex Optimization

    Jul 28, 2026Wentao ZhangConvex OptimizationDynamic Regret Bounds

  22. Extracting Voice Styles from Frozen TTS Models via Gradient-Based Inverse Optimization

    Jul 28, 2026Gyeongmin KimText-To-SpeechWav2Vec

  23. Global Convergence of DGM and PINN Algorithms for Solving Nonlinear PDEs

    Jul 27, 2026Justin Sirignano, Konstantinos Spiliopoulos, Samuel CohenPartial Differential EquationsSolution Operator

  24. Greedy dynamical meta-learning

    Jul 27, 2026Aria YomMeta-LearningGradient-Based Optimization

  25. Sparse Gaussian-Mixture-Model Q-Functions via Hadamard Overparametrization for Online Reinforcement Learning

    Jul 26, 2026Minh Vu, Konstantinos SlavakisRiemannian OptimizationOver-Parameterized Neural Networks

  26. Learning from the Descent Direction: Adaptive Gradient Descent under One-Sided Hölder Regularity

    Jul 24, 2026Arzu Ahmadova, Ismail HuseynovGradient DescentDescent

  27. Beyond Negative-Ridge Endpoints: Mixed-Sign Spectral Regularization via Negative-Shifted Gradient Descent

    Jul 24, 2026Peng ZhaoRegularization MethodsQuadratic Slack Penalty

  28. Complexity Bounds and Approaches to Learning Projected Gradient Descent Solver Iterates

    Jul 24, 2026Anjian Li, Ryne BeesonGradient DescentClassical Numerical Solver

  29. Energy Manifold Natural Gradient Descent: Riemannian Optimization for Neural PDE Solvers

    Jul 24, 2026Zhangyong Liang, Huanhuan GaoNeural Partial Differential Equation SolversRiemannian Optimization

  30. Optimizing Regret

    Jul 21, 2026Irene AldridgeDynamic Regret BoundsRegret

  31. Enhanced Neural Quantum State via Annealed Gradient Descent

    Jul 21, 2026Shiwei Zhou, Yiming Huang, Xiao Yuan +1Quantum LearningCosine Annealing

  32. Exact Network Surgery: Functional Invariance and Gradient Plasticity in Reactive Computational Graphs

    Jul 18, 2026Abdallah KhemaisResidual ConnectionsGraph Neural Network Inference

  33. Gradient Flow Dynamics and Implicit Bias of Diagonal Linear Networks under Infinitesimal Initialization

    Jul 14, 2026Jiajie Zhao, Jianxing Wang, Junjie Yang +2Gradient DescentTwo-Layer Neural Networks

  34. LayerNorm as Implicit Gain Control in Looped Transformers

    Jul 12, 2026Matthias M. M. BuehlmaierBatch NormalizationRecurrent Model

  35. Dynamics of Gradient Descent with Large Step Size Near a Manifold of Flat Minima

    Jul 9, 2026Lachlan Ewen MacDonald, René VidalGradient DescentFlat Minima

  36. Causal Optimizer Interaction Calculus: Hidden Geometric Relaxation and Identifiable Interventions

    Jul 8, 2026Zavier LiShape OptimizationAdaptive Optimizers

  37. Hybrid Least Squares/Gradient Descent Methods for MIONets

    Jul 8, 2026Jun Choi, Chang-Ock Lee, Minam MoonGradient DescentLeast Squares

  38. Converge to Surprise: Evolutionary Self-supervised Image Clustering

    Jul 8, 2026Canlin Zhang, Xiuwen LiuClusteringDeep Learning Models

  39. Differentially Private Natural Gradient Descent

    Jul 7, 2026Pan Li, Kai Chen, Shuai Chang +3Differential PrivacyDp-Sgd

  40. The Map Behind the Flow: Finite-Step Gradient Descent as a Dynamical System

    Jul 6, 2026Thomas HofmannGradient DescentWasserstein Gradient Flows

  41. Why can genetic algorithms work in high-dimensional search spaces?

    Jun 29, 2026Stephen WhitelamGenetic AlgorithmsHessian Eigenvalue

  42. ITSPACE: Monotone Gaussian Optimal Transport Updates

    Jun 29, 2026Woojoo Na, Jennifer DyOptimal TransportCovariance Estimation

  43. Difference of Convex Programming in the Wasserstein Space with Applications to MMD Optimization

    Jun 26, 2026Clément Bonet, Pierre-Cyril Aubin-Frankowski, Youssef MrouehWasserstein DistanceMaximum Mean Discrepancy

  44. Fast algorithms for learning a Gaussian under halfspace truncation with optimal sample complexity

    Jun 25, 2026Haitong Liu, Deepak Narayanan Sridharan, David Steurer +1Optimal Sample ComplexityGaussian Mixture Models

  45. Parametric Open Source Games

    Jun 25, 2026Aleksandar Todorov, Jesse ten Napel, Alexander MüllerNash EquilibriaSelf-Play

  46. \chisao{}: A GPU-Native Parallel Optimizer for Multimodal Black-Box Functions via Convergence-Anticonvergence Oscillation

    Jun 24, 2026Ira WolfsonBlack-Box OptimizationMulti-Gpu Systems

  47. Convergence of Gradient Descent for General Neural Network Architectures Beyond the NTK Regime

    Jun 22, 2026Yuqing WangNeural Tangent KernelGradient Descent

  48. Alternate loss functions and regression models that achieve robustness to outliers by modulating the learning rate

    Jun 20, 2026Mathew Mithra Noel, Arindam Banerjee, Yug D. Oswal +2OutliersLoss Function

  49. Gradient-Descent Steps to Success over Mean Accuracy: A Paradigm Shift for ML

    Jun 20, 2026Riccardo Poli, Ahmet YilmazHyperparameterGradient Descent

  50. Finite-Sample Performance of Gradient Descent in Logistic Regression with Gaussian Design

    Jun 19, 2026Junren Chen, Arya MazumdarFinite-SampleGradient Descent

  51. On the Oracle Complexity of Interpolation-Based Gradient Descent

    Jun 18, 2026Dongmin Lee, William Lu, Anuran MakurGradient DescentEmpirical Risk Minimization

  52. Global Convergence of Gradient Descent for Score Matching in Gaussian Mixtures via Reverse Fisher Divergence

    Jun 18, 2026Alexander TyurinGaussian Mixture ModelsScore-Based Diffusion Model

  53. Edge Flow: A Tractable and Predictive Continuous-Time Model for Gradient Descent at the Edge of Stability

    Jun 16, 2026Pierre MarionGradient DescentSharpness-Aware Minimization

  54. Functional Gradient Descent with Adaptive Representations

    Jun 15, 2026Daniel Csillag, Rodrigo Schuller, Pedro Dall'Antonia +3Gradient DescentConvex Loss

  55. A Bifurcation Theory Framework for Gradient Descent on the Edge of Stability

    Jun 14, 2026Eric GanGradient DescentBifurcation

  56. FastMix: Fast Data Mixture Optimization via Gradient Descent

    Jun 12, 2026Haoru Tan, Sitong Wu, Yanfeng Chen +5Model PretrainingMixture Models

  57. Muon^p: Muon with Fractional Spectral Powers

    Jun 11, 2026Yihe Dong, Will SawinMuonSpectral Descent

  58. Generalization in Nonlinear Least Squares via Learned Feature Geometry

    Jun 7, 2026Ayub Kharel, Ilja Kuzborskij, Patrick Rebeschini +1Generalization BoundsLeast Squares

  59. Uniform Stability and Generalization Error of GD and SGD on Fixed-Point Parameters

    Jun 5, 2026Jonghyun Shin, Sejun ParkStochastic Gradient DescentGeneralization Bounds

  60. Optimal Rates for Generalization of Gradient Descent Methods with Deep Neural Networks

    Jun 4, 2026Junyu Zhou, Puyu Wang, Yunwen Lei +2Gradient DescentStochastic Gradient Descent