Gradient Descent

Ask about this topic

Recent momentum

+0%

16 papers in the last 28 days · 0.3% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Also in Stochastic Gradient Descent

Neighbouring directions

Stochastic Gradient DescentGradient FlowGradient-Based OptimizationWasserstein Gradient FlowsGradient ConcentrationMini-Batch Stochastic Gradient Descent

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-21

7 new papers

A weekly snapshot of new work published in Gradient Descent.

Period ending 2026-09-14

5 new papers

A weekly snapshot of new work published in Gradient Descent.

Period ending 2026-09-07

3 new papers

A weekly snapshot of new work published in Gradient Descent.

155 papers

Latest in Gradient Descent

Browse all topics →
CardsList

  1. Dynamic Scaled Gradient Descent for Stable Fine-Tuning for Classifications

    Apr 30, 2026Nghia Bui, Lijing WangGradient DescentClass Imbalance

  2. Attractor FCM

    Apr 30, 2026Alexis KafantarisAttractorsGradient Descent

  3. Learning to Forget: Continual Learning with Adaptive Weight Decay

    Apr 29, 2026Aditya A. Ramesh, Alex Lewandowski, Jürgen SchmidhuberLearning RatesContinual Learning Methods

  4. State-Dependent Lyapunov Analysis of Rank-1 Matrix Factorization

    Apr 28, 2026Jaehong MoonMatrix FactorizationLyapunov Function

  5. The Origin of Edge of Stability

    Apr 22, 2026Elon LitmanCondition NumberGradient Descent

  6. FG2^2-GDN: Enhancing Long-Context Gated Delta Networks with Doubly Fine-Grained Control

    Apr 21, 2026Pingwei Sun, Yuxuan Hu, Jianchao Tan +6Global Attention MechanismEfficient Long-Context Inference

  7. Random Matrix Theory of Early-Stopped Gradient Flow: A Transient BBP Scenario

    Apr 20, 2026Florentin Coeurdoux, Grégoire Ferré, Jean-Philippe BouchaudGradient FlowGradient Descent

  8. Centre manifold theorem for maps along manifolds of fixed points

    Apr 20, 2026Lachlan Ewen MacDonaldManifoldsGradient Descent

  9. Compiling Deterministic Structure into SLM Harnesses

    Apr 19, 2026Zan Kai Chong, Hiroyuki Ohsaki, Bryan NgSmall Language ModelsDeterministic

  10. Lower Bounds and Proximally Anchored SGD for Non-Convex Minimization Under Unbounded Variance

    Apr 17, 2026Arda Fazla, Ege C. Kaya, Antesh Upadhyay +1Stochastic Smooth Convex OptimizationStochastic Gradient Descent

  11. Natural gradient descent with momentum

    Apr 16, 2026Anthony Nouy, Agustín SomacalGradient DescentLoss Function

  12. Marginal-Contribution Policy Gradients under Filtered Feedback for Multi-Agent LLMs

    Apr 3, 2026Elai Ben-Gal, Stela TongCredit AssignmentMulti-Agent Reinforcement Learning

  13. DISCOVER: A Solver for Distributional Counterfactual Explanations

    Mar 17, 2026Yikai Gu, Lele Cao, Bo Zhao +2Counterfactual ExplanationCounterfactual Reasoning

  14. First-Order Softmax Weighted Switching Gradient Method for Distributed Stochastic Minimax Optimization with Stochastic Constraints

    Mar 6, 2026Zhankun Luo, Antesh Upadhyay, Sang Bin Moon +1Stochastic OptimizationFederated Learning

  15. Non-Euclidean Gradient Descent Operates at the Edge of Stability

    Mar 5, 2026Rustem Islamov, Michael Crawshaw, Jeremy Cohen +1Gradient DescentSharpness-Aware Minimization

  16. The GRADIEND Python Package: An End-to-End System for Gradient-Based Feature Learning

    Feb 27, 2026Jonathan Drechsel, Steffen HerboldFeature LearningContrastive Decoding

  17. To Grok Grokking: Provable Grokking in Ridge Regression

    Jan 27, 2026Mingyue Xu, Gal Vardi, Itay SafranOver-Parameterized Neural NetworksKernel Ridge Regression

  18. Bridging Mechanistic Interpretability and Prompt Engineering with Gradient Ascent for Interpretable Persona Control

    Jan 6, 2026Harshvardhan Saini, Yiming Tang, Dianbo LiuPersonaMechanistic Interpretability

  19. Basic Inequalities for First-Order Optimization with Applications to Statistical Risk Analysis

    Dec 31, 2025Seunghoon Paik, Kangjie Zhou, Matus Telgarsky +1InequalitiesZeroth-Order Optimization

  20. Gradient Descent as Implicit EM in Distance-Based Neural Models

    Dec 31, 2025Alan OurslandExpectation-MaximizationGradient Descent

  21. Convergence Analysis of the ProbAbilistic Gradient Estimator Algorithm for Weakly Convex Finite-Sum Optimization

    Aug 31, 2025Laurent Condat, Peter RichtárikConvergence AnalysisConvex Sets

  22. Douglas-Rachford Splitting for Group-Sparse Feedback Linear-Quadratic Control

    Jul 26, 2025Lechen Feng, Xun Li, Yuan-Hua NiLinear Quadratic RegulatorSparsity

  23. When majority rules, minority loses: bias amplification of gradient descent

    May 19, 2025François Bachoc, Jérôme Bolte, Ryan Boustany +1MinorityEmpirical Risk Minimization

  24. The Dynamics of Generalization in Deep Learning

    Apr 23, 2025Rubing Yang, Pratik ChaudhariOver-Parameterized Neural NetworksGradient Descent

  25. Derivation of effective gradient flow equations and dynamical truncation of training data in Deep Learning

    Jan 13, 2025Thomas ChenGradient DescentWasserstein Gradient Flows

  26. A Mechanistic Study of Transformers Training Dynamics

    Oct 31, 2024Ambroise Odonnat, Wassim Bouaziz, Vivien CabannesTransformer ArchitecturesModel Pretraining

  27. Improved Finite-Particle Convergence Rates for Stein Variational Gradient Descent

    Sep 13, 2024Sayan Banerjee, Krishnakumar Balasubramanian, Promit GhosalGradient DescentConvergence

  28. Learning rate adaptive stochastic gradient descent optimization methods: numerical simulations for deep learning methods for partial differential equations and convergence analyses

    Jun 20, 2024Steffen Dereich, Arnulf Jentzen, Adrian RiekertStochastic Gradient DescentAdam

  29. Gradient Descent on Logistic Regression with Non-Separable Data and Large Step Sizes

    Jun 7, 2024Si Yi Meng, Antonio Orvieto, Daniel Yiming Cao +1Large Step SizesGradient Descent

  30. On Regularization via Early Stopping for Least Squares Regression

    Jun 6, 2024Rishi Sonthalia, Jackie Lok, Elizaveta RebrovaKernel Ridge RegressionEarly Stopping

  31. Fisher-Rao Gradient Flows of Linear Programs and State-Action Natural Policy Gradients

    Mar 28, 2024Johannes Müller, Semih Çaycı, Guido MontúfarWasserstein Gradient FlowsGradient Flow

  32. Privacy-Preserving Logistic Regression Training with A Faster Gradient Variant

    Jan 26, 2022John ChiangPrivacy-Preserving FrameworkGradient Descent

  33. REAL-Q: E2E LLM Quantization via Dynamic Gradient Descent

    Date pendingQian Zhang, Yaoming Li, Zhewen Tan +9Large Language Model QuantizationWeight-Only Quantization

  34. Generalization Guarantees on Data-Driven Tuning of Gradient Descent with Langevin Updates

    Date pendingSaumya Goyal, Rohith Rongali, Ritabrata Ray +1Gradient DescentHyperparameter Optimization

  35. Silver Rate Is (Almost) Optimal for Gradient Descent

    Date pendingYuhan Ye, Kaizhao LiuLarge Step SizesGradient Descent