Stochastic Gradient Descent

Ask about this topic

Recent momentum

-17%

10 papers in the last 28 days · 0.3% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Also in Gradient Descent

Neighbouring directions

GradientGradient DescentDp-SgdGradcamGradient-Based OptimizationSpectral DescentLarge Step Sizes

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-14

5 new papers

A weekly snapshot of new work published in Stochastic Gradient Descent.

Period ending 2026-09-07

3 new papers

A weekly snapshot of new work published in Stochastic Gradient Descent.

161 papers

Latest in Stochastic Gradient Descent

Open your feed →
CardsList

  1. Gradient Descent with Stochastic Subspaces via Persistence of Memory

    Sep 16, 2026Subhroshekhar Ghosh, Clement Z. Q. Ng, Pierre-Louis Poirion +1Stochastic Gradient Descent

  2. Same Flow, Different Paths: Variance Reduction in Flow Matching

    Sep 15, 2026Alexander TyurinConditional Flow MatchingStochastic Gradient Descent

  3. Convergence of Stochastic Gradient Methods under Heavy-Tailed Noise and H\"{o}lder Smoothness

    Sep 14, 2026Misbah Uz Zaman, Anirbit MukherjeeStochastic Gradient Descent

  4. Stochastic Gradient Descent over P2

    Sep 11, 2026Maria Oprea, Qin Li, Yunan YangStochastic Gradient Descent

  5. Almost Sure Convergence Analysis of Stochastic Gradient Methods with Clipping and Additive Noise

    Sep 10, 2026Amartya Mukherjee, Jun LiuStochastic Gradient Descent

  6. Learning with Synthetic Data via SGD in High-Dimensional Linear Regression

    Sep 10, 2026Jichu li, Difan ZouSynthetic DataTraining Models

  7. The Exact Time-Uniform Rate Frontier for Stochastic Gradient Descent on Smooth Convex Objectives

    Sep 8, 2026Ruijie Li, Kang Chen, Tianyu WangStochastic Gradient DescentOptimization Convergence Analysis

  8. Speed Limit for Information Acquisition in Stochastic Learning Dynamics

    Sep 8, 2026Shuta Kobayashi, Andreas DechantStochastic Gradient DescentInformation-Theoretic Limits

  9. Sparse Data Augmentation for Optimization with Provable Guarantees

    Sep 8, 2026Behrooz Tahmasebi, Melanie WeberConvex OptimizationData Augmentation

  10. SGD in Multiclass Logistic Regression: Sequential Learning and Scaling Laws

    Sep 7, 2026Konstantinos Christopher Tsiolis, Denny Wu, Christos Thrampoulidis +1Multiclass ClassificationScaling Laws

  11. High-Dimensional Learning Dynamics of Attention-Indexed Models

    Sep 3, 2026Yizhou Xu, Margarita Sagitova, Lenka Zdeborová +1Stochastic Gradient Descent

  12. Convergence Theory of Knowledge Distillation in Asynchronous P2P Gossip Learning Network

    Sep 1, 2026Lucas Qingyang Fang, Tiyao Liu, Jinhao Jing +4Heterogeneous Knowledge DistillationPeer-To-Peer

  13. Subspace Levenberg Marquardt Algorithms in Training Neural Networks

    Sep 1, 2026M. Duc HoangArtificial Neural NetworksNewton

  14. Adaptive Bregman Proximal Stochastic Gradient with a Stabilized Barzilai--Borwein Step Size

    Aug 12, 2026Chenhan Jin, Shengze Xu, Binghui Xie +4Stochastic Gradient DescentConvex Optimization

  15. Optimized Certainty Equivalent Risk Minimization Using Samples: Algorithms, Convergence Rates, and Applications

    Aug 7, 2026Sumedh Gupte, Prashanth L. A., Sanjay P. BhatStochastic OptimizationConditional-Value-At-Risk

  16. Stochastic gradient descent with discontinuity across a manifold

    Aug 7, 2026Vivek S. BorkarStochastic Gradient DescentManifolds

  17. On MUON optimization: From non-convergence to an error analysis with Polar Express and the Newton-Schulz polynomial from implementations

    Aug 5, 2026Thang Do, Steffen Dereich, Arnulf JentzenStochastic Gradient DescentNewton

  18. A Counterexample to Fourier Alignment in Single-Neuron Modular Addition

    Aug 5, 2026Gautam Neelakantan MemanaModular AdditionCounterexample

  19. Convergence and Regret of the Policy Gradient for Multi-Armed Bandits in Diffusion Environment

    Jul 31, 2026Yanwei Jia, Du OuyangPolicy GradientOptimal Policies

  20. Adaptivity via a Parallel Architecture for Stochastic Gradient Methods

    Jul 31, 2026Bin FuStochastic Gradient DescentGradient Descent

  21. A Trust-region Framework for Moment Estimation

    Jul 26, 2026Oluwasegun A. SomefunTrust RegionStochastic Gradient Descent

  22. Mini-batch Noise Lowers Sharpness via Dominant-Subspace Fluctuations

    Jul 25, 2026Junho So, Dongwook ShinStochastic Gradient DescentSharpness-Aware Minimization

  23. An Isotropy-Preserving Spectral Cap for Muon: Theory and Three Case Studies

    Jul 22, 2026Jiachun LiMuonSpectral Norm

  24. RELTA-SGLD: Relative-Growth Localized Taming for Nonconvex Stochastic-Gradient Langevin Learning

    Jul 21, 2026Yiwei Zhou, Ziheng ChenStochastic Gradient DescentNonconvex

  25. Optimizing the Preconditioner: A Black-box Online-to-Nonconvex Conversion with Static Regret Minimization Oracles

    Jul 20, 2026Haichen Hu, David Simchi-LeviConvex OptimizationStochastic Gradient Descent

  26. Scaling Limits of Constant-Stepsize SGD at Flat Minima

    Jul 17, 2026Jingyi Zhang, Cheng Mao, Debankur MukherjeeLarge Step SizesStochastic Gradient Descent

  27. What's in a Smoothness Constant? Tighter Rates for Local SGD with Bounded Second-order Heterogeneity

    Jul 16, 2026Kumar Kshitij Patel, Rustem Islamov, Sebastian U Stich +3Stochastic Gradient DescentConvex Optimization

  28. Efficient Tuning Before Low-Bit Post-Training Quantization for Stochastic Gradient Descent-optimized Models

    Jul 13, 2026Peng Xia, Junbiao Pang, Muhammad Ayub SabirPost-Training QuantizationQuantization Effects

  29. Understanding Schedule-Free Methods in Nonconvex Optimization: Rate Guarantees and Escaping Saddles

    Jul 10, 2026Jiseok Chae, Donghwan KimConvex OptimizationStochastic Gradient Descent

  30. Dynamics of Gradient Descent with Large Step Size Near a Manifold of Flat Minima

    Jul 9, 2026Lachlan Ewen MacDonald, René VidalGradient DescentFlat Minima

  31. Vanilla SGD with Momentum Survives Heavy-Tailed Noise: Convergence Analysis without Gradient Clipping or Normalization

    Jul 9, 2026Ryusei Yamada, Naoki Sato, Hideaki IidukaStochastic Gradient DescentHeavy-Tailed Noise

  32. Leveraging Extragradient for Effective Sharpness-Aware Minimization in Deep Learning

    Jul 7, 2026Yao Fu, Chunxia Zhang, Junmin Liu +3Stochastic Gradient DescentSharpness-Aware Minimization

  33. Non-asymptotic Convergence of Stochastic Gradient Descent in Score-based Generative Models

    Jul 6, 2026Stanislas Strasman, Sobihan Surendran, Sylvain Le CorffStochastic Gradient DescentFinite Time Analysis

  34. Implicit Bias of SGD in Multivariate ReLU Networks: Effective Width Collapse

    Jul 3, 2026Shuang Liang, Tom Jacobs, Guido MontúfarRectified Linear Unit NetworksStochastic Gradient Descent

  35. Can Model Merging Improve Aggregation in DiLoCo?

    Jul 3, 2026Stefan Horoi, Benjamin Thérien, Guy Wolf +1Continual Model MergingModel Merging

  36. Homogenization of \ell_2-Adversarial Training in High-Dimensions: Exact Dynamics under Stochastic Gradient Descent

    Jun 30, 2026Fabrizzio SabelliStochastic Gradient DescentAdversarial Transferability

  37. Random Reshuffling Dominates Stochastic Gradient Descent

    Jun 30, 2026Zijian LiuStochastic Gradient DescentOptimization Convergence Analysis

  38. Revisiting the Volume Hypothesis

    Jun 30, 2026Ari Pakman, Lior Kreimer, Yakir BerchenkoStochastic Gradient DescentLow-Data Regimes

  39. SGD at the Edge of Stability: Stochastic Stabilization with Large Learning Rates

    Jun 29, 2026Konstantinos Emmanouilidis, Lachlan MacDonald, Salma Tarmoun +1Stochastic Gradient DescentLearning Rates

  40. Curvature-Weighted Gradient Diversity: A Noise Measure for Geometry-Adaptive SGD Schedules

    Jun 29, 2026Muhammad Hamza, Ayush GoelStochastic Gradient DescentHessian Matrix

  41. SGD Provably Prioritizes a Shortcut Spurious Feature in the XOR Model

    Jun 29, 2026Tyler LaBonte, Vidya MuthukumarFeature LearningRectified Linear Unit Networks

  42. How AI settled the complexity of the oldest SGD algorithm

    Jun 28, 2026Michał Dereziński, Xiaoyu DongStochastic Gradient DescentWorst-Case Sensitivity

  43. Variance Reduction for Stochastic Gradient Generalized Non-reversible Langevin Monte Carlo Algorithms

    Jun 27, 2026Bingye Ni, Xiaoyu Wang, Yingli Wang +1Langevin DynamicsVariance Reduction

  44. Stochastic Gradient Optimization with Model-Assisted Sampling

    Jun 25, 2026Jonne Pohjankukka, Jukka HeikkonenStochastic Gradient DescentVariance Reduction

  45. Sketched Linear Contrastive Learning: Approximation, Optimization, and Statistical Scaling

    Jun 25, 2026Ziyan Chen, Zhongzhu Zhou, Ding-Xuan ZhouScaling LawsContrastive Learning

  46. GRAIN: Group Aggregation via Min-Norm Objective

    Jun 22, 2026Nghia Bui, Jiarui Yao, Lijing WangStochastic Gradient DescentDeep Learning Models

  47. Parameterized Representations via Implicit Stochastic Modulation for High-Dimensional and High-Order Neural PDE Solvers

    Jun 20, 2026Zhangyong Liang, Huanhuan GaoNeural Partial Differential Equation SolversPartial Differential Equations

  48. Fisher-Geometric Sharpness and the Implicit Bias of SGD toward Flat Minima

    Jun 18, 2026Md Sakir Ahmed, Kumaresh Sarmah, Hemen DuttaFlat MinimaStochastic Gradient Descent

  49. Weight Adaptation for Improving Parallel Performance of Adaptive Stochastic Natural Gradient

    Jun 18, 2026Yutaro Yamada, Kento Uchida, Shinichi ShirakawaEvolutionary AlgorithmsStochastic Gradient Descent

  50. Compute Efficiency and Serial Runtime Tradeoffs for Stochastic Momentum Methods

    Jun 17, 2026Depen Morwani, Alexandru Meterez, Pranav Nair +1Stochastic Gradient DescentBatch Size

  51. Mixed-Precision Communication-Avoiding SGD for Generalized Linear Models on GPUs

    Jun 16, 2026Aditya Devarakonda, Irene Simó Muñoz, Giulia GuidiDp-SgdStochastic Gradient Descent

  52. A Link between Shock-wave Theory and Symmetry-reduced Stochastic Gradient Descent for Artificial Neural Networks

    Jun 16, 2026Taiki MiyagawaStochastic Gradient DescentArtificial Neural Networks

  53. Variance Reduction for Non-Log-Concave Sampling with Applications to Inverse Problems

    Jun 15, 2026M. Berk Sahin, Ahmet Ege Tanriverdi, Behzad Sharif +1Variance ReductionPosterior Sampling

  54. Clipping Makes Distributed and Federated Asynchronous SGD Robust to Stragglers

    Jun 11, 2026Samuel Erickson, Mikael JohanssonGradient ClippingStochastic Gradient Descent

  55. Simplicity Suffices for Parameter Noise Injection in Stochastic Gradient Descent

    Jun 10, 2026Benjamin Leblanc, Louis-Jacob Lebel, Teddy Kana +1Stochastic Gradient DescentSimplification

  56. Renewable Lasso without Batch-Number Constraints: A Gradient-Enhanced Approach

    Jun 10, 2026Junzhuo Gao, Ling Peng, Xu Guo +1LassoSurrogate Gradient

  57. Fourier fractal dimension to predict the generalization of deep neural networks

    Jun 6, 2026Joao B. Florindo, Davi Wanderley MisturiniDeep Neural NetworkStochastic Gradient Descent

  58. Accelerated Decentralized Stochastic Gradient Descent for Strongly Convex Optimization

    Jun 5, 2026Ming Sun, Kun YuanStochastic Gradient DescentConvex Optimization

  59. Second-Order Path Kernel Interpolation Formulas in Machine Learning

    Jun 5, 2026Jin Guo, Roy Y. He, Jean-Michel MorelStochastic Gradient DescentInterpolation

  60. Beyond Linear and Overcomplete Regimes: A Mean-Field Analysis of Bottleneck Autoencoders

    Jun 5, 2026Santanu Das, Ramyak Bilas, Pascal Esser +1AutoencoderLow-Data Regimes