Stochastic Gradient Descent

Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

21 new papers

A weekly snapshot of new work published in Stochastic Gradient Descent.

Period ending 2026-09-14

17 new papers

A weekly snapshot of new work published in Stochastic Gradient Descent.

Period ending 2026-09-07

12 new papers

A weekly snapshot of new work published in Stochastic Gradient Descent.

Inside this field

Focused directions

541 papers

Latest in Stochastic Gradient Descent

Browse all topics →
CardsList

  1. Scaling Laws of SignSGD in Linear Regression: When Does It Outperform SGD?

    Mar 2, 2026Jihwan Kim, Dogyoon Song, Chulhee YunStochastic Gradient DescentScaling Laws

  2. The GRADIEND Python Package: An End-to-End System for Gradient-Based Feature Learning

    Feb 27, 2026Jonathan Drechsel, Steffen HerboldFeature LearningContrastive Decoding

  3. LoMime: Query-Efficient Membership Inference using Model Extraction in Label-Only Settings

    Feb 21, 2026Abdullah Caglar Oksuz, Anisa Halimi, Erman AydayMembership Inference AttacksMembership Inference

  4. Efficient privacy loss accounting for subsampling and random allocation

    Feb 19, 2026Vitaly Feldman, Moshe ShenfeldDifferential PrivacyEfficient Privacy Loss

  5. Limitations of SGD for Multi-Index Models Beyond Statistical Queries

    Feb 5, 2026Daniel Barzilai, Ohad ShamirStochastic Gradient Descent

  6. Gradient Flow Through Diagram Expansions: Learning Regimes and Explicit Solutions

    Feb 4, 2026Dmitry Yarotsky, Eugene Golikov, Yaroslav GusevGradient FlowWasserstein Gradient Flows

  7. Decentralized SGD with Controlled Disagreement Finds Flatter Minima

    Feb 2, 2026Zesen Wang, Mikael JohanssonStochastic Gradient DescentDistributional Learning

  8. Not All Preferences Deserve Gradients: Understanding Gradient Utility in Offline Reasoning Alignment

    Feb 1, 2026Hui Wu, Hengyi Cai, Jinman Zhao +6Gradient Concentration

  9. To Grok Grokking: Provable Grokking in Ridge Regression

    Jan 27, 2026Mingyue Xu, Gal Vardi, Itay SafranOver-Parameterized Neural NetworksKernel Ridge Regression

  10. ZENITH: Automated Gradient Norm Informed Stochastic Optimization

    Jan 21, 2026Dhrubo SahaStochastic Gradient DescentRegularization Techniques

  11. Performance and Complexity Trade-off Optimization of Speech Models During Training

    Jan 20, 2026Esteban Gómez, Tom BäckströmLarge ModelsStochastic Gradient Descent

  12. Bridging Mechanistic Interpretability and Prompt Engineering with Gradient Ascent for Interpretable Persona Control

    Jan 6, 2026Harshvardhan Saini, Yiming Tang, Dianbo LiuPersonaMechanistic Interpretability

  13. SGD-Based Knowledge Distillation with Bayesian Teachers: Theory and Guidelines

    Jan 4, 2026Itai Morad, Nir Shlezinger, Yonina C. EldarKnowledge DistillationStochastic Gradient Descent

  14. Basic Inequalities for First-Order Optimization with Applications to Statistical Risk Analysis

    Dec 31, 2025Seunghoon Paik, Kangjie Zhou, Matus Telgarsky +1InequalitiesZeroth-Order Optimization

  15. Gradient Descent as Implicit EM in Distance-Based Neural Models

    Dec 31, 2025Alan OurslandExpectation-MaximizationGradient Descent

  16. An operator splitting analysis of Wasserstein--Fisher--Rao gradient flows

    Nov 22, 2025Francesca Romana Crucinio, Sahani PathirajaWasserstein Gradient FlowsFisher Information

  17. On the Gradient Complexity of Private Optimization with Private Oracles

    Nov 17, 2025Michael Menart, Aleksandar NikolovDifferential PrivacyStochastic Gradient Descent

  18. Non-Euclidean SGD for Structured Optimization: Unified Analysis and Improved Rates

    Nov 14, 2025Dmitry Kovalev, Ekaterina BorodichStochastic Gradient DescentConvergence Analysis

  19. Can Stationary Distributions of Scale-Invariant Neural Networks Be Described by the Thermodynamics of an Ideal Gas?

    Nov 10, 2025Ildus Sadrtdinov, Ekaterina Lobacheva, Ivan Klimov +3Stochastic Gradient DescentThermodynamics

  20. Adam symmetry theorem: characterization of the convergence of the stochastic Adam optimizer

    Nov 10, 2025Steffen Dereich, Thang Do, Arnulf Jentzen +1AdamStochastic Smooth Convex Optimization

  21. Exact Dynamics of Multi-class Stochastic Gradient Descent

    Oct 15, 2025Elizabeth Collins-Woodfin, Inbar SeroussiStochastic Gradient DescentCovariance Estimation

  22. A Gradient Flow Approach to Solving Inverse Problems with Latent Diffusion Models

    Sep 23, 2025Tim Y. J. Wang, O. Deniz AkyildizLatent Diffusion ModelIll-Posed Inverse Problem

  23. Graph Coloring for Multi-Task Learning

    Sep 21, 2025Santosh Patapati, Ian NoronhaMulti--Task LearningGraph Representation Learning

  24. Convergence Analysis of the ProbAbilistic Gradient Estimator Algorithm for Weakly Convex Finite-Sum Optimization

    Aug 31, 2025Laurent Condat, Peter RichtárikConvergence AnalysisConvex Sets

  25. Convergence of Stochastic Gradient Methods for Wide Two-Layer Physics-Informed Neural Networks for the Poisson Equation

    Aug 29, 2025Bangti Jin, Longjun WuConventional Physics-Informed Neural NetworksStochastic Gradient Descent

  26. Douglas-Rachford Splitting for Group-Sparse Feedback Linear-Quadratic Control

    Jul 26, 2025Lechen Feng, Xun Li, Yuan-Hua NiLinear Quadratic RegulatorSparsity

  27. Optimizing Canaries for Privacy Auditing with Metagradient Descent

    Jul 21, 2025Matteo Boglioni, Terrance Liu, Andrew Ilyas +1Differential PrivacyStochastic Gradient Descent

  28. A statistical physics framework for optimal learning

    Jul 10, 2025Francesca Mignacco, Francesco MoriStochastic Gradient DescentStatistical Mechanics

  29. On the Effectiveness of the z-Transform Method in Quadratic Optimization

    Jul 4, 2025Francis BachConvergence AnalysisZeroth-Order Optimization

  30. Local Learning Rules for Out-of-Equilibrium Physical Generative Models

    Jun 23, 2025Cyrill Bösch, Geoffrey Roeder, Marc Serra-Garcia +1Statistical MechanicsGenerative Models

  31. Scaling Textual Gradients via Sampling-Based Momentum

    May 31, 2025Zixin Ding, Junyuan Hong, Zhan Shi +6Prior Prompt Optimization TechniquesAutomatic Prompt Optimization

  32. Private Rate-Constrained Optimization with Applications to Fair Learning

    May 28, 2025Mohammad Yaghini, Tudor Cebere, Michael Menart +2Differential PrivacyFairness-Aware Learning

  33. Online simultaneous inference for quantiles via smoothed stochastic gradient descent

    May 19, 2025Likai Chen, Georg Keilbar, Wei Biao WuQuantile RegressionEstimators

  34. When majority rules, minority loses: bias amplification of gradient descent

    May 19, 2025François Bachoc, Jérôme Bolte, Ryan Boustany +1MinorityEmpirical Risk Minimization

  35. Adaptive GoGI-Skip: Coupling Goal-Gradient Importance with Dynamic Uncertainty for Efficient Reasoning

    May 13, 2025Ren ZhuangChain-of-Thought ReasoningSkip

  36. Negative Stepsizes Make Gradient-Descent-Ascent Converge

    May 2, 2025Henry Shugart, Jason M. AltschulerLarge Step SizesFlat Minima

  37. The Dynamics of Generalization in Deep Learning

    Apr 23, 2025Rubing Yang, Pratik ChaudhariOver-Parameterized Neural NetworksGradient Descent

  38. Towards Weaker Variance Assumptions for Stochastic Optimization

    Apr 14, 2025Ahmet Alacaoglu, Yura Malitsky, Stephen J. WrightStochastic Smooth Convex OptimizationStochastic Optimization

  39. Invariance Pair Guidance: Robustness to Spurious Correlations via Corrective Gradients

    Feb 26, 2025Martin Surner, Abdelmajid Khelil, Ludwig BothmannTime-Step-Wise Learnable Surrogate Gradient FunctionSpurious Correlations

  40. Preconditioned Inexact Stochastic ADMM for Deep Model

    Feb 15, 2025Shenglong Zhou, Ouya Wang, Ziyan Luo +2Stochastic Gradient DescentAlternating Direction Method Of Multiplier

  41. Derivation of effective gradient flow equations and dynamical truncation of training data in Deep Learning

    Jan 13, 2025Thomas ChenGradient DescentWasserstein Gradient Flows

  42. Inclusive KL Gradient Flows: Otto-Wasserstein, Fisher-Rao-Gaussian, and Local-Estimator Dynamics

    Oct 31, 2024Jia-Jie ZhuWasserstein Gradient FlowsVariational Inference

  43. A Mechanistic Study of Transformers Training Dynamics

    Oct 31, 2024Ambroise Odonnat, Wassim Bouaziz, Vivien CabannesTransformer ArchitecturesModel Pretraining

  44. Koopman-informed recurrent neural networks

    Oct 30, 2024Erik Lien Bolager, Ana Čukarska, Iryna Burak +2Recurrent Neural NetworksKoopman Operator

  45. Local Flow Matching Generative Models

    Oct 3, 2024Chen Xu, Xiuyuan Cheng, Yao XieGenerative Flow NetworksConditional Flow Matching

  46. Improved Finite-Particle Convergence Rates for Stein Variational Gradient Descent

    Sep 13, 2024Sayan Banerjee, Krishnakumar Balasubramanian, Promit GhosalGradient DescentConvergence

  47. Learning rate adaptive stochastic gradient descent optimization methods: numerical simulations for deep learning methods for partial differential equations and convergence analyses

    Jun 20, 2024Steffen Dereich, Arnulf Jentzen, Adrian RiekertStochastic Gradient DescentAdam

  48. Gradient Descent on Logistic Regression with Non-Separable Data and Large Step Sizes

    Jun 7, 2024Si Yi Meng, Antonio Orvieto, Daniel Yiming Cao +1Large Step SizesGradient Descent

  49. On Regularization via Early Stopping for Least Squares Regression

    Jun 6, 2024Rishi Sonthalia, Jackie Lok, Elizaveta RebrovaKernel Ridge RegressionEarly Stopping

  50. The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication

    May 19, 2024Kumar Kshitij Patel, Margalit Glasgow, Ali Zindari +5Mini-Batch Stochastic Gradient DescentStochastic Gradient Descent

  51. Fisher-Rao Gradient Flows of Linear Programs and State-Action Natural Policy Gradients

    Mar 28, 2024Johannes Müller, Semih Çaycı, Guido MontúfarWasserstein Gradient FlowsGradient Flow

  52. A Functional-Space Mean-Field Theory of Partially-Trained Three-Layer Neural Networks

    Oct 28, 2022Zhengdao Chen, Eric Vanden-Eijnden, Joan BrunaTwo-Layer Neural NetworksMean-Field Limit

  53. Learning Non-Vacuous Generalization Bounds from Optimization

    Jun 9, 2022Chengli Tan, Jiangshe Zhang, Junmin Liu +1Generalization BoundsStochastic Gradient Descent

  54. Privacy-Preserving Logistic Regression Training with A Faster Gradient Variant

    Jan 26, 2022John ChiangPrivacy-Preserving FrameworkGradient Descent

  55. Attention by Synchronization in Coupled Oscillator Networks

    Date pendingFabio Pasqualetti, Taosha GuoOscillatory DynamicsActivation Functions

  56. Can SGD Select Good Fishermen? Local Convergence under Self-Selection Biases

    Date pendingAlkis Kalavasis, Anay Mehrotra, Felix ZhouEstimatorsOptimal Sample Complexity

  57. Generalization Guarantees on Data-Driven Tuning of Gradient Descent with Langevin Updates

    Date pendingSaumya Goyal, Rohith Rongali, Ritabrata Ray +1Gradient DescentHyperparameter Optimization

  58. Prediction--Loss Alignment for Sampler--Robust Flow Matching Training

    Date pendingJiadong Hong, Lei Liu, Xinyu Bian +2Diffusion SamplingStochastic Gradient Descent

  59. Adapt or Forget: Provable Tradeoffs Between Adam and SGD in Nonstationary Optimization

    Date pendingSharan Sahu, Abir Sarkar, Cameron J. Hogan +1AdamStationarity

  60. REAL-Q: E2E LLM Quantization via Dynamic Gradient Descent

    Date pendingQian Zhang, Yaoming Li, Zhewen Tan +9Large Language Model QuantizationWeight-Only Quantization