Stochastic Gradient Descent

Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

21 new papers

A weekly snapshot of new work published in Stochastic Gradient Descent.

Period ending 2026-09-14

17 new papers

A weekly snapshot of new work published in Stochastic Gradient Descent.

Period ending 2026-09-07

12 new papers

A weekly snapshot of new work published in Stochastic Gradient Descent.

Inside this field

Focused directions

541 papers

Latest in Stochastic Gradient Descent

Browse all topics →
CardsList

  1. Efficient Tuning Before Low-Bit Post-Training Quantization for Stochastic Gradient Descent-optimized Models

    Jul 13, 2026Peng Xia, Junbiao Pang, Muhammad Ayub SabirWeight-Only QuantizationQuantization

  2. LayerNorm as Implicit Gain Control in Looped Transformers

    Jul 12, 2026Matthias M. M. BuehlmaierBatch NormalizationTransformer Layers

  3. Conditional Optimal Bridge for Riemannian Activation Steering

    Jul 12, 2026Seyed Arshan Dalili, Ajay Narayanan Sridhar, Vijaykrishnan Narayanan +1Activation SteeringModel Activations

  4. Sharper Analysis of Single-Loop Methods for Bilevel Optimization

    Jul 11, 2026Yubo Zhou, Jun Shu, Luo Luo +4Bilevel OptimizationConvergence Analysis

  5. Understanding Schedule-Free Methods in Nonconvex Optimization: Rate Guarantees and Escaping Saddles

    Jul 10, 2026Jiseok Chae, Donghwan KimConvex OptimizationConvergence Analysis

  6. Learning-enabled Parameter Synthesis for Nonlinear Systems from Signal Temporal Logic

    Jul 9, 2026Alex Beaudin, Hanna Krasowski, Eric Palanques-Tost +2Signal Temporal LogicGradient-Based Optimization

  7. Beyond Backpropagation: Monte Carlo Method Can Train Deep Neural Networks

    Jul 9, 2026Hong ZhaoBackpropagationNeural Network

  8. Dynamics of Gradient Descent with Large Step Size Near a Manifold of Flat Minima

    Jul 9, 2026Lachlan Ewen MacDonald, René VidalGradient DescentCondition Number

  9. Vanilla SGD with Momentum Survives Heavy-Tailed Noise: Convergence Analysis without Gradient Clipping or Normalization

    Jul 9, 2026Ryusei Yamada, Naoki Sato, Hideaki IidukaStochastic Gradient DescentConvergence Analysis

  10. GradInf: Gradient Estimation as Probabilistic Inference

    Jul 8, 2026Gaurav Arya, Mathieu Huot, Moritz Schauer +2Probabilistic InferenceGradient Flow

  11. Causal Optimizer Interaction Calculus: Hidden Geometric Relaxation and Identifiable Interventions

    Jul 8, 2026Zavier LiInformation-Geometric OptimizationGradient Concentration

  12. Widest-Path Reachability Fields for Connectivity-Preserving Slender Structure Segmentation

    Jul 8, 2026Youcheng Zong, Runda Jia, Minxuan Hu +2Gradient ConcentrationReachability

  13. Hybrid Least Squares/Gradient Descent Methods for MIONets

    Jul 8, 2026Jun Choi, Chang-Ock Lee, Minam MoonGradient DescentLeast Squares

  14. Converge to Surprise: Evolutionary Self-supervised Image Clustering

    Jul 8, 2026Canlin Zhang, Xiuwen LiuImage ClusteringCluster Structure

  15. Dithered Gaussian Mechanism for Randomness-Efficient Differential Privacy

    Jul 7, 2026Nikita P. Kalinin, Rasmus PaghDifferential PrivacyRandom

  16. Leveraging Extragradient for Effective Sharpness-Aware Minimization in Deep Learning

    Jul 7, 2026Yao Fu, Chunxia Zhang, Junmin Liu +3Stochastic Gradient DescentSharpness-Aware Minimization

  17. K-ABENA: K-Adaptive Backpropagation with Error-based N-exclusion Algorithm : (Compensated Loss-Based Sample Exclusion with Unbiased Gradient Estimation)

    Jul 7, 2026Jean-Francois BonbhelStochastic Gradient DescentGradient Descent

  18. Differentially Private Natural Gradient Descent

    Jul 7, 2026Pan Li, Kai Chen, Shuai Chang +3Differential PrivacyStochastic Gradient Descent

  19. The Map Behind the Flow: Finite-Step Gradient Descent as a Dynamical System

    Jul 6, 2026Thomas HofmannGradient DescentWasserstein Gradient Flows

  20. Non-asymptotic Convergence of Stochastic Gradient Descent in Score-based Generative Models

    Jul 6, 2026Stanislas Strasman, Sobihan Surendran, Sylvain Le CorffStochastic Gradient DescentGenerative Models

  21. Wasserstein Residuals: Learning Gradient Flows from Population Dynamics

    Jul 6, 2026Markus Heinonen, Yair Shenfeld, Ricardo Baptista +4Wasserstein Gradient FlowsWasserstein Distance

  22. Unified convergence analysis for gradient descent optimization methods in the training of deep neural networks

    Jul 5, 2026Shokhrukh Ibragimov, Arnulf JentzenNeural Network OptimizationAdam

  23. Implicit Bias of SGD in Multivariate ReLU Networks: Effective Width Collapse

    Jul 3, 2026Shuang Liang, Tom Jacobs, Guido MontúfarRectified Linear Unit NetworksStochastic Gradient Descent

  24. Can Model Merging Improve Aggregation in DiLoCo?

    Jul 3, 2026Stefan Horoi, Benjamin Thérien, Guy Wolf +1Continual Model MergingModel Merging

  25. Homogenization of ℓ2\ell_2-Adversarial Training in High-Dimensions: Exact Dynamics under Stochastic Gradient Descent

    Jun 30, 2026Fabrizzio SabelliStochastic Gradient DescentAdversarial Transferability

  26. Random Reshuffling Dominates Stochastic Gradient Descent

    Jun 30, 2026Zijian LiuStochastic Gradient DescentConvergence Analysis

  27. Revisiting the Volume Hypothesis

    Jun 30, 2026Ari Pakman, Lior Kreimer, Yakir BerchenkoStochastic Gradient DescentGeneralisation

  28. SGD at the Edge of Stability: Stochastic Stabilization with Large Learning Rates

    Jun 29, 2026Konstantinos Emmanouilidis, Lachlan MacDonald, Salma Tarmoun +1Stochastic Gradient DescentLearning Rates

  29. Gradient Smoothing: Coupling Layer-wise Updates for Improved Optimization

    Jun 29, 2026Haoming Meng, Anton Sugolov, Vardan PapyanGradient FlowTransformer Layers

  30. Why can genetic algorithms work in high-dimensional search spaces?

    Jun 29, 2026Stephen WhitelamGenetic AlgorithmsGradient Descent

  31. ITSPACE: Monotone Gaussian Optimal Transport Updates

    Jun 29, 2026Woojoo Na, Jennifer DyOptimal TransportCovariance Estimation

  32. Muon learns balanced solutions in matrix factorization without slow saddle-to-saddle dynamics

    Jun 29, 2026Mark Rhee, Jamie Simon, Dhruva KarkadaMuon OptimizerMatrix Factorization

  33. Curvature-Weighted Gradient Diversity: A Noise Measure for Geometry-Adaptive SGD Schedules

    Jun 29, 2026Muhammad Hamza, Ayush GoelStochastic Gradient DescentMini-Batch Stochastic Gradient Descent

  34. SGD Provably Prioritizes a Shortcut Spurious Feature in the XOR Model

    Jun 29, 2026Tyler LaBonte, Vidya MuthukumarFeature LearningRectified Linear Unit Networks

  35. ACPO: Agent-Chained Policy Optimization for Multi-Agent Reinforcement Learning

    Jun 29, 2026Daiki E. Matsunaga, Junho Na, Tri Wahyu Guntara +4Multi-Agent Reinforcement LearningDecentralization

  36. How AI settled the complexity of the oldest SGD algorithm

    Jun 28, 2026Michał Dereziński, Xiaoyu DongStochastic Gradient DescentWorst-Case

  37. Knowing in Advance When an Evolutionary Outer Loop Will Not Help: A Pre-Registered Cheap-Baseline Screening Rule

    Jun 28, 2026Ramchand KumaresanScreeningEarly Stopping

  38. Compositional Dynamics in Learning and Mechanics

    Jun 27, 2026David I. SpivakGradient-Based OptimizationHamiltonian Physics

  39. Variance Reduction for Stochastic Gradient Generalized Non-reversible Langevin Monte Carlo Algorithms

    Jun 27, 2026Bingye Ni, Xiaoyu Wang, Yingli Wang +1Langevin DynamicsVariance Reduction

  40. Closed-Form Steepest Descent Direction toward Flat Minima: Reducing Upper Bounds on the Loss Hessian Eigenspectrum in Neural Networks

    Jun 27, 2026Yuto Omae, Kazuki Sakai, Yohei Kakimoto +3Condition NumberGradient Descent

  41. Replica Symmetry Breaking and Algorithmic Thresholds in Empirical Risk Minimization under Multi-Index Model

    Jun 26, 2026Andrea Montanari, Kangjie ZhouEmpirical Risk MinimizationOptimal Sample Complexity

  42. GBC: Gradient-Based Connections for Optimizing Multi-Agent Systems

    Jun 26, 2026Xiaocheng Yang, Abdulrahman Alrabah, Dilek Hakkani-Tür +1Multi-Agent SystemsCredit Assignment

  43. Mosaic: A Benchmark Suite for Differentiable Physics Solvers

    Jun 26, 2026Andrin Rehmann, Heiko Zimmermann, Dion HäfnerPartial Differential Equation BenchmarksMosaic

  44. Difference of Convex Programming in the Wasserstein Space with Applications to MMD Optimization

    Jun 26, 2026Clément Bonet, Pierre-Cyril Aubin-Frankowski, Youssef MrouehWasserstein DistanceMaximum Mean Discrepancy

  45. Fast algorithms for learning a Gaussian under halfspace truncation with optimal sample complexity

    Jun 25, 2026Haitong Liu, Deepak Narayanan Sridharan, David Steurer +1Optimal Sample ComplexityGaussian Mixture Models

  46. Stochastic Gradient Optimization with Model-Assisted Sampling

    Jun 25, 2026Jonne Pohjankukka, Jukka HeikkonenStochastic Gradient DescentMini-Batch Stochastic Gradient Descent

  47. Parametric Open Source Games

    Jun 25, 2026Aleksandar Todorov, Jesse ten Napel, Alexander MüllerGame TheoryNash Equilibrium

  48. Geometric Gradient Rectification for Safe Open-Set Semi-Supervised Learning

    Jun 25, 2026Jiahe Chen, Qian Shao, Qiyuan Chen +4Semi-Supervised LearningRectification

  49. Escaping Iterative Parameter-Space Noise: Differentially Private Learning with a Hypernetwork

    Jun 25, 2026Naoki Nishikawa, Shokichi Takakura, Satoshi HasegawaDifferential PrivacyStochastic Gradient Descent

  50. Zero-Shot Size Transfer for Neural ODEs on Sparse Random Graphs: Graphon Limits and Adjoint Convergence

    Jun 25, 2026Mingsong Yan, Zhida Wang, Sui TangNeural Ordinary Differential EquationsGraph Neural Networks

  51. Sketched Linear Contrastive Learning: Approximation, Optimization, and Statistical Scaling

    Jun 25, 2026Ziyan Chen, Zhongzhu Zhou, Ding-Xuan ZhouContrastive LearningScaling Laws

  52. High-Probability PL-SGD with Markovian Noise: Optimal Mixing and Tail Dependence

    Jun 24, 2026Dhruv Sarkar, Aprameyo Chakrabartty, Vaneet AggarwalStochastic Gradient DescentLong-Tailed Distribution

  53. \chisao{}: A GPU-Native Parallel Optimizer for Multimodal Black-Box Functions via Convergence-Anticonvergence Oscillation

    Jun 24, 2026Ira WolfsonBlack-Box OptimizationMulti-Gpu Systems

  54. Revealing Training Data Exposure in Vision Language Large Models via Parameter Gradients

    Jun 23, 2026Zhihao Zhu, Hongyi Tang, Yi Yang +1Vision-LanguageTraining Data

  55. The Geometry Behind Diffusion and Flow Matching: Gradient Flows and Geodesics in Wasserstein Space

    Jun 23, 2026Yian Yao, Weiwei ZhangWasserstein Gradient FlowsWasserstein Distance

  56. Convergence of Gradient Descent for General Neural Network Architectures Beyond the NTK Regime

    Jun 22, 2026Yuqing WangNeural Tangent KernelGradient Descent

  57. FORGE: Fused On-Register Gradient Elimination for Memory-Efficient LLM Training

    Jun 22, 2026Dikshant Kukreja, Kritarth Prasad, Avinash Anand +6Gradient FlowOptimizer Design

  58. GRAIN: Group Aggregation via Min-Norm Objective

    Jun 22, 2026Nghia Bui, Jiarui Yao, Lijing WangMini-Batch Stochastic Gradient DescentLarge Models

  59. Generative Robust Optimisation

    Jun 21, 2026Yuhui Yin, Vassilis M. CharitopoulosRobust OptimizationGenerative Models

  60. Federated learning with heavy-tailed gradient noise and communication noise: a variance-reduction based algorithm

    Jun 21, 2026Shengchao Zhao, Yongchao LiuFederated LearningFedavg