Stochastic Gradient Descent

Follow this field Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

21 new papers

A weekly snapshot of new work published in Stochastic Gradient Descent.

Period ending 2026-09-14

17 new papers

A weekly snapshot of new work published in Stochastic Gradient Descent.

Period ending 2026-09-07

12 new papers

A weekly snapshot of new work published in Stochastic Gradient Descent.

Inside this field

Focused directions

Focused topic

Stochastic Gradient Descent

Within Stochastic Gradient Descent

175 papersExplore

Focused topic

Gradient Descent

Within Stochastic Gradient Descent

155 papersExplore

Focused topic

Gradient Flow

Within Stochastic Gradient Descent

78 papersExplore

Focused topic

Gradient-Based Optimization

Within Stochastic Gradient Descent

58 papersExplore

Focused topic

Wasserstein Gradient Flows

Within Stochastic Gradient Descent

55 papersExplore

Focused topic

Gradient Concentration

Within Stochastic Gradient Descent

48 papersExplore

Focused topic

Mini-Batch Stochastic Gradient Descent

Within Stochastic Gradient Descent

24 papersExplore

541 papers

Latest in Stochastic Gradient Descent

Browse all topics →

On the Stability of Spherical Hellinger-Kantorovich Flows and Their Implications for Differential Privacy
May 22, 2026Aratrika Mustafi, Soumya MukherjeeDifferential Privacy Wasserstein Gradient Flows
Move on Muon : A Hamiltonian probability gradient flow perspective of Muon optimizer
May 22, 2026Aratrika Mustafi, Soumya Mukherjee, Bharath K. SriperumbudurMuon Optimizer Muon
B-GRTO: Bootstrapped Group Relative Tool Optimization for Referring Segmentation
May 22, 2026Mario Markov, Stefan Maria Ailuro, Mohammad Mahdi +2Agentic Optimization Group
Operationalizing Individual Fairness via Gradient Descent and Bradley-Terry Models
May 22, 2026Conlan Olson, Linjun Zhang, Zhun Deng +1Gradient Descent Hyperparameter Optimization
Steered Generation via Gradient-Based Optimization on Sparse Query Features
May 21, 2026Sumanta Bhattacharyya, Pedram RooshenasActivation Steering Steering
Why SGD is not Brownian Motion: A New Perspective on Stochastic Dynamics
May 21, 2026Igor Ignashin, Anna Radovskaya, Andrew Semenov +7Stochastic Gradient Descent Stochastic Differential Equations
A note on convergence of Wasserstein policy optimization
May 21, 2026David Šiška, Yufei ZhangWasserstein Gradient Flows Markov Decision Processes
Implicit Regularization of Mini-Batch Training in Graph Neural Networks
May 21, 2026Clement Wang, Antoine Vialle, Robin Vaysse +1Graph Neural Networks Mini-Batch Stochastic Gradient Descent
Uniform-in-Time Weak Propagation-of-Chaos in Shallow Neural Networks
May 21, 2026Margalit Glasgow, Joan BrunaTwo-Layer Neural Networks Wasserstein Gradient Flows
Large-Step Training Dynamics of a Two-Factor Linear Transformer Model
May 20, 2026Krishnakumar BalasubramanianTransformer Architectures Gradient Descent
A Rigorous, Tractable Measure of Model Complexity
May 20, 2026Oskar Allerbo, Thomas B. SchönModel Complexity Random Forest
A Typed Tensor Language for Federated Learning
May 20, 2026Theofilos Mailis, Kalliopi-Christina Despotidou, Konstantinos Filippopolitis +6Federated Learning Tensor Programs
LOSCAR-SGD: Local SGD with Communication-Computation Overlap and Delay-Corrected Sparse Model Averaging
May 20, 2026Yassine Maziane, Ammar Mahran, Artavazd Maranjyan +1Distributed Optimization Federated Learning
SMA-DP: Spectral Memory-Aware Differential Privacy for Deep Learning
May 19, 2026Mohammad Partohaghighi, Roummel MarciaDifferential Privacy Stochastic Gradient Descent
SDM: A Powerful Tool for Evaluating Model Robustness
May 19, 2026Xinlei Liu, Tao Hu, Jichao Xie +3Adversarial Examples Stochastic Gradient Descent
Training Neural Networks with Optimal Double-Bayesian Learning
May 19, 2026Vy Bui, Hang Yu, Karthik Kantipudi +2Learning Rates Stochastic Gradient Descent
From SGD to Muon: Adaptive Optimization via Schatten-p Norms
May 19, 2026Thomas Massena, Corentin Friedrich, Mathieu SerrurierMuon Optimizer Optimizer Design
Increasing Missingness to Reduce Bias: Richardson-SGD with Missing Data
May 19, 2026Ferdinand Genans, Erwan ScornetMissing Data Imputation
Adynamical systems view of training generativemodels and the memorization phenomenon
May 19, 2026Siva Athreya, Chiranjib Bhattacharya, Vivek S. BorkarStochastic Gradient Descent Memorization
Implicit Bias of Mirror Flow in Homogeneous Neural Networks: Sparse and Dense Feature Learning
May 19, 2026Tom Jacobs, Guido MontufarFeature Learning Gradient Flow
OmniISR: A Unified Framework for Centralized and Federated Learning via Intermediate Supervision and Regularization
May 19, 2026Wei-Bin Kou, Guangxu Zhu, Ming Tang +4Federated Learning Heterogeneous Edge Devices
Factor Augmented High-Dimensional SGD
May 19, 2026Shubo Li, Yuefeng Han, Xiufan YuStochastic Gradient Descent Factor
Provably Data-driven Lagrangian Relaxation for Mixed Integer Linear Programming
May 18, 2026Tung Quoc Le, Anh Tuan Nguyen, Viet Anh NguyenMixed-Integer Programming Semidefinite Programming
Can Adaptive Gradient Methods Converge under Heavy-Tailed Noise? A Case Study of AdaGrad
May 18, 2026Zijian LiuConvergence Analysis Adam
Perfect Parallelization in Mini-Batch SGD with Classical Momentum Acceleration
May 18, 2026Sachin Garg, Michał DerezińskiMini-Batch Stochastic Gradient Descent Stochastic Gradient Descent
Scale-Invariant Neural Network Optimization: Norm Geometry and Heavy-Tailed Noise
May 18, 2026Jiayu Zhang, Tianyi LinNeural Network Optimization Stochastic Gradient Descent
The Symmetries of Three-Layer ReLU Networks
May 18, 2026Johanna Marie Gegenfurtner, Moritz Grillo, Guido MontúfarRectified Linear Unit Networks Symmetry
Canonical Regularisation of Wide Feature-Learning Neural Networks
May 18, 2026George Whittle, Pranav Vaidhyanathan, Juliusz Ziomek +2Feature Learning Regularization Techniques
New Insight of Variance reduce in Zero-Order Hard-Thresholding: Mitigating Gradient Error and Expansivity Contradictions
May 18, 2026Xinzhe Yuan, William de Vazelhes, Bin Gu +1Variance Reduction Zeroth-Order Optimization
Scenario Generation in Roundabouts with Adjustable Interaction Intensity
May 18, 2026Li Li, Till Temmen, Tobias Brinkmann +3Scenario Generation Safety-Critical Scenarios
Adaptive Stochastic Natural Gradient Method for Safe Optimization on Binary Space
May 18, 2026Kento Uchida, Ryoki Hamano, Masahiro Nomura +1Evolutionary Search Stochastic Gradient Descent
Efficient Bilevel Optimization for Meta Label Correction in Noisy Label Learning
May 18, 2026Ba Hoang Anh Nguyen, Viet Cuong TaNoisy Labels Bilevel Optimization
A Unified Framework for Data-Free One-Step Sampling via Wasserstein Gradient Flows
May 18, 2026Chenguang Wang, Tianshu YuWasserstein Gradient Flows Diffusion Sampling
Feature Learning in Linear-Width Two-Layer Networks: Two vs. One Step of Gradient Descent
May 18, 2026Behrad Moniri, Hamed HassaniTwo-Layer Neural Networks Feature Learning
Form and Function: Machine Unlearning as a Problem of Misaligned States
May 17, 2026Kennon StewartMachine Unlearning Learning-Augmented Algorithms
FishBack: Pullback Fisher Geometry for Optimal Activation Steering in Transformers
May 17, 2026Sihan Wang, Jiayi ZhaoActivation Steering Steering
High-dimensional Limit of SGD for Diagonal Linear Networks
May 16, 2026Begoña García Malaxechebarría, Courtney Paquette, Maryam Fazel +1Stochastic Gradient Descent Stochastic Differential Equations
Global Convergence of Sampling-Based Nonconvex Optimization through Diffusion-Style Smoothing
May 15, 2026Zeji Yi, Chaoyi Pan, Guanya Shi +1Convergence Analysis Smoothing
Multi-Headed Transformer Architectures as Time-dependent Wasserstein Gradient Flows
May 15, 2026Alex Massucco, Leonardo Del Grande, Marcello Carioni +2Transformer Architectures Wasserstein Gradient Flows
Accelerated Gradient Descent for Faster Convergence with Minimal Overhead
May 15, 2026Manuel Graca, L. Miguel Silveira, Arlindo Oliveira +1Mini-Batch Stochastic Gradient Descent Gradient Descent
Grokking as Structural Inference: Transformers Need Bayesian Lottery Tickets
May 15, 2026Kai Hidajat, Solden Stoll, Joseph AnAttention-Based Models Transformer Architectures
Rethinking Neural Network Learning Rates: A Stackelberg Perspective
May 15, 2026Sihan Zeng, Sujay Bhatt, Sumitra GaneshLearning Rates Two-Layer Neural Networks
$φ$ -Balancing for Mixture-of-Experts Training
May 14, 2026Lizhang Chen, Jonathan Li, Qi Wang +5Mixture-of-Experts Models Balanced Learning
Beyond Bounded Variance: Variance-Reduced Normalized Methods for Nonconvex Optimization under Blum-Gladyshev Noise
May 14, 2026Antesh Upadhyay, Arda Fazla, Abolfazl HashemiStochastic Smooth Convex Optimization Convex Optimization
Gradient-Free Training of Spiking Neural Networks via Low-Rank Evolution Strategies
May 14, 2026Dhruv Patankar, Sachit Ramesha GowdaSpiking Neural Networks Neuromorphic Hardware
Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity
May 13, 2026Ammar Mahran, Artavazd Maranjyan, Peter RichtárikStochastic Gradient Descent Distributed Optimization
Genetic algorithm vs. gradient descent for training a neural network architecture dedicated to low data regimes in small medical datasets
May 13, 2026Amine Boukhari, Boglarka Ecsedi, Laszlo Papp +1Neural Architectures Gradient Descent
State-of-art minibatches via novel DPP kernels: discretization, wavelets, and rough objectives
May 13, 2026Hoang-Son Tran, Pranav Gupta, Rémi Bardenet +1Mini-Batch Stochastic Gradient Descent Subsampling
The Mechanism of Weak-to-Strong Generalization: Feature Elicitation from Latent Knowledge
May 13, 2026Ryoya Awano, Taiji SuzukiFeature Learning Weak Supervision
AGOP as Explanation: From Feature Learning to Per-Sample Attribution in Image Classifiers
May 12, 2026Raj Kiran Gupta KatakamTraining Data Attribution Feature Learning
Inference-Time Machine Unlearning via Gated Activation Redirection
May 12, 2026Vinícius Conte Turani, Otávio Parraga, João Vitor Boer Abitante +7Training-Free Inference Framework Inference-Time
State-Space NTK Collapse Near Bifurcations
May 12, 2026James Hazelden, Eric Shea-BrownNeural Tangent Kernel Bifurcation
Population Risk Bounds for Kolmogorov-Arnold Networks Trained by DP-SGD with Correlated Noise
May 12, 2026Puyu Wang, Jan Schuchardt, Nikita Kalinin +4Differential Privacy Stochastic Gradient Descent
OmniNFT: Modality-wise Omni Diffusion Reinforcement for Joint Audio-Video Generation
May 12, 2026Guohui Zhang, XiaoXiao Ma, Jie Huang +9Joint Audio-Video Generation Pre-Trained Video Diffusion Models
A proximal gradient algorithm for composite log-concave sampling
May 12, 2026Linghai Liu, Sinho ChewiNon-Log-Concave Proximal Gradient Descent
Not How Many, But Which: Parameter Placement in Low-Rank Adaptation
May 12, 2026Arijit Sehanobish, Charles LoveringLow-Rank Adaptation Adapters Gradient Concentration
Emergence of Frontier Superposition: Möbius attractor and Cascade Supervision
May 12, 2026Hongyu Gu, Jingwen FuSingle Forward Pass Gradient Concentration
Holder Policy Optimisation
May 12, 2026Yuxiang Chen, Dingli Liang, Yihang Chen +8Group Relative Policy Optimization Gradient Concentration
Sobolev Regularized MMD Gradient Flow
May 12, 2026Chenyang Tian, Bharath K. Sriperumbudur, Arthur Gretton +1Maximum Mean Discrepancy Wasserstein Gradient Flows
Multi-Timescale Conductance Spiking Networks: A Sparse, Gradient-Trainable Framework with Rich Firing Dynamics for Enhanced Temporal Processing
May 12, 2026Alex Fulleda-Garcia, Saray Soldado-Magraner, Josep Maria Margarit-TauléSpiking Neural Networks Binary Spikes

May 22, 2026 · stat.MLJ/K move · Enter open · S save

On the Stability of Spherical Hellinger-Kantorovich Flows and Their Implications for Differential Privacy

Aratrika Mustafi, Soumya Mukherjee

Department of Statistics, Pennsylvania State University

Gradient-flow sampling interprets a Gibbs distribution as the minimizer of an energy functional over probability measures and generates dynamics converging to this target. Under spherical Hellinger-Kantorovich (SHK) geometry, the flow couples transport and reaction and coincides with birth-death Langevin dynamics. In this work, we develop a perturbation theory for SHK gradient flows. For two potentials

V

and

V^{\prime}

, we compare the associated flows from a common initialization and quantify how potential discrepancies propagate over time. A uniform perturbation bound yields dimension-free, pointwise control of the log-likelihood ratio and Rényi divergence, while additional structure allows us to derive bounds for the KL divergence as well. We apply these results to approximate sampling for the exponential mechanism in differential privacy. The likelihood-ratio control provides explicit time-dependent Pure-DP guarantees for SHK-based samplers, while the KL bound yields Approximate-DP certificates via hockey-stick divergence. We also derive a utility bound separating intrinsic exponential-mechanism suboptimality from finite-time sampling error.

Differential Privacy Wasserstein Gradient Flows Langevin Dynamics Optimality Perturbations

Open paper Ask about it PDF