Stochastic Gradient Descent

Follow this field Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

21 new papers

A weekly snapshot of new work published in Stochastic Gradient Descent.

Period ending 2026-09-14

17 new papers

A weekly snapshot of new work published in Stochastic Gradient Descent.

Period ending 2026-09-07

12 new papers

A weekly snapshot of new work published in Stochastic Gradient Descent.

Inside this field

Focused directions

Focused topic

Stochastic Gradient Descent

Within Stochastic Gradient Descent

175 papersExplore

Focused topic

Gradient Descent

Within Stochastic Gradient Descent

155 papersExplore

Focused topic

Gradient Flow

Within Stochastic Gradient Descent

78 papersExplore

Focused topic

Gradient-Based Optimization

Within Stochastic Gradient Descent

58 papersExplore

Focused topic

Wasserstein Gradient Flows

Within Stochastic Gradient Descent

55 papersExplore

Focused topic

Gradient Concentration

Within Stochastic Gradient Descent

48 papersExplore

Focused topic

Mini-Batch Stochastic Gradient Descent

Within Stochastic Gradient Descent

24 papersExplore

541 papers

Latest in Stochastic Gradient Descent

Browse all topics →

One-Step Generative Modeling via Wasserstein Gradient Flows
May 12, 2026Jiaqi Han, Puheng Li, Qiushan Guo +3Discrete Diffusion Model Generative Models
A Mixture Autoregressive Image Generative Model on Quadtree Regions for Gaussian Noise Removal via Variational Bayes and Gradient Methods
May 12, 2026Shota Saito, Yuta Nakahara, Kohei Horinouchi +3Image Denoising Autoregressive Image Generation
Metric-Gradient Projection for Stable Multi-Agent Policy Learning
May 12, 2026Zuyuan Zhang, Sizhe Tang, Mahdi Imani +1Multi-Agent Reinforcement Learning Wasserstein Gradient Flows
Gradient-Free Noise Optimization for Reward Alignment in Generative Models
May 12, 2026Jeongsol Kim, Hongeun Kim, Jian Wang +1Reward Gradients Gradient-Based Optimization
Policy Gradient Methods for Non-Markovian Reinforcement Learning
May 11, 2026Avik Kar, Siddharth Chandak, Rahul Singh +4Markov Decision Processes Policy Gradient
On the global convergence of gradient descent for wide shallow models with bounded nonlinearities
May 11, 2026Romain Petit, Clarice Poon, Gabriel PeyréTwo-Layer Neural Networks Gradient Descent
Kernel-Gradient Drifting Models
May 11, 2026Maria Esteban-Casadevall, Jorge Carrasco-Pollo, Max Welling +3Wasserstein Gradient Flows Generative Models
Exact Fixed-Point Constraints in Neural-ODEs with Provable Universality
May 11, 2026Feliciano Giuseppe Pacifico, Duccio Fanelli, Lorenzo Buffoni +3Neural Ordinary Differential Equations Fixed-Point Iteration
DP-LAC: Lightweight Adaptive Clipping for Differentially Private Federated Fine-tuning of Language Models
May 11, 2026Haaris Mehmood, Jie Xu, Karthikeyan Saravanan +2Federated Learning Differential Privacy
The Benefits of Temporal Correlations: SGD Learns k-Juntas from Random Walks Efficiently
May 11, 2026Elisabetta Cornacchia, Dan Mikulincer, Elchanan MosselSpatio-Temporal Correlations Stochastic Gradient Descent
Extended Wasserstein-GAN Approach to Causal Distribution Learning: Density-Free Estimation and Minimax Optimality
May 11, 2026Shu Tamano, Masaaki ImaizumiCausal Inferences Conditional Distribution
Unveiling High-Probability Generalization in Decentralized SGD
May 11, 2026Jiahuan Wang, Ping Luo, Ziqing Wen +2Stochastic Gradient Descent Generalization Bounds
Deep Learning under Fractional-Order Differential Privacy
May 11, 2026Mohammad Partohaghighi, Roummel MarciaDifferential Privacy Stochastic Gradient Descent
Flag Varieties: A Geometric Framework for Deep Network Alignment
May 11, 2026Jingchuan Xiao, Xinyi Sui, Cihan RuanNeural Collapse Orthogonality
Pretraining large language models with MXFP4 on Native FP4 Hardware
May 11, 2026Musa Cim, Poovaiah Palangappa, Miro Hodak +3W4A4 Quantization Fp8
Parameter-Efficient Neuroevolution for Diverse LLM Generation: Quality-Diversity Optimization via Prompt Embedding Evolution
May 10, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuLarge Language Models(Llms Diversity
EvoPref: Multi-Objective Evolutionary Optimization Discovers Diverse LLM Alignments Beyond Gradient Descent
May 10, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuLarge Language Model Alignment Evolutionary Search
Dual-Path Hyperprior Informed Deep Unfolding Network for Image Compressive Sensing
May 10, 2026Tianyi Lu, Wenxue Cui, Shaohui LiuDeep Unfolding Networks Sparse Recovery
Phases of Muon: When Muon Eclipses SignSGD
May 10, 2026Elliot Paquette, Noah Marshall, Lucas Benigni +3Muon Optimizer Muon
Outlier-Robust Diffusion Solvers for Inverse Problems
May 10, 2026Yang Zheng, Jiahua Liu, Tongyao Pang +2Ill-Posed Inverse Problem Inverse Problem
Quantitative Local Convergence of Mean-Field Stein Variational Gradient Flow
May 10, 2026Lénaïc Chizat, Maria Colombo, Roberto Colombo +1Wasserstein Gradient Flows Convergence Analysis
Transfer Learning of Multiobjective Indirect Low-Thrust Trajectories Using Diffusion Models and Markov Chain Monte Carlo
May 9, 2026Jannik Graebner, Ryne BeesonTrajectory Optimization Multi-Objective Optimization
Probability-Flow Distillation: Exact Wasserstein Gradient Flow for High-Fidelity 3D Generation
May 9, 2026Rohith Ramanan, A. N. Rajagopalan3D Generative Models Diffusion Priors
Towards Scalable Persistence-Based Topological Optimization
May 9, 2026Abderrahim Bendahi, Alexandre Duplessis, Arnaud FickingerTopology Optimization Persistent Homology
Rennala MVR: Improved Time Complexity for Parallel Stochastic Optimization via Momentum-Based Variance Reduction
May 9, 2026Zhirayr Tovmasyan, Artavazd Maranjyan, Peter RichtárikVariance Reduction Stochastic Gradient Descent
A Call to Lagrangian Action: Learning Population Mechanics from Temporal Snapshots
May 8, 2026Vincent Guan, Lazar Atanackovic, Kirill NeklyudovLagrangian Methods Population Dynamics
INO-SGD: Addressing Utility Imbalance under Individualized Differential Privacy
May 8, 2026Xiao Tian, Jue Fan, Rachael Hwee Ling Sim +1Differential Privacy Class Imbalance
Consistency Regularised Gradient Flows for Inverse Problems
May 8, 2026Alessio Spagnoletti, Tim Y. J. Wang, Marcelo Pereyra +1Ill-Posed Inverse Problem Inverse Problem
Characterizing and Correcting Effective Target Shift in Online Learning
May 8, 2026Ziyan Li, Naoki HirataniLearning-Augmented Algorithms Online Learning
Approximation-Free Differentiable Oblique Decision Trees
May 8, 2026Subrat Prasad Panda, Blaise Genest, Arvind EaswaranDifferentiable Optimization Decision Trees
When Losses Align: Gradient-Based Composite Loss Weighting for Efficient Pretraining
May 8, 2026Ivan Karpukhin, Andrey SavchenkoModel Pretraining Hyperparameter Optimization
Drifting Field Policy: A One-Step Generative Policy via Wasserstein Gradient Flow
May 8, 2026Juil Koo, Mingue Park, Jiwon Choi +2Wasserstein Gradient Flows Behavior Cloning
Robust stochastic first order methods in heavy-tailed noise via medoid mini-batch gradient sampling
May 8, 2026Manojlo Vukovic, Dusan JakoveticStochastic Gradient Descent Zeroth-Order Optimization
SGD for Variational Inference: Tackling Unbounded Variance via Preconditioning and Dynamic Batching
May 8, 2026Hippolyte Labarrière, Cesare Molinari, Silvia Villa +1Variational Inference Mini-Batch Stochastic Gradient Descent
Decentralized Time-Varying Optimization for Streaming Data via Temporal Weighting
May 7, 2026Muhammad Faraz Ul Abrar, Nicolò Michelusi, Erik G. LarssonStochastic Gradient Descent Time-Varying Systems
Locally Near Optimal Piecewise Linear Regression in High Dimensions via Difference of Max-Affine Functions
May 7, 2026Haitham Kanj, Kiryung LeeProximal Gradient Descent Stochastic Gradient Descent
Attribution-Based Neuron Utility for Plasticity Restoration in Deep Networks
May 7, 2026Patrick Elisii, Lucas Beauchemin, Dawer JamshedEfficient Continual Learning Plasticity
A Rod Flow Model for Adam at the Edge of Stability
May 7, 2026Eric Regis, Sinho ChewiAdam Gradient Descent
When and Why SignSGD Outperforms SGD: A Theoretical Study Based on $\ell_1$ -norm Lower Bounds
May 7, 2026Hongyi Tao, Dingzhi Yu, Lijun ZhangStochastic Gradient Descent Riemannian Optimization
Transformers Efficiently Perform In-Context Logistic Regression via Normalized Gradient Descent
May 7, 2026Chenyang Zhang, Yuan CaoIn-Context Learning Transformer Architectures
Trade-off Functions for DP-SGD with Subsampling based on Random Shuffling: Tight Upper and Lower Bounds
May 7, 2026Marten van Dijk, Murat Bilgehan ErtanDifferential Privacy Stochastic Gradient Descent
Learning Discrete Autoregressive Priors with Wasserstein Gradient Flow
May 7, 2026Bowen Zheng, Yihong Luo, Tianyang HuAutoregressive Image Generation Generative Priors
Fast Gauss-Newton for Multiclass Cross-Entropy
May 7, 2026Mikalai Korbit, Mario ZanonCross-Entropy Losses Multiclass Classification
Medical Imaging Classification with Cold-Atom Reservoir Computing using Auto-Encoders and Surrogate-Driven Training
May 7, 2026Nuno Batista, Ana Morgado, Oscar Ferraz +3Hybrid Quantum Neural Networks Reservoir Computing
Architecture Shape Governs QNN Trainability: Jacobian Null Space Growth and Parameter Efficiency
May 7, 2026Michael Poppel, David Bucher, Maximilian Zorn +5Variational Quantum Circuits Jacobian
SOPE: Stabilizing Off-Policy Evaluation for Online RL with Prior Data
May 7, 2026Carlo Romeo, Girolamo Macaluso, Alessandro Sestini +1Off-Policy Evaluation Learning-Augmented Algorithms
MDN: Parallelizing Stepwise Momentum for Delta Linear Attention
May 7, 2026Yulong Huang, Xiang Liu, Hongxiang Huang +5Quadratic Attention Large Models
Optimal Confidence Band for Kernel Gradient Flow Estimator
May 7, 2026Yuqian Cheng, Zhuo Chen, Qian LinConfidence Intervals Wasserstein Gradient Flows
Distributionally Robust Multi-Objective Optimization
May 7, 2026Yufeng Yang, Fangning Zhuo, Ziyi Chen +2Multi-Objective Optimization Proximal Gradient Descent
Energy Generative Modeling: A Lyapunov-based Energy Matching Perspective
May 7, 2026Yixuan Wang, Wenqian Xue, Warren E. DixonLangevin Dynamics Generative Models
GRALIS: A Unified Canonical Framework for Linear Attribution Methods via Riesz Representation
May 6, 2026Raimondo FanaleTraining Data Attribution Shapley Value
On the Wasserstein Gradient Flow Interpretation of Drifting Models
May 6, 2026Arthur Gretton, Li Kevin Wenliang, Alexandre Galashov +3Wasserstein Gradient Flows Generative Models
Rethinking Local Learning: A Cheaper and Faster Recipe for LLM Post-Training
May 6, 2026Hengyu Shi, Tianyang Han, Peizhe Wang +3Large Language Model Post-Training LLM Post-Training Methods
Gradient Flow Structure and Quantitative Dynamics of Multi-Head Self-Attention
May 5, 2026Ayan PendharkarMulti-Head Self-Attention Transformer Attention
On Adaptivity in Zeroth-Order Optimization
May 5, 2026Hassan Dbouk, Nidham Gazagnadou, Matthias Reisser +1Zeroth-Order Optimization Continual Fine-Tuning
Complex Equation Learner: Rational Symbolic Regression with Gradient Descent in Complex Domain
May 5, 2026Sergei Garmaev, Maurice Gauché, Olga FinkSymbolic Regression Regularization Techniques
Distributed Learning with Adversarial Gradient Perturbations
May 5, 2026Nawapon Sangsiri, Yufei TaoDistributional Learning Optimal Sample Complexity
Private Speech Classification without Collapse: Stabilized DP Training and Offline Distillation
May 4, 2026Yadi Wen, Tianxin Li, Enji Liang +2Speaker Recognition Efficient Privacy Loss
Robust and Fast Training via Per-Sample Clipping
May 4, 2026Davide Nobile, Philipp GrohsGradient Clipping Stochastic Gradient Descent
Deciphering Shortcut Learning from an Evolutionary Game Theory Perspective
May 4, 2026Xiayang Li, Kuo Gai, Shihua ZhangStochastic Gradient Descent Shortcuts

May 12, 2026 · cs.LGJ/K move · Enter open · S save

One-Step Generative Modeling via Wasserstein Gradient Flows

Jiaqi Han, Puheng Li, Qiushan Guo, Renyuan Xu+2

Stanford University · ByteDance

Diffusion models and flow-based methods have shown impressive generative capability, especially for images, but their sampling is expensive because it requires many iterative updates. We introduce W-Flow, a framework for training a generator that transforms samples from a simple reference distribution into samples from a target data distribution in a single step. This is achieved in two steps: we first define an evolution from the reference distribution to the target distribution through a Wasserstein gradient flow that minimizes an energy functional; second, we train a static neural generator to compress this evolution into one-step generation. We instantiate the energy functional with the Sinkhorn divergence, which yields an efficient optimal-transport-based update rule that captures global distributional discrepancy and improves coverage of the target distribution. We further prove that the finite-sample training dynamics converge to the continuous-time distributional dynamics under suitable assumptions. Empirically, W-Flow sets a new state of the art for one-step ImageNet 256

\times

256 generation, achieving 1.29 FID, with improved mode coverage and domain transfer. Compared to multi-step diffusion models with similar FID scores, our method yields approximately 100

\times

faster sampling. These results show that Wasserstein gradient flows provide a principled and effective foundation for fast and high-fidelity generative modeling.

Discrete Diffusion Model Generative Models Wasserstein Distance Gradient Flow Imagenet

Open paper Ask about it PDF