Stochastic Gradient Langevin Dynamics

SGLD combines minibatch gradient estimates with Langevin noise to sample approximately from probability distributions, especially Bayesian posteriors that are costly to explore with full-gradient methods. Its main variants use preconditioning, momentum, variance reduction, taming or robust gradient estimates, and non-reversible updates to improve stability, sampling speed, or resistance to noisy gradients. The methods are studied in Bayesian regression and classification, latent-variable models, energy-based image models, neural-network training, and scientific inverse problems, alongside analyses of discretization error and uncertainty estimates. A central tension is that minibatch noise and finite step sizes can distort the target distribution, so practical scalability must be balanced against reliable sampling guarantees and careful tuning.