Differentially Private Stochastic Gradient Descent

DP-SGD studies how to train useful machine-learning models while limiting what their updates reveal about individual training examples. Its standard recipe clips per-example gradients and adds calibrated noise; subsampling and privacy accountants quantify the resulting guarantee, while adaptive clipping, alternative optimizers, and structured or correlated noise aim to improve utility. Researchers evaluate these methods on image and language benchmarks, large language model fine-tuning, and federated or healthcare settings, balancing accuracy, computation, and privacy. A central tension is that stronger protection can impair learning unevenly across tasks or groups, and formal privacy guarantees do not by themselves establish resistance to every empirical attack or ensure fair performance.