Differentially Private Stochastic Gradient Descent

Also known as DP-SGD

Follow this topic Ask about this topic

Momentum

2 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 32

All topics

Closed-Form Noise Calibration Against Membership Inference for Random-Allocation DP-SGD
Oct 7, 2026Murat Bilgehan Ertan, Marten van DijkDifferentially Private Stochastic Gradient Descent Membership Inference Attacks
Affiliated with Vrije Universiteit Amsterdam.
Reward-Driven Learning under Prompt-Level Differential Privacy
Oct 5, 2026Jiachen Zhao, Antonia Januszewicz, Taeho JungPrivacy-Preserving Language Models Differentially Private Stochastic Gradient Descent
University of Notre Dame
Is Weight Tying Still Beneficial for Decoder-Only LLMs in Private Settings Under DP-SGD?
Sep 30, 2026Razan El Mais, Ali Chehab, Ibrahim Issa +1Privacy-Preserving Language Models Differentially Private Stochastic Gradient Descent
Department of Electrical and Computer Engineering American University of Beirut Beirut, Lebanon
Feature Suppression and Differential Privacy for Residential Traffic Classification: A Two-Home Federated Study
Sep 20, 2026Márton Pál Lipcsey-Magyar, Adrian PekarDifferentially Private Stochastic Gradient Descent Federated Learning
Budapest University of Technology and Economics, Hungary · CUJO LLC Hungary
StraightDP: Geometry-Aware Differential Privacy for Rectified-Flow Transformers
Jul 31, 2026Xujun Che, Depeng Xu, Xintao WuDiffusion Transformer Differentially Private Stochastic Gradient Descent
Department of Cybersecurity, University of North Carolina at Charlotte, Charlotte, NC, USA · Department of Electrical Engineering and Computer Science, University of Arkansas, Fayetteville, AR, USA
Decomposing Memorization Reduction in Privacy-Preserving Fine-Tuning of SLMs for CSIRTs
Jun 26, 2026Cristhian Kapelinski, Diego KreutzPrivacy-Preserving Language Models Memorization in Language Models
Federal University of Pampa (UNIPAMPA), Alegrete, Brazil
From Gradient Clipping to Structural Refinement: Improving DPSGD for Medical Image Segmentation
Jun 19, 2026Shiva Parsarad, Parth Shandilya, Isabel WagnerDifferentially Private Stochastic Gradient Descent Medical Image Segmentation
University of Basel
Private Learning with Public Feature Conditioning
Jun 17, 2026Shuli Jiang, Walid Krichene, Nicolas MayorazDifferentially Private Stochastic Gradient Descent Privacy-Preserving ML
1AWS Agentic AI · 2Microsoft · 3Google Research
Let's Ask Gauss: Improved One-Run Privacy Auditing
Jun 10, 2026Adya Agrawal, Yu Wei, Jaspal Singh +2Privacy Auditing Differentially Private Stochastic Gradient Descent
Georgia Institute of Technology · Purdue University · Rensselaer Polytechnic Institute
DP-MacAdam: Differentially Private Mechanism with Adaptive Clipping and Adaptive Momentum
Jun 3, 2026Naima Tasnim, Lalitha Sankar, Oliver KosutDifferentially Private Stochastic Gradient Descent Gradient Clipping
Arizona State University, Tempe, AZ, United States
Revisiting Privacy Amplification by Subsampling in Selective Release DPSGD
Jun 3, 2026Xiaobo Huang, Fang XieDifferentially Private Stochastic Gradient Descent Differential Privacy
Guangdong Provincial Key Laboratory of IRADS, Beijing · Normal-Hong Kong Baptist University · Zhuhai, China
When Do Fewer Coordinates Suffice in DP-SGD?
Jun 3, 2026Huiqi Zhang, Fang XieDifferentially Private Stochastic Gradient Descent Nonconvex Stochastic Optimization
Guangdong Provincial Key Laboratory of IRADS, Beijing Normal-Hong Kong Baptist University Zhuhai, China
PRISM: Gauge-Invariant Tangent-Space Differentially Private LoRA
May 31, 2026Shihao Wang, Xueru ZhangDifferentially Private Stochastic Gradient Descent Low-Rank Adaptation
Department of Computer Science and Engineering, The Ohio State University, Columbus, OH, USA.
Revisiting ML Training under Fully Homomorphic Encryption: Convergence Guarantees, Differential Privacy, and Efficient Algorithms
May 27, 2026Yvonne Zhou, Mingyu Liang, Ivan Brugere +5Differentially Private Stochastic Gradient Descent Homomorphic Encryption
University of Maryland, College Park, MD 20742 · J.P. Morgan AI Research, New York, NY, 10017 · AlgoCRYPT CoE
From Privacy to Generalization: Linear Max-Information Bounds for Differentially Private Learning Algorithms
May 25, 2026Christoph H. Lampert, Max Cairney-Leeming, Hossein ZakeriniaPAC-Bayesian Generalization Bounds Differentially Private Stochastic Gradient Descent
Institute of Science and Technology Austria (ISTA) Klosterneuburg, Austria
Efficient DP-SGD for LLMs with Randomized Clipping
May 24, 2026Enayat Ullah, Sai Aparna Aketi, Devansh Gupta +2Privacy-Preserving Language Models Differentially Private Stochastic Gradient Descent
Meta Platforms Inc. · University of Southern California
SMA-DP: Spectral Memory-Aware Differential Privacy for Deep Learning
May 19, 2026Mohammad Partohaghighi, Roummel MarciaDeep Learning Optimization Differentially Private Stochastic Gradient Descent
Department of Electrical Engineering and Computer Science University of California, Merced Merced, CA, USA · Department of Applied Mathematics University of California, Merced Merced, CA, USA
Beyond Square Roots: Explicit Memory-Efficient Factorization for Multi-Epoch Private Learning
May 18, 2026Nikita P. Kalinin, Aki Rehn, Joel Daniel Andersson +2Differentially Private Stochastic Gradient Descent Differential Privacy
Institute of Science and Technology Austria · University of Helsinki
DP-KFC: Data-Free Preconditioning for Privacy-Preserving Deep Learning
May 13, 2026Marc Molina Van den Bosch, Riccardo Taiello, Albert Sund Aillet +3Differentially Private Stochastic Gradient Descent Privacy-Preserving ML
CERN, Geneva, Switzerland · BCN MedTech, Department of Engineering, Universitat Pompeu Fabra, Barcelona, Spain · ICREA, Barcelona, Spain
Population Risk Bounds for Kolmogorov-Arnold Networks Trained by DP-SGD with Correlated Noise
May 12, 2026Puyu Wang, Jan Schuchardt, Nikita Kalinin +4Differentially Private Stochastic Gradient Descent Nonconvex Stochastic Optimization
RPTU Kaiserslautern-Landau, Kaiserslautern, Germany · Machine Learning Research, Morgan Stanley · Institute of Science and Technology, Klosterneuburg, Austria +1
DP-LAC: Lightweight Adaptive Clipping for Differentially Private Federated Fine-tuning of Language Models
May 11, 2026Haaris Mehmood, Jie Xu, Karthikeyan Saravanan +2Privacy-Preserving Language Models Differentially Private Stochastic Gradient Descent
Samsung R&D Institute UK (SRUK) · Samsung AI Centre Cambridge
Deep Learning under Fractional-Order Differential Privacy
May 11, 2026Mohammad Partohaghighi, Roummel MarciaDifferentially Private Stochastic Gradient Descent Privacy-Preserving ML
MESA Lab, Dept. of EECS University of California, Merced Merced, CA 95343 · Dept. of Applied Mathematics University of California, Merced Merced, CA 95343
INO-SGD: Addressing Utility Imbalance under Individualized Differential Privacy
May 8, 2026Xiao Tian, Jue Fan, Rachael Hwee Ling Sim +1Differentially Private Stochastic Gradient Descent Differential Privacy
Department of Computer Science, National University of Singapore · Agency for Science, Technology and Research (A*STAR), Singapore
Less Random, More Private: What is the Optimal Subsampling Scheme for DP-SGD?
May 8, 2026Andy Dong, Ayfer ÖzgürDifferentially Private Stochastic Gradient Descent Differential Privacy
Stanford University
Trade-off Functions for DP-SGD with Subsampling based on Random Shuffling: Tight Upper and Lower Bounds
May 7, 2026Marten van Dijk, Murat Bilgehan ErtanDifferentially Private Stochastic Gradient Descent Differential Privacy
CWI Amsterdam · ∗Affiliated with Vrije Universiteit Amsterdam.
FIBER: A Differentially Private Optimizer with Filter-Aware Innovation Bias Correction
May 5, 2026Duc Dm, Thao Do, Minh Son Hoang +3Differentially Private Stochastic Gradient Descent Privacy-Preserving ML
School of Computing, Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea · Center for Environmental Intelligence (CEI), VinUniversity, Hanoi, Vietnam · Khoury College of Computer Sciences, Northeastern University, USA
Private Speech Classification without Collapse: Stabilized DP Training and Offline Distillation
May 4, 2026Yadi Wen, Tianxin Li, Enji Liang +2Differentially Private Stochastic Gradient Descent Learning Using Privileged Information
Shanxi Key Laboratory of Industrial Internet Security, Taiyuan University of Technology, China · University of Michigan, Ann Arbor, USA
Class-Aware Adaptive Differential Privacy in Deep Learning for Sensor-Based Fall Detection
May 3, 2026Joydeb Kumar SanaDifferentially Private Stochastic Gradient Descent Privacy-Preserving ML
Institute of Information and Communication Technology, Bangladesh University of Engineering and Technology, Dhaka, Bangladesh
On the Gradient Complexity of Private Optimization with Private Oracles
Nov 17, 2025Michael Menart, Aleksandar NikolovQuery Complexity Oracle Complexity
Department of Computer Science, University of Toronto, Vector Institute · Department of Computer Science, University of Toronto
Optimizing Canaries for Privacy Auditing with Metagradient Descent
Jul 21, 2025Matteo Boglioni, Terrance Liu, Andrew Ilyas +1Privacy Auditing Differentially Private Stochastic Gradient Descent
Carnegie Mellon University · Stanford Statistics

Oct 7, 2026, cs.LGJ/K move · Enter open · S save

Closed-Form Noise Calibration Against Membership Inference for Random-Allocation DP-SGD

Murat Bilgehan Ertan, Marten van Dijk

Affiliated with Vrije Universiteit Amsterdam.

Differentially Private Stochastic Gradient Descent Membership Inference Attacks

Open paper AskPDF arXiv

DP-SGD protects training data by adding Gaussian noise to clipped gradients. The amount of noise is usually chosen by running a numerical privacy accountant inside a search. We study DP-SGD with random allocation, where each epoch uses every record once, at a randomly chosen step. For this setting we give a one-line formula that bounds the accuracy of every membership inference attack (MIA) on the trained model. With

M

steps per epoch,

E

epochs and noise multiplier

σ

, and with membership and non-membership equally likely a priori, the attack accuracy is at most

\frac12+\frac14\sqrt{(1+(e^{1/σ^2}-1)/M)^E-1}

. The formula comes from the chi-square divergence between a Gaussian distribution and a Gaussian mixture that dominates random allocation. It is interpretable and gives

σ

in about a microsecond. Where applicable, our formula needs at most about half the noise of the state-of-the-art closed-form bound. To measure how close the bound is, we also derive an exact expression for the attack accuracy of these two distributions and evaluate it numerically. Calibrating to this exact expression requires

13.0\%

20.2\%

less noise than the formula in our main experiments, and since it is exact, no accountant that knows only

M

E

and

σ

can certify a smaller

σ

. In training, the resulting

σ

outperforms the formula and matches a published accountant in test accuracy. It is found in seconds and certified in minutes, whereas every search we ran with that accountant took longer or returned at least

0.62\%

more noise. We show that MIAs on the trained models stay below the bound.

Momentum

Related topics

Latest papers 32