Revisiting the Provable-Auditable Privacy Gap of DP-SGD

Saloni Modi; Srivi Balaji; Yusong Zhu; Gautam Kamath; Kevin Tian

cs.LGAug 28, 2026

Revisiting the Provable-Auditable Privacy Gap of DP-SGD

Authors: Saloni Modi, Srivi Balaji, Yusong Zhu, Gautam Kamath, Kevin Tian

Organizations: University of Texas at Austin · University of Waterloo and Vector Institute

Abstract

Differential privacy (DP) has traditionally been used to provide theoretical upper bounds on an algorithm's stability to changing its training data. In modern private machine learning applications, achieving strong tradeoffs between utility and theoretical privacy is challenging, and thus one may optimistically hope that existing theoretical privacy analyses are loose. Recent work on privacy auditing has adopted a dual viewpoint, instead lower bounding the true privacy of an algorithm by constructing empirical distinguishing events. The auditing literature has thus far yielded a pessimistic outlook on the looseness of theoretical privacy bounds for DP-SGD, the de facto private training method in modern ML, as nearly-matching empirical lower bounds have been achieved under various threat models [NHSBTJCT23, AC24, CBP25]. In this work, we propose the empirical privacy lower bound of an algorithm as a concrete metric to optimize for, complementary to the theoretical upper bound. We give a lightweight defense framework that generically augments optimization methods in the ML pipeline to have significantly-improved empirical privacy on standard benchmarks. Moreover, we show that our framework comes at no theoretical privacy cost when augmenting DP-SGD, unlike previously-proposed defenses against membership inference attacks. We evaluate our defense against a broad range of audit constructions, models, and datasets to demonstrate its flexibility.

Figures & tables

Figure 1: $\varepsilon_{\textup{lb}}$ across varying $\varepsilon_{\textup{ub}}$ budgets.

Dataset (Model)	Theoretical $\boldsymbol{\varepsilon}$	Train Acc (%)	Test Acc (%)
MNIST (CNN)	2	97.47/96.36	97.40/96.57
MNIST (CNN)	4	98.42/97.23	98.37/97.52
MNIST (CNN)	6	98.65/97.44	98.53/97.74
MNIST (CNN)	8	98.75/97.54	98.61/97.86
MNIST (CNN)	10	98.80/97.59	98.64/97.89
Purchase (MLP)	2	77.82/76.79	74.51/73.19

Table 1: Defense performance on private models across privacy budgets

\varepsilon\in\{2,4,6,8,10\}

. Training and testing accuracies are reported in the table as No Defense/Defense .

Attack	Dataset (Model)	Defense Setting	Train Acc (%)	Test Acc (%)	Empirical $\varepsilon$
FGSM	CIFAR-10 (CNN)	No Defense	76.06	70.75	1.14
FGSM	CIFAR-10 (CNN)	Defense	74.95	70.14	0.07
FGSM	MNIST (CNN)	No Defense	98.78	98.57	3.03
FGSM	MNIST (CNN)	Defense	97.58	97.78	0
Mislabeled	CIFAR-10 (CNN)	No Defense	76.31	71.37	0.47
Mislabeled	CIFAR-10 (CNN)	Defense	75.05	70.61	0

Table 2: Empirical privacy loss under non-blank input space attacks for

\varepsilon_{\textup{ub}}=10

$\lambda$	Average drop epoch
0.0	5.7
0.0	6.0
0.1	16.7
0.5	14.0
1.0	15.0
5.0	13.0

Table 3: Defense-aware canary optimization attempts across different values of

\lambda

Defense Setting	Group-Privacy $\varepsilon_{\mathrm{lb}}$
No Defense	4.43
Defense	27.26

Table 4: Group-privacy

\varepsilon_{\mathrm{lb}}

under the gradient canceling attack on MNIST.

Appendix figures & tables28 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 2 — Figure 2: Finite-sample effect of Clopper–Pearson (CP) correction under different sample sizes $k$ , with $\gamma=0.05$ and $\delta=10^{-5}$ . Each row corresponds to a fixed $k$ . The left column compares the distributions of the raw lower bound $\varepsilon_{\rm raw}$ and the CP-corrected lower bound $\varepsilon_{\rm cp}$ ; the middle column shows the distribution of the correction gap $\varepsilon_{\rm raw}-\varepsilon_{\rm cp}$ ; and the right column shows the distribution of raw lower bounds that collapse to $0$ after CP correction.

majority_pct	c0_red (maj)	c0_blue (min)	c1_red (min)	c1_blue (maj)
0.995	$1.2\pm 0.0\%$	$100.0\pm 0.0\%$	$100.0\pm 0.0\%$	$1.1\pm 0.0\%$
0.950	$0.3\pm 0.0\%$	$27.6\pm 0.2\%$	$21.4\pm 0.2\%$	$0.6\pm 0.0\%$
0.900	$0.5\pm 0.0\%$	$12.2\pm 0.1\%$	$8.6\pm 0.1\%$	$0.8\pm 0.0\%$
0.850	$0.6\pm 0.0\%$	$7.9\pm 0.1\%$	$5.4\pm 0.1\%$	$0.9\pm 0.0\%$
0.800	$0.7\pm 0.0\%$	$5.7\pm 0.1\%$	$4.1\pm 0.1\%$	$1.0\pm 0.0\%$
0.750	$0.8\pm 0.0\%$	$4.3\pm 0.1\%$	$3.2\pm 0.1\%$	$1.1\pm 0.0\%$

Appendix

Table 5: Defense Removal Rate (%): mean

\pm

std over repetitions

Experiment	Train Acc (%)	Test Acc (%)	$\varepsilon_{\mathrm{lb}}$
Mislabeled Single Run Audit, No Defense	75.62	70.81	0.20
Mislabeled Single Run Audit, Defense	74.73	70.42	0.10
Mislabeled Group Privacy Audit, No Defense	76.13	71.65	3.54
Mislabeled Group Privacy Audit, Defense	74.69	70.62	2.67

Appendix

Table 6: Empirical privacy loss under individual and group privacy audits.

Scoring Function	Train Acc (%)	Test Acc (%)	$\varepsilon_{\mathrm{lb}}$
Cosine similarity ( $\theta_{0}$ )	75.85	71.05	1.11
Grad norm clipped ( $L_{2}$ )	75.14	70.47	1.37
Grad norm clipped ( $L_{\infty}$ )	75.28	70.54	0
Grad norm unclipped ( $L_{2}$ )	75.25	70.73	1.19
Grad norm unclipped ( $L_{\infty}$ )	75.20	70.58	0
Gradient kurtosis unclipped	75.49	70.62	0

Appendix

Table 7: Scoring function ablation on CIFAR-10 (CNN).

Figure 3: $\varepsilon_{\textup{lb}}$ with/without defense on MNIST (CNN) across Poisson and shuffled sampling methods.

Dataset	Defense Setting	$\alpha=0$	$\alpha=0.25$	$\alpha=0.5$	$\alpha=0.75$	$\alpha=1.0$
CIFAR-10	No Defense	0.46	0.42	0.30	0.35	0.73
CIFAR-10	Defense	0	0	0	0.11	0
MNIST	No Defense	2.45	2.45	2.50	2.45	2.50
MNIST	Defense	0.59	0.58	0.62	0.71	0.69

Appendix

Table 8: Empirical

\varepsilon_{\mathrm{lb}}

across interpolated canary strengths on CIFAR-10 and MNIST (CNN).

Figure 4: Distribution of empirical $\varepsilon_{\textup{lb}}$ for non-canary training samples (post-defense) vs. canary $\varepsilon_{\textup{lb}}$ (pre-defense) across two auditing methods. In both cases, 0.0% of non-canary samples exceed the canary reference.

Figure 5: Empirical $\varepsilon_{\textup{lb}}$ of the worst-case non-canary sample post-defense vs. canary $\varepsilon_{\textup{lb}}$ pre-defense, under GDP 50% holdout and CP 50% holdout. The worst-case non-canary sample is selected as the sample achieving the highest empirical $\varepsilon_{\textup{lb}}$ under no defense; its empirical $\varepsilon_{\textup{lb}}$ drops to 0.000 post-defense under both auditing methods.

Defense Setting	Train Acc (%)	Test Acc (%)	$\varepsilon_{\textup{lb}}$ (400 Trials)	$\varepsilon_{\textup{lb}}$ (1000 Trials)
No Defense	98.80	98.64	3.71	4.01
Defense	97.60	97.89	0	0.09

Appendix

Table 9: Empirical privacy loss on MNIST (CNN) with 1000 shadow models.

Global Filtering $k$ Value	Train Acc (%)	Test Acc (%)	$\varepsilon_{\textup{lb}}$
10	0.984	0.984	0
25	0.980	0.981	0.015
50	0.975	0.977	0.090

Appendix

Table 10: Global filtering ablation on MNIST (CNN).

Local Filtering $k$ Value	Train Acc (%)	Test Acc (%)	$\varepsilon_{\mathrm{lb}}$
1	76.01	71.07	0.44
2	75.92	71.11	0.52
3	75.52	70.62	0.43
4	75.28	70.60	0.12
5	75.27	70.77	0.36

Appendix

Table 11: Local filtering bandwidth ablation on CIFAR-10 (CNN).

Epochs Before Filtering	Train Acc (%)	Test Acc (%)	$\varepsilon_{\textup{lb}}$
1	75.21	70.77	0.43
5	75.74	70.73	1.21
10	76.19	71.13	1.39
20	76.06	70.89	1.19

Appendix

Table 12: Effect of filtering frequency on CIFAR-10 (CNN).

Dataset (Model)	$\varepsilon_{\mathrm{ub}}$	Setting	GDP 25%	GDP 75%	CP no holdout	CP 75%
MNIST (CNN)	2	No Defense	0.000	0.000	0.000	0.000
		Defense	0.000	0.000	0.015	0.000
	4	No Defense	0.000	0.011	0.398	0.000
		Defense	0.000	0.000	0.000	0.000
	6	No Defense	0.575	0.688	0.600	0.138
		Defense	0.000	0.000	0.000	0.000

Appendix

Table 13: Full empirical privacy results across privacy budgets

\varepsilon_{\mathrm{ub}}\in\{2,4,6,8,10\}

, datasets, and model architectures. See Section 4.2 .

Scoring Function	Setting	GDP 50%	GDP 75%	CP no holdout
Cosine similarity ( $\theta_{0}$ )	Defense	0.000	0.063	0.357
Grad norm clipped ( $L_{2}$ )	Defense	0.394	0.000	0.477
Grad norm clipped ( $L_{\infty}$ )	Defense	0.000	0.000	0.000
Grad norm unclipped ( $L_{2}$ )	Defense	0.000	0.023	0.391
Grad norm unclipped ( $L_{\infty}$ )	Defense	0.000	0.000	0.000
Gradient kurtosis unclipped	Defense	0.000	0.000	0.000

Appendix

Table 14: Scoring function ablation on CIFAR-10 (CNN): full empirical privacy results across auditing methods. See Section D .

Dataset	Setting	GDP 25%	GDP 50%	GDP 75%	CP no holdout	CP 25%	CP 50%	CP 75%
MNIST (CNN)	No Defense	1.856	2.313	2.867	1.437	0.133	0.891	1.157
	Defense	0.000	0.000	0.000	0.000	0.000	0.000	0.000

Appendix

Table 15: Empirical privacy loss on MNIST (CNN) with 1000 shadow models, full results across auditing methods. See Section H .

Sampling Method	Setting	GDP 25%	GDP 50%	GDP 75%	CP no holdout	CP 25%	CP 50%	CP 75%
Shuffled	No Defense	0.000	0.617	1.335	1.372	0.000	0.025	0.368
	Defense	0.000	0.000	0.000	0.000	0.000	0.000	0.000
Poisson	No Defense	0.773	1.580	2.708	1.066	0.071	0.184	0.669
	Defense	0.000	0.000	0.000	0.000	0.000	0.000	0.000

Appendix

Table 16:

\varepsilon_{\mathrm{lb}}

with/without defense on MNIST (CNN) across Poisson and shuffled sampling methods, full results across auditing methods. See Section E .

Audit Type	Dataset	Setting	GDP 25%	GDP 50%	GDP 75%	CP no holdout	CP 25%	CP 50%	CP 75%
Group privacy (multi-canary)	CIFAR-10	No Defense	0.453	1.180	2.563	1.376	0.000	0.449	0.701
		Defense	0.000	1.425	1.415	1.146	0.062	0.394	0.409

Appendix

Table 17: Full empirical privacy results under group privacy (multi-canary) mislabeled audits. See Section C.2 .

Canary Type	Setting	GDP 25%	GDP 50%	GDP 75%	CP no holdout	CP 25%	CP 50%	CP 75%
Majority canary	No Defense	0.000	0.000	0.000	0.000	0.000	0.000	0.000
Minority canary	No Defense	0.000	0.000	0.000	0.296	0.000	0.000	0.000

Appendix

Table 18: Full empirical privacy results for the fairness audit on Colored MNIST, under the no-defense setting. See Section B .

Dataset	Setting	GDP 25%	GDP 50%	GDP 75%	CP no holdout	CP 25%	CP 50%	CP 75%
MNIST (CNN)	No Defense	0.153	1.732	2.663	1.052	0.199	0.000	0.152
	Defense	0.000	0.000	0.000	0.000	0.000	0.000	0.000
CIFAR-10 (CNN)	No Defense	2.845	1.519	1.809	0.767	0.000	0.253	0.491
	Defense	0.000	0.000	0.000	0.000	0.000	0.000	0.000

Appendix

Table 19: Full empirical privacy results under ClipBKD attacks on MNIST and CIFAR-10 (CNN). See Section 4.2 .

Dataset	Setting	GDP 25%	GDP 50%	GDP 75%	CP no holdout	CP 50%	CP 75%
MNIST (CNN)	No Defense	0.000	1.452	2.404	1.306	0.340	0.543
	Defense	0.000	0.000	0.000	0.000	0.000	0.000
CIFAR-10 (CNN)	No Defense	0.018	0.269	0.000	0.362	0.000	0.000
	Defense	0.000	0.000	0.000	0.000	0.000	0.000

Appendix

Table 20: Full empirical privacy results under FGSM attacks on MNIST and CIFAR-10 (CNN). See Section 4.2 .

Dataset	Setting	GDP 25%	GDP 50%	GDP 75%	CP no holdout	CP 25%	CP 50%	CP 75%
MNIST (CNN)	No Defense	0.701	1.576	1.318	0.806	0.065	0.152	0.223
	Defense	0.829	0.000	0.000	0.124	0.000	0.000	0.000
CIFAR-10 (CNN)	No Defense	0.008	0.000	0.000	0.068	0.000	0.000	0.000
	Defense	0.000	0.000	0.000	0.000	0.000	0.000	0.000

Appendix

Table 21: Full empirical privacy results under mislabeled input space attacks on MNIST and CIFAR-10 (CNN). See Section 4.2 .

Setting	GDP 25%	GDP 50%	GDP 75%	CP no holdout	CP 25%	CP 50%	CP 75%
No Defense	1.081	2.476	2.410	1.760	0.150	0.872	0.861
Defense	19.564	24.378	27.130	4.185	2.785	3.493	3.903

Appendix

Table 22: Full empirical privacy results under the gradient canceling attack on MNIST (CNN). See Section 4.5 .

Dataset	$\alpha$	Setting	CP no holdout
CIFAR-10	0.00	No Defense	0.062
		Defense	0.000
	0.25	No Defense	0.060
		Defense	0.000
	0.50	No Defense	0.024
		Defense	0.000

Appendix

Table 23: Full empirical privacy results across interpolated canary strengths (

\alpha

) on CIFAR-10 and MNIST (CNN). See Section F .

Epochs Between Filtering	GDP 50%	GDP 75%	CP no holdout	CP 75%
1	0.000	0.000	0.066	0.000
5	0.086	0.302	0.391	0.013
10	0.266	0.000	0.449	0.000
20	0.000	0.000	0.391	0.000

Appendix

Table 24: Full empirical privacy results across filter frequencies on CIFAR-10 (CNN). See Section I.3 .

Local Filtering $k$ Value	GDP 25%	GDP 50%	GDP 75%	CP no holdout	CP 25%	CP 50%	CP 75%
1	0.000	0.000	0.000	0.055	0.000	0.000	0.000
2	0.000	0.000	0.000	0.102	0.000	0.000	0.000
3	0.000	0.000	0.000	0.065	0.000	0.000	0.000
4	0.000	0.000	0.000	0.000	0.000	0.000	0.000
5	0.000	0.000	0.000	0.035	0.000	0.000	0.000

Appendix

Table 25: Full empirical privacy results across local filtering bandwidth values on CIFAR-10 (CNN). See Section I.2 .

Attack	Setting	GDP 25%	GDP 50%	GDP 75%	CP no holdout	CP 25%	CP 50%	CP 75%
Mislabeled	No Defense	0.701	1.576	1.318	0.806	0.065	0.152	0.223
	Defense	0.849	0.000	0.000	0.153	0.000	0.000	0.000
Defense-aware	No Defense	0.000	0.672	1.346	0.531	0.000	0.106	0.333
	Defense	0.000	0.000	0.000	0.000	0.000	0.000	0.000

Appendix

Table 26: Full empirical privacy results for the defense-aware audit on MNIST (CNN), comparing standard mislabeled canaries against the fixed-point attack. See Section 4.4 .

Global Filtering $k$ Value	GDP 25%	GDP 50%	GDP 75%	CP no holdout	CP 25%	CP 50%	CP 75%
10	0.000	0.000	0.000	0.000	0.000	0.000	0.000
25	0.000	0.000	0.000	0.000	0.000	0.000	0.000
50	0.000	0.000	0.000	0.000	0.000	0.000	0.000

Appendix

Table 27: Full empirical privacy results for global filtering ablation on MNIST (CNN). See Section I.1 .

Setting	GDP 25%	GDP 50%	GDP 75%	CP no holdout	CP 25%	CP 50%	CP 75%
No Defense	17.625	24.378	27.130	4.185	2.749	3.493	3.903
Defense	17.625	24.378	25.311	4.185	2.749	3.493	3.892

Appendix

Table 28: Full empirical privacy results under the gradient bandwidth attack on MNIST (CNN). See Section 4.5 .

Explore similar work

Jun 10, 2026

Let's Ask Gauss: Improved One-Run Privacy Auditing

Privacy auditing provides an important safeguard by estimating the actual information leaked by a model, thus ensuring that theoretical privacy guarantees hold in practice. We study empirical privacy auditing for differentially private (DP) machine learning, focusing on efficient one-run methods for mechanisms such as DP-SGD. Prior one-run approaches threshold training examples or "canaries" into binary membership guesses, which discards useful information. We show that, in the white-box DP-SGD setting, canary-aligned signals naturally form a sequence of random variables whose normalized sum is asymptotically Gaussian. Leveraging this distributional perspective, we develop a DP-auditing framework that leads to tighter privacy lower bounds from a single training run.

Adya Agrawal, Yu Wei, Jaspal Singh +2

Georgia Institute of Technology · Purdue University · Rensselaer Polytechnic Institute

Privacy Auditing Differentially Private Stochastic Gradient Descent Differential Privacy

Ask PDF

Jul 21, 2025

Optimizing Canaries for Privacy Auditing with Metagradient Descent

In this work we study black-box privacy auditing, where the goal is to lower bound the privacy parameter of a differentially private learning algorithm using only the algorithm's outputs (i.e., final trained model). For DP-SGD (the most successful method for training differentially private deep learning models), the canonical auditing approach uses membership inference - an auditor comes with a small set of special "canary" examples, inserts a random subset of them into the training set, and then tries to discern which of their canaries were included in the training set (typically via a membership inference attack). The auditor's success rate then provides a lower bound on the privacy parameters of the learning algorithm. Our main contribution is a method for optimizing the auditor's canary set to improve privacy auditing, leveraging recent work on metagradient optimization (Engstrom et al., 2025). Our empirical evaluation demonstrates that in certain instances, using such optimized canaries can improve empirical lower bounds for differentially private image classification models by several times when compared to canaries proposed in prior work. Furthermore, we demonstrate that our method is DP-SGD agnostic and efficient: canaries optimized for non-private SGD with a small model architecture remain effective when auditing larger models trained with DP-SGD.

Matteo Boglioni, Terrance Liu, Andrew Ilyas +1

Carnegie Mellon University · Stanford Statistics

Privacy Auditing Differentially Private Stochastic Gradient Descent Black-Box Model Auditing

Ask PDF

Mar 3, 2026

Adaptive Methods Are Preferable in High Privacy Settings: An SDE Perspective

Differential Privacy (DP) is becoming central to large-scale training as privacy regulations tighten. We revisit how DP noise interacts with adaptivity in optimization through the lens of stochastic differential equations, providing the first SDE-based analysis of private optimizers. Focusing on DP-SGD and DP-SignSGD under per-example clipping, we show a sharp contrast under fixed hyperparameters: DP-SGD converges at a Privacy-Utility Trade-Off of

\mathcal{O}(1/\varepsilon^2)

with speed independent of

\varepsilon

, while DP-SignSGD converges at a speed linear in

\varepsilon

with an

\mathcal{O}(1/\varepsilon)

trade-off, dominating in high-privacy or large batch noise regimes. By contrast, under optimal learning rates, both methods achieve comparable theoretical asymptotic performance; however, the optimal learning rate of DP-SGD scales linearly with

\varepsilon

, while that of DP-SignSGD is essentially

\varepsilon

-independent. This makes adaptive methods far more practical, as their hyperparameters transfer across privacy levels with little or no re-tuning. Empirical results confirm our theory across training and test metrics, and empirically extend from DP-SignSGD to DP-Adam.

Enea Monzio Compagnoni, Alessandro Stanghellini, Rustem Islamov +2

University of Basel, Switzerland · University of Zürich, Switzerland

Differentially Private Stochastic Gradient Descent Differential Privacy Adaptive Gradient Methods

Ask PDF