Many differentially private (DP) data release systems either output DP synthetic data and leave analysts to perform inference as usual, which can lead to severe miscalibration, or output a DP point estimate without a principled way to do uncertainty quantification. This paper develops a clean and tractable middle ground for exponential families: release only DP sufficient statistics, then perform noise-calibrated likelihood-based inference and optional parametric synthetic data generation as post-processing. Our contributions are: (1) a general recipe for approximate-DP release of clipped sufficient statistics under the Gaussian mechanism; (2) asymptotic normality, explicit variance inflation, and valid Wald-style confidence intervals for the plug-in DP MLE; (3) a noise-aware likelihood correction that is first-order equivalent to the plug-in but supports bootstrap-based intervals; and (4) a matching minimax lower bound showing the privacy distortion rate is unavoidable. The resulting theory yields concrete design rules and a practical pipeline for releasing DP synthetic data with principled uncertainty quantification, validated on three exponential families and real census data.
Figures & tables
Figure 1: Pipeline overview. The noisy sufficient statistic Sˉ is the only DP-protected release; all downstream tasks inherit the same (ε,δ) -DP guarantee by post-processing.
Figure 2: Empirical versus theoretical variance for the DP plug-in estimator in Gaussian mean estimation. Each point corresponds to one (n,ε) configuration, with color indicating ε and marker shape indicating n . The close alignment with the identity line validates the finite-sample relevance of Theorem 2 .
Method
ε=0.1
ε=0.5
ε=1.0
ε=5.0
ε=10
Non-private MLE
0.947
0.958
0.954
0.948
0.948
DP plug-in Wald
0.950
0.947
0.951
0.950
0.955
DP noise-aware Wald
0.950
0.947
0.951
0.950
0.955
DP param. bootstrap
0.944
0.945
0.945
0.945
0.948
Naive DP synthetic
0.146
0.499
0.679
0.848
0.831
Table 1: Summary: 95% CI Coverage across Methods and Privacy Levels (Gaussian Model, n=1000 ).
Figure 3: Coverage of 95% intervals across privacy levels for Gaussian, logistic, and Poisson models, each at two sample sizes. The shaded band marks an acceptable calibration range around nominal coverage. Noise-calibrated DP methods remain near nominal while naive synthetic analysis undercovers in the low- ε regime.
Figure 4: Average confidence-interval length versus ε for the same settings as Figure 3 . Noise-aware methods are wider at strong privacy (small ε ) and contract as ε increases, reflecting the expected privacy-accuracy trade-off. Naive synthetic intervals remain narrow but are miscalibrated.
Figure 5: Logistic-regression clipping study comparing DP plug-in and DP noise-aware estimators. Left: average absolute bias versus clipping radius B . Right: empirical 95% coverage versus B . Both methods exhibit a U-shaped bias curve: too-small B causes clipping bias while too-large B increases noise through higher sensitivity. The noise-aware estimator provides no advantage over plug-in, consistent with Proposition 1 .
Figure 6: Log-log MSE scaling for Gaussian mean estimation. Solid lines are empirical MSE for each ε ; dashed color-matched lines are theoretical predictions; the thin black dashed line is non-private 1/n . The observed two-regime behavior matches the predicted 1/n+1/(n2ε2) structure.
Figure 7: Power curves and type-I error diagnostics for Gaussian Wald tests across four privacy levels. At Δ=0 , calibrated DP methods remain near the nominal 0.05 level while naive synthetic testing is inflated. Power increases with effect size and approaches non-private behavior as ε grows.
Figure 8: Real-data ACSIncome evaluation across (n,ε) configurations. The top panel reports average coefficient coverage and the bottom panel reports average CI length. DP plug-in and noise-aware methods maintain substantially better calibration than naive synthetic analysis, with the largest gains in low- n , low- ε settings.
Figure 9: Coverage comparison for direct DP inference, noise-aware synthetic analysis, and naive synthetic analysis across synthetic sample-size ratios. Direct and noise-aware approaches remain near nominal coverage, while naive analysis systematically undercovers. Larger synthetic datasets do not eliminate the need for privacy-aware uncertainty correction.
Differential privacy (DP) provides robust privacy guarantees for statistical inference, but this can lead to unreliable results and biases in downstream applications. While several noise-aware approaches have been proposed which integrate DP perturbation into the inference, they are limited to specific types of simple probabilistic models. In this work, we propose a novel method for noise-aware approximate Bayesian inference based on stochastic gradient variational inference which can also be applied to high-dimensional and non-conjugate models. We also propose a more accurate evaluation method for noise-aware posteriors. Empirically, our inference method has similar performance to existing methods in the domain where they are applicable. Outside this domain, we obtain accurate coverages on high-dimensional Bayesian linear regression and well-calibrated predictive probabilities on Bayesian logistic regression with the UCI Adult dataset.
Differentially private (DP) pipelines often combine heterogeneous mechanisms, yet their guarantees are best analyzed using different notions of privacy (e.g., bounded range, zCDP, ADP). When composing them, we have to analyze the guarantees through a reduced common representation that loses accounting precision. This issue is particularly relevant in DP synthetic data generation, where data preprocessing, generation algorithms, and DP-SGD could appear within the same pipeline. We develop novel tools for tight end-to-end accounting of such compositions using privacy loss distributions (PLD), f-DP, and Gaussian DP (GDP). In particular, we derive PLD and GDP analyses of bounded-range mechanisms such as the Exponential mechanism. Building on these theoretical results, we introduce AIM-GDP, a GDP-native variant of AIM (a state-of-the-art DP synthetic data mechanism) that enables easy composability with complex pipelines via PLD, attack-aware risk calibration, and tight end-to-end auditing while consistently achieving better utility; and MixCal, an algorithm for calibrating adaptive mixed compositions using PLD accounting. Overall, our tools enable tighter end-to-end accounting and thus less noise at the same privacy level in DP synthetic data pipelines.
Differential privacy (DP) has become the gold standard for ensuring the privacy protection of machine learning and statistical algorithms in recent decades. A plethora of algorithms and methods have been developed to enhance the utility of DP algorithms while maintaining the same level of DP. However, these are often overly complex or computationally ineffective. We propose a novel approach focusing on denoising the output of the simple additive Gaussian mechanism by adopting the idea of empirical Bayes estimation. We highlight that the empirical Bayes approach can reduce the mean-squared error solely by taking the output of the Gaussian mechanism as input. Our numerical studies show that this simple yet powerful approach can be applied to improve upon various statistical problems, including histogram release, principal component analysis, and linear regression, often outperforming existing private algorithms.
Minwoo Kim, Junyong Park, Sungkyu Jung
Department of Statistics Seoul National University Seoul, 08826, South Korea · Department of Statistics and Institute for Data Innovation in Science Seoul National University Seoul, 08826, South Korea