Organizations: Department of Mathematics, Imperial College London, London SW7 2AZ, UK · Institute for Mathematical Stochastics, University of Münster, Münster, Germany · Department of Statistics, University of Oxford, Oxford OX1 3LB, UK
Rènyi differential privacy, based on Rènyi divergence, has become a notion of choice for privacy guarantees. However, outside the Gaussian setting, Rènyi divergence can be difficult to compute. In particular, Rènyi differential privacy guarantees that exploit task-specific information, as in Rènyi Pufferfish privacy, can be challenging. Here we use kernel Stein discrepancies based on escort distributions to provide certificates for Rènyi differential privacy. By designing the kernel to focus on the relevant privacy directions, we also derive task-aware privacy certificates. We provide exact identities for Gaussian and Laplace mechanisms and bounds for exponential mechanisms, with separate task-aware guarantees. A real data set example illustrates how the protected comparison set and the released subspace jointly determine the resulting certificates.
Figures & tables
α
Dα
est. KSD bound ( 37 )
bound / Dα
fallback ( 38 )
tD (s)
tKSD (s)
CH
1 (KL)
5.588e−04
1.300e−03
2.327
1.385e−03
1.75
0.57
0.4810
sd 2.98e−05
sd 4.69e−05
2
1.137e−03
2.868e−03
2.521
3.086e−03
2.83
0.56
0.5078
sd 5.12e−05
sd 1.03e−05
4
2.463e−03
7.171e−03
2.911
7.812e−03
5.21
0.58
0.5713
sd 6.60e−05
sd 2.65e−04
Table 1: Dense-quartic order study; 50 independent MCMC repetitions. The order α=1 row is the KL limit. The KSD bound is the estimated normalized task-aware privacy certificate ( 37 ) (with the MCMC estimate for the escort moment). The fallback option is the cruder bound ( 38 ). The tD and tKSD columns show the runtimes.
Figure 1: The full Gaussian release and its one-dimensional task-aware release using the same Gaussian noise. Solid curves show the analytic certificates, and dashed curves show the empirical 95th percentiles over sampled replace-one Texas training pairs.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 2: Numerical validation of Theorem 4.3 : varying the size of the displacement
Figure 3: Numerical validation of Theorem 4.3 : varying the angle
Figure 4: The KSD approximation to RDP for the quartic exponential family. The “exact” RDP is an approximation obtained by numerical quadrature.
Figure 5: Rényi divergence for a fixed neighbouring natural-parameter displacement δ=d1(1,…,1)⊤ in the coupled quartic exponential family, with d=20 . The solid and dashed curves are numerical RDP estimates obtained from the mathematically equivalent variance-path and mean-path thermodynamic-integration identities ( 36 ) and ( 35 ). Their agreement provides an MCMC and quadrature diagnostic. The dash-dotted curve is the rigorous strong-convexity/KSD upper bound α∥δ∥22/(2μ) . The shaded region is an approximate Monte Carlo uncertainty interval for the mean-path estimate.
Figure 6: Privacy bounds; top: full release; bottom: projected release. Solid curves show clipping-based upper bounds that are valid for all admissible fixed contexts. Dashed curves show exact maxima of KSD2/(2α)=Dα over the evaluated contexts and specified secret comparisons.