Differentially Private Synthetic Data via Generalized Moment Matching

Rundong Ding; Yiyun He; Yizhe Zhu

stat.MLOct 21, 2026

Differentially Private Synthetic Data via Generalized Moment Matching

Authors: Rundong Ding, Yiyun He, Yizhe Zhu

Organizations: Department of Mathematics, University of Southern California · Department of Mathematics, University of California San Diego

Abstract

We study $(\varepsilon,δ)$ -differentially private synthetic data for arbitrary databases of $n$ records in $[-1,1]^d$ under a specified integral probability metric (IPM). We introduce generalized moment matching, which estimates moments privately, fits a probability measure, and samples synthetic records. A general error theorem applies when moment error controls the target IPM. This framework extends Chebyshev moment matching for $1$ -Wasserstein distance and smooth queries. We apply the framework to maximum mean discrepancy with Matern and Laplace kernels and a two-layer ReLU neural tangent kernel. We also obtain guarantees for sliced Wasserstein- $1$ and the IPM induced by norm-bounded ReLU networks. With sufficiently many synthetic records,these rates are minimax optimal up to logarithmic factors for fixed dimension.

Figures & tables

\|v_{q}-\widehat{\theta}\|_{*}\leq\min_{v\in\mathcal{C}_{G}}\|v-\widehat{\theta}\|_{*}+e_{\mathrm{opt}}.

Algorithm 1 Differentially private generalized moment matching

Kernel or IPM	Rate up to logarithmic factors	Moment choice
Gaussian; polynomial, fixed degree	$(n\varepsilon)^{-1}$	Orthogonal feature projections
Matérn, $\nu>0$	$(n\varepsilon)^{-\min\{1,\,\frac{1}{2}+\frac{\nu}{d}\}}$	$\left.\begin{array}[]{@{}c@{}}\rule{0.0pt}{8.13748pt}\\[-1.54996pt] \rule{0.0pt}{8.13748pt}\\[-1.54996pt] \rule{0.0pt}{8.13748pt}\end{array}\right\}$ Wavelet moments
Laplace and ReLU NTK	$(n\varepsilon)^{-\frac{d+1}{2d}}$
Sliced Wasserstein- $1$ , $d\geq 2$	$(n\varepsilon)^{-\frac{d+1}{2d}}$
ReLU IPMs	$(n\varepsilon)^{-\min\{1,\frac{d+3}{2d}\}}$	ReLU moments on a public net

Table 1: Moment choices and minimax rates for fixed dimension and query parameters, up to logarithmic factors in

n\varepsilon

and

1/\delta

Kernel or IPM	Upper-bound ingredients	Lower-bound ingredients
Gaussian / polynomial	Orthogonal kernel features (Corollary 3.1 )	General kernel lower bound (Theorem C.2 )
Matérn	Wavelet moments, with $\ell_{1}$ denoising for $0<\nu<d/2$ (Theorem 3.3 )	General kernel lower bound & geometric construction (Appendix E.4 )
Laplace	Matérn upper bound with $\nu=1/2$	Matérn lower bound with $\nu=1/2$
ReLU NTK	Matérn upper bound with $\nu=1/2$ via RKHS norm equivalence (Lemma E.7 )	Matérn lower bound with $\nu=1/2$ via RKHS norm equivalence (Lemma E.7 )
Sliced Wasserstein- $1$	Laplace MMD upper bound via ( 28 )	Separate information-theoretic argument (Appendix F )
ReLU IPM	Adaptive private moment estimation (Appendix G ) and Theorem 1.2	Matérn lower bound with $\nu=3/2$ via IPM comparison (Appendix G )

Table 2: Proof dependencies for the upper and lower bounds. All upper bounds use the general moment-matching theorem.

Differentially Private Synthetic Data Generation

See all 23 papers

Geometry-Adaptive Mechanisms for Private Synthetic Data
Sep 27, 2026Raoof Zare Moayedi, Amir R. Asadi, Mohammad Hossein Yassaee +1Intrinsic Dimensionality Differentially Private Synthetic Data Generation
Sharif University of Technology · University of Birmingham · The Alan Turing Institute
Fixed-Parameter Tractability of Private Synthetic Data Generation
Jun 9, 2026Badih Ghazi, Cristóbal Guzmán, Pritish Kamath +3Differentially Private Synthetic Data Generation Synthetic Data Generation
Google Deepmind · Institute for Mathematical and Computational Engineering, Faculty of Mathematics and School of Engineering, Pontificia Universidad Cat´olica de Chile · Google Research
Tight End-to-End Accounting for Differentially Private Synthetic Data
Oct 12, 2026Georgi Ganev, Vadym Doroshenko, Arun Ganesh +3Differential Privacy Differentially Private Synthetic Data Generation
UCL · SAS · Google +3

Sep 27, 2026J/K move · Enter open · S save

Geometry-Adaptive Mechanisms for Private Synthetic Data

Raoof Zare Moayedi, Amir R. Asadi, Mohammad Hossein Yassaee, Gholamali Aminian

Sharif University of Technology · University of Birmingham · The Alan Turing Institute

Intrinsic Dimensionality Differentially Private Synthetic Data Generation Synthetic Data Generation Privacy-Preserving ML Differential Privacy Wasserstein Distance

Open paper AskPDF arXiv

Generating differentially private synthetic data with meaningful Wasserstein utility guarantees is challenging in high dimensions. For datasets of size

n

[0,1]^d

with

d\ge2

, existing pure

\varepsilon

-differentially private mechanisms achieve expected

1

-Wasserstein error of order

(\varepsilon n)^{-1/d}

, reflecting the curse of dimensionality. While this rate is optimal in the worst case, it can be overly pessimistic when the data are supported on a lower-dimensional set. We formalize this through a multiscale packing-growth dimension

k

, which captures the geometric complexity of the support via the growth of packing numbers across scales. We propose Adaptive Pruned-PMM, a pure

\varepsilon

-differentially private mechanism that combines private depth selection with our pruned variant of the Private Measure Mechanism (PMM) of He et al.\ (2023). The mechanism supports deeper, geometry-adapted hierarchies with expected running time

O\!\left(d(n+d)\log(\varepsilon n)\right)

, which is near-linear in

n

for fixed dimension and privacy budget. Under an external multiscale packing-growth condition with dimension

k

, we show that, for fixed positive privacy budgets and fixed geometry, the expected

1

-Wasserstein error is of order

(\varepsilon n)^{-1/k}

for

k>1

n

grows. We also prove a lower bound under a corresponding internal packing-growth condition, showing that the exponent

1/k

is sharp within this framework.

Abstract

Differentially Private Synthetic Data Generation

Explore similar work

Geometry-Adaptive Mechanisms for Private Synthetic Data

Fixed-Parameter Tractability of Private Synthetic Data Generation

Tight End-to-End Accounting for Differentially Private Synthetic Data