stat.MLNov 19, 2026

Differentially Private Score Estimation with Applications to Score-Based Generative Models

Authors: Mengxiao Gao, Elly K. H. Hung, Yuchen Wu, Yi Yu

Organizations: School of Operations Research and Information Engineering, Cornell University · Department of Statistics, University of Warwick

Abstract

Estimating the score function of a distribution plays a central role in statistics and machine learning, with applications in distribution estimation, sampling, and score-based generative modeling. However, learning scores from sensitive data raises privacy concerns. These concerns are particularly pronounced in score-based generative modeling, where trained models may memorize individual training examples and expose sensitive information through membership inference and data extraction attacks. Although non-private score estimation has been extensively studied, the statistical limits of its private counterpart remain less understood. In this work, we study score estimation under (α,β)(α,β)-differential privacy for dd-dimensional sub-Gaussian distributions with locally γγ-Hölder-continuous scores. We develop a sampling-based private score estimator using quadratic B-splines and the exponential mechanism, and we implement our proposal through projected Langevin Monte Carlo with runtime polynomial in the sample size nn. The proposed estimator satisfies (α,β)(α,β)-differential privacy, and achieves expected density-weighted squared error \mboxpolylog(n)max⁡(n−2γ/(d+2γ+2), (nα)−2γ/(d+γ+1))\mbox{polylog}(n) \max(n^{-2γ/(d+2γ+2)},\,(nα)^{-2γ/(d+γ+1)}). We also establish a nearly matching private minimax lower bound, thereby characterizing the privacy-accuracy trade-off for score estimation. As an application, we construct a differentially private diffusion model whose privacy guarantee is independent of the number of denoising steps. We show that, with an appropriate discretization scheme, the resulting diffusion model attains nearly minimax optimal expected total variation error among all private distribution estimators. We validate our theoretical findings through experiments on both synthetic and real-world datasets.

Figures & tables

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Sobolev Regularized Score Difference Estimation in Diffusion Models

    Aug 18, 2026Chenghan Xie, Jose Blanchet, Renyuan XuDiffusion ModelsScore Matching

  2. Diffusion-based Denoising Beats Vanilla Score Matching in Parameter Estimation: A Theoretical Explanation

    May 21, 2026Benedikt Lütke Schwienhorst, Nadja Klein, Johannes LedererScore MatchingParameter Estimation

  3. Robustness of Diffusion Models under Distribution Shift

    Sep 23, 2026Wei Luo, Neil K. Chada, Shijie Zhang +1Distribution Shift RobustnessDiffusion Sampling