cs.LGFeb 18, 2026

Learning with Locally Private Examples by Inverse Weierstrass Private Stochastic Gradient Descent

Authors: Jean Dufraiche, Paul Mangold, Michaël Perrot, Marc Tommasi

Organizations: Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189 - CRIStAL, F-59000 Lille, France · CMAP, CNRS, École polytechnique, Institut Polytechnique de Paris, 91120 Palaiseau, France

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Releasing data once and for all under noninteractive Local Differential Privacy (LDP) enables complete data reusability, but the resulting noise may create bias in subsequent analyses. In this work, we leverage the Weierstrass transform to characterize this bias in binary classification. We prove that inverting this transform leads to a bias-correction method to compute unbiased estimates of nonlinear functions on examples released under LDP. We then build a novel stochastic gradient descent algorithm called Inverse Weierstrass Private SGD (IWP-SGD). It converges to the true population risk minimizer at a rate of O(1/n)\mathcal{O}(1/n), with nn the number of examples. We empirically validate IWP-SGD on binary classification tasks using synthetic and real-world datasets.

Figures & tables

Figure 1: Comparison of SGD convergence of the exponential loss under (2,10−5)(2,10^{-5}) -LDP for the 2-dimensional synthetic data and (5,10−5)(5,10^{-5}) -LDP for the 10-dimensional synthetic data.
Figure 2: Comparison of SGD convergence of the exponential loss under (2,10−5)(2,10^{-5}) -LDP on ACSPublicCoverage and ACSIncome. Left hand plots show the averaged loss of fitted model (θn)\left(\theta_{n}\right) while right hand plots are showing the loss of the averaged model (Eθn)\left({\mathbb{E}}\theta_{n}\right) over random draws.
Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Approximation of the truncation error biasKbias_{K} for K∈{1,2,3}K\in\{1,2,3\} .
Figure 4: Comparison of Accuracy convergence of the model fitted on exp loss under (2,10−5)(2,10^{-5}) -LDP on ACSPublicCoverage and ACSIncome.
Figure 5: Comparison of SGD convergence of the log loss under (2,10−5)(2,10^{-5}) -LDP for the 2-dimensional synthetic data and (5,10−5)(5,10^{-5}) -LDP for the 10-dimensional synthetic data.
Figure 6: Comparison of SGD convergence of the log loss under (2,10−5)(2,10^{-5}) -LDP on ACSPublicCoverage and ACSIncome.
Figure 7: Comparison of Accuracy convergence of the model fitted on log loss under (2,10−5)(2,10^{-5}) -LDP on ACSPublicCoverage and ACSIncome.
Figure 8: Comparison of SGD convergence under (2,10−5)(2,10^{-5}) -LDP on ACSIncome linear regression variant.
More in this topic

Differential Privacy

Differential privacy limits how much an algorithm’s outputs can reveal about any one person’s data, with formal guarantees.

See all 276 papers

Explore similar work

CardsList

  1. PRISM: Gauge-Invariant Tangent-Space Differentially Private LoRA

    May 31, 2026Shihao Wang, Xueru ZhangDifferentially Private Stochastic Gradient DescentLow-Rank Adaptation

    Department of Computer Science and Engineering, The Ohio State University, Columbus, OH, USA.

  2. Efficient Agnostic Learning of Univariate Densities Under Local Differential Privacy

    Oct 15, 2026Mohammad Afzali, Hassan Ashtiani, Sivan SabatoDifferential PrivacyDensity Estimation

    University of British Columbia · McMaster University · Vector Institute +1

  3. Jacobian-Guided Anisotropic Noise Reshaping for Enhancing Representation Utility under Local Differential Privacy

    May 16, 2026Youngmok Ha, Viktor Schlegel, Yidan Sun +1Privacy-Preserving MLDifferential Privacy

    Department of Bioengineering, Imperial College London, United Kingdom · Artificial Intelligence Computing Research Laboratory, Electronics and Telecommunications Research Institute (ETRI), Daejeon, Republic of Korea · Imperial Global Singapore, Imperial College London, Singapore +1