cs.LGMar 5, 2026

Differential Privacy in Two-Layer Networks: How DP-SGD Harms Fairness and Robustness

Authors: Ruichen Xu, Kexin Chen

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Differentially private learning is essential for training models on sensitive data, but empirical studies consistently show that it can degrade performance, introduce fairness issues like disparate impact, and reduce adversarial robustness. The theoretical underpinnings of these phenomena in modern, non-convex neural networks remain largely unexplored. This paper introduces a unified feature-centric framework to analyze the feature learning dynamics of differentially private stochastic gradient descent (DP-SGD) in two-layer ReLU convolutional neural networks. Our analysis establishes test loss bounds governed by a crucial metric: the feature-to-noise ratio (FNR). We demonstrate that the noise required for privacy leads to suboptimal feature learning, and specifically show that: 1) imbalanced FNRs across classes and subpopulations cause disparate impact; 2) even in the same class, noise has a greater negative impact on semantically long-tailed data; and 3) noise injection exacerbates vulnerability to adversarial attacks. Furthermore, our analysis reveals that the popular paradigm of public pre-training and private fine-tuning does not guarantee improvement, particularly under significant feature distribution shifts between datasets. Experiments on synthetic and real-world data corroborate our theoretical findings.

Figures & tables

Figure 1: Illustration of the privacy-utility phase transition between benign and harmful privacy protection.
Figure 2: Model standard test loss and adversarial test loss.
Figure 3: Visualization of correctly classified (left) and misclassified (right) images.
Padding ratio0%26%50%62%75%
MNIST97%97%95%94%86%
CIFAR-1058%54%51%48%46%
MNIST (adv)95%89%50%20%1%
CIFAR-10 (adv)3%2%1%0%0%
Table 4
Rotation angle0∘0^{\circ}22.5∘22.5^{\circ}45∘45^{\circ}67.5∘67.5^{\circ}
MNIST99%97%94%94%
CIFAR-10 (CNN)70%59%51%49%
CIFAR-10 (ResNet)91%66%43%37%
Table 5
Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Examples on using image padding to control feature sizes. For digits and objects, we pad the images with their background color.
Class 5 Size10050010003000
Accuracy (%)71939697
Accuracy (Adv) (%)60839697
Appendix
Table 7
Padding ratio0%26%50%75%
MNIST93%93%93%91%
MNIST (adv)78%77%66%20%
Appendix
Table 8
 With freezingWithout freezing
Accuracy96.71%95.11%
Appendix
Table 9
More in this topic

Differentially Private Stochastic Gradient Descent

Differentially private stochastic gradient descent (DP-SGD) protects training data by clipping per-example gradients, adding calibrated noise, and accounting for privacy loss.

See all 50 papers

Explore similar work

CardsList

  1. Escaping Iterative Parameter-Space Noise: Differentially Private Learning with a Hypernetwork

    Jun 25, 2026Naoki Nishikawa, Shokichi Takakura, Satoshi HasegawaPrivacy-Preserving MLDifferentially Private Federated Learning

    The University of Tokyo · LY Corporation

  2. DP-KFC: Data-Free Preconditioning for Privacy-Preserving Deep Learning

    May 13, 2026Marc Molina Van den Bosch, Riccardo Taiello, Albert Sund Aillet +3Differentially Private Stochastic Gradient DescentKronecker-Factored Approximation

    CERN, Geneva, Switzerland · BCN MedTech, Department of Engineering, Universitat Pompeu Fabra, Barcelona, Spain · ICREA, Barcelona, Spain

  3. Deep Learning under Fractional-Order Differential Privacy

    May 11, 2026Mohammad Partohaghighi, Roummel MarciaDifferentially Private Stochastic Gradient DescentPrivacy-Preserving ML

    MESA Lab, Dept. of EECS University of California, Merced Merced, CA 95343 · Dept. of Applied Mathematics University of California, Merced Merced, CA 95343