Mitigating Membership Inference in Intermediate Representations with Differentially Private Training
Authors: Jiayang Meng, Tao Huang, Chen Hou, Guolong Zheng, Hong Chen
Organizations: School of Information, Renmin University of China, Beijing, China · School of Computer Science and Big Data, Minjiang University, Fuzhou, Fujian, China
In Embedding-as-an-Interface (EaaI) settings, pre-trained models are queried for Intermediate Representations (IRs). The distributional properties of IRs can leak training-set membership signals, enabling Membership Inference Attacks (MIAs) whose strength varies across layers. Although Differentially Private Stochastic Gradient Descent (DP-SGD) mitigates such leakage, existing implementations employ per-example gradient clipping and a uniform, layer-agnostic noise multiplier, ignoring heterogeneous layer-wise MIA vulnerability. This paper introduces Layer-wise MIA-risk-aware DP-SGD (LM-DP-SGD), which adaptively allocates privacy protection across layers in proportion to their MIA risk. Specifically, LM-DP-SGD trains a shadow model on a public shadow dataset, extracts per-layer IRs from its train/test splits, and fits layer-specific MIA adversaries, using their attack error rates as MIA-risk estimates. Leveraging the cross-dataset transferability of MIAs, these estimates are then used to reweight each layer's contribution to the globally clipped gradient during private training, providing layer-appropriate protection under a fixed noise magnitude. We further establish theoretical guarantees on both privacy and convergence of LM-DP-SGD. Extensive experiments show that, under the same privacy budget, LM-DP-SGD reduces the peak IR-level MIA risk while preserving utility, yielding a superior privacy-utility trade-off.
Figures & tables
Figure 1: An overview of LM-DP-SGD, which comprises two components: (i) Layer-wise MIA-risk estimation (Section 4.1 ), which trains layer-specific adversaries on a public shadow dataset to assess MIA risk of each layer; and (ii) Private training via LM-DP-SGD (Section 4.3 ), a differentially private optimization procedure that leverages these estimated risks to perform layer-wise reweighted clipping to per-example gradients before noise injection, allocating protection proportionally to layer vulnerability.
Figure 2: MIA accuracy using intermediate representations from different convolutional layers.
DP-SGD Method
Gradient Norm Constraint Mechanism
DP-SGD
Fixed clipping
Auto-S/NSGD
Adaptive clipping/Normalization
DP-PSAC
Adaptive clipping/Normalization
LM-DP-SGD (Ours)
Layer-wise reweighted clipping
Table 1: Conceptual comparison of gradient norm constraint mechanisms across DP-SGD methods. bt captures the expected deviation between the constrained and true gradients.
Training
Target
Shadow
Privacy
Sampling
Learning
Clipping
Layer-wise MIA-Risk Heterogeneity
Dataset D
Model F
Dataset Dshadow
Budget (ε,δ)
Probability q
Rate ηt
Threshold C
Emphasis Factor r
MNIST
S-CNN
FashionMNIST
(5.0,1×10−5)
0.01
0.08
1.0
5.0
CIFAR10
D-CNN
ImageNet
(8.0,1×10−5)
0.01
0.10
2.0
3.0
CIFAR100
ResNet-18
ImageNet
(8.0,1×10−5)
0.01
0.10
3.0
2.0
CelebA
VGG-16
FFHQ
(8.0,1×10−5)
0.01
0.08
3.0
2.0
Table 2: Hyperparameter settings. Specifically, for MNIST, we follow the setup in ( Shamsabadi and Papernot, 2023 ) , employing a 6-layer Shallow-CNN (S-CNN). For CIFAR10, we adopt the configuration from ( Papernot et al., 2021 ) , using a 12-layer Deep-CNN (D-CNN).
Model
DP-SGD
Index of Intermediate
MIA Accuracy
MIA Accuracy
Maximum MIA Accuracy across
(Dataset)
Method
Convolutional-Layer
under SGD (%)
under DP-SGD (%)
Layers under DP-SGD (%)
DP-SGD
1, 2
72.5, 75.0
67.2, 70.9
70.9
S-CNN
Auto-S/NSGD
66.6, 70.0
70.0
(MNIST)
DP-PSAC
67.7, 71.8
71.8
LM-DP-SGD (Ours)
68.3, 69.2
69.2
DP-SGD
1, 3, 5, 7, 8
59.6, 64.9, 68.3, 71.6, 73.1
57.0, 63.2, 65.5, 67.3, 69.8
69.8
Table 3: Evaluation of MIA accuracy on partial intermediate convolutional-layer representations. The maximum MIA accuracy across layers indicates the maximum IR-level MIA risk, reflecting the overall vulnerability to MIAs at the IR level. Bolded values highlight the lowest peak MIA accuracy across layers among all methods, identifying the most effective method for mitigating IR-level MIA risk.
Figure 3: Evolution of test accuracy during training. The optimal layer-wise reweighting coefficients for convergence, {wt(l)∗}l=1L , are derived via Lagrange multipliers (see Appendix D.2 ). For clarity, all curves are smoothed using a Savitzky-Golay filter. The shaded regions denote performance within 2% below the test accuracy of the baseline which achieves the maximum final accuracy. The results show that our method preserves model utility relative to the baselines. Importantly, this does not imply superior test accuracy; rather, performance remains comparable. This conclusion is supported by two observations. First, due to curve smoothing, minor deviations should be interpreted as smoothing artifacts rather than as meaningful performance differences; Second, once training converges, LM-DP-SGD’s performance consistently lies within the shaded regions. Overall, these results indicate that our method preserves utility.
Figure 4: Evolution of the ℓ2 -norm of the bias term bt , ∥bt∥2 . For visual clarity, all curves are smoothed using a Savitzky-Golay filter. We observe that the magnitude of ∥bt∥2 remains within the same order across all methods throughout training. Compared to the baselines, our method does not introduce excessive bias; in fact, it yields a relatively lower gradient bias over the majority of training.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Symbol
Description
F(x;W)
Target model with input x and parameters W .
Fshadow(x;W)
Shadow model with input x and parameters W .
I(l)(x)
Intermediate representation of input x at layer l of the trained shadow model.
Fadv(l)(I(l)(x);W(l))
Adversarial model targeting layer- l representations for membership inference attacks.
D
Private training set with N examples.
Dshadow
Shadow dataset with Ns examples.
Appendix
Table 4: Key symbols and descriptions used in LM-DP-SGD.
Shadow Dataset
Index of Intermediate Convolutional-Layer
Layer-wise MIA Error Rates (%)
ImageNet
1, 3, 5, 7, 8
40.4, 35.1, 31.7, 28.4, 26.9
SVHN
40.2, 35.0, 31.4, 28.1, 26.7
MNIST
39.9, 34.8, 31.4, 28.1, 26.5
VGGFace2
40.8, 35.5, 31.9, 28.7, 27.2
Appendix
Table 5: Layer-wise MIA error rates across shadow datasets.
Figure 5: Impact of privacy budget ε .
Figure 6: Impact of clipping threshold C .
Figure 7: Impact of heterogeneity emphasis factor r .
Heterogeneity Emphasis Factor r
Index of Intermediate Convolutional-Layer
MIA Accuracy under LM-DP-SGD (%)
1.0
1, 3, 5, 7, 8
57.4, 63.7, 65.5, 67.5, 69.4
2.0
57.8, 63.9, 65.4, 67.2, 69.1
3.0
59.0, 64.3, 65.2, 66.5, 67.9
4.0
60.3, 64.8, 64.9, 65.8, 67.2
5.0
60.9, 65.2, 64.6, 65.4, 66.9
Appendix
Table 6: MIA accuracy under varying r values.
Dataset
Model
DP-SGD
Auto-S/NSGD
DP-PSAC
LM-DP-SGD
MNIST
S-CNN
0.0984
0.1071
0.1049
0.1078
CIFAR10
D-CNN
0.1516
0.1569
0.1536
0.1632
CIFAR100
ResNet-18
0.2735
0.2845
0.2857
0.2860
CelebA
VGG-16
0.2557
0.2593
0.2615
0.2654
Appendix
Table 7: Average running time (seconds) per iteration.
Differentially private stochastic gradient descent (DP-SGD) protects training data by clipping per-example gradients, adding calibrated noise, and accounting for privacy loss.
Recent work in the privacy literature shows that sample-targeted membership inference attacks (MIAs) significantly outperform untargeted approaches by a wide margin. Motivated by this observation, we address the following question: can the privacy vulnerability of individual training points be assessed without training shadow models? We show that per-sample exposure to MIA is governed not only by a point's loss, but also by a data-dependent geometric measure. In the linear setting, we derive a closed-form decomposition of individual black-box MIA vulnerability into a population leverage score and a residual loss term, making explicit how sample-dependent geometry translates into privacy exposure. Since the final layer of most modern architectures is linear, we extend this framework to deep networks and propose a surrogate score operating on last-layer representations that requires only a single trained model and no shadow models. Empirical evaluations across diverse datasets and architectures show that our score outperforms loss and gradient-norm baselines at identifying the highest-risk points under state-of-the-art attacks, providing a computationally efficient and theoretically grounded tool for per-sample privacy risk assessment.
Modern AI models are not static. They go through multiple updates in their lifecycles. We propose to design Sequential Membership Inference (SeMI) attacks leading to tighter privacy audits by exploiting the sequence of models and injecting a target canary at a controlled insertion time. First, for empirical mean computation, we develop SeMI*, an {optimal SeMI attack to identify the presence of a target inserted at a specific insertion step}. We derive the power of SeMI* to show that accessing the model sequence yields more powerful MI attacks than scrutinising only the final model. SeMI* exhibits an isolation property -- its power depends on the statistics obtained right before and after insertion of the target. Leveraging this insight, we develop practical white-box (accessing model gradients) and black-box (accessing loss) SeMI attacks against models trained with (DP-)SGD. Across datasets and models trained with (DP-)SGD, our experiments show that SeMI attacks achieve higher powers than snapshot-independent baselines, and yield tighter privacy audits thanks to (a) control over the insertion time and (b) observations across the model sequence.
Thomas Michel, Debabrota Basu, Emilie Kaufmann
Univ. Lille, Inria, CNRS, Centrale Lille UMR 9189-CRIStAL, France
While Federated Learning (FL) mitigates direct data exposure, the resulting trained models remain susceptible to membership inference attacks (MIAs). This paper presents an empirical evaluation of Differential Privacy (DP) as a defense mechanism against MIAs in FL, leveraging the environment of the 2025 NIST Genomics Privacy-Preserving Federated Learning (PPFL) Red Teaming Event. To improve inference accuracy, we propose a stacking attack strategy that ensembles seven black-box estimators to train a meta-classifier on prediction probabilities and cross-entropy losses. We evaluate this methodology against target models under three privacy configurations: an unprotected convolutional neural network (CNN, ε=∞), a low-privacy DP model (ε=200), and a high-privacy DP model (ε=10). The attack outperforms all baselines in the No DP and Low Privacy settings and, critically, maintains measurable membership leakage at ε=200 where a single-signal LiRA baseline collapses. Evaluated on an independent third-party benchmark, these results provide an empirical characterisation of how stacking-based inference degrades across calibrated DP tiers in FL.