CutClean: Neural Network Pruning for Privacy-Preserving Inference

Leonardo Magliolo; Vito Paolo Pastore; Giuseppe Valenzise; Enzo Tartaglione

cs.LGAug 13, 2026

CutClean: Neural Network Pruning for Privacy-Preserving Inference

Authors: Leonardo Magliolo, Vito Paolo Pastore, Giuseppe Valenzise, Enzo Tartaglione

Organizations: LTCI, Télécom Paris, Institut Polytechnique de Paris, France · MaLGa-DIBRIS, University of Genoa, Italy · Istituto Italiano di Tecnologia, Italy · Universite Paris-Saclay, CNRS, CentraleSupélec, L2S, France

Abstract

Neural networks are increasingly deployed in high-stakes applications with growing privacy leakage concerns. We show that this privacy leakage can occur even in the absence of representation imbalances that lead to traditional dataset biases. This poses significant privacy risks when deploying models that process sensitive attributes. In this context, we propose CutClean, a privacy-aware pruning method that allows to reduce privacy information flow through the network, while increasing its sparsity. Our approach employs auxiliary linear privacy heads placed at each network's block to quantify information leakage, and further applies increasing levels of sparsity to remove the private attribute leakage, measured in terms of the accuracy of the privacy head attached to the last block. Experiments on synthetic and real-world datasets demonstrate that our approach effectively minimizes private information flow while achieving high sparsity rates and preserving classification target accuracy.

Figures & tables

Figure 1: Schematic overview of CutClean.

$\gamma$	Target Accuracy (%)	Last PH accuracy (%)
0	$91.71\pm 0.84$	$90.61\pm 0.45$
0.01	$92.17\pm 0.79$	$87.68\pm 0.58$
0.10	$89.93\pm 0.58$	$74.13\pm 1.69$
1	$89.15\pm 1.85$	$71.34\pm 2.38$
5	$89.47\pm 0.75$	$71.15\pm 0.67$
10	$89.61\pm 0.26$	$69\pm 0.66$

Table 1: Target and last privacy head accuracy on the validation set for different values of

\gamma

for Celeb-A "blonde" attribute.

Model	Target Accuracy (%)	Last PH Accuracy (%)	Sparsity (%)
Baseline	$91.81\pm 0.85$	$87.59\pm 1.14$	0
CutClean	$90.32\pm 0.64$	$62.73\pm 1.24$	60
CutClean w/o fine-tuning	$88.84\pm 1.16$	$61.02\pm 2.36$	5

Table 2: Target accuracy, last privacy head accuracy on the test set and sparsity level on Celeb-A "blonde" attribute.

$\gamma$	Target Accuracy (%)	Last PH accuracy (%)
0	$74.07\pm 3.46$	$81.48\pm 7.29$
0.01	$78.70\pm 1.31$	$85.19\pm 7.29$
0.10	$74.07\pm 5.71$	$78.70\pm 4.72$
1	$73.15\pm 4.72$	$76.85\pm 1.31$
$\gamma^{\star}=5$	$75\pm 3.93$	62.04 $\pm$ 14.76
10	$70.37\pm 2.62$	$63.89\pm 3.93$

Table 3: Target and last privacy head accuracy on the validation set for different values of

\gamma

for Celeb-A "Heavy make-up" attribute.

Model	Target Accuracy (%)	Last PH Accuracy (%)	Sparsity (%)
Baseline	$77.65\pm 4.76$	$80.30\pm 0.53$	0
CutClean	$73.11\pm 6.29$	$64.39\pm 4.98$	30
CutClean w/o fine-tuning	$55.30\pm 9.19$	$59.47\pm 7.56$	40

Table 4: CutClean performance on Celeb-A, "Heavy make-up" attribute

$\gamma$	Target Accuracy (%)	Last PH accuracy (%)
0	$83.89\pm 0.27$	$64.41\pm 0.81$
0.01	$84.05\pm 0.51$	$63.92\pm 0.84$
0.10	$82.07\pm 0.59$	$41.93\pm 0.54$
1	$80.09\pm 0.13$	$25.44\pm 0.79$
5	$79.53\pm 0.58$	$19.67\pm 0.51$
10	$78.11\pm 0.49$	$17.69\pm 0.54$

Table 5: Target and last privacy head accuracy on the validation set for different values of

\gamma

for Corrupted-CIFAR10.

Model	Target Accuracy (%)	Last PH Accuracy (%)	Sparsity (%)
Baseline	$84.23\pm 0.49$	$62.87\pm 1.16$	0
CutClean	$81.36\pm 0.36$	$13.33\pm 0.50$	10
CutClean w/o fine-tuning	$75.54\pm 0.33$	$14.59\pm 0.97$	0

Table 6: Target accuracy, last privacy head accuracy on the test set and sparsity level on Corrupted-CIFAR10.

$\gamma$	Target Accuracy (%)	Last PH accuracy (%)
0	$90.31\pm 0.07$	$91.47\pm 0.26$
0.01	$90.46\pm 0.51$	$90.36\pm 0.93$
0.10	$89.46\pm 0.61$	$71.64\pm 1.47$
$\gamma^{\star}=1$	$89.26\pm 0.19$	69.23 $\pm$ 3.50
5	$88.20\pm 0.84$	$75.05\pm 0.94$
10	$88\pm 0.70$	$77.51\pm 2.86$

Table 7: Target and last privacy head accuracy on the validation set for different values of

\gamma

for Waterbirds.

Model	Target Accuracy (%)	Last PH Accuracy (%)	Sparsity (%)
Baseline	$88.76\pm 0.51$	$88.96\pm 0.61$	0
CutClean	$86.55\pm 0.75$	$64.56\pm 1.48$	20
ClutClean w/o fine-tuning	$50.00\pm 0.01$	$50.00\pm 0.01$	95

Table 8: Target accuracy, last privacy head accuracy on the test set and sparsity level on waterbirds.

$\gamma$	Target Accuracy (%)	Last PH accuracy (%)
0	$90.20\pm 1.60$	$86.86\pm 0.39$
0.01	$89.38\pm 0.97$	$86.08\pm 1.04$
0.10	$87.13\pm 1.19$	$76.28\pm 2.69$
1	$87.96\pm 0.26$	$73.58\pm 1.80$
5	$87.41\pm 1.13$	$76.42\pm 2.64$
10	$87.59\pm 1.92$	$70.42\pm 1.72$

Table 9: Target and last privacy head accuracy on the validation set for different values of

\gamma

for Celeb-A "blonde" attribute, employing a ViTb-16 model.

Model	Target Accuracy (%)	Last PH Accuracy (%)	Sparsity (%)
Baseline	$90.19\pm 1.04$	$86.90\pm 1.91$	0
Without retrain after pruning	$79.03\pm 3.01$	$64.02\pm 4.02$	10
With retrain after pruning	$84.72\pm 2.14$	$61.73\pm 2.84$	50

Table 10: Target accuracy, last privacy head accuracy on the test set and sparsity level on Celeb-A "blonde" attribute, employing a ViTb-16 model.

Figure 2: Target and privacy head accuracy on the test set for the investigated datasets for CutClean (Left) and CutClean without fine-tuning (Right). Results are averaged across three runs, standard deviations in the shaded area.

Privacy-Preserving ML

Privacy-preserving machine learning develops methods to learn, share, and run models while limiting exposure of sensitive data.

See all 433 papers

Prune the Leaky Neurons: Mitigating Private Data Leakage in Fine-Tuned LLMs via Joint-Embedding-Guided Pruning
Oct 16, 2026Mohamed Shaaban, Mohamed ElmahallawyPrivacy Leakage in Language Models Privacy-Preserving Language Models
School of Engineering and Applied Sciences, Washington State University, Richland, WA 99354, USA
Hyperflux: Pruning Reveals Importance
Apr 6, 2025Eugen Barbulescu, Antonio Alexoaie, Lucian BusoniuSparse Neural Networks Neural Network Training Dynamics
Department of Computer Science Technical University of Cluj-Napoca · Department of Automation Technical University of Cluj-Napoca
Pruning Deep Neural Networks via the Marchenko--Pastur Distribution
May 23, 2026Leonid Berlyand, Theo Bourdais, Houman Owhadi +1Structured Pruning Sparse Neural Networks
Department of Mathematics Pennsylvania State University University Park, PA 16802, USA · Department of Computing and Mathematical Sciences California Institute of Technology Pasadena, CA 91125, USA

Oct 16, 2026J/K move · Enter open · S save

Prune the Leaky Neurons: Mitigating Private Data Leakage in Fine-Tuned LLMs via Joint-Embedding-Guided Pruning

Mohamed Shaaban, Mohamed Elmahallawy

School of Engineering and Applied Sciences, Washington State University, Richland, WA 99354, USA

Privacy Leakage in Language Models Privacy-Preserving Language Models LLM Pruning Neural Network Pruning

Open paper AskPDF arXiv

Fine-tuning large language models (LLMs) on proprietary records enables domain adaptation but introduces two challenges: (i) privacy leakage of memorized private data, such as personally identifiable information (PII), and (ii) balancing private data removal with utility preservation. Existing defenses, including data sanitization and differential privacy, often degrade performance, while privacy-neuron editing lacks a PII-free reference to distinguish identity-specific neurons from those supporting useful text. We propose JEPA-Prune, a post-hoc neuron pruning framework guided by a model trained exclusively on PII-free views. Alongside the fine-tuned private model, JEPA-Prune, trains a public model on surrogate and masked views of the same records using a joint-embedding predictive architecture (JEPA), capturing domain knowledge without identifiers. A cross-model representation probe quantifies deviations between private representations of leaked records and this PII-free reference. Integrated gradients and a scale-invariant specificity score then identify PII-responsive neurons, which are pruned without retraining. Experiments show that pruning just 1.6% of neurons reduces PII inference accuracy by up to 7.4x and PII extraction recall by up to 13.9x, while keeping unseen-text perplexity within 16% of that of the unpruned model for most LLMs. In contrast, privacy-neuron editing increases perplexity by at least 26x.

Abstract

Privacy-Preserving ML

Explore similar work

Prune the Leaky Neurons: Mitigating Private Data Leakage in Fine-Tuned LLMs via Joint-Embedding-Guided Pruning

Hyperflux: Pruning Reveals Importance

Pruning Deep Neural Networks via the Marchenko--Pastur Distribution