Privacy-Preserving ML

ML: Machine Learning

The field studies how to train, evaluate, and deploy machine-learning systems while reducing disclosure of personal data, model information, or sensitive patterns. Main approaches include differential privacy during optimization, federated learning with secure aggregation, homomorphic encryption and secure multiparty computation, as well as synthetic data, representation obfuscation, and machine unlearning. Methods are tested across clinical, language, vision, and graph tasks, using measures such as prediction quality, reconstruction and membership-inference attacks, communication, and runtime. A central tension is that federated learning or transformed data do not guarantee privacy on their own, while stronger formal protections can reduce utility or add substantial computational and communication costs.

Latest papers

433 papers37 papers in the last four weeks, down 3% on the four weeks before. 0.2% of all new papers.

May 28, 2025

Private Rate-Constrained Optimization with Applications to Fair Learning

Many problems in trustworthy ML can be expressed as constraints on prediction rates across subpopulations, including group fairness constraints (demographic parity, equalized odds, etc.). In this work, we study such constrained minimization problems under differential privacy (DP). Standard DP optimization techniques like DP-SGD rely on objectives that decompose over individual examples, enabling per-example gradient clipping and noise addition. Rate constraints, however, depend on aggregate statistics across groups, creating inter-sample dependencies that violate this decomposability. To address this, we develop RaCO-DP, a DP variant of Stochastic Gradient Descent-Ascent (SGDA) that solves the Lagrangian formulation of rate constraint problems. Through careful design, the extra privacy cost incurred by incorporating these constraints in our approach is limited to that of privately estimating a histogram over each mini-batch at every step. We prove the convergence of our algorithm through a novel analysis of SGDA that leverages the linear structure of the dual parameter. Empirical results show that our method Pareto-dominates existing private learning approaches under group fairness constraints and also achieves strong privacy-utility-fairness performance on neural networks.

Mohammad Yaghini, Tudor Cebere, Michael Menart +2

University of Toronto · Vector Institute · PreMeDICaL team +3

Primal-Dual Optimization Group Fairness Privacy-Preserving ML Differential Privacy Constrained Optimization Stochastic Gradient Descent

Ask PDF

May 28, 2025

Inclusive Federated Learning Through Compliance-Weighted Noise Allocation in Healthcare AI

Background: Federated learning (FL) enables collaborative training of clinical AI models without centralizing patient data, but adoption is limited by privacy concerns, heterogeneous institutional compliance, and resource disparities; standard differential privacy (DP) applies uniform noise to all clients, penalizing well-compliant or under-resourced institutions. Objective: We introduce a compliance-aware FL framework that adapts DP to institutional compliance, letting lower-compliance sites participate without uniformly penalizing others. Methods: A compliance scoring tool aligned with HIPAA, GDPR, NIST, ISO, and HL7/FHIR maps each client score to a per-step Gaussian noise scale for server-side DP-SGD on a small aggregator dataset. The formal

(ε,δ)

bound applies to the aggregator dataset under a semi-honest aggregator; client-level DP needs secure aggregation (future work). We evaluate five FL strategies on PneumoniaMNIST and BreastMNIST (16 clients, 50 rounds, five seeds); the cumulative aggregator-dataset

ε

is about 1434 (Breast) and 513 (Pneumonia) at

δ=10^{-5}

. Results: Including 12 lower-compliance clients (Experiment 1) versus a compliant-only baseline (Experiment 4) changed BreastMNIST accuracy by +4.5 (FedAvg), +6.8 (FedMedian), +5.2 (FedProx), +1.6 (FedYogi), and -4.1 (FedAdam) percentage points (pooled +2.8 pp; not significant at n=5; up to +17 pp per configuration); compliance-weighted allocation matched uniform server-side DP at equal mean noise (+0.1 pp), carrying no utility penalty, and first-round noise cost 1.3 pp (Breast) and 2.5 pp (Pneumonia, FedAvg). Conclusions: Compliance-weighted server-side DP lets lower-compliance institutions join FL without degrading performance, giving auditable per-site noise control at no utility cost; formal guarantees apply to the aggregator dataset, with client-level DP requiring secure aggregation.

Santhosh Parampottupadam, Melih Coşğun, Sarthak Pati +7

German Cancer Research Center (DKFZ), Division of Medical Image Computing, Heidelberg, Germany · Department of Computer Engineering, Bilkent University, Universiteler 06800 Çankaya/Ankara, Türkiye · Medical Research Group, MLCommons, San Francisco, CA, USA +2

Federated Learning Aggregation Differentially Private Stochastic Gradient Descent Healthcare Privacy-Preserving ML Differentially Private Federated Learning Federated Learning+1

Ask PDF

Apr 2, 2025

CO-DEFEND: Continuous Decentralized Federated Learning for Secure DoH-Based Threat Detection

The use of DNS over HTTPS (DoH) tunneling by an attacker to hide malicious activity within encrypted DNS traffic poses a serious threat to network security, as it allows malicious actors to bypass traditional monitoring and intrusion detection systems while evading detection by conventional traffic analysis techniques. ML techniques can be used to detect DoH tunnels; however, their effectiveness relies on large datasets containing both benign and malicious traffic. Sharing such datasets across entities is challenging due to privacy concerns. In this work, we propose CO-DEFEND framework that enables multiple entities to collaboratively train a classification machine learning model for DoH threat detection while preserving data privacy, enhancing scalability and resilience against single points of failure. The proposed DFL framework provides a realistic implementation for DoH threat detection, enabling multiple entities to train their local models online with incoming DoH flows in real-time batches as they are processed - an approach that fits naturally within modern Internet architectures. This framework adapts four classical machine learning algorithms, Support Vector Machines, Logistic Regression, Decision Trees, and Random Forest, for federated scenarios and efficient training. In addition, a key methodological feature of CO-DEFEND is the use of DT and RF as model selection rather than aggregation mechanisms, allowing each participant to retain interpretable and locally optimal decision structures while benefiting from collective updates. We compare our proposed method by using the dataset CIRA-CIC-DoHBrw-2020 with existing machine learning approaches, including more computationally complex alternatives such as neural networks, to demonstrate its effectiveness in detecting malicious DoH tunnels while improving scalability and computational efficiency.

Diego Cajaraville-Aboy, Marta Moure-Garrido, Carlos Beis-Penedo +5

atlanTTic Research Center – ICLAB – Universidade de Vigo, Vigo, 36310, Spain · Department of Telematic Engineering, Universidad Carlos III de Madrid, Spain

Network Intrusion Detection Encrypted Network Traffic Analysis Privacy-Preserving ML Federated Learning

Ask PDF

Mar 15, 2025

From Laboratory to Real World: A New Benchmark Towards Privacy-Preserved Visible-Infrared Person Re-Identification

Aiming to match pedestrian images captured under varying lighting conditions, visible-infrared person re-identification (VI-ReID) has drawn intensive research attention and achieved promising results. However, in real-world surveillance contexts, data is distributed across multiple devices/entities, raising privacy and ownership concerns that make existing centralized training impractical for VI-ReID. To tackle these challenges, we propose L2RW, a benchmark that brings VI-ReID closer to real-world applications. The rationale of L2RW is that integrating decentralized training into VI-ReID can address privacy concerns in scenarios with limited data-sharing regulation. Specifically, we design protocols and corresponding algorithms for different privacy sensitivity levels. In our new benchmark, we ensure the model training is done in the conditions that: 1) data from each camera remains completely isolated, or 2) different data entities (e.g., data controllers of a certain region) can selectively share the data. In this way, we simulate scenarios with strict privacy constraints which is closer to real-world conditions. Intensive experiments with various server-side federated algorithms are conducted, showing the feasibility of decentralized VI-ReID training. Notably, when evaluated in unseen domains (i.e., new data entities), our L2RW, trained with isolated data (privacy-preserved), achieves performance comparable to SOTAs trained with shared data (privacy-unrestricted). We hope this work offers a novel research entry for deploying VI-ReID that fits real-world scenarios and can benefit the community.

Yan Jiang, Hao Yu, Xu Cheng +3

Center for Machine Vision and Signal Analysis, University of Oulu, Oulu FI-90014, Finland · School of Computer Science, Nanjing University of Information Science and Technology, Nanjing 210044, China

Privacy-Preserving ML Visible-Infrared Person Re-Identification Person Re-Identification

Ask PDF

Mar 10, 2025

Split-n-Chain: Privacy-Preserving Multi-Node Split Learning with Blockchain-Based Auditability

Deep learning, when integrated with a large amount of training data, has the potential to outperform machine learning in terms of high accuracy. Recently, privacy-preserving deep learning has drawn significant attention of the research community. Different privacy notions in deep learning include privacy of data provided by data-owners and privacy of parameters and/or hyperparameters of the underlying neural network. Federated learning is a popular privacy-preserving execution environment where data-owners participate in learning the parameters collectively without leaking their respective data to other participants. However, federated learning suffers from certain security/privacy issues. In this paper, we propose Split-n-Chain, a variant of split learning where the layers of the network are split among several distributed nodes. Split-n-Chain achieves several privacy properties: data-owners need not share their training data with other nodes, and no nodes have access to the parameters and hyperparameters of the neural network (except that of the respective layers they hold). Moreover, Split-n-Chain uses blockchain to audit the computation done by different nodes. Our experimental results show that: Split-n-Chain is efficient, in terms of time required to execute different phases, and the training loss trend is similar to that for the same neural network when implemented in a monolithic fashion.

Mukesh Sahani, Binanda Sengupta

Indian Institute of Technology (Indian School of Mines) Dhanbad, India · Indian Institute of Engineering Science and Technology, Shibpur, India

Privacy-Preserving ML Distributed Training

Ask PDF

Feb 8, 2025

Differentially Private Synthetic Data via APIs 3: Using Simulators Instead of Foundation Model

Differentially private (DP) synthetic data enables the use of sensitive datasets while providing strong privacy guarantees. Recently, Private Evolution (PE) has emerged as a promising framework for generating DP synthetic data using only inference APIs of foundation models. However, suitable foundation models are not always available for every private data domain. In this paper, we show that PE is more general than previously understood and can incorporate APIs beyond foundation models. In particular, we show that state-of-the-art non-neural-network data synthesizers, including computer graphics-based image generators and physics simulation tools (which we refer to as simulators), integrate naturally into PE. We call the resulting method Sim-PE. Sim-PE broadens the applicability of PE while improving performance. For image synthesis, Sim-PE improves downstream classification accuracy by up to 3x, reduces FID by up to 80%, and is significantly more efficient. We further show that simulators and foundation models can be seamlessly combined within Sim-PE for additional gains. Beyond performance improvements, Sim-PE enables new DP applications that are difficult or impossible for existing methods, including generating DP control signals whose execution reproduces the distribution of private observations. The code is open-sourced in the Private Evolution Python library: https://github.com/microsoft/DPSDA

Zinan Lin, Tadas Baltrusaitis, Wenyu Wang +1

Microsoft Research Redmond, WA, USA · Microsoft Cambridge, UK · Redmond, WA, USA

Differentially Private Synthetic Data Generation Synthetic Data Generation Privacy-Preserving ML Differential Privacy

Ask PDF

Dec 8, 2024

Batch effects can impair federated learning in multi-center omics studies

Federated learning (FL) enables collaborative analysis of biomedical data without exchanging sensitive patient-level information, but its performance in multi-center studies may be compromised by batch effects which can obscure biological signals. Here, we systematically assess the impact of uncorrected batch effects on FL outcomes using four multi-center omics datasets, including transcriptomic, proteomic, and metabolomic data, and two representative algorithms: federated k-means clustering and federated random forest classification. Our results demonstrate that uncorrected batch effects undermine unsupervised FL and can substantially degrade supervised FL performance, indicating that privacy-aware batch-effect correction is essential for reliable FL. To enable privacy-preserving BEC in distributed bulk omics data, we introduce fedRBE ( https://featurecloud.ai/app/fedrbe ), a federated implementation of limma's removeBatchEffect() method enhanced by secure multi-party computation, suitable for datasets with missing values and non-identical feature sets across clients, including proteomics and metabolomics data.

Yuliya Burankova, Julian Klemm, Jens J. G. Lohmann +5

Institute for Computational Systems Biomedicine, University of Hamburg, 22761 Hamburg, Germany · Chair of Proteomics and Bioanalytics, TUM School of Life Sciences, Technical University of Munich, 85354 Freising, Germany · Department of Mathematics and Computer Science, University of Southern Denmark, 5230 Odense, Denmark +3

Batch Effect Correction Secure Multi-Party Computation Privacy-Preserving ML Federated Learning

Ask PDF

Apr 20, 2024

DNA: Differentially private Neural Augmentation for contact tracing

The COVID19 pandemic had enormous economic and societal consequences. Contact tracing is an effective way to reduce infection rates by detecting potential virus carriers early. However, this was not generally adopted in the recent pandemic, and privacy concerns are cited as the most important reason. We substantially improve the privacy guarantees of the current state of the art in decentralized contact tracing. Whereas previous work was based on statistical inference only, we augment the inference with a learned neural network and ensure that this neural augmentation satisfies differential privacy. In a simulator for COVID19, even at epsilon=1 per message, this can significantly improve the detection of potentially infected individuals and, as a result of targeted testing, reduce infection rates. This work marks an important first step in integrating deep learning into contact tracing while maintaining essential privacy guarantees.

Rob Romijnders, Christos Louizos, Yuki M. Asano +1

University of Amsterdam · Qualcomm AI research

Privacy-Preserving ML Differential Privacy

Ask PDF

Mar 10, 2024

PPFedIT: Towards Privacy-Preserving Federated Instruction Tuning with Few-shot Local Examples

Instruction tuning aligns large language models (LLMs) with human intentions but requires diverse, high-quality data that are difficult to collect in privacy-sensitive domains. Federated instruction tuning (FedIT) enables collaborative training across data owners, yet existing methods typically assume sufficient local data. In realistic few-shot settings, limited samples can cause overfitting, degrade performance, and increase vulnerability to training data extraction attacks. We propose PPFedIT, a federated algorithm that improves both model performance and privacy protection in federated few-shot learning. It comprises three client-side steps: (1) synthetic data generation, which uses LLMs to diversify and enrich local data; (2) parameter isolation training, which updates the shared global LLM on synthetic data and local LLMs on private local data to mitigate synthetic-data noise; and (3) local aggregation then sharing, which mixes global and local model parameters before uploading them for server aggregation to mitigate data extraction attacks. Experiments on three open-source datasets show that PPFedIT improves model performance by an average of 8.4% and reduces the risk of data extraction attacks by approximately 20% in challenging federated few-shot settings.

Zhuo Zhang, Jingyuan Zhang, Jintao Huang +6

Harbin Institute of Technology, Shenzhen, China · Peng Cheng Lab, Shenzhen, China · Kuaishou, Beijing, China +1

Privacy Leakage in Language Models Few-Shot Learning Privacy-Preserving ML Federated Learning

Ask PDF

Mar 9, 2024

PEARL: Structural Privacy-Utility Control in Human-Centric CPS via Personalized Early-Exit Deep Reinforcement Learning

In human-centric Cyber-Physical Systems (CPS), personalized Deep Reinforcement Learning (DRL) agents must share fine-grained control actions with cloud services, exposing sensitive private states to inference attacks by honest-but-curious adversaries. Static privacy models fail to address the dynamic nature of human interactions. This paper introduces PEARL (Personalized Early-exit Adaptive Reinforcement Learning), a novel framework that addresses this challenge through structural privacy control rather than data perturbation. PEARL deploys a dual-path Early-Exit Deep Q-Network (EE-DQN) at the edge, using Mutual Information (MI) between private states and observable actions to train per-branch binary labels: Utility Confidence Labels (UCL), verifying action quality, and Privacy Confidence Labels (PCL), verifying MI leakage remains below a user-defined threshold. At inference, PEARL selects the shallowest exit branch satisfying both UCL and PCL, structurally limiting shared action descriptive power without noise injection. An MI-based feedback loop tracks behavioral drift and triggers retraining when privacy-utility profiles shift, ensuring long-term robustness. Validated on a personalized smart-home HVAC system and a VR smart classroom, PEARL reduces adversarial state-inference accuracy by 25.67% on average with a controlled 10-16% utility cost, establishing a practical, dynamically enforceable privacy-utility tradeoff.

Mojtaba Taherisadr, Salma Elmalaki

University of California, Irvine, USA

RL Control Cyber-Physical Systems Dynamic Neural Networks Privacy-Preserving ML Early-Exit Neural Networks Privacy-Utility Tradeoff

Ask PDF

Oct 16, 2023

Label Differential Privacy via Aggregation

This paper explores the use of linear aggregation to protect the privacy of sensitive training labels through the concept of label differential privacy (label-DP) while maintaining regression task utility. Our key finding is that weighted linear aggregation of training instances with i.i.d.

N(0, 1)

weights can achieve

(\varepsilon, \delta)

-label-DP with

m = O\left(n/(\log(1/\delta))\right)

. Unlike prior methods, our approach relies on the minimum linear regression loss rather than the minimum singular value of the data matrix, resulting in better practical bounds on real datasets. We also examine real-world mechanisms involving disjoint sets or bags of instances. We demonstrate that aggregating labels from sub-sampled disjoint

k

-sized bags using i.i.d.

N(0,1)

weights achieves

(\varepsilon,\delta)

-label-DP with

k \geq \Omega\left(\left((1/\varepsilon)\log\left(1/\delta\right)\right)^2\right)

. In both scenarios, the optimal linear mse-regressor on the aggregated data approximates the original dataset's optimum with high probability, without needing additive label noise. Furthermore, we show that adding

N(0,1)

noise to any constant fraction of labels allows for similar label-DP guarantees when aggregating labels over random disjoint bags, while preserving the utility of Lipschitz-bounded neural mse-regression tasks.

Anand Brahmbhatt, Rishi Saket, Shreyas Havaldar +3

Google Research India.

Privacy-Preserving ML Differential Privacy

Ask PDF

Jul 24, 2023

A Differentially Private Weighted Empirical Risk Minimization Procedure and its Application to Outcome Weighted Learning

Data used to train predictive models via empirical risk minimization (ERM) often contain sensitive personal information. While differential privacy (DP) provides mathematically provable bounds to protect such data, previous work has focused almost exclusively on unweighted ERM. We consider weighted ERM (wERM) -- an important generalization where individual contributions to the objective function vary. We propose the first DP algorithm for general wERM with formal privacy guarantees and derive both its empirical and population excess risk bounds. Crucially, this general wERM framework provides a pathway for deriving privacy-preserving learning methods for individualized treatment rules, including the popular outcome-weighted learning (OWL) approach. We evaluate DP-wERM applied to OWL in simulated and real data experiments. Our empirical results demonstrate that training OWL models via wERM provides strong DP guarantees while maintaining robust performance, proving the method is practical for sensitive, real-world data.

Spencer Giddens, Yiwang Zhou, Kevin R. Krull +3

University of Notre Dame, Notre Dame, IN 46556, USA · St. Jude Children’s Research Hospital, Memphis, TN 38138, USA · University of Michigan, Ann Arbor, MI 48109, USA

Privacy-Preserving ML Differential Privacy Statistical Learning Theory Empirical Risk Minimization

Ask PDF

Jan 26, 2022

Privacy-Preserving Logistic Regression Training with A Faster Gradient Variant

Training logistic regression over encrypted data has emerged as a prominent approach to addressing security concerns in recent years. In this paper, we introduce an efficient gradient variant, termed the quadratic gradient, which is specifically designed for privacy-preserving logistic regression while remaining equally effective in plaintext optimization. By incorporating this quadratic gradient, we enhance Nesterov's Accelerated Gradient (NAG), Adaptive Gradient (AdaGrad), and Adam algorithms. We evaluate these enhanced algorithms across various datasets, with experimental results demonstrating state-of-the-art convergence rates that significantly outperform traditional first-order gradient methods. Furthermore, we apply the enhanced NAG method to implement homomorphic logistic regression training, achieving comparable performance within only four iterations. The proposed quadratic-gradient approach offers a unified framework that synergizes the advantages of first-order gradient methods and second-order Newton-type methods, suggesting broad applicability to diverse numerical optimization tasks.

John Chiang

Logistic Regression Gradient Descent Privacy-Preserving ML Adaptive Gradient Methods Accelerated Gradient Methods

Ask PDF