Privacy Preserving Training

Privacy-preserving training (PPT) focuses on developing machine learning methods that protect sensitive data during model training, addressing concerns about data breaches and privacy violations. Current research emphasizes techniques like federated learning, differential privacy, and homomorphic encryption, applied to various architectures including transformers and diffusion models, to achieve this goal. The field is driven by the need to enable the use of sensitive data (e.g., medical images) for AI development while maintaining strong privacy guarantees, impacting both the ethical application of AI and the advancement of machine learning algorithms themselves.

Papers

October 23, 2024

Differentially Private Learning Needs Better Model Initialization and Self-Distillation
Ivoline C. Ngong, Joseph P. Near, Niloofar Mireshghallah
Language Model Self Distillation Private Learning Privacy Preserving Training

May 30, 2024

Gradient Inversion of Federated Diffusion Models
Jiyue Huang, Chi Hong, Lydia Y. Chen, Stefanie Roos
Diffusion Model Gradient Inversion Attack Gradient Inversion Inversion Attack Privacy Preserving Training

February 14, 2024

I can't see it but I can Fine-tune it: On Encrypted Fine-tuning of Transformers using Fully Homomorphic Encryption
Prajwal Panzade, Daniel Takabi, Zhipeng Cai
Transformer Megatron Decepticons Homomorphic Encryption Privacy Preserving Machine Learning Private Fine Tuning Encrypted Model Privacy Preserving Training

November 15, 2023

Scalable Federated Learning for Clients with Different Input Image Sizes and Numbers of Output Categories
Shuhei Nitta, Taiji Suzuki, Albert Rodríguez Mulet, Atsushi Yaguchi, Ryusuke Hirai
Federated Learning Image Classification Structured Output Numerical Data Client Collaboration Scalable Neural Privacy Preserving Training Image Size

August 21, 2023

Unlocking Accuracy and Fairness in Differentially Private Image Classification
Leonard Berrada, Soham De, Judy Hanwen Shen, Jamie Hayes, Robert Stanforth, David Stutz, Pushmeet Kohli, Samuel L. Smith, Borja Balle
Differential Privacy Procedural Fairness Privacy Preserving Image Enhancing Accuracy Privacy Preserving Training

August 18, 2023

Privacy-Preserving 3-Layer Neural Network Training
John Chiang
Neural Network Homomorphic Encryption Classification Problem Privacy Preserving Training

March 27, 2023

Privacy-preserving machine learning for healthcare: open challenges and future perspectives
Alejandro Guerra-Manzanares, L. Julian Lechuga Lopez, Michail Maniatakos, Farah E. Shamout
Machine Learning New Machine Healthcare System Open Challenge Medical Data Future Perspective Inference Service Privacy Preserving Training Healthcare Prediction Task

February 3, 2023

Private, fair and accurate: Training large-scale, privacy-preserving AI models in medical imaging
Soroosh Tayebi Arasteh, Alexander Ziller, Christiane Kuhl, Marcus Makowski, Sven Nebelung, Rickmer Braren, Daniel Rueckert, Daniel Truhn, Georgios Kaissis
Differential Privacy Medical Imaging Privacy Preserving Large Scale Training Privacy Preserving Training Fairness Estimation

January 30, 2023

Extracting Training Data from Diffusion Models
Nicholas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramèr, Borja Balle, Daphne Ippolito, Eric Wallace
Diffusion Model Training Data GAN Model Synthetic Image Image Diffusion Model Generative Prior Privacy Preserving Training

January 24, 2023

When does the student surpass the teacher? Federated Semi-supervised Learning with Teacher-Student EMA
Jessica Zhao, Sayan Ghosh, Akash Bharadwaj, Chih-Yao Ma
Semi Supervised Learning Semi Supervised Good Teacher Student Friendly Knowledge SSL Algorithm Privacy Preserving Training Semi Supervised Teacher Student Exponential Moving Average

December 18, 2022

Plankton-FL: Exploration of Federated Learning for Privacy-Preserving Training of Deep Neural Networks for Phytoplankton Classification
Daniel Zhang, Vikram Voleti, Alexander Wong, Jason Deglint
Deep Neural Network Training Data Environment Exploration Privacy Preserving Training Phytoplankton Image

November 19, 2022

Personalized Federated Learning with Hidden Information on Personalized Prior
Mingjia Shi, Yuhao Zhou, Qing Ye, Jiancheng Lv
Personalized Federated Learning Bayesian Learning Hidden Knowledge Bregman Divergence Privacy Preserving Training pFL Method Personalized Prior

September 22, 2022

In Differential Privacy, There is Truth: On Vote Leakage in Ensemble Private Learning
Jiaqi Wang, Roei Schuster, Ilia Shumailov, David Lie, Nicolas Papernot
Differential Privacy Differential Privacy Guarantee Great Truth Private Aggregation Privacy Preserving Training Election Manipulation Private Ensemble

May 25, 2022

Additive Logistic Mechanism for Privacy-Preserving Self-Supervised Learning
Yunhao Yang, Parham Gohari, Ufuk Topcu
Self Supervised Learning Membership Inference Attack Privacy Attack Private Datasets Differential Privacy Mechanism Privacy Preserving Training

May 1, 2022

A New Dimensionality Reduction Method Based on Hensel's Compression for Privacy Protection in Federated Learning
Ahmed El Ouadrhiri, Ahmed Abdelhadi
Differential Privacy Dimensionality Reduction Linear Compression PRIvacy Leakage Privacy Protection Privacy Preserving Training Dataset Level Privacy

Privacy Preserving Training

Papers

Differentially Private Learning Needs Better Model Initialization and Self-Distillation

Gradient Inversion of Federated Diffusion Models

I can't see it but I can Fine-tune it: On Encrypted Fine-tuning of Transformers using Fully Homomorphic Encryption

Scalable Federated Learning for Clients with Different Input Image Sizes and Numbers of Output Categories

Unlocking Accuracy and Fairness in Differentially Private Image Classification

Privacy-Preserving 3-Layer Neural Network Training

Privacy-preserving machine learning for healthcare: open challenges and future perspectives

Private, fair and accurate: Training large-scale, privacy-preserving AI models in medical imaging

Extracting Training Data from Diffusion Models

When does the student surpass the teacher? Federated Semi-supervised Learning with Teacher-Student EMA

Plankton-FL: Exploration of Federated Learning for Privacy-Preserving Training of Deep Neural Networks for Phytoplankton Classification

Personalized Federated Learning with Hidden Information on Personalized Prior

In Differential Privacy, There is Truth: On Vote Leakage in Ensemble Private Learning

Additive Logistic Mechanism for Privacy-Preserving Self-Supervised Learning

A New Dimensionality Reduction Method Based on Hensel's Compression for Privacy Protection in Federated Learning