Private Machine Learning

Private machine learning (PML) aims to train machine learning models on sensitive data while rigorously guaranteeing individual privacy, typically using differential privacy (DP) mechanisms. Current research focuses on improving the accuracy of DP models by enhancing algorithms like DP-SGD, exploring techniques like noise-tolerant pre-training and adaptive hyperparameter optimization, and leveraging public data to reduce the impact of privacy-preserving noise. These advancements are crucial for enabling the responsible use of sensitive data in various applications, addressing the inherent trade-off between privacy and model utility, and fostering trust in data-driven technologies.

Papers

May 24, 2024

Scaling up the Banded Matrix Factorization Mechanism for Differentially Private ML
Ryan McKenna
Matrix Factorization Privacy Preserving Machine Learning Privacy Amplification Private Machine Learning Noise Optimization

May 23, 2024

Nearly Tight Black-Box Auditing of Differentially Private Machine Learning
Meenatchi Sundaram Muthu Selva Annamalai, Emiliano De Cristofaro
Black Box Membership Inference Attack PRIvacy Leakage Private Stochastic Gradient Descent Privacy Analysis Private Machine Learning

March 5, 2024

DPAdapter: Improving Differentially Private Deep Learning through Noise Tolerance Pre-training
Zihao Wang, Rui Zhu, Dongruo Zhou, Zhikun Zhang, John Mitchell, Haixu Tang, XiaoFeng Wang
Differentially Private Private Machine Learning Parameter Aware Robustness

March 1, 2024

Shifted Interpolation for Differential Privacy
Jinho Bok, Weijie Su, Jason M. Altschuler
Differential Privacy Interpolation Regime Convex Loss Private Machine Learning

November 15, 2023

Are Normalizing Flows the Key to Unlocking the Exponential Mechanism?
Robert A. Bridges, Vandy J. Tombs, Christopher B. Stanley
Differential Privacy Empirical Privacy Exponential Mechanism Private Machine Learning Private Optimization

October 24, 2023

Private Learning with Public Features
Walid Krichene, Nicolas Mayoraz, Steffen Rendle, Shuang Song, Abhradeep Thakurta, Li Zhang
Private Algorithm Private Learning Private Machine Learning Personalization Task Private Feature Recommendation Benchmark

October 2, 2023

Online Sensitivity Optimization in Differentially Private Learning
Filippo Galli, Catuscia Palamidessi, Tommaso Cucinotta
Optimization Purpose Gradient Descent Hyperparameter Tuning Privacy Analysis Differentially Private Private Machine Learning

August 11, 2023

Private Distribution Learning with Public Data: The View from Sample Compression
Shai Ben-David, Alex Bie, Clément L. Canonne, Gautam Kamath, Vikrant Singhal
Available Datasets Best View Private Learning Distributionally Robust Private Machine Learning Sample Compression

August 4, 2023

Large-Scale Public Data Improves Differentially Private Image Generation Quality
Ruihan Wu, Chuan Guo, Kamalika Chaudhuri
Generative Adversarial Network Image Generation Data Distribution Private Machine Learning

June 9, 2023

DP-HyPO: An Adaptive Private Hyperparameter Optimization Framework
Hua Wang, Sheng Gao, Huanyu Zhang, Weijie J. Su, Milan Shen
Hyperparameter Optimization Preference Optimization Private Machine Learning Private Hyperparameter

May 29, 2023

Unleashing the Power of Randomization in Auditing Differentially Private ML
Krishna Pillutla, Galen Andrew, Peter Kairouz, H. Brendan McMahan, Alina Oprea, Sewoong Oh
Differential Privacy Real Power Sample Complexity Private Machine Learning Randomization Program Privacy Preserving Computation Privacy Model Higher Order Correlation

May 15, 2023

Shared and Private Information Learning in Multimodal Sentiment Analysis with Deep Modal Alignment and Self-supervised Multi-Task Learning
Songning Lai, Jiakang Li, Guinan Guo, Xifeng Hu, Yulong Li, Yuan Tan, Zichen Song, Yutong Liu, Zhaoxia Ren, Chun Wan, Danmin Miao, Zhi Liu
Self Supervised Cross Modal Hierarchical Representation Multimodal Sentiment Analysis Sharing Matter Private Machine Learning Multimodal Sentiment

May 10, 2023

DPMLBench: Holistic Evaluation of Differentially Private Machine Learning
Chengkun Wei, Minghu Zhao, Zhikun Zhang, Min Chen, Wenlong Meng, Bo Liu, Yuan Fan, Wenzhi Chen
Differential Privacy PRIvacy Leakage Differential Privacy SGD Differentially Private Private Machine Learning Holistic Evaluation Differential Analysis

April 3, 2023

On Mitigating the Utility-Loss in Differentially Private Learning: A new Perspective by a Geometrically Inspired Kernel Approach
Mohit Kumar, Bernhard A. Moser, Lukas Fischer
Kernel Method New Perspective Reproducing Kernel Hilbert Space Private Learning Privacy Utility Kernel Based Privacy Sensitive Private Machine Learning

March 2, 2023

Choosing Public Datasets for Private Machine Learning via Gradient Subspace Distance
Xin Gu, Gautam Kamath, Zhiwei Steven Wu
Gradient Descent Available Datasets Private Stochastic Private Machine Learning Private Model Gradient Subspace

February 15, 2023

Tight Auditing of Differentially Private Machine Learning
Milad Nasr, Jamie Hayes, Thomas Steinke, Borja Balle, Florian Tramèr, Matthew Jagielski, Nicholas Carlini, Andreas Terzis
Differential Privacy Black Box Privacy Guarantee Empirical Privacy Private Machine Learning

December 26, 2022

Packing Privacy Budget Efficiently
Pierre Tholoniat, Kelly Kostopoulou, Mosharaf Chowdhury, Asaf Cidon, Roxana Geambasu, Mathias Lécuyer, Junfeng Yang
Differential Privacy Privacy Budget Private Machine Learning

December 13, 2022

Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining
Florian Tramèr, Gautam Kamath, Nicholas Carlini
Differential Privacy Product Specific Position Information Private Learning Crucial Consideration Large Scale Pretraining Private Machine Learning Non Private

November 12, 2022

Multi-Epoch Matrix Factorization Mechanisms for Private Machine Learning
Christopher A. Choquette-Choo, H. Brendan McMahan, Keith Rush, Abhradeep Thakurta
Gradient Based Differential Privacy SGD Private Machine Learning Nonnegative Matrix Factorization

October 16, 2022

A General Framework for Auditing Differentially Private Machine Learning
Fred Lu, Joseph Munoz, Maya Fuchs, Tyler LeBlond, Elliott Zaresky-Williams, Edward Raff, Francis Ferraro, Brian Testa
Poisoning Attack General Framework Membership Inference Audit Evidence Private Machine Learning Privacy Level