Private Learning

Private learning aims to train machine learning models on sensitive data while preserving individual privacy, primarily using techniques like differential privacy and federated learning. Current research focuses on improving the accuracy and efficiency of private learning algorithms, exploring methods such as buffered linear Toeplitz mechanisms, adaptive clipping, and the incorporation of public data or pre-trained models to enhance utility. This field is crucial for enabling the use of sensitive data in machine learning applications across various domains, while mitigating privacy risks and fostering trust.

Papers

October 10, 2023

Correlated Noise Provably Beats Independent Noise for Differentially Private Learning
Christopher A. Choquette-Choo, Krishnamurthy Dvijotham, Krishna Pillutla, Arun Ganesh, Thomas Steinke, Abhradeep Thakurta
Private Learning Additive Noise Correlated Noise Correlation Function Private Learning Algorithm

August 11, 2023

Private Distribution Learning with Public Data: The View from Sample Compression
Shai Ben-David, Alex Bie, Clément L. Canonne, Gautam Kamath, Vikrant Singhal
Available Datasets Best View Private Learning Distributionally Robust Private Machine Learning Sample Compression

June 15, 2023

ViP: A Differentially Private Foundation Model for Computer Vision
Yaodong Yu, Maziar Sanjabi, Yi Ma, Kamalika Chaudhuri, Chuan Guo
Vision Transformer Computer Vision Differential Privacy Foundation Model Supervised ImageNet Supervised Autoencoder Private Learning

June 9, 2023

Differentially Private Sharpness-Aware Training
Jinseong Park, Hoki Kim, Yujin Choi, Jaewook Lee
Differential Privacy Sharpness Aware Minimization Private Learning Sharpness Aware Training

June 8, 2023

Differentially Private Image Classification by Learning Priors from Random Processes
Xinyu Tang, Ashwinee Panda, Vikash Sehwag, Prateek Mittal
Stochastic Process Private Learning Privacy Utility Private Stochastic Gradient Descent Privacy Preserving Image Privacy Preserving Machine Learning Learned Prior

May 23, 2023

Selective Pre-training for Private Fine-tuning
Da Yu, Sivakanth Gopi, Janardhan Kulkarni, Zinan Lin, Saurabh Naik, Tomasz Lukasz Religa, Jian Yin, Huishuai Zhang
Language Model Private Learning Private Datasets Private Fine Tuning

April 3, 2023

On Mitigating the Utility-Loss in Differentially Private Learning: A new Perspective by a Geometrically Inspired Kernel Approach
Mohit Kumar, Bernhard A. Moser, Lukas Fischer
Kernel Method New Perspective Reproducing Kernel Hilbert Space Private Learning Privacy Utility Kernel Based Privacy Sensitive Private Machine Learning

March 29, 2023

On the Query Complexity of Training Data Reconstruction in Private Learning
Prateeti Mukherjee, Satya Lokam
Training Data White Box Private Learning Query Complexity Data Reconstruction Attack DP Algorithm

March 10, 2023

Optimal and Private Learning from Human Response Data
Duc Nguyen, Anderson Y. Zhang
Sample Complexity Private Learning Item Response Theory Spectral Algorithm Aggregate Response

March 7, 2023

Polynomial Time and Private Learning of Unbounded Gaussian Mixture Models
Jamil Arbas, Hassan Ashtiani, Christopher Liaw
Polynomial Time Gaussian Mixture Private Algorithm Private Learning Gaussian Distribution Dimensional Gaussian Mixture

December 31, 2022

A Comparative Study of Image Disguising Methods for Confidential Outsourced Learning
Sagar Sharma, Yuechun Gu, Keke Chen
Comparative Study Private Learning Image Encryption Model Attack Information Hiding

December 13, 2022

Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining
Florian Tramèr, Gautam Kamath, Nicholas Carlini
Differential Privacy Product Specific Position Information Private Learning Crucial Consideration Large Scale Pretraining Private Machine Learning Non Private

December 12, 2022

Collaborating Heterogeneous Natural Language Processing Tasks via Federated Learning
Chenhe Dong, Yuexiang Xie, Bolin Ding, Ying Shen, Yaliang Li
Language Understanding NLP Field Federated Setting Private Learning

December 9, 2022

Near-Optimal Differentially Private Reinforcement Learning
Dan Qiao, Yu-Xiang Wang
Private Learning DP Algorithm Private Reinforcement Learning

December 3, 2022

Exploring the Limits of Differentially Private Deep Learning with Group-wise Clipping
Jiyan He, Xuechen Li, Da Yu, Huishuai Zhang, Janardhan Kulkarni, Yin Tat Lee, Arturs Backurs, Nenghai Yu, Jiang Bian
Continuum Limit Private Learning Private Optimization Adaptive Clipping

December 1, 2022

Differentially Private Learning with Per-Sample Adaptive Clipping
Tianyu Xia, Shuheng Shen, Su Yao, Xinyi Fu, Ke Xu, Xiaolong Xu, Xing Fu
Differential Privacy Privacy Preserving Private Learning Adaptive Clipping Gradient Descent Batch Size

November 15, 2022

Differentially Private Sampling from Distributions
Sofya Raskhodnikova, Satchit Sivakumar, Adam Smith, Marika Swanberg
Differential Privacy Product Distribution Private Learning Private Data Analysis

November 11, 2022

\~Optimal Differentially Private Learning of Thresholds and Quasi-Concave Optimization
Edith Cohen, Xin Lyu, Jelani Nelson, Tamás Sarlós, Uri Stemmer
Sample Complexity Near Optimality Private Learning Upper Bound Canonical Threshold Threshold Function Quasi Convex Optimization

October 9, 2022

Performances of Symmetric Loss for Private Data from Exponential Mechanism
Jing Bi, Vorapong Suppakitpaisarn
Native Robustness Private Data Private Learning Exponential Mechanism Symmetric Loss

October 7, 2022

Differentially Private Deep Learning with ModelMix
Hanshen Xiao, Jun Wan, Srinivas Devadas
Private Learning Private Stochastic Gradient Descent Privacy Analysis Dp SGD