Private Training

Private training in machine learning aims to develop and deploy models without directly exposing sensitive training data. Current research heavily focuses on mitigating privacy risks in federated learning and other distributed training paradigms, exploring techniques like differential privacy, random data augmentation, and secure multi-party computation, often applied to vision transformers and other deep neural networks. This field is crucial for enabling the use of sensitive data in machine learning applications across healthcare, finance, and other sectors while addressing significant ethical and regulatory concerns around data privacy.

Papers

October 21, 2024

Subword Embedding from Bytes Gains Privacy without Sacrificing Accuracy and Complexity
Mengjiao Zhang, Jia Xu
Deep Neural Network Complexity Matter Sub Word Private Training

September 27, 2024

In-depth Analysis of Privacy Threats in Federated Learning for Medical Data
Badhan Chandra Das, M. Hadi Amini, Yanzhao Wu
Medical Image Privacy Risk Medical Data Depth Analysis Sensitive Patient Private Training Privacy Threat

September 2, 2024

Defending against Model Inversion Attacks via Random Erasing
Viet-Hung Tran, Ngoc-Bao Nguyen, Son T. Mai, Hans Vandierendonck, Ngai-man Cheung
Model Inversion Attack Model Inversion Private Training High Dimensional Image

August 2, 2024

Privacy-Preserving Split Learning with Vision Transformers using Patch-Wise Random and Noisy CutMix
Seungeun Oh, Sihun Baek, Jihong Park, Hyelin Nam, Praneeth Vepakomma, Ramesh Raskar, Mehdi Bennis, Seong-Lyun Kim
Vision Transformer Membership Inference Attack Split Learning Random Participation Differentially Private Private Training Cross Modal Manifold CutMix

June 18, 2024

Federated Learning with a Single Shared Image
Sunny Soni, Aaqib Saeed, Yuki M. Asano
Knowledge Distillation Private Training Local Client Model Image SR

June 3, 2024

Seeing the Forest through the Trees: Data Leakage from Partial Transformer Gradients
Weijun Li, Qiongkai Xu, Mark Dras
Differential Privacy Tree Specie Data Leakage Forest Environment Gradient Inversion Attack Private Training Part Aware Transformer Training Data Leakage

May 6, 2024

Federated Learning Privacy: Attacks, Defenses, Applications, and Policy Landscape - A Survey
Joshua C. Zhao, Saurabh Bagchi, Salman Avestimehr, Kevin S. Chan, Somali Chaterji, Dimitris Dimitriadis, Jiacheng Li, Ninghui Li, Arash Nourian, Holger R. Roth
Deep Learning Financial Application Federated Learning New Attack Privacy Attack Privacy Preservation Private Training Policy Space

April 8, 2024

SoK: Gradient Leakage in Federated Learning
Jiacheng Du, Jiahui Hu, Zhibo Wang, Peng Sun, Neil Zhenqiang Gong, Kui Ren
Gradient Inversion Attack Defense Method Private Training Gradient Leakage

April 2, 2024

Federated Distillation: A Survey
Lin Li, Jianping Gou, Baosheng Yu, Lan Du, Zhang Yiand Dacheng Tao
Timely Survey Knowledge Distillation Knowledge Transfer Private Training Federated Distillation

March 26, 2024

Leak and Learn: An Attacker's Cookbook to Train Using Leaked Data from Federated Learning
Joshua C. Zhao, Ahaan Dabholkar, Atul Sharma, Saurabh Bagchi
Private Training Gradient Inversion Data Reconstruction Attack Pseudo Victim AttackER Leakage Attack

March 5, 2024

FLGuard: Byzantine-Robust Federated Learning via Ensemble of Contrastive Models
Younghan Lee, Yungi Cho, Woorim Han, Ho Bae, Yunheung Paek
Diverse Ensemble State of the Art Defense Private Training Byzantine Robust Federated Learning Contrastive Model

February 6, 2024

Privacy Leakage on DNNs: A Survey of Model Inversion Attacks and Defenses
Hao Fang, Yixiang Qiu, Hongyao Yu, Wenbo Yu, Jiawei Kong, Baoli Chong, Bin Chen, Xuan Wang, Shu-Tao Xia, Ke Xu
Timely Survey High Fidelity PRIvacy Leakage Model Inversion Attack Adversary Agent Model Inversion Private Training

December 21, 2023

Fine-grained Disentangled Representation Learning for Multimodal Emotion Recognition
Haoqin Sun, Shiwan Zhao, Xuechen Wang, Wenjia Zeng, Yong Chen, Yong Qin
Multimodal Emotion Recognition Private Training

October 16, 2023

Passive Inference Attacks on Split Learning via Adversarial Regularization
Xiaochen Zhu, Xinjian Luo, Yuncheng Wu, Yangfan Jiang, Xiaokui Xiao, Beng Chin Ooi
Split Learning Inference Attack Private Training Adversarial Regularization Malicious Model

October 12, 2023

Why Train More? Effective and Efficient Membership Inference via Memorization
Jihye Choi, Shruti Tople, Varun Chandrasekaran, Somesh Jha
Membership Inference Attack Limited Memorization Membership Inference Adversary Agent Private Training Box Membership Inference Attack

October 4, 2023

Efficient Federated Prompt Tuning for Black-box Large Pre-trained Models
Zihao Lin, Yan Sun, Yifan Shi, Xueqian Wang, Lifu Huang, Li Shen, Dacheng Tao
Fine Tuning Pre Trained Model Black Box Large Pre Trained Model Private Training Federated Prompt

February 24, 2023

Active Membership Inference Attack under Local Differential Privacy in Federated Learning
Truc Nguyen, Phung Lai, Khang Tran, NhatHai Phan, My T. Thai
Membership Inference Attack Local Differential Privacy Membership Inference Private Training Differential Privacy Noise

December 16, 2022

Planting and Mitigating Memorized Content in Predictive-Text Language Models
C. M. Downey, Wei Dai, Huseyin A. Inan, Kim Laine, Saurabh Naik, Tomasz Religa
Language Model Differential Privacy Sensitive Data Privacy Guarantee Plant Monitoring Private Training Verbatim Memorization

August 21, 2022

Cluster Based Secure Multi-Party Computation in Federated Learning for Histopathology Images
S. Maryam Hosseini, Milad Sikaroudi, Morteza Babaei, H. R. Tizhoosh
Differential Privacy Histopathology Image Cluster Structure Secure Approach Private Training Secure Multiparty Computation Party Computation Patient Privacy

August 9, 2022

Privacy-Aware Adversarial Network in Human Mobility Prediction
Yuting Zhan, Hamed Haddadi, Afra Mashhadi
Mobility Data Private Training Privacy Utility Trade Identification Risk Human Mobility Prediction Adversarial Privacy