Privacy Attack

Privacy attacks target machine learning models to illicitly extract sensitive information from training data or model parameters. Current research focuses on developing and evaluating these attacks across various model architectures, including large language models, federated learning systems, and recommender systems, often employing techniques like membership inference, attribute inference, and model stealing. Understanding the effectiveness and limitations of these attacks is crucial for developing robust privacy-preserving techniques and ensuring responsible deployment of machine learning in sensitive applications. This research directly impacts the security and trustworthiness of AI systems, influencing both the design of privacy-preserving algorithms and the development of effective countermeasures.

Papers

March 7, 2024

Membership Inference Attacks and Privacy in Topic Modeling
Nico Manzonelli, Wanrong Zhang, Salil Vadhan
Large Language Model Membership Inference Attack Privacy Policy Topic Modeling Topic Model Privacy Attack Latent Dirichlet Allocation

February 26, 2024

Pandora's White-Box: Precise Training Data Detection and Extraction in Large Language Models
Jeffrey G. Wang, Jason Wang, Marvin Li, Seth Neel
Membership Inference Attack Process Extraction White Box Privacy Attack Model Stealing PanDORA Dataset Detector Training

February 22, 2024

Closed-Form Bounds for DP-SGD against Record-level Inference
Giovanni Cherubin, Boris Köpf, Andrew Paverd, Shruti Tople, Lukas Wutschitz, Santiago Zanella-Béguelin
Privacy Attack Inference Attack Differential Privacy SGD Sharp Bound Driven Inference

February 15, 2024

Privacy Attacks in Decentralized Learning
Abdellah El Mrini, Edwige Cyffers, Aurélien Bellet
Decentralized Learning Privacy Attack Graph Network Reconstruction Attack Decentralized Gradient Neighboring Node

February 6, 2024

A Survey of Privacy Threats and Defense in Vertical Federated Learning: From Model Life Cycle Perspective
Lei Yu, Meng Han, Yiming Li, Changting Lin, Yao Zhang, Mingyang Zhang, Yan Liu, Haiqin Weng, Yuseok Jeon, Ka-Ho Chow, Stacy Patterson
Timely Survey Federated Learning Vertical Federated Learning Privacy Risk Privacy Attack Privacy Threat

January 30, 2024

Security and Privacy Challenges of Large Language Models: A Survey
Badhan Chandra Das, M. Hadi Amini, Yanzhao Wu
Timely Survey Security Related Privacy Attack LLM Safety Language Pattern Privacy Issue

December 18, 2023

Model Stealing Attack against Recommender System
Zhihao Zhu, Rui Fan, Chenwang Wu, Yi Yang, Defu Lian, Enhong Chen
Adversarial Attack Recommender System Privacy Attack Model Privacy Model Stealing Attack

December 10, 2023

Beyond Gradient and Priors in Privacy Attacks: Leveraging Pooler Layer Inputs of Language Models in Federated Learning
Jianwei Li, Sheng Liu, Qi Lei
Large Language Model Language Model Natural Gradient PRIvacy Leakage Privacy Attack Modern Language Model Pooling Layer

October 31, 2023

Verification of Neural Networks Local Differential Classification Privacy
Roie Reshef, Anan Kabaha, Olga Seleznova, Dana Drachsler-Cohen
Differential Privacy Verification Task Local Differential Privacy Privacy Attack Network Structure Local Robustness

October 28, 2023

Where have you been? A Study of Privacy Risk for Point-of-Interest Recommendation
Kunlin Cai, Jinghuai Zhang, Will Shand, Zhiqing Hong, Guang Wang, Desheng Zhang, Jianfeng Chi, Yuan Tian
Study Feature Privacy Preserving Privacy Risk Privacy Attack Mobility Data

October 26, 2023

Where you go is who you are -- A study on machine learning based semantic privacy attacks
Nina Wiedemann, Ourania Kounadi, Martin Raubal, Krzysztof Janowicz
New Machine Study Feature Privacy Attack Semantic Attack Location Data

October 24, 2023

The Janus Interface: How Fine-Tuning in Large Language Models Amplifies the Privacy Risks
Xiaoyi Chen, Siyuan Tang, Rui Zhu, Shijun Yan, Lei Jin, Zihao Wang, Liya Su, Zhikun Zhang, XiaoFeng Wang, Haixu Tang
Sensitive Data PRIvacy Leakage Privacy Risk Privacy Attack

October 16, 2023

Privacy in Large Language Models: Attacks, Defenses and Future Directions
Haoran Li, Yulin Chen, Jinglong Luo, Yan Kang, Xiaojin Zhang, Qi Hu, Chunkit Chan, Yangqiu Song
Large Language Model Privacy Policy Future Direction New Attack Privacy Attack Powerful Language Model

October 12, 2023

Defending Our Privacy With Backdoors
Dominik Hintersdorf, Lukas Struppek, Daniel Neider, Kristian Kersting
Backdoor Attack Privacy Policy Privacy Attack

September 27, 2023

Identifying and Mitigating Privacy Risks Stemming from Language Models: A Survey
Victoria Smith, Ali Shahin Shamsabadi, Carolyn Ashurst, Adrian Weller
Language Model Timely Survey Privacy Risk Privacy Attack Data Memorization

September 20, 2023

Information Leakage from Data Updates in Machine Learning Models
Tian Hui, Farhad Farokhi, Olga Ohrimenko
Machine Learning Model Privacy Attack Data Leakage Inference Attack Attack Method Incremental Data Information Leakage

August 31, 2023

A Survey on Privacy in Graph Neural Networks: Attacks, Preservation, and Applications
Yi Zhang, Yuying Zhao, Zhaoqing Li, Xueqi Cheng, Yu Wang, Olivera Kotevska, Philip S. Yu, Tyler Derr
Graph Neural Network Financial Application Privacy Policy New Attack Privacy Attack Privacy Preservation

June 23, 2023

Deconstructing Classifiers: Towards A Data Reconstruction Attack Against Text Classification Models
Adel Elmahdy, Ahmed Salem
Natural Language Processing Privacy Attack Interpretable Classifier Text Classification Model Data Reconstruction Attack

March 7, 2023

Client-specific Property Inference against Secure Aggregation in Federated Learning
Raouf Kerkouche, Gergely Ács, Mario Fritz
Differential Privacy Secure Aggregation Privacy Attack Property Inference

February 28, 2023

Membership Inference Attack for Beluga Whales Discrimination
Voncarlos Marcelo Araújo, Sébastien Gambs, Clément Chion, Robert Michaud, Léo Schneider, Hadrien Lautraite
Membership Inference Attack Underwater Environment Privacy Attack Cyber Attack Whistleblower Re Identification ORCa Behavior

Privacy Attack

Papers

Membership Inference Attacks and Privacy in Topic Modeling

Pandora's White-Box: Precise Training Data Detection and Extraction in Large Language Models

Closed-Form Bounds for DP-SGD against Record-level Inference

Privacy Attacks in Decentralized Learning

A Survey of Privacy Threats and Defense in Vertical Federated Learning: From Model Life Cycle Perspective

Security and Privacy Challenges of Large Language Models: A Survey

Model Stealing Attack against Recommender System

Beyond Gradient and Priors in Privacy Attacks: Leveraging Pooler Layer Inputs of Language Models in Federated Learning

Verification of Neural Networks Local Differential Classification Privacy

Where have you been? A Study of Privacy Risk for Point-of-Interest Recommendation

Where you go is who you are -- A study on machine learning based semantic privacy attacks

The Janus Interface: How Fine-Tuning in Large Language Models Amplifies the Privacy Risks

Privacy in Large Language Models: Attacks, Defenses and Future Directions

Defending Our Privacy With Backdoors

Identifying and Mitigating Privacy Risks Stemming from Language Models: A Survey

Information Leakage from Data Updates in Machine Learning Models

A Survey on Privacy in Graph Neural Networks: Attacks, Preservation, and Applications

Deconstructing Classifiers: Towards A Data Reconstruction Attack Against Text Classification Models

Client-specific Property Inference against Secure Aggregation in Federated Learning

Membership Inference Attack for Beluga Whales Discrimination