Training Data Leakage

Training data leakage refers to the vulnerability of machine learning models, particularly large language models (LLMs) and deep neural networks, to revealing sensitive information from their training datasets. Current research focuses on understanding how various attack vectors, including gradient inversion and exploiting specific character patterns, can extract this data, even from seemingly secure training methods like differential privacy. This is a significant concern for privacy and intellectual property, impacting the development and deployment of machine learning systems across diverse applications, driving efforts to develop more robust training and defense mechanisms.

Papers

November 17, 2024

Stealing Training Graphs from Graph Neural Networks
Minhua Lin, Enyan Dai, Junjie Xu, Jinyuan Jia, Xiang Zhang, Suhang Wang
Graph Neural Network Training Graph Graph Diffusion Training Data Leakage

October 9, 2024

PII-Scope: A Benchmark for Training Data PII Leakage Assessment in LLMs
Krishna Kanth Nakka, Ahmed Frikha, Ricardo Mendes, Xue Jiang, Xuebing Zhou
Adversarial Attack New Benchmark Medical LLM Privacy Attack QuEry Based Attack Data Extraction Attack Training Data Leakage Identifiable Information

June 3, 2024

Seeing the Forest through the Trees: Data Leakage from Partial Transformer Gradients
Weijun Li, Qiongkai Xu, Mark Dras
Differential Privacy Tree Specie Data Leakage Forest Environment Gradient Inversion Attack Private Training Part Aware Transformer Training Data Leakage

May 9, 2024

Special Characters Attack: Toward Scalable Training Data Extraction From Large Language Models
Yang Bai, Ge Pei, Jindong Gu, Yong Yang, Xingjun Ma
Large Language Model Training Corpus Scale Pre Trained Language Model Training Data Leakage Training Data Extraction

October 12, 2023

When Machine Learning Models Leak: An Exploration of Synthetic Training Data
Manel Slokom, Peter-Paul de Wolf, Martha Larson
Machine Learning Model Synthetic Data Environment Exploration Synthetic Training Data Sensitive Attribute Propensity Score Training Data Leakage

August 8, 2023

Accurate, Explainable, and Private Models: Providing Recourse While Minimizing Training Data Leakage
Catherine Huang, Chelse Swoopes, Christina Xiao, Jiaqi Ma, Himabindu Lakkaraju
PRIvacy Leakage Algorithmic Recourse Robust Recourse Private Model Training Data Leakage

October 20, 2022

Analysing Training-Data Leakage from Gradients through Linear Systems and Gradient Matching
Cangxiong Chen, Neill D. F. Campbell
Gradient Based Attack Linear System Gradient Matching Gradient Leakage Attack Training Data Leakage

November 19, 2021

Understanding Training-Data Leakage from Gradients in Neural Networks for Image Classification
Cangxiong Chen, Neill D. F. Campbell
Neural Network Deep Learning Model Image Classification Deep Network Gradient Based Attack Training Data Leakage

Training Data Leakage

Papers

Stealing Training Graphs from Graph Neural Networks

PII-Scope: A Benchmark for Training Data PII Leakage Assessment in LLMs

Seeing the Forest through the Trees: Data Leakage from Partial Transformer Gradients

Special Characters Attack: Toward Scalable Training Data Extraction From Large Language Models

When Machine Learning Models Leak: An Exploration of Synthetic Training Data

Accurate, Explainable, and Private Models: Providing Recourse While Minimizing Training Data Leakage

Analysing Training-Data Leakage from Gradients through Linear Systems and Gradient Matching

Understanding Training-Data Leakage from Gradients in Neural Networks for Image Classification