Data Auditing

Data auditing in machine learning focuses on verifying the ethical and responsible use of data in model training, aiming to detect unauthorized or biased data usage. Current research emphasizes developing methods to identify data provenance, particularly within federated learning and for various model types (including image classifiers and language models), often employing membership inference attacks and techniques from information theory and causal inference. These auditing techniques are crucial for ensuring model fairness, accountability, and transparency, impacting both the development of responsible AI and the legal compliance of data-driven applications.

Papers

September 28, 2024

Subject Data Auditing via Source Inference Attack in Cross-Silo Federated Learning
Jiaxin Li, Marco Arazzi, Antonino Nocera, Mauro Conti
Membership Inference Attack Data Leakage Inference Attack Cross Silo Federated Learning Data Auditing

July 21, 2024

A General Framework for Data-Use Auditing of ML Models
Zonghao Huang, Neil Zhenqiang Gong, Michael K. Reiter
Machine Learning Model General Framework Machine Learning Task Machine Learning Practitioner Box Membership Inference Attack Data Auditing

November 10, 2023

Autoregressive Language Models For Estimating the Entropy of Epic EHR Audit Logs
Benjamin C. Warner, Thomas Kannampallil, Seunghwan Kim
Electronic Health Record Autoregressive Language Model Stable Entropy Healthcare Workflow Data Auditing

April 6, 2023

Data AUDIT: Identifying Attribute Utility- and Detectability-Induced Bias in Task Models
Mitchell Pavlak, Nathan Drenkow, Nicholas Petrick, Mohammad Mehdi Farhangi, Mathias Unberath
Absolute Stance Bias Medical Image Datasets Model Bias Skin Lesion Algorithm Auditing Task Model Data Auditing

April 5, 2023

FACE-AUDITOR: Data Auditing in Facial Recognition Systems
Min Chen, Zhikun Zhang, Tianhao Wang, Michael Backes, Yang Zhang
Face Recognition Model Facial Recognition System Data Auditing

November 23, 2022

Unsupervised User-Based Insider Threat Detection Using Bayesian Gaussian Mixture Models
Simon Bertrand, Nadia Tawbi, Josée Desharnais
User Modeling Data Auditing Insider Threat Insider Threat Detection

October 25, 2022

Unsupervised Anomaly Detection for Auditing Data and Impact of Categorical Encodings
Ajay Chawda, Stefanie Grimm, Marius Kloft
Anomaly Detection Global Impact Unsupervised Anomaly Detection Benchmark Datasets Insurance Data Categorical Encoding Data Auditing

June 20, 2022

The Right Tool for the Job: Open-Source Auditing Tools in Machine Learning
Cherie M Poland
Machine Learning BRIO Tool Fairness Assessment Algorithm Auditing Audit Model Data Auditing

Data Auditing

Papers

Subject Data Auditing via Source Inference Attack in Cross-Silo Federated Learning

A General Framework for Data-Use Auditing of ML Models

Autoregressive Language Models For Estimating the Entropy of Epic EHR Audit Logs

Data AUDIT: Identifying Attribute Utility- and Detectability-Induced Bias in Task Models

FACE-AUDITOR: Data Auditing in Facial Recognition Systems

Unsupervised User-Based Insider Threat Detection Using Bayesian Gaussian Mixture Models

Unsupervised Anomaly Detection for Auditing Data and Impact of Categorical Encodings

The Right Tool for the Job: Open-Source Auditing Tools in Machine Learning