Algorithmic Auditing

Ask about this topic

Overview

Algorithmic auditing studies how to test automated decisions and the systems around them, especially when errors or disparities affect people. Common approaches compare subgroup outcomes, use counterfactual or causal tests, probe model behavior under perturbations, and apply statistical methods such as sequential testing and uncertainty analysis. Audits increasingly examine deployed pipelines, data and component interactions, strategic responses, and evidence trails—not just a model’s aggregate accuracy. A central challenge is making conclusions reliable and meaningful despite limited access, shifting conditions, incomplete data, and metrics that can be gamed or fail to capture the harms that matter.

Latest papers

CardsList

143 papers27 papers in the last four weeks, up 59% on the four weeks before. 0.2% of all new papers.

  1. Who Audits the Auditor? Tamper-Proof Fraud Detection with Blockchain-Anchored Explainable ML

    Apr 23, 2026Zhaohui WangData ProvenanceAlgorithmic Auditing

    Viterbi School of Engineering University of Southern California Los Angeles, CA, USA

  2. Participatory provenance as representational auditing for AI-mediated public consultation

    Apr 22, 2026Sachit MahajanData ProvenanceAlgorithmic Auditing

    Computational Social Science, D-GESS · ETH Zurich, Rämistrasse 101, 8092 Zurich, Switzerland

  3. Fairness Audits of Institutional Risk Models in Deployed ML Pipelines

    Apr 21, 2026Kelly McConvey, Dipto Das, Maya Ghai +3Algorithmic AuditingEducational Technology

    University of Toronto, Toronto, Ontario, Canada

  4. FairTree: Subgroup Fairness Auditing of Machine Learning Models with Bias-Variance Decomposition

    Apr 21, 2026Rudolf DebelakAlgorithmic AuditingAlgorithmic Fairness

    University of Zurich, Switzerland · EPFL, Switzerland

  5. Perturbation Sensitivity of Maximum-Likelihood Pairwise Ranking in Computational Decision Systems

    Apr 20, 2026Junyi Yao, Zihao Zheng, Jiayu LongAlgorithmic AuditingLearning to Rank

    Washington University in St. Louis USA

  6. Shapley Value-Guided Adaptive Ensemble Learning for Explainable Financial Fraud Detection with U.S. Regulatory Compliance Validation

    Apr 14, 2026Mohammad Nasir Uddin, Md Munna AzizFinancial Fraud DetectionEnsemble Learning

    Visual Data Analyst and Applied AI Researcher, Westcliff University, Irvine, CA, USA · Data Analyst and Applied AI Researcher, Westcliff University, Irvine, CA, USA

  7. AI Integrity: A New Paradigm for Verifiable AI Governance

    Apr 13, 2026Seulki LeeAI GovernanceAI Assurance

    AI Integrity Organization (AIO), Geneva, Switzerland

  8. Compliant But Unsatisfactory: The Gap Between Auditing Standards and Practices for Probabilistic Genotyping Software

    Apr 13, 2026Angela Jin, Alexander Asemota, Dan E. Krane +2Algorithmic Auditing

    University of California, Berkeley, Berkeley, California, USA · Wright State University, Dayton, Ohio, USA · Forensic Bioinformatic Services, Inc., Fairborn, Ohio, USA +1

  9. Why Aggregate Accuracy is Inadequate for Evaluating Fairness in Law Enforcement Facial Recognition Systems

    Mar 30, 2026Khalid Adnan AlsayedAlgorithmic FairnessGroup Fairness

    Founder, Ducaltus | AI Assurance & Governance · BSc (Hons) Artificial Intelligence · School of Computing, Engineering & Digital Technologies

  10. Synthetic Designed Experiments for Diagnosing Vision Model Failure

    Mar 30, 2026Krisanu SarkarSynthetic Data EvaluationAlgorithmic Auditing

    Indian Institute of Technology Bombay, Mumbai, India

  11. Explainability and Certification of AI-Generated Educational Assessments

    Mar 18, 2026Antoun Yaacoub, Zainab Assaghir, Anuradha KarEducational AssessmentGenerative AI in Education

    aivancity School for Technology, Business & Society, 151 Bd Maxime Gorki, 94800 Villejuif, France · Faculty of Science, Lebanese University, Hadath Campus, Hadath, Lebanon

  12. Anterior's Approach to Fairness Evaluation of Automated Prior Authorization System

    Mar 15, 2026Sai P. Selvaraj, Khadija Mahmoud, Anuj IravaneAlgorithmic FairnessGroup Fairness

    Anterior, Inc.

  13. Representation Fidelity:Auditing Algorithmic Decisions About Humans Using Self-Descriptions

    Mar 5, 2026Theresa Elstner, Martin PotthastAlgorithmic AuditingAlgorithmic Accountability

    Kassel University, Germany

  14. Algorithmic Compliance and Regulatory Loss in Digital Assets

    Mar 4, 2026Khem Raj Bhatt, Krishna SharmaAnti-Money LaunderingAnti-Money Laundering Detection

  15. A multimodal slice discovery framework for systematic failure detection and explanation in medical image classification

    Feb 27, 2026Yixuan Liu, Kanwal K. Bhatia, Ahmed E. FetitAlgorithmic AuditingMedical Image Classification

    Department of Computing, Imperial College London, UK · Aival, London, UK

  16. Global Sequential Testing for Multi-Stream Auditing

    Feb 25, 2026Beepul Bharti, Ambar Pal, Jeremias SulamSequential Hypothesis TestingMultiple Hypothesis Testing

    Johns Hopkins University · Amazon Responsible AI

  17. Governing AI Forgetting: Auditing for Machine Unlearning Compliance

    Feb 16, 2026Qinqi Lin, Ningning Ding, Lingjie Duan +1Algorithmic AuditingMachine Unlearning

    School of Science and Engineering, Shenzhen Institute of Artificial Intelligence and Robotics for Society, The Chinese University of Hong Kong, Shenzhen, China · Data Science and Analytics Thrust, Information Hub, Hong Kong University of Science and Technology (Guangzhou), China · Internet of Things Thrust, Information Hub, Hong Kong University of Science and Technology (Guangzhou), China

  18. Falsifying Discriminant Validity of Predictive Algorithms

    Jan 23, 2026Amanda CostonAlgorithmic AuditingPsychometric Validation

    Statistics Department University of California, Berkeley Berkeley, CA, USA

  19. White-Box Sensitivity Auditing with Steering Vectors

    Jan 23, 2026Hannah Cyberey, Yangfeng Ji, David EvansAlgorithmic AuditingLLM Auditing

    University of Virginia

  20. One Permutation Is All You Need: Fast, Deterministic Feature Importance and Model Stress-Testing

    Dec 15, 2025Albert DoradorPerturbation-Based Feature AttributionAlgorithmic Auditing

  21. A framework for auditing grounding claims

    Dec 5, 2025Daniel Quigley, Eric MaynardAlgorithmic Auditing

    Center for Possible Minds Indiana University Bloomington Bloomington, IN 47408 · Eruditis Milwaukee, WI 53207

  22. A global log for medical AI

    Oct 5, 2025Ayush Noori, Aaron E. Boussina, Hai Ho Bich +48Algorithmic AuditingHealthcare

  23. How Reliable are Fairness Audits with Unreliable Data?

    Jun 28, 2025Yash Vardhan TomarAlgorithmic AuditingAlgorithmic Fairness

    Purdue University