Overview

Algorithmic auditing studies how to test automated decisions and the systems around them, especially when errors or disparities affect people. Common approaches compare subgroup outcomes, use counterfactual or causal tests, probe model behavior under perturbations, and apply statistical methods such as sequential testing and uncertainty analysis. Audits increasingly examine deployed pipelines, data and component interactions, strategic responses, and evidence trails—not just a model’s aggregate accuracy. A central challenge is making conclusions reliable and meaningful despite limited access, shifting conditions, incomplete data, and metrics that can be gamed or fail to capture the harms that matter.

Latest papers

CardsList

159 papers20 papers in the last four weeks, down 9% on the four weeks before. 0.2% of all new papers.

  1. Action-outcome coupling: a pre-investment feasibility audit for offline reinforcement learning in supply chain operations

    Nov 17, 2026Tung Dang, The Hung Phung, Son Lam Nguyen +1Offline RLAlgorithmic Auditing

  2. Optimistic Verifiers Get Hacked: Reward Hacking Under an Exact Process Oracle

    Nov 16, 2026Arunkumar V, Manoranjan Gandhudi, Gangadharan G. R. +2Reward HackingReinforcement Learning with Verifiable Rewards

  3. A Crowd-Audited Atlas of Implicit Bias in LLM-Powered Agents

    Nov 16, 2026Jason Alexander, Yuxuan Li, Michael Licamele +2Social Bias in Language ModelsDemographic Bias in Language Models

  4. Verifiable Random Sampling of Private Data Without a Trusted Single Party

    Nov 16, 2026Muhammed Karamuk, Mikołaj Glinka, Ruben WolffAlgorithmic Auditing

  5. Calibrated Functional Prediction from Overlapping Nonlinear Observations

    Nov 16, 2026Peida LiKernel RegressionAlgorithmic Auditing

  6. When Does an AUROC Comparison Survive a Change in Population Mix?

    Nov 15, 2026Mahtab Movahhedrad, Hamidreza Hasani Balyani, Seyed Pouyan Mousavi DavoudiBenchmark AuditingUncertainty Quantification

  7. Algorithmic Opacity as a Principal-Agent Problem

    Nov 13, 2026Ignacio Cofone, Katherine J. StrandburgAlgorithmic Auditing

  8. Hiding a Gap Does Not Close It: The Price of Removing the Mean Group Contrast from Learned Models of Learning Dynamics

    Nov 11, 2026Sosthène Bidojesussi Hounzinhin, Ahmed Ratnani, Abdeladim El AkriGroup FairnessAlgorithmic Auditing

  9. Too Big to Inspect? Ethically Auditing Large Datasets Through Small Samples

    Nov 7, 2026Giovana Marchetti, Jhessica Silva, Tiago Torrent +2AI EthicsDataset Bias

  10. Navigating Disagreement Among Criteria for Evaluating Explanation Faithfulness

    Nov 4, 2026Grant Shanklin, Jiří Němeček, Asia BiegaExplainability EvaluationAlgorithmic Auditing

  11. Sequential auditing of connected forecast errors under partial observations

    Nov 2, 2026Yuanbo Zhou, Junjie XiongAlgorithmic AuditingGraph-Based Anomaly Detection

  12. Auditing Representation-Space Fraud Detection: A Bayesian IRL Framework for Calibrated Uncertainty and Its Limits

    Nov 2, 2026Shivam Swarup, Divya Prakash Shrivastava, Rakesh ThakurRecommender SystemsAlgorithmic Auditing

  13. Queering the Audits: Community-Based Auditing of AI Harms to Queer Communities

    Oct 30, 2026Organizers of QueerInAI, :, A Pranav +8Algorithmic AuditingParticipatory AI

  14. Reliable AI-Assisted Decisions via Selective Auditing

    Oct 22, 2026Isaac Gibbs, Elise L. Han, Ryan J. Tibshirani +1AI-Assisted Decision MakingHuman Oversight of AI

  15. Auditing the Hand-Written Inputs of Learned Access Control

    Oct 22, 2026Arsalan Jalil Sarghaleh, Mehdi HazratiAccess ControlAlgorithmic Auditing

  16. The Price of Choice in Selective Prediction

    Oct 21, 2026Daniel Cieślak, Andrzej CzyżewskiSelective PredictionAlgorithmic Auditing

  17. Better Policies, Worse Verifier Audits

    Oct 21, 2026Daniel Cieślak, Jacek ŁubińskiAlgorithmic AuditingPolicy Evaluation

  18. A Cue's Hidden Price: Auditing Seller Influence on AI Shoppers

    Oct 19, 2026Felix Buecheler, Gerardo Berbeglia, Mingyu Guoconversational shopping assistantsAI Agent Auditing

  19. Transparent Information Flow Modeling for Role-Aware AI Act Assessment

    Oct 12, 2026Mattias Brännström, Themis Dimitra Xanthopoulou, Lili JiangAI GovernanceAlgorithmic Auditing