Algorithmic Auditing

Follow this topic Ask about this topic

Momentum

15 papers in the last four weeks, up 50% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 96

All topics

Robust Decentralized Fairness Auditing
Oct 7, 2026Sayan Biswas, Jade Garcia Bourrée, Anne-Marie Kermarrec +2Algorithmic Auditing LLM Auditing
EPFL, Switzerland
Shared and structured inputs undermine collective random choice by reasoning AI agents
Oct 7, 2026Takahiro Ezaki, Naoto Imura, Katsuhiro NishinariAI Agent Evaluation Selection Bias
Research Center for Advanced Science and Technology, The University of Tokyo 4-6-1 Komaba, Meguro-ku, Tokyo 153-8904, Japan · Department of Aeronautics and Astronautics, School of Engineering, The University of Tokyo 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656, Japan
CHARTER: Auditing Reference Substitution in Hierarchical Compact-Evidence Evaluation for Computational Pathology
Oct 6, 2026Hyun Do Jung, Jungwon Choi, Soojung Choi +2Computational Pathology Algorithmic Auditing
Yonsei University · KAIST · Hallym University Chuncheon Sacred Heart Hospital
Tolerance-Based Fairness Auditing: Violation Certification and Sensitivity Screening
Oct 1, 2026Jie Tang, Chuanlong Xie, Lixing ZhuAlgorithmic Auditing Algorithmic Fairness
School of Statistics, Beijing Normal University, Zhuhai, Guangdong 519087, China
A Rank Graduation metric for Algorithmic fairness
Sep 30, 2026Dalia Atif, Paolo GiudiciAlgorithmic Auditing Algorithmic Fairness
Fairness Theatre: Evaluating Post-Hoc Fairness Interventions in Vendor-Controlled Early Warning Systems
Sep 29, 2026Kelly McConvey, Angelina Zhai, Rebecca Li +1Algorithmic Auditing Algorithmic Fairness
University of Toronto, Toronto, Ontario, Canada · Georgia Institute of Technology, Atlanta, Georgia, USA
When Does Randomized Oversight Align AI Agents That Can Conceal?
Sep 29, 2026Joshua S. Gans, Richard HoldenAlgorithmic Auditing AI Agent Auditing
UNSW Business School
VStress: Correlation-Aware Auditing and Adaptive Budget Allocation for Repeated Verifiers
Sep 29, 2026Miaobo Hu, Shuhao Hu, Xiaobo Guo +5Algorithmic Auditing Black-Box Model Auditing
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China
Audit-First VAPO: Risk-Certified Selective Updates under Imperfect Verification
Sep 27, 2026Miaobo Hu, Shuhao Hu, Xiaobo Guo +5Algorithmic Auditing Reinforcement Learning with Verifiable Rewards
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China
When Post-Processing Fairness Constraints Help and When They Harm: Evidence from Eight Cross-Domain Evaluations
Sep 22, 2026Nithin Raghava Ramachandra NarlaAlgorithmic Auditing Algorithmic Fairness
Independent Researcher, Dallas-Fort Worth, Texas, United States
Governance-as-Code: Translating EU AI Act Technical Requirements into Executable Compliance Pipelines for Generative AI Systems
Sep 17, 2026Rudrendu Kumar Paul, Sourav NandyAlgorithmic Auditing AI Governance
Boston University, Boston, MA, USA
Anomaly Detection in General Ledger Data: Results from a Hybrid Approach
Sep 16, 2026Jan Gronewald, Alexander Michael Rombach, Sebastian Stephan +1Algorithmic Auditing Anomaly Detection
German Research Center for Artificial Intelligence (DFKI) GmbH
A Decision-Support Audit Protocol for Supervision Drift in Proxy-Labeled Credit-Risk Prediction
Sep 14, 2026Mehrdad Shoeibi, Muhammad Shabanpour, Waldemar Karwowski +1AI Risk Management Algorithmic Auditing
Department of Industrial Engineering and Management Systems University of Central Florida, Orlando, FL, USA · Department of Economics, Northeastern University, Boston, MA, USA
When Agent Metrics Measure Different Things: An Evidence-Grounded Audit of the Praxa AI Pipeline
Sep 14, 2026Stefan G. Creadore, Peyton WoakzAlgorithmic Auditing Benchmark Auditing
Tampa, FL, USA
Interpolation Is Not Invariance: Pair Count Is Not Coverage in Transformation Audits
Sep 14, 2026Mohammed Ahnouch, Lotfi ElaachakAlgorithmic Auditing Black-Box Model Auditing
Université Paris 1 Paris, France · Faculty of Science and Technology of Tangier Abdelmalek Essaadi University Tangier, Morocco
Characterizing Bluesky Content Moderation Service: From Automation of Service to Landscape of Harms
Sep 12, 2026Pushpdeep Singh, Sayeh Jarollahi, Ayan Majumdar +5Algorithmic Auditing Content Moderation
Max Planck Institute for Software Systems (MPI-SWS), Germany · Saarland University, Germany · Indian Institute of Technology Kharagpur, India
Do AI Coding Assistants Check Before They Install? A Pre-Registered Demand-Side Audit of Trust Signals in the Research Software Supply Chain
Sep 7, 2026Pengyin ShanAlgorithmic Auditing AI Coding Agents
National Center for Supercomputing Applications University of Illinois Urbana-Champaign Urbana, IL, USA
Scores Alone Do Not Prove Discovery: The Discovery Certification Protocol for Auditing AI Research Agents
Sep 7, 2026Jingjie Ning, Shanshan Zhong, Xiaochuan Li +1AI Agents for Scientific Discovery Algorithmic Auditing
School of Computer Science, Carnegie Mellon University
Agent Flight Recorder: Tamper-Evident Audit Trails with On-Chain Anchoring for Long-Horizon Tool-Using Agents
Sep 1, 2026Laurent Bindschaedler, Quentin Botha, Christoph SiebenbrunnerData Provenance Algorithmic Auditing
Max Planck Institute for Software Systems · Research Institute for Cryptoeconomics, Vienna University of Economics and Business
When Can a Machine Trust a Statute? A Survival Certificate for Machine-Extracted Legal Logic
Sep 1, 2026Surya SakaAlgorithmic Auditing Legal Document Analysis
JudicialMind
Causal Evidentiary Governance for High-Risk Machine Learning Systems
Sep 1, 2026Samah Kareem, Barış ÇeliktaşAlgorithmic Auditing Algorithmic Accountability
Department of Computer Engineering, Isik University, Istanbul, Türkiye. · Department of Computer Engineering, I¸sık University, Istanbul, Türkiye.
VFR-Audit: Verdict-Level Reliability for Fairness Audits in Hospital Length-of-Stay Prediction
Aug 31, 2026Md Jannatul Rakib Joy, Viet Vo, Caslon ChuaAlgorithmic Auditing
School of Science, Computing and Emerging Technologies
The Acknowledgment Point Is the System: Durable Policy-Decision Receipts for AI Audit Evidence
Aug 17, 2026Neeraj Kumar Singh BeshaneData Provenance Algorithmic Auditing
Independent Researcher, Fremont, California
Sampling Luck Masquerades as Allocation Gain: Auditing Test-Time Budget Allocation for Neural Combinatorial Optimization
Aug 13, 2026Jinhyung BaeAlgorithmic Auditing Inference-Time Compute Allocation
Department of Industrial and Management Engineering and the AI Data Convergence program, Hankuk University of Foreign Studies, Yongin, Republic of Korea
Correct Is Not Governed: Provenance Integrity in Agentic Workflows
Aug 13, 2026Jesus SalasData Provenance Algorithmic Auditing
Independent Researcher · The author is employed by Microsoft.
Co-constructing sociotechnical AI governance: participatory system mapping using algorithm registers
Aug 12, 2026Íñigo de Troya, Maurus Enbergs, Neelke Doorn +1AI-Assisted Decision Making Algorithmic Auditing
TU Delft
Governing Agentic AI in FinTech
Aug 11, 2026Henry HanAlgorithmic Auditing AI Agent Governance
Data Science and Artificial Intelligence Innovation Laboratory, Baylor University, USA
Who Gets Heeded? An Obligation-Level Audit of Responsiveness in EPA Rulemaking
Aug 11, 2026Jianing Fan, Yue YaoAlgorithmic Auditing Legal Document Analysis
Columbia University New York, NY, USA
When Can Fraud Operations Authorize Automation? A Decision-Support Framework for Fresh Audit Evidence and Review Workload
Aug 9, 2026Jie DengAI-Assisted Decision Making Algorithmic Auditing
Tongji University, Shanghai, China
Beyond Aggregate Calibration: Decomposing Income-Conditional Recall Disparities in Automated Credit Default Prediction
Aug 8, 2026Sai Srikar BoddupalliAlgorithmic Auditing Algorithmic Fairness
Independent Researcher
Innovation-Residual Auditing of Autonomous Analysis Agents: Localization, Detection Limits, Error Control, and Identifiability
Aug 6, 2026Ahmed Hassoon, Mark DredzeAlgorithmic Auditing FDR Control
Johns Hopkins University
A Chain Is Only as Strong as Its Weakest Link: A Scoping Review of System Integration Audits in AI
Aug 5, 2026Leah Davis, Dominic Martin, AJung MoonAI Assurance Algorithmic Auditing
McGill University, Montr´eal, QC · Universit´e du Qu´ebec `a Montr´eal, Montr´eal, QC
Local Violation Certification for Linear Predict-Then-Optimize Pipelines
Aug 5, 2026Ş. İlker Birbil, Wenhao ChiAlgorithmic Auditing Predict-Then-Optimize
Faculty of Economics and Business, University of Amsterdam, The Netherlands · Yau Mathematical Sciences Center, Tsinghua University
GoT-CD: Graph-of-Thoughts Causal Discovery and the Fragility of Post-hoc Path-Specific Fairness Audits
Aug 3, 2026Nitish Nagesh, Elahe Khatibi, Thomas Dean Hughes +3Algorithmic Auditing Algorithmic Fairness
Department of Computer Science, University of California, Irvine, Irvine, CA, USA · University of Orléans, Orléans, France
Auditing Semantic Gains in Sequential Recommendation: A Lightweight Recovery Test
Aug 2, 2026Kong Wang, Zhongke He, Xiang Chen +4Sequential Recommendation Algorithmic Auditing
Hunan University · Dalian University of Technology · Tongji University +3
ZK-SR117: A Chunked Zero-Knowledge Attestation Design for Aggregated Fair-Lending Metrics, with a Control Mapping toward Full SR 11-7 Coverage
Aug 1, 2026Mohammad Nasir Uddin, Rahnuma Tabassum Orpita, Eklachur Rahman Bhuiyan +1AI Risk Management Algorithmic Auditing
1DBA-BIDA (Business Intelligence and Data Analytics) Program, Westcliff University, USA · Department of Computer Science and Engineering, Northern University Bangladesh, Bangladesh · 3Washington University of Science and Technology, Alexandria, Virginia, USA +1
Fairness Auditing: Lower Bounds on Company Manipulation
Aug 1, 2026Rachit Verma, Padala Manisha, Sujit GujarAlgorithmic Auditing Algorithmic Fairness
Indian Institute of Technology Gandhinagar · International Institute of Information Technology Hyderabad
ALIVE: Warnings Before Exclusion in Budgeted Multi-Source Learning
Jul 31, 2026Xiyang Zhang, Hongzhi Wang, Yuanhe TianAlgorithmic Auditing
1Harbin Institute of Technology · 2Zhongguancun Academy
Scaling, Lock-In, and Proxy Compliance: A Political Economy of Responsible AI
Jul 30, 2026Florian A. D. Burnat, Brittany I. DavidsonAlgorithmic Auditing Algorithmic Accountability
University of Bath, Bath, UK.
Collusion with Competitive Marginals: Price-Level Audits Are Blind by Construction
Jul 29, 2026Xin Xu, Chengrui Wu, Jiayu Lu +3Algorithmic Auditing Automated Bidding
Carnegie Mellon University · Zhejiang University
A Computational Ethical Framework for Financial Digital Phenotyping for Mental Health
Jul 27, 2026Oluwadara Adedeji, Michael Mayowa Farayola, Jeff Brozena +3Algorithmic Auditing
School of Computer Science, University College Dublin, Dublin, Ireland. · School of Computing, Dublin City University, Dublin, Ireland. · College of Information Sciences and Technology, Pennsylvania State University,University Park, PA, USA. +1
Finite-Sample Coverage Audits for High-Recall Candidate Generation: Certification and Learning-Theoretic Design
Jul 23, 2026Martin Anthony, Kaveh Salehzadeh NobariAlgorithmic Auditing
Data Science Institute, London School of Economics and Political Science · Department of Mathematics, London School of Economics and Political Science · The Inclusion Initiative, London School of Economics and Political Science
Operational Identity: A Finite Audit of Declared and Implemented Rules of Sameness
Jul 22, 2026Denise M. CaseEntity Resolution Algorithmic Auditing
School of Computer Science and Information Systems Northwest Missouri State University, Maryville, MO, USA
Toward Auditable Fraud Detection: Combining Graph Features, Model Explanations, and Agentic Case Investigation
Jul 21, 2026Rahil SharmaAlgorithmic Auditing Tabular Anomaly Detection
Vrije Universiteit Amsterdam
Predictive Extrema, Unprofitable Policies: An AI-Assisted Audit of Candle-Based Binance Spot Timing Models
Jul 21, 2026Ayoub JadouliQuantitative Finance Algorithmic Auditing
Computer Science and Smart Systems, Faculty of Sciences and Technology Abdelmalek Essaâdi University, Tangier, Morocco
Can We Steer the Black-Box? Towards Controllability-Centric Evaluation of Recommender Systems with Collaborative Agents
Jul 15, 2026Jiwen Zhou, Xiang Liu, Mingming Li +5Algorithmic Auditing Recommender Systems
Institute of Information Engineering, Chinese Academy of Sciences, China · School of Cyber Security, University of Chinese Academy of Sciences, China
Auditing the Risk Claims of Distributional Reinforcement Learning
Jul 13, 2026Hari PrasadDistributional RL Algorithmic Auditing
From Neural Network Decisions to Training Cases: An Exact Account via Case-Based Decision Theory
Jul 13, 2026Manli Yan, Yuebin Lin, Yaowen Yu +1Algorithmic Auditing Neural Network Interpretability
Huazhong University of Science and Technology Wuhan, China
When the Reward Suite Is Leaky: A Preregistered Causal Contrast of Natural Verifier False Positives in RLVR
Jul 13, 2026Chuyifei ZhangAlgorithmic Auditing Reinforcement Learning with Verifiable Rewards
Beijing Jiaotong University
SolarChain-Eval: A Physics-Constrained Benchmark for Trustworthy Economic Agents in Decentralized Energy Markets
Jul 9, 2026Shilin Ou, Yifan Xu, Luyao ZhangAlgorithmic Auditing AI Agent Evaluation
Duke Kunshan University Kunshan, Jiangsu, China · Digital Innovation Research Center and Social Science Division, Duke Kunshan University, Address: Duke Avenue No.8, Kunshan, Suzhou, Jiangsu, China, 215316
The Power of Backdoor Absorption in Community Training
Jul 7, 2026Issam Seddik, Sami Souihi, Mohamed Tamaazousti +1Algorithmic Auditing Backdoor Attacks
Université Paris-Saclay, CEA LIST, Palaiseau, France
Governed Caste Reassignment in Heterogeneous Swarms: An Asymmetric-Trust Protocol with Audited Operator Countersignature
Jul 6, 2026Xue Qin, Simin Luan, Cong Yang +1Algorithmic Auditing Swarm Robotics
School of Software, Harbin Institute of Technology, Harbin, China. · School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China. · School of Future Science and Engineering, Soochow University, Suzhou, China.
The Eticas AI Risk Taxonomy: Open Infrastructure for Operationalizing AI Audits
Jul 2, 2026Gemma Galdon Clavell, Pablo Accuosto, Usman GoharAI Assurance AI Risk Management
Gaming Consensus: Coordinated Manipulation in Crowdsourced Fact-Checking
Jul 2, 2026Nikil Roashan Selvam, Jay Baxter, Sophie Hilgard +4Social Media Analysis Algorithmic Auditing
Stanford University, Stanford, CA, USA · X Community Notes, xAI, Palo Alto, USA
Using AI Agents to Automate Black-Box Audits of Personalization Algorithms at Scale
Jun 29, 2026Alessandro Morosini, Sarah H. Cen, Andrew Ilyas +3Algorithmic Auditing Algorithmic Bias
Massachusetts Institute of Technology · Carnegie Mellon University
Pooled Leaderboards Hide System-Specific Winners: A Reporting-Protocol Audit of Offline Root-Cause Analysis Benchmarks
Jun 28, 2026Lining Hu, Ting Liu, Yuzhuo FuPairwise Comparison Algorithmic Auditing
School of Electronic, Information and Electrical Engineering Shanghai Jiao Tong University Shanghai, China
Liquidity-Based Audit of Algorithmic Trading Strategies
Jun 27, 2026Irene AldridgeFinancial Services Quantitative Finance
The Two Genie Game: Adoption and Welfare in Audit-Grounded AI Governance
Jun 27, 2026Darrell Lewis-SandyAlgorithmic Auditing AI Agent Safety
Statistical and Structural Approaches to Algorithmic Fairness
Jun 24, 2026Antonio FerraraAlgorithmic Auditing Algorithmic Fairness
Graz University of Technology · Institute of Human-Centred Computing · Graz
Certification of Machine Learning Models via Directional Sharpness
Jun 23, 2026Gefei Tan, Adria Gascon, Sarah Meiklejohn +1Neural Network Generalization Neural Network Verification
Northwestern University & Google · Google · University College London

Oct 7, 2026, cs.LGJ/K move · Enter open · S save

Robust Decentralized Fairness Auditing

Sayan Biswas, Jade Garcia Bourrée, Anne-Marie Kermarrec, Palak+1

EPFL, Switzerland

Algorithmic Auditing LLM Auditing Algorithmic Fairness

Open paper AskPDF arXiv

Emerging legislation requires large language models (LLMs) to be audited for compliance with regulatory standards, particularly fairness. Such black-box audits typically assume a single auditor with access to a large, representative set of queries. In practice, it can be difficult for an auditor to obtain such a query set, but multiple auditors can together cover the relevant demographic groups by auditing the LLM collaboratively with their individual query sets. However, relying on multiple auditors raises a fundamental trust problem, as they may act on behalf of the LLM provider to portray a misleading appearance of fairness, i.e., fairwashing. We propose Auditopus, a novel approach for robust decentralized fairness auditing. In Auditopus, auditing proceeds in rounds without a central server. In each round, every auditor issues a fixed number of queries to the LLM, and sends only cumulative statistics vectors of its query results to other auditors instead of sensitive queries in clear. The fairness of the audited LLM is then estimated by aggregating all the vectors. We show theoretically and empirically that even a single adversarial auditor in the network can steer this estimate by fabricating the vectors it sends, making an unfair LLM appear fair. To address this threat, Auditopus has each honest auditor locally down-weight any auditor whose cumulative statistics vectors are statistically inconsistent with previous ones. We implement Auditopus and compare it to robust aggregation baselines on two datasets with two pre-trained LLMs. Against an attacker that optimizes the vectors it sends to make the LLM appear fair, Auditopus reduces audit error by up to 78% on average relative to no defense and at least 62% relative to the robust aggregation baselines. Even when 49% of the auditors are adversarial, Auditopus never lets a very unfair or moderately unfair LLM pass as fair.

Momentum

Related topics

Latest papers 96