Black-Box Model Auditing

Ask about this topic

Overview

Black-box model auditing studies what can be verified about an AI system when its weights, training data, or internal processes are inaccessible. Auditors use statistical tests and membership inference, active or sequential query selection, counterfactual and causal probes, behavioral fingerprints, and sometimes cryptographic verification to examine privacy, fairness, safety, robustness, explanations, and model identity. These methods are applied to settings from hosted language-model APIs and recommendation systems to scientific predictors, often using controlled benchmarks, matched interventions, and repeated trials. A central tension is that limited or manipulable outputs can conceal rare failures and make latent properties hard to establish, so audit conclusions depend on query access, measurement reliability, and how closely tests reflect deployment.

Latest papers

CardsList

96 papers11 papers in the last four weeks, down 35% on the four weeks before. 0.1% of all new papers.

  1. Auditor-Assisted Summary-Channel Verification for Hosted LLM Identity Substitution

    Oct 19, 2026Ziyi Zhang, Ziyao Wang, Guoheng Sun +2LLM AuditingLLM Watermarking

    Stony Brook University · University of Maryland, College Park

  2. Blackbox Performance Analysis of LLM-based Network Control

    Oct 17, 2026Mahdi Alizadeh, Pooria Namyar, Arvin Ghavidel +1Black-Box Model AuditingLanguage Model-Based Control

    University of Southern California · Microsoft Research

  3. Dataset Inference for Data Provenance and Privacy Auditing in Tabular Foundation Models

    Oct 16, 2026Dariush Wahdany, Jesse Cresswell, Naiqing Guan +3Training Data AttributionTabular Foundation Models

    CISPA Helmholtz Center for Information Security · Layer 6 AI

  4. PROBE: Learning to Audit Policy Compliance in Tool-Using LLM Agents

    Oct 15, 2026Kshitij Mishra, Abhijith Sharma, Nils Lukas +1AI Agent AuditingLLM Agent Evaluation

    Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE

  5. PairAudit: Guiding Human Review with Graph Tokens under Distribution Shift

    Oct 7, 2026Jiran Tao, Binyan JiangGraph-Based Anomaly DetectionNetwork Intrusion Detection

    The Hong Kong Polytechnic University

  6. What Does an Observability Foundation Model Know?

    Oct 4, 2026Dhyey Dharmendrakumar Mavani, Rian Atri, Tairan JiTime Series ForecastingRepresentation Probing

    Independent · Keiji AI

  7. Confidence-Controlled XAI Auditing for Pedestrian Detection under Domain Shift

    Oct 1, 2026Ruben Dario Florez-ZelaExplainability EvaluationBlack-Box Model Auditing

    Universidad Nacional de San Agustín de Arequipa, Arequipa, Peru

  8. Beyond Answer Confidence: A Controlled Audit of Self-Knowledge in a Black-Box Decision Model

    Oct 1, 2026Sharath M Shankaranarayana, Davor Runje, Jan JanninkProbability CalibrationBlack-Box Model Auditing

  9. Efficient Active Auditing of Multi-Group Fairness with Bias Probes

    Sep 30, 2026Ayoub Ajarra, Debabrota BasuAlgorithmic FairnessGroup Fairness

    ´Equipe Scool, Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189- CRIStAL

  10. VStress: Correlation-Aware Auditing and Adaptive Budget Allocation for Repeated Verifiers

    Sep 29, 2026Miaobo Hu, Shuhao Hu, Xiaobo Guo +4Algorithmic AuditingBlack-Box Model Auditing

    School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China

  11. PhysioTRACE: Provenance-Aware Stress Tests for Physiological Foundation Models

    Sep 28, 2026Ayana Mussabayeva, Anuar Aimoldin, Olivier Oullier +2Data ProvenanceElectrocardiogram Foundation Models

    Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), Abu Dhabi, UAE · McGill University, Montreal, Canada · Carnegie Mellon University, Pittsburgh, USA

  12. Auditing Agent Actions through Query-Conditioned Attribution

    Sep 27, 2026Yifan Liu, Praveen Venkateswaran, Abdulhamid Adebayo +1Gradient-Based AttributionAI Agent Benchmarks

    University of Illinois Urbana-Champaign · IBM

  13. Black-Box Auditing of Epistemic Reliability in Multi-Agent Debate Distillation

    Sep 26, 2026Derui Wang, Zewei Shi, Rayne Holland +4AI Agent ReliabilityBlack-Box Model Auditing

    CSIRO · The University of Melbourne

  14. Auditing System-1 Models on Biosecurity-Relevant Benchmarks: Calibration, Selective Prediction, and Permutation Instability in a Non-Generative Model

    Sep 24, 2026Kimon Antonios Provatas, Ilias Georgakopoulos-SoaresLanguage Model CalibrationSelective Prediction

    Division of Pharmacology and Toxicology, College of Pharmacy The University of Texas at Austin, Dell Pediatric Research Institute Austin, TX, USA

  15. When Temporal Perturbations Act Like Sensor Biases: Label-Free Auditing of Wearable Activity Recognizers

    Sep 24, 2026Qingyu Wu, Yuan Wei, Renju Liu +1Human Activity RecognitionAdversarial Attacks

    Defense Innovation Institute, Academy of Military Science, Beijing, China · School of Software Engineering, South China University of Technology, Guangzhou, China

  16. The Delegation Blind Spot: Auditing Product Decisions from Agent Choices

    Sep 22, 2026Shivam GuptaAI Agent AuditingPartial Identification

    Independent Research

  17. Trust, but Validate the Instrument: Auditing AI-Generated RTL Verification Plans on Authored Security-Regression Proxies

    Sep 17, 2026Hang Xiao, Chuhong Xu, Kainan Zhou +2Formal Hardware VerificationLLM Auditing

    Fortinet, Inc. Sunnyvale, CA, USA · Sony Corporate of America San Jose, CA, USA · Google LLC Mountain View, CA, USA

  18. Interpolation Is Not Invariance: Pair Count Is Not Coverage in Transformation Audits

    Sep 14, 2026Mohammed Ahnouch, Lotfi ElaachakAlgorithmic AuditingBlack-Box Model Auditing

    Université Paris 1 Paris, France · Faculty of Science and Technology of Tangier Abdelmalek Essaadi University Tangier, Morocco

  19. Operational Range Bounding in Spectroscopy: A Safety Cage Framework for Machine Learning Models

    Sep 11, 2026Nikki Grens, Luís F. Simões, Kai Hou Yip +1Scientific MLSelective Prediction

    ML Analytics, 2660-329 Lisbon, Portugal · Department of Space Engineering, Delft University of Technology, 2600 AA Delft, The Netherlands · Department of Physics, King’s College London, University of London, WC2R 2LS, London, United Kingdom +1

  20. Membership Inference via Pairwise Likelihood Ratios

    Sep 11, 2026Shengjie Niu, Zebin Yun, Yeheng Ge +1Membership Inference AttacksLikelihood Ratio Testing

    The Hong Kong Polytechnic University, Hong Kong, China · Tel Aviv University, Israel

  21. A Low Grounding Score Is Not an Ungrounded Judge: Identifying the Perceptibility Confound in Multimodal Oversight

    Sep 9, 2026Rasul Khanbayov, Hasan KurbanMultimodal Model EvaluationVLM Evaluation

    College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar

  22. The Audit Decides the Verdict: Instrument Effects Rival Demographic Bias in LLM Decision Audits

    Sep 8, 2026Siddharth Vohra, Manikandan RavikiranLanguage Model Bias EvaluationDemographic Bias in Language Models

    Carnegie Mellon University Amazon Web Services AI Native Pittsburgh, PA, USA · Indian Institute of Technology Mandi, India

  23. Black-Box Membership Inference via Word-Level Probability Estimation

    Sep 8, 2026Shengjie Niu, Yeheng Ge, Jian HuangMembership Inference AttacksPrivacy Leakage in Language Models

    Department of Data Science and Artificial Intelligence The Hong Kong Polytechnic University, Hong Kong SAR, China · Department of Applied Mathematics The Hong Kong Polytechnic University, Hong Kong SAR, China

  24. Evaluating and Recalibrating Probabilistic Classifiers from One-Bit Threshold Queries

    Sep 4, 2026Roman Plaud, Antoine Saillenfest, Matthieu Labeau +2Language Model CalibrationUncertainty Calibration

    Institut Polytechnique de Paris · Onepoint, France · Ghent University

  25. Clean Engineering, Unstable Measurement: A Preregistered Reliability Failure of Black-Box LLM Observers on Shared Endpoints

    Sep 3, 2026Haoyaun Zhu, Jie ZhangLLM EvaluationLLM-as-a-Judge

    School of Electronic and Electrical Engineering, University of Sheffield, Sheffield S10 2TN, UK · R&D Department, Ranplan Wireless Network Design Ltd., Cambridge CB23 3UY, UK · R&D Department, Cambridge AI+ Ltd., Cambridge CB23 3UY, UK

  26. Rent-a-RAG: Embedding-Space Watermarks for Auditing Third-Party RAG

    Sep 3, 2026Alexandr Goultiaev Tolstokorov, Kyriakos Mouratidis, Javad Dogani +1Dataset WatermarkingRAG Security

    IMDEA Networks Institute, Madrid, Spain · Universidad Carlos III de Madrid, Madrid, Spain · Singapore Management University, Singapore

  27. Auditing Anonymous AI Models: A Four-Stage Protocol for Black-Box Identity Verification

    Aug 31, 2026Yisen XiLanguage Model FingerprintingBlack-Box Model Auditing

    Independent Researcher, Beijing, China

  28. BLOOM-WILT: Logit Tilting for Behaviour Elicitation in Automated LLM Auditing

    Aug 31, 2026Adrians Skapars, Edoardo ManinoLanguage Model SteeringLanguage Model Safety Evaluation

    University of Manchester, United Kingdom

  29. Not to Break, but to Attest: Adversarial Probes for Privacy-Preserving LLM Verification

    Aug 28, 2026Cameron Wilding, Mina Shaker, Fatemeh GanjiLLM AuditingZero-Knowledge ML

    Worcester Polytechnic Institute Worcester, MA, USA

  30. CleanScore: Black-Box Benchmark Audits with Negative Controls and Sensitivity Bounds

    Aug 27, 2026Jeffery Opoku, David BanaheneBenchmark AuditingBlack-Box Model Auditing

    School of Mathematical and Statistical Sciences The University of Texas Rio Grande Valley, Edinburg, Texas, USA · Department of Biostatistics, Robert Stempel College of Public Health and Social Work Florida International University, Miami, Florida, USA