Algorithm Auditing

Ask about this topic

Recent momentum

-60%

41 papers in the last 28 days · 1.1% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-14

18 new papers

A weekly snapshot of new work published in Algorithm Auditing.

Period ending 2026-09-07

6 new papers

A weekly snapshot of new work published in Algorithm Auditing.

436 papers

Latest in Algorithm Auditing

Open your feed →
CardsList

  1. Challenges of Auditing: Variability in Outputs of Large Language Models for Health

    Sep 15, 2026Yuan Pu, Yewon Chang, Furong Jia +4Trustworthy Medical Artificial IntelligenceAlgorithm Auditing

  2. A Decision-Support Audit Protocol for Supervision Drift in Proxy-Labeled Credit-Risk Prediction

    Sep 14, 2026Mehrdad Shoeibi, Muhammad Shabanpour, Waldemar Karwowski +1Credit Risk PredictionAlgorithm Auditing

  3. Clean Scores, Buried Evidence, and Confident Wrong: A Receipt-Based Audit of Frontier Agentic QA

    Sep 14, 2026Luis M. SánchezAlgorithm Auditing

  4. DepthBenchCAD: When Does Deeper Auditing Yield More Reliable Conclusions?

    Sep 14, 2026Hongye Yang, Zhihao Xie, Shengjun Xiong +1Algorithm Auditing

  5. Four Ledgers, Not One Score: Responsible Communication of LLM-Judge Calibration in Biomedical ML

    Sep 14, 2026Sidi Chang, Peiying ZhuLlm-As-A-JudgeAlgorithm Auditing

  6. Interpolation Is Not Invariance: Pair Count Is Not Coverage in Transformation Audits

    Sep 14, 2026Mohammed Ahnouch, Lotfi ElaachakAlgorithm AuditingOptimal Ergodic Coverage

  7. Crypto Accounting Bench: Evaluating Frontier and Open-Weight Models on Crypto-Asset Accounting Tasks

    Sep 13, 2026Kareem Khattab, Omar Khattab, Mohamed IbrahemBlockchainAlgorithm Auditing

  8. LLMs as Post-hoc Auditors of Physiological Plausibility in Symbolic Regression: A Clinician-Evaluated Case Study

    Sep 12, 2026Jorge López-Varela, J. Ignacio Hidalgo, José-Manuel Muñoz +6Symbolic RegressionExplainability

  9. Characterizing Bluesky Content Moderation Service: From Automation of Service to Landscape of Harms

    Sep 12, 2026Pushpdeep Singh, Sayeh Jarollahi, Ayan Majumdar +5Content ModerationAlgorithm Auditing

  10. Published Unlearning Numbers Move Per Checkpoint, and Not Because the Removed Data Survives: An Audit of 263 Released Batch-Normalized Checkpoints

    Sep 12, 2026Junlong Shen Xingyu LiModel CheckpointsUnlearning Aims

  11. GANDR: Claim Auditing for Verifiable Legal Answer Generation

    Sep 9, 2026Chen Qian, Yimeng Wang, Yu Chen +2Llm-Based Statutory Question AnsweringVerdict

  12. Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets

    Sep 9, 2026Judith Bernett, Anton Spannagl, Joel Ås +2ProteinShortcuts

  13. Reference-Based Bias Detection in LLMs via Relative Representations of Hidden States

    Sep 9, 2026Marek Jeliński, Jan Dubiński, Maciej Chrabaszcz +1Large Language Model BiasAlgorithm Auditing

  14. AgentAudit: An Open, Extensible Framework for Full-Lifecycle Trust Evaluation of AI Agents

    Sep 9, 2026Shrey Nag, Sachita, Abhishek Kumar Singh +2Agentic EvaluationsAlgorithm Auditing

  15. Subgroup Membership Inference Audits of Differentially Private Synthetic Text

    Sep 9, 2026Yidan Sun, Viktor Schlegel, Srinivasan Nandakumar +2Membership Inference AttacksMembership Inference

  16. Decision Shifts, Lost Label Functionality, and an Inconclusive Grounding Audit in Correctness-Gated Multi-Teacher Distillation

    Sep 9, 2026Xiaofei FengMulti-Teacher DistillationTeacher

  17. Efficient Fairness Auditing Across Guidance Scales in Text-to-Image Diffusion Models via Causal Abstraction

    Sep 8, 2026Nabila Tasfiha Rahman, Rajatsubhra Chakraborty, Depeng Xu +1Text-To-Image Diffusion ModelsDiffusion Models

  18. The Audit Decides the Verdict: Instrument Effects Rival Demographic Bias in LLM Decision Audits

    Sep 8, 2026Siddharth Vohra, Manikandan RavikiranLarge Language Model BiasAlgorithm Auditing

  19. When Models Defer to Wrong Answers: A Robustness Audit of Source-Attributed Cues in Multiple-Choice QA

    Sep 8, 2026Manikandan Ravikiran, Siddharth VohraMultiple-Choice QuestionsCues

  20. Benchmark Scores Are Pipeline-Dependent: A Reliability Audit of Cybersecurity LLM Benchmarks

    Sep 8, 2026Aymene Berriche, Cathrine Shalby, Mohannad Alhanahnah +1Large Language Model SafetyUnified Benchmark

  21. What Eviction Destroys: A Restore-Counterfactual Audit of Forgetting in Agent Memory

    Sep 8, 2026Chen ShenEvictionAgent Memory

  22. Do AI Coding Assistants Check Before They Install? A Pre-Registered Demand-Side Audit of Trust Signals in the Research Software Supply Chain

    Sep 7, 2026Pengyin ShanCode QualityCoding Agents

  23. Audit Without Verification: When LLM Accountability Layers Relay Rather Than Check

    Sep 7, 2026Paul-Peter ArslanAlgorithm AuditingStep-Level Failure Attribution

  24. A Removal Based Approach to Improve LLM Faithfulness at Test-Time

    Sep 7, 2026Qinglan Luo, S M A Nahian, John Guttag +2Large Language Model ReliabilityTest Time

  25. Does the Selected Object Reach the Reader? Auditing Identity Handoffs in Grounded Language-Model Pipelines

    Sep 4, 2026Siddharth Vohra, Runmin Jiang, Xiaomo Li +1ReadingAlgorithm Auditing

  26. The Dice Roll Method: A Standardized Protocol for Repeated-Query Auditing of Large Language Model Brand Recommendations

    Sep 3, 2026Dmitrij ŻatuchinAlgorithm AuditingEffect Sizes

  27. Common-Witness Certificates and Sharp Feature Bounds for Counterfactual Image Auditing

    Sep 3, 2026Usef Faghihi, Amir SakiVersion WitnessAlgorithm Auditing

  28. Rent-a-RAG: Embedding-Space Watermarks for Auditing Third-Party RAG

    Sep 3, 2026Alexandr Goultiaev Tolstokorov, Kyriakos Mouratidis, Javad Dogani +1Hievi-RagAlgorithm Auditing

  29. Agent Flight Recorder: Tamper-Evident Audit Trails with On-Chain Anchoring for Long-Horizon Tool-Using Agents

    Sep 1, 2026Laurent Bindschaedler, Quentin Botha, Christoph SiebenbrunnerAlgorithm AuditingRollback

  30. Measuring consistency via ensemble margin and local prediction variability: Auditing decision systems in the presence of predictive multiplicity

    Sep 1, 2026Sinjini Banerjee, Tim Marrinan, Anand D. SarwateEnsembleCalibrated Predictors

  31. Where the Verifier Fails: A Category-Level Audit of Reward Signals in RLVR

    Sep 1, 2026Esther XinVerifiable RewardsVerification

  32. Drift-Aware LLM Routing with Sparse Contexts and Shared Budgets

    Sep 1, 2026Cheung Hao Lee, Patrick WongLarge Language Model RoutingLarge Language Model Serving

  33. Self-Reports Are Not Verification: Environment-Grounded Auditing of LLM Operators in Evolutionary Search

    Sep 1, 2026Enrong Pan, Ryan Zhou, Ting HuModel Self-AssessmentAlgorithm Auditing

  34. Auditing Anonymous AI Models: A Four-Stage Protocol for Black-Box Identity Verification

    Aug 31, 2026Yisen XiAlgorithm AuditingAnonymization

  35. BLOOM-WILT: Logit Tilting for Behaviour Elicitation in Automated LLM Auditing

    Aug 31, 2026Adrians Skapars, Edoardo ManinoLarge Language Model SafetyAlgorithm Auditing

  36. LLM Judges Verify Presence, Not Absence: Omission Blindness in AI Clinical Notes and What Recovers It

    Aug 31, 2026Sebastian Fox, Luke Markham, Ryan Lail +1Algorithm AuditingLarge Language Model Judges

  37. Selection-Aware Stress Testing for Interactive Agents

    Aug 31, 2026Yang Xu, Chenang Li, Jiefu Zhang +3Agentic EvaluationsAgent Decisions

  38. VFR-Audit: Verdict-Level Reliability for Fairness Audits in Hospital Length-of-Stay Prediction

    Aug 31, 2026Md Jannatul Rakib Joy, Viet Vo, Caslon ChuaFairness AuditsAlgorithm Auditing

  39. Designing an Auditable LLM-Supported Workflow for Qualitative Thematic Analysis

    Aug 31, 2026Nadia Jul Jeldtoft, Tariq YousefAlgorithm AuditingInterpretive Openness

  40. On the Recoverability of Private Information Unlearning in Large Language Models

    Aug 30, 2026Shicheng Hu, Runzhi Tian, Ziqiao Wang +1Large Language Model UnlearningMachine Unlearning

  41. Influence Is Not Authority: When Causal Guardrail Signals Make Legitimate Tool Use Look Like an Attack in Tool-Using LLM Agents

    Aug 30, 2026Tanzim Ahad, Ismail Hossain, Md Jahangir Alam +3Streaming GuardrailsTool-Schema Quality Predicts Tool Use

  42. Source-Dependent Deference in Medical Imaging Agents Under Falsified Findings: A Pilot Audit

    Aug 30, 2026Ridam Roy, Md Shahriar Rashid, Md. Rajib MiaFalsificationFalsehood

  43. Not to Break, but to Attest: Adversarial Probes for Privacy-Preserving LLM Verification

    Aug 28, 2026Cameron Wilding, Mina Shaker, Fatemeh GanjiCanonical Action Verification And AttestationAlgorithm Auditing

  44. ICON Decomposition: Auditing deep neural networks for shortcuts by decomposing layer-wise representations using concepts

    Aug 26, 2026Roshan Prakash Rane, Marco Simnacher, Manuel Pfeuffer +7Concept-Based ExplainabilityDeep Neural Network

  45. Do Vision-Language Models Agree on the Affective Qualities of Shape? A Cross-Model Audit for Generative Design Interfaces

    Aug 26, 2026Luca Bux, Thiago Rios, Ingo Scholtes +2Visual GenerationComputer-Aided Design

  46. CatchBench: When Can an Agent Failure Be Caught?

    Aug 24, 2026Yue ZhaoAlgorithm AuditingSecurity Analysis

  47. Verifiable abstention makes AI leak diagnosis accountable in urban water distribution networks

    Aug 19, 2026Tianwei Mu, Yue Wang, Mingzhe Yuan +8LeakageAlgorithm Auditing

  48. Beyond Local Accuracy: A Protocol-Level Identifiability Audit for Controlled LLM Reasoning Evaluation

    Aug 13, 2026Junhao Luo, Ning Huang, Ziqi Sha +2Algorithm AuditingValidity

  49. Labels Are Not Endpoints: Treatment Leakage and Construct Validity in MCP Agent Security Evaluation

    Aug 13, 2026Rana Muhammad Ahmed, Sabahat AbbasSecurity AnalysisAlgorithm Auditing

  50. Excess Separability: Nuisance-Controlled Residual-Stream Probing for Benchmark Contamination Detection

    Aug 12, 2026Florian BraunSeparabilityAlgorithm Auditing

  51. When Explanations Betray Backdoors: Black-Box Auditing for Language Model Classifiers

    Aug 12, 2026Yang Liu, Ran ZouText ClassifierAlgorithm Auditing

  52. GRPO for Financial Advice Generation: Outperforming Commercial LLMs under CATE Evaluation

    Aug 12, 2026Ofir Ben Shoham, Shrutendra Harsola, Vignesh Subrahmaniam +3Finance DomainAlgorithm Auditing

  53. Who Would You Vote For? Auditing Political Alignment in LLMs: An Italian Case-Study

    Aug 12, 2026Simone MungariPolitical DiscoursePreference Pairs

  54. Unmasking Toxic Mimicry in Medical Offline Reinforcement Learning for ICU Sepsis Management via Counterfactual Clinical Audits

    Aug 11, 2026Hangqi Ren, Junyi LiaoMimic-Iv DatasetsSepsis

  55. Measure, Don't Optimize: Forecasting Recovery in LLM Unlearning

    Aug 11, 2026Zirui Song, Huaxing Liu, Xiang Wang +8Large Language Model UnlearningUnlearning Aims

  56. Governing Agentic AI in FinTech

    Aug 11, 2026Henry HanArtificial Intelligence GovernanceBanking

  57. Auditing Chinese Web-scale Corpora via Sampled BPE Token Statistics

    Aug 11, 2026Qingjie Zhang, Ziqi Tang, Jie Zhang +7CorpusByte-Pair Encoding

  58. Who Gets Heeded? An Obligation-Level Audit of Responsiveness in EPA Rulemaking

    Aug 11, 2026Jianing Fan, Yue YaoRegulationAlgorithm Auditing

  59. Similarity Gates Approve Reversals: A Validity Audit of Embedding-Cosine Thresholds in Agent Systems

    Aug 10, 2026Scott E. FriasGatingAlgorithm Auditing

  60. When Repository Labels Are Not Image-Level Truth: A Supervision Auditing Framework for Chest Radiograph AI

    Aug 10, 2026Yesika Alexandra Agudelo-Londoño, Jhon Wilmer Pino-Román, Brahian Carrera Rodríguez +9Chest X-RayMedical Imaging Datasets