AI Agent Auditing

Follow this topic Ask about this topic

AI agent auditing studies whether autonomous systems follow their instructions and policies as they use tools, exchange information, and change external state. Researchers combine trace analysis, runtime monitoring and authorization, static and dynamic security testing, adversarial benchmarks, and tamper-evident logs to detect violations and reconstruct how they occurred. A central lesson is that correct final answers or benchmark scores do not establish safe or faithful execution; audits increasingly examine intermediate decisions, permissions, evidence provenance, and effects on the environment. The field still faces challenges in obtaining complete, independent records and reliable evaluators, while balancing stronger oversight with task utility, cost, and agent autonomy.

Latest papers

269 papers79 papers in the last four weeks, up 39% on the four weeks before. 0.5% of all new papers.

SearchAuditor: Auditing and Attributing Failures in Long-Horizon Search Agents
Aug 5, 2026Zhixiang Liang, Yifei Liu, Yidan Huang +5Agent Failure Analysis AI Agent Reliability
University of Illinois Urbana-Champaign · 2Joy Future Academy, JD
SkillTrace: Multi-Trace Provenance Auditing for LLM-Agent Skill Reuse
Aug 5, 2026Jialuo Chen, Minghe Wang, Lingqi Jiang +7Data Provenance AI Agent Auditing
Ant Group · Zhejiang University · Hangzhou Dianzi University, China +2
Coding Agents as Test-Suite Auditors: Finding What Official Suites Miss While Approaching What They Catch
Aug 3, 2026Shuyang Xie, Shuxiao Xie, Feng Zhu +2Automated Software Testing Benchmark Auditing
Harbin Institute of Technology · Fudan University · Sun Yat-sen University
Auditing Discovery Claims: A Two-Sided Criterion for Agentic Science, with the Negative Side Decidable
Aug 2, 2026Wenhui Chen, Jianlin Chen, Ziyao Lin +1Reward Hacking AI for Science
University of Macau · 2South China University of Technology
One Human, $N$ Agents: Audit-Budget Allocation for LLM Agent Fleets under Miscalibrated, Correlated Confidence
Jul 30, 2026Cesare Zavattari, Alessandro Tommasi, Giuseppe PrencipeScalable Oversight AI Agent Reliability
Dipartimento di Informatica, Università di Pisa, Pisa, Italy
Auditing Emergent LLM-Agent Collaboration through Cooperation-Obligation Coupling
Jul 29, 2026Zuyuan Zhang, Hanqing Yang, Carlee Joe-Wong +1Multi-Agent LLM Systems AI Agent Auditing
Department of Electrical and Computer Engineering, The George Washington University · Department of Electrical and Computer Engineering, Carnegie Mellon University
Success Is Not Self-Explanatory: Auditing Success Provenance in Agent Evaluation
Jul 27, 2026Jingkun Luo, Da-Tian PengData Provenance Benchmark Auditing
Northwestern Polytechnical University
Do Agent Benchmarks Measure Capability? Protocol Validity in the Age of Agentic AI
Jul 24, 2026Jiaqi Shao, Hanck Chen, Wei Zhang +2Computer-Use Agent Benchmarks Reward Hacking
1Hunyuan Team, Tencent · 2The Hong Kong University of Science and Technology · 3Duke Kunshan University
Auditing Provenance Sensitivity in LLM Agent Action Selection
Jul 23, 2026Junchi LiaoAI Agent Auditing LLM Auditing
Evaluating Frontier AI Agents as Autonomous Clinical Security Auditors
Jul 15, 2026Michael O. EnioladeAI Agent Auditing AI Agent Evaluation
University of the Cumberlands
Automated Textbook Auditing with Multi-Agent LLM Systems
Jul 13, 2026Ciprian Cristescu, Adrian-Marius Dumitran, Angela-Liliana Dumitran +1AI Agent Auditing Generative AI in Education
University of Bucharest, Bucharest, Romania · Dimitrie Cantemir Christian University, Bucharest, Romania
ANCHOR: Automated Alignment Auditing for CLI Agents on Real-World Harm
Jul 11, 2026Kefan Song, Yanjun QiAI Agent Auditing LLM Auditing
University of Virginia, Charlottesville, VA, USA
Capability Gates Are Not Authorization: Confused-Deputy Failures in LLM Agent Frameworks
Jun 27, 2026David Mellafe ZuvicAI Agent Auditing LLM Auditing
Independent Security Research Chile
SkillAudit: From Fixed-Suite Benchmarking to Skill-Centered Assessment
Jun 21, 2026Dexu Yu, Youhua Li, Zhaoyang Guan +12AI Agent Auditing Automated Evaluation
Northeastern University · City University of Hong Kong · DeciLix Lab +8
AgentRiskBOM: A Risk-Scoping Security Bill of Materials for Agentic AI Systems
Jun 20, 2026Srimonti Dutta, Akshata Kishore MoharirAI Risk Management AI Agent Auditing
WAI USA Research Labs · WAI USA · Austin, USA +1
Local LLM Agents as Vulnerable Runtimes:A Source-Code Audit of the Agent Runtime Layer
Jun 19, 2026Zhengsong Zhang, Zongze Li, Jiawei Guo +1AI Agent Auditing Software Vulnerability Detection
Department of Computer Science and Engineering University at Buffalo, SUNY
PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections
Jun 10, 2026Pengfei He, Lesly Miculicich, Vishesh Sharma +5AI Agent Auditing LLM Auditing
Google Cloud AI Research · Google · Michigan State University
Runtime Skill Audit: Targeted Runtime Probing for Agent Skill Security
Jun 10, 2026Tu Lan, Chaowei XiaoAI Agent Auditing LLM Agent Security
Johns Hopkins University
The Arbiter Agent: Continually Monitoring Multi-Agent Conversations to Detect Emergent Misalignment
Jun 9, 2026Filippo Tonini, Federico Torrielli, Anton Danholt Lautrup +3AI Agent Auditing LLM Auditing
University of Southern Denmark Odense, Denmark · University of Turin Turin, Italy · University of Southern Denmark Vejle, Denmark
Observability for Delegated Execution in Agentic AI Systems
Jun 8, 2026Abhinav Mishra, Kumar SharadAI Agent Auditing AI Agent Monitoring
Splunk, Cisco Inc
Provably Auditable and Safe LLM Agents from Human-Authored Ontologies
Jun 3, 2026Aaron SterlingAI Agent Auditing LLM Auditing
Thistleseeds
PrivacyPeek: Auditing What LLM-Based Agents Acquire, Not Just What They Say
May 29, 2026Mingxuan Zhang, Jiahui Han, Dadi Guo +5Data Leakage Privacy Auditing
Shanghai Artificial Intelligence Laboratory · Southeast University
Gram: Assessing sabotage propensities via automated alignment auditing
May 28, 2026David Lindner, Victoria Krakovna, Sebastian FarquharAI Coding Agents AI Agent Auditing
1Google DeepMind
Beyond Final Answers: Auditing Trajectory-Level Hallucinations in Multi-Agent Industrial Workflows
May 22, 2026Harshada Badave, Santosh Borse, Andrea Gomez +6AI Agent Reliability AI Agent Auditing
1IBM · 2Columbia University
ProofAgent Harness: Open Infrastructure for Adversarial Evaluation of AI Agents
May 22, 2026Fouad BousetouaneAI Agent Reliability AI Agent Auditing
The University of Chicago, USA
MemAudit: Post-hoc Auditing of Poisoned Agent Memory via Causal Attribution and Structural Anomaly Detection
May 22, 2026Zhewen Tan, Yilun Yao, Huiyan Jin +9AI Agent Auditing LLM Auditing
Institute of Information Engineering, Chinese Academy of Sciences · Qiyuan Tech · Laboratory of Multimedia Information Processing, School of Computer Science, Peking University +1
The Log is the Agent: Event-Sourced Reactive Graphs for Auditable, Forkable Agentic Systems
May 21, 2026Yohei NakajimaAI Agent Auditing Agent Memory Management
Untapped Capital
AgentAtlas: Beyond Outcome Leaderboards for LLM Agents
May 19, 2026Parsa Mazaheri, Kasra MazaheriLLM Agent Reliability Computer-Use Agent Benchmarks
University of California, Santa Cruz · Massachusetts Institute of Technology
Pramana: A Protocol-Layer Treatment of Claim Verification in Autonomous Agent Networks
May 19, 2026Ravi Kiran KadaboinaAI Agent Reliability Claim Verification
Independent Researcher
Auditing Agent Harness Safety
May 14, 2026Chengzhi Liu, Yichen Guo, Yepeng Liu +8AI Agent Auditing LLM Auditing
University of California, Santa Barbara · University of California, Berkeley · Microsoft Research +2

Aug 5, 2026J/K move · Enter open · S save

SearchAuditor: Auditing and Attributing Failures in Long-Horizon Search Agents

Zhixiang Liang, Yifei Liu, Yidan Huang, Haozhe Zhao+4

University of Illinois Urbana-Champaign · 2Joy Future Academy, JD

Agent Failure Analysis AI Agent Reliability Deep Research Agents AI Agent Auditing LLM Auditing Long-Horizon Agent Evaluation+1

Open paper AskPDF arXiv

Deep search agents tackle challenging questions through long-horizon web interactions, a process that is both complex and fragile: small reasoning errors may propagate through long, noisy trajectories into fluent but incorrect answers. Diagnosing such failures is difficult, requiring the manual inspection of extremely long execution traces, which could be beyond human capacity. We therefore introduce SearchAuditBench, a benchmark that evaluates whether LLM auditors can localize, attribute, and repair these failures, thereby reducing the human burden. SearchAuditBench comprises 1,243 failed trajectories, averaging 73.1 messages and 65.1K tokens, collected from eight open-weight models on five deep-search benchmarks, each expert-annotated with the critical error step, a search-specific root cause, and a reference repair with grading rubrics. We further propose SearchAuditor, a multi-perspective auditing framework that effectively localizes, attributes, and repairs search-agent failures through evidence-grounded adjudication. Experimental results show that even the strongest baseline, when powered by a frontier model like GPT-5.5, attains only a 26.6% end-to-end pass rate. In contrast, our SearchAuditor consistently outperforms all baselines across different frontier models, achieving an end-to-end pass rate of 32.3%, and resuming failed runs with its repairs enables agents to better recover from errors.