Overview

AI agent auditing studies whether autonomous systems follow their instructions and policies as they use tools, exchange information, and change external state. Researchers combine trace analysis, runtime monitoring and authorization, static and dynamic security testing, adversarial benchmarks, and tamper-evident logs to detect violations and reconstruct how they occurred. A central lesson is that correct final answers or benchmark scores do not establish safe or faithful execution; audits increasingly examine intermediate decisions, permissions, evidence provenance, and effects on the environment. The field still faces challenges in obtaining complete, independent records and reliable evaluators, while balancing stronger oversight with task utility, cost, and agent autonomy.

Latest papers

CardsList

312 papers54 papers in the last four weeks, down 44% on the four weeks before. 0.4% of all new papers.

  1. Foundations for Investigations of Agentic AI Systems from the Forensic Analysis of OpenClaw

    Apr 7, 2026Jan Gruber, Jan-Niclas HilgertAgentic AIDigital Forensics

  2. Auditable Agents

    Apr 7, 2026Yi Nian, Aojie Yuan, Haiyue Zhang +7AI AccountabilityAI Agent Auditing

  3. Explainable Model Routing for Agentic Workflows

    Apr 4, 2026Mika Okamoto, Ansel Kaplan Erol, Mark RiedlLLM Agent RoutingAgentic Workflows

  4. UK AISI Alignment Evaluation Case-Study

    Apr 1, 2026Alexandra Souly, Robert Kirk, Jacob Merizian +2AI Agent SafetyAI Safety Evaluation

  5. Near-Miss: Latent Policy Failure Detection in Agentic Workflows

    Mar 31, 2026Ella Rabinovich, David Boaz, Naama Zwerdling +1LLM Agent EvaluationAI Agent Auditing

  6. Emergence WebVoyager: Toward Consistent and Transparent Evaluation of (Web) Agents in The Wild

    Mar 30, 2026Deepak Akkil, Mowafak Allaham, Amal Raj +2Agent EvaluationWeb Agent Benchmarks

  7. "What Did It Actually Do?": Understanding Risk Awareness and Traceability for Computer-Use Agents

    Mar 30, 2026Zifan Peng, Mingchen LiComputer-Use Agent SafetyComputer-Use Agents

  8. From Inference Routing to Agent Orchestration: Declarative Policy Compilation with Cross-Layer Verification

    Mar 28, 2026Huamin Chen, Xunzhuo Liu, Bowei He +1LLM RoutingLLM Agent Orchestration

  9. Clawed and Dangerous: Can We Trust Open Agentic Systems?

    Mar 27, 2026Shiping Chen, Qin Wang, Guangsheng Yu +2LLM Agent SecurityAI Agent Governance

  10. The Stochastic Gap: A Markovian Framework for Pre-Deployment Reliability and Oversight-Cost Auditing in Agentic Artificial Intelligence

    Mar 25, 2026Biplab Pal, Santanu BhattacharyaAI Agent ReliabilityAI Agent Auditing

  11. MedOpenClaw and MedFlowBench: Auditing Medical Agents in Full-Study Workflows

    Mar 25, 2026Weixiang Shen, Chengzhi Shen, Yanzhu Hu +12Medical VLMsMedical Image Benchmarks

  12. Willful Disobedience: Automatically Detecting Failures in Agentic Traces

    Mar 25, 2026Reshabh K Sharma, Shraddha Barke, Benjamin ZornAI Agent EvaluationAI Agent Auditing

  13. Agent Audit: A Security Analysis System for LLM Agent Applications

    Mar 24, 2026Haiyue Zhang, Yi Nian, Yue ZhaoAI Agent AuditingLLM Agent Security

  14. When Only the Final Text Survives: Implicit Execution Tracing for Multi-Agent Auditing

    Mar 18, 2026Yi Nian, Haosen Cao, Shenzhe Zhu +4Data Provenance in Language ModelsAI Agent Auditing

  15. Ablation Study of a Fairness Auditing Agentic System for Bias Mitigation in Early-Onset Colorectal Cancer Detection

    Mar 17, 2026Amalia Ionescu, Jose Guadalupe Hernandez, Jui-Hsuan Chang +4Algorithmic FairnessAI Agent Auditing

  16. Compatibility at a Cost: Systematic Discovery and Exploitation of MCP Clause-Compliance Vulnerabilities

    Mar 10, 2026Nanzi Yang, Weiheng Bai, Kangjie LuMCP SecurityAI Agent Security

  17. Arbiter: Detecting Interference in LLM Agent System Prompts

    Mar 9, 2026Tony MasonAI Agent AuditingAI Coding Agents

  18. Give Them an Inch and They Will Take a Mile:Understanding and Measuring Caller Identity Confusion in MCP-Based AI Systems

    Mar 8, 2026Yuhang Huang, Boyang Ma, Biwei Yan +5MCP SecurityAI Agent Security