AI Agent Auditing

Follow this topic Ask about this topic

AI agent auditing studies whether autonomous systems follow their instructions and policies as they use tools, exchange information, and change external state. Researchers combine trace analysis, runtime monitoring and authorization, static and dynamic security testing, adversarial benchmarks, and tamper-evident logs to detect violations and reconstruct how they occurred. A central lesson is that correct final answers or benchmark scores do not establish safe or faithful execution; audits increasingly examine intermediate decisions, permissions, evidence provenance, and effects on the environment. The field still faces challenges in obtaining complete, independent records and reliable evaluators, while balancing stronger oversight with task utility, cost, and agent autonomy.

Latest papers

269 papers79 papers in the last four weeks, up 39% on the four weeks before. 0.5% of all new papers.

Do Androids Dream of Breaking the Game? Systematically Auditing AI Agent Benchmarks with BenchJack
May 12, 2026Hao Wang, Hanchen Li, Qiuyang Mang +3Computer-Use Agent Benchmarks Reward Hacking
UC Berkeley
Iterative Audit Convergence in LLM-Managed Multi-Agent Systems: A Case Study in Prompt-Engineering Quality Assurance
May 12, 2026Elias CalboreanuMulti-Agent LLM Systems AI Agent Auditing
Swift AI Lab, The Swift Group, LLC, Reston, VA 20190, USA
Property-Level Reconstructability of Agent Decisions: An Anchor-Level Pilot Across Vendor SDK Adapter Regimes
May 12, 2026Oleg SolozobovAI Agent Auditing Agent Evaluation
Independent Researcher (Global)
Behavioral Integrity Verification for AI Agent Skills
May 12, 2026Yuhao Wu, Tung-Ling Li, Hongliang LiuAgent Reliability AI Agent Auditing
Palo Alto Networks
AuditRepairBench: A Paired-Execution Trace Corpus for Evaluator-Channel Ranking Instability in Agent Repair
May 6, 2026Yuelin Hu, Zhenbo Yu, Zhengxue Cheng +2AI Agent Auditing Agent Evaluation
Shanghai Jiao Tong University · Shanghai Maritime University
Semia: Auditing Agent Skills via Constraint-Guided Representation Synthesis
May 1, 2026Hongbo Wen, Ying Li, Hanzhi Liu +4AI Agent Auditing Static Code Analysis
University of California, Santa Barbara Santa Barbara, CA, USA · University of California, Los Angeles Los Angeles, CA, USA · Fuzzland USA +1
TRUST: A Framework for Decentralized AI Service v.0.1
Apr 29, 2026Yu-Chao Huang, Zhen Tan, Mohan Zhang +3AI Agent Reliability AI Accountability
UNITES Lab, University of North Carolina at Chapel Hill · Arizona State University · Columbia University
Structured Security Auditing and Robustness Enhancement for Untrusted Agent Skills
Apr 28, 2026Lijia Lv, Xuehai Tang, Jie Wen +2AI Agent Auditing AI Agent Security
Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences
Agentic Witnessing: Pragmatic and Scalable TEE-Enabled Privacy-Preserving Auditing
Apr 27, 2026Antony RowstronPrivacy Auditing AI Agent Auditing
Advanced Research and Invention Agency (ARIA) London, UK
HINTBench: Horizon-agent Intrinsic Non-attack Trajectory Benchmark
Apr 15, 2026Jiacheng Wang, Jinchang Hou, Fabian Wang +3AI Agent Auditing Long-Horizon Agent Evaluation
Baidu Inc., Beijing, China · Beijing Institute of Technology, Beijing, China
Honeypot Protocol
Apr 14, 2026Najmul HasanAI Agent Monitoring AI Safety Evaluation
University of North Carolina at Pembroke · With Apart Research
Detecting Safety Violations Across Many Agent Traces
Apr 13, 2026Adam Stein, Davis Brown, Hamed Hassani +2AI Agent Auditing LLM Agent Safety
University of Pennsylvania
Governed Reasoning for Institutional AI
Apr 12, 2026Mamadou SeckAI Agent Governance AI-Assisted Decision Making
Independent Researcher
Toward Explanatory Equilibrium: Verifiable Reasoning as a Coordination Mechanism under Asymmetric Information
Apr 10, 2026Feliks Bańka, Jarosław A. ChudziakMulti-Agent LLM Systems Multi-Agent Coordination
The Faculty of Electronics and Information Technology, Warsaw University of Technology, Warsaw, Poland
Verify Before You Commit: Towards Faithful Reasoning in LLM Agents via Self-Auditing
Apr 9, 2026Wenhao Yuan, Chenchen Lin, Jian Chen +3Faithfulness of Language Model Reasoning LLM Agent Reliability
The University of Hong Kong · Sun Yat-sen University
LogAct: Enabling Agentic Reliability via Shared Logs
Apr 9, 2026Mahesh Balakrishnan, Ashwin Bharambe, Davide Testuggine +7LLM Agent Reliability AI Agent Monitoring
Meta
From Business Events to Auditable Decisions: Ontology-Governed Graph Simulation for Enterprise AI
Apr 8, 2026Hongyin Zhu, Jinming Liang, Mengjun Hou +5Ontology-Based Reasoning LLM Decision-Making
Yonyou AI Lab · Yonyou Network Technology Co., Ltd.
OpenKedge: Governing Agentic Mutation with Execution-Bound Safety and Evidence Chains
Apr 7, 2026Jun He, Deying YuAI Agent Governance Runtime Enforcement for AI Agents
Foundations for Investigations of Agentic AI Systems from the Forensic Analysis of OpenClaw
Apr 7, 2026Jan Gruber, Jan-Niclas HilgertAgentic AI Digital Forensics
KASTEL Security Research Labs, Karlsruhe Institute of Technology, Am Fasanengarten 5, 76131 Karlsruhe, Germany · Fraunhofer Institute for Communication, Information Processing and Ergonomics FKIE, Zanderstr. 5, 53177, Bonn, Germany
Auditable Agents
Apr 7, 2026Yi Nian, Aojie Yuan, Haiyue Zhang +7AI Accountability AI Agent Auditing
University of Southern California · Arizona State University · University of Tennessee, Knoxville +1
Undetectable Conversations Between AI Agents via Pseudorandom Noise-Resilient Key Exchange
Apr 6, 2026Vinod Vaikuntanathan, Or ZamirLanguage Model Steganography AI Agent Security
MIT · Tel Aviv University
Springdrift: An Auditable Persistent Runtime for LLM Agents with Case-Based Memory, Normative Safety, and Ambient Self-Perception
Apr 6, 2026Seamus BradyLLM Agents Long-Horizon LLM Agents
Explainable Model Routing for Agentic Workflows
Apr 4, 2026Mika Okamoto, Ansel Kaplan Erol, Mark RiedlLLM Agent Routing Agentic Workflows
Georgia Institute of Technology, USA
Coupled Control, Structured Memory, and Verifiable Action in Agentic AI (SCRAT -- Stochastic Control with Retrieval and Auditable Trajectories): A Comparative Perspective from Squirrel Locomotion and Scatter-Hoarding
Apr 3, 2026Maximiliano Armesto, Christophe KolbAI Agent Reliability Memory-Augmented Agents
Taller Technologies
UK AISI Alignment Evaluation Case-Study
Apr 1, 2026Alexandra Souly, Robert Kirk, Jacob Merizian +2AI Agent Safety AI Safety Evaluation
Near-Miss: Latent Policy Failure Detection in Agentic Workflows
Mar 31, 2026Ella Rabinovich, David Boaz, Naama Zwerdling +1LLM Agent Evaluation AI Agent Auditing
IBM Research
Emergence WebVoyager: Toward Consistent and Transparent Evaluation of (Web) Agents in The Wild
Mar 30, 2026Deepak Akkil, Mowafak Allaham, Amal Raj +2Agent Evaluation Web Agent Benchmarks
Emergence AI · Northwestern University
"What Did It Actually Do?": Understanding Risk Awareness and Traceability for Computer-Use Agents
Mar 30, 2026Zifan Peng, Mingchen LiComputer-Use Agent Safety Computer-Use Agents
The Hong Kong University of Science and Technology (Guangzhou), China · University of North Texas, The United States
From Inference Routing to Agent Orchestration: Declarative Policy Compilation with Cross-Layer Verification
Mar 28, 2026Huamin Chen, Xunzhuo Liu, Bowei He +1LLM Routing LLM Agent Orchestration
vLLM Semantic Router Project · MBZUAI · McGill University +1
Clawed and Dangerous: Can We Trust Open Agentic Systems?
Mar 27, 2026Shiping Chen, Qin Wang, Guangsheng Yu +2LLM Agent Security AI Agent Governance
CSIRO Data61 · University of Technology Sydney

May 12, 2026J/K move · Enter open · S save

Do Androids Dream of Breaking the Game? Systematically Auditing AI Agent Benchmarks with BenchJack

Hao Wang, Hanchen Li, Qiuyang Mang, Alvin Cheung+2

UC Berkeley

Computer-Use Agent Benchmarks Reward Hacking Benchmark Auditing AI Agent Auditing AI Agent Security Benchmarks AI Agent Benchmarks+3

Open paper AskPDF arXiv

Agent benchmarks have become the de facto measure of frontier AI competence, guiding model selection, investment, and deployment. However, reward hacking, where agents maximize a score without performing the intended task, emerges spontaneously in frontier models without overfitting. We argue that benchmarks must be secure by design. From past incidents of reward hacks, we derive a taxonomy of eight recurring flaw patterns and compile them into the Agent-Eval Checklist for benchmark designers. We condense the insights into BenchJack, an automated red-teaming system that drives coding agents to audit benchmarks and identify possible reward-hacking exploits in a clairvoyant manner. Moreover, we extend BenchJack to an iterative generative-adversarial pipeline that discovers new flaws and patches them iteratively to improve benchmark robustness. We apply BenchJack to 10 popular agent benchmarks spanning software engineering, web navigation, desktop computing, and terminal operations. BenchJack synthesizes reward-hacking exploits that achieve near-perfect scores on most of the benchmarks without solving a single task, surfacing 219 distinct flaws across the eight classes. Moreover, BenchJack's extended pipeline reduces the hackable-task ratio from near 100% to under 10% on four benchmarks without fatal design flaws, fully patching WebArena and OSWorld within three iterations. Our results show that evaluation pipelines have not internalized an adversarial mindset, and that proactive auditing could help close the security gap for the fast-paced benchmarking space.