AI Agent Security Benchmarks

Overview

AI agent security benchmarks test whether agents remain safe as they use tools, retrieve information, manage memory, and act across multi-step workflows. Common approaches use sandboxed environments, realistic attack scenarios, adaptive red-teaming, and executable checks of tool calls or resulting system states; cyber ranges also assess vulnerability discovery, exploitation, and repair. The field is moving beyond single-turn prompt-injection tests to cover persistent memory, third-party skills, privacy, multi-agent interactions, and real-world interfaces. A central challenge is making results both realistic and comparable: static tests may overstate robustness, while attack success alone can miss harm severity, authorization failures, and the cost to legitimate task utility.

Latest papers

CardsList

336 papers43 papers in the last four weeks, down 27% on the four weeks before. 0.3% of all new papers.

  1. Replication of Agentic Security Events: Audit and Investigation (ROGUE-AI)

    Nov 19, 2026Mengxiao Wang, Mahbub Alam, Kalyan Nakka +1AI Agent SecurityAI Agent Auditing

  2. Kaer: Security-Aware Replanning for Multi-Agent Systems

    Nov 18, 2026Tushin Mallick, Evan Rose, Matthew Laws +2Multi-Agent System SecurityDynamic Replanning

  3. HEAMS: Holistic Evaluation of Agent Memory Systems

    Nov 15, 2026Chirag Goel, Jacob Lavoie, Amirhossein Naghshzan +7Memory Agent BenchmarksAgent Memory Governance

  4. We Have All the Cards: Securing Agent Cards at the Operational Trust Boundary of the Agent2Agent Protocol

    Nov 13, 2026Miguel A. Guirao Aguilera, Satyajayant Misra, Efren Lopez-MoralesAI Agent SecurityAI Agent Interoperability

  5. Singularity-Bench: Can Agents Build Agents?

    Nov 12, 2026Yang Zhou, Krish Agarwal, Zhuoming Chen +16AI Agent BenchmarksLong-Horizon Agent Evaluation

  6. DEPUTYBENCH: Evaluating Contextual Safety in Enterprise Agents

    Nov 9, 2026Hao Zhao, Minghao LinAI Agent Security BenchmarksLLM Agent Safety

  7. ShadowBench-MCP: Measuring Shadow Invocations in MCP-Style Tool Routing

    Nov 9, 2026Eyhab Al-Masri, Wenjun YangLLM Agent SecurityLLM Routing

  8. Treating the Model as an Untrusted Component: Deterministic Policy Enforcement, Human Approval and Taint Tracking Against Prompt Injection in Tool-Using Agents, with Validation on Open-Weights Models

    Nov 8, 2026Mustafa S. Aljumaily, Nawar S. Alseelawi, Raghad AlshabandarPrompt Injection DefenseLLM Agent Safety

  9. CyberBinGym: Can AI Agents Turn Binary Patches into Real Exploits?

    Nov 7, 2026Lambang Akbar Wijayadi, Zhaoqi Xiao, Zhun Wang +5Automated Exploit GenerationAI Agent Security Benchmarks

  10. What Do Collusion Probes Read? Three Pitfalls in Evaluating Multi-Agent Collusion Monitors

    Nov 7, 2026Mainul Hasan Mohsin, Mohammad Shahidur RahmanAI Agent MonitoringMulti-Agent System Security

  11. Toward Safety-Aware AI Agents in Realistic Social Engineering Attack Scenarios

    Nov 5, 2026Zhitao He, Chengru Han, Chi Him Zhang +1AI Agent SecurityAI Agent Security Benchmarks

  12. Multimodal-SEA: Benchmarking AI Agent Security under Multimodal Social Engineering Attacks

    Nov 5, 2026Zhitao He, Chi Him Zhang, Chengru Han +1AI Agent SecurityAI Agent Security Benchmarks

  13. MetaPI: Constructing Prompt Injection Benchmarks from LLM Agent Benchmark

    Nov 5, 2026Peiran Wang, Chong Xiang, Wenjie Qu +4Prompt Injection Attacks on AI AgentsAI Agent Security Benchmarks

  14. ScamWorld: An Interactive Environment for Evaluating Web Agents Against Online Fraud

    Nov 3, 2026Yutao Wu, Xiao Liu, Ming Wen +3Web AgentsLLM Agent Evaluation

  15. How Much Can Tool Calls Reveal About You? Privacy Leakage from Tool-Call Traces

    Oct 30, 2026Donghang Duan, Qingyu Liu, Yifei Liu +3User Attribute Inference AttacksLLM Agent Security

  16. ANGLER: Verifier-Grounded Benchmarking with an Adaptive Attacker for Web Agents

    Oct 30, 2026Yini Huang, Aiwen LU, Zhen Sun +3AI Agent Security BenchmarksAutomated Red Teaming

  17. "Shall We Approve?" Consequence-Aware Approval for Autonomous Coding Agents

    Oct 29, 2026Weiheng Wu, Junkai Chen, Zhensu Sun +3AI Agent SafetyTool Access Control for LLM Agents

  18. Judging Before Propagation: Task-Graph Contextual Integrity for Multi-Agent Privacy

    Oct 29, 2026Hengyu Wu, Blaise Delattre, Yang CaoPrivacy Protection for LLM AgentsContextual Integrity

  19. Where You Inject Matters: The Trust Region Effect in LLM Agents

    Oct 28, 2026Chengquan Guo, Xiaodong Liu, Weiwei Yang +7Prompt InjectionPrompt Injection Attacks on AI Agents

  20. Scan Before You Install: Detecting and Mitigating Local and Composite Vulnerabilities in Skills

    Oct 27, 2026Parth A. Gandhi, Avital Aviv, Prasanna N. Wudali +2Agent Skill Security AuditingSoftware Vulnerability Detection

  21. Beyond QASecClaw: Agent Scaling, Verifier Feedback, and Evidence Reliability in Software Assurance

    Oct 27, 2026S M Abdulla Al Mamun, Akif Islam, Mohd. Ruhul AmeenLLM Agent VerificationLLM Reliability

  22. The Agent in the Middle: How AI-Mediated Access Reshapes Web Vulnerability Exploitation

    Oct 26, 2026Ruozhao Yang, Jiongchi Yu, Ming Hu +1AI Agent SecurityAI Agent Security Benchmarks