AI Agent Security Benchmarks

Overview

AI agent security benchmarks test whether agents remain safe as they use tools, retrieve information, manage memory, and act across multi-step workflows. Common approaches use sandboxed environments, realistic attack scenarios, adaptive red-teaming, and executable checks of tool calls or resulting system states; cyber ranges also assess vulnerability discovery, exploitation, and repair. The field is moving beyond single-turn prompt-injection tests to cover persistent memory, third-party skills, privacy, multi-agent interactions, and real-world interfaces. A central challenge is making results both realistic and comparable: static tests may overstate robustness, while attack success alone can miss harm severity, authorization failures, and the cost to legitimate task utility.

Latest papers

CardsList

336 papers43 papers in the last four weeks, down 27% on the four weeks before. 0.3% of all new papers.

  1. A Survey on Agentic Security: Applications, Threats and Defenses

    Oct 7, 2025Asif Shahriar, Md Nafiu Rahman, Sadif Ahmed +2AI Agent SecurityAutonomous Cyber Defense

  2. Breaking the Code: Security Assessment of AI Code Agents Through Systematic Jailbreaking Attacks

    Oct 1, 2025Shoumik Saha, Jifan Chen, Sam Mayers +3AI Coding AgentsAI Agent Security

  3. Measuring Harmfulness of Computer-Using Agents

    Jul 31, 2025Aaron Xuxiang Tian, Ruofan Zhang, Janet Tang +3LLM Agent SafetyComputer-Use Agent Benchmarks

  4. Meta-SecAlign: Training LLMs against Prompt Injection for Robust Agents

    Jul 3, 2025Sizhe Chen, Arman Zharmagambetov, David Wagner +1LLM Agent SecurityAI Agent Security Benchmarks

  5. LLMail-Inject: A Dataset from a Realistic Adaptive Prompt Injection Challenge

    Jun 11, 2025Sahar Abdelnabi, Aideen Fay, Ahmed Salem +22LLM SecurityIndirect Prompt Injection