AI Agent Security Benchmarks

Follow this topic Ask about this topic

AI agent security benchmarks test whether agents remain safe as they use tools, retrieve information, manage memory, and act across multi-step workflows. Common approaches use sandboxed environments, realistic attack scenarios, adaptive red-teaming, and executable checks of tool calls or resulting system states; cyber ranges also assess vulnerability discovery, exploitation, and repair. The field is moving beyond single-turn prompt-injection tests to cover persistent memory, third-party skills, privacy, multi-agent interactions, and real-world interfaces. A central challenge is making results both realistic and comparable: static tests may overstate robustness, while attack success alone can miss harm severity, authorization failures, and the cost to legitimate task utility.

Latest papers

309 papers56 papers in the last four weeks, up 51% on the four weeks before. 0.3% of all new papers.

When the Manual Lies: A Realistic Benchmark to Evaluate MCP Poisoning Attacks for LLM Agents
May 22, 2026Shi Liu, Xuehai Tang, Xikang Yang +4MCP Security LLM Agent Security
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China
Boiling the Frog: A Multi-Turn Benchmark for Agentic Safety
May 21, 2026Piercosma Bisconti, Matteo Prandi, Federico Pierucci +11LLM Safety Benchmarks Computer-Use Agent Benchmarks
Icaro Foundation · Sapienza University of Rome · Sant’Anna School of Advanced Studies +6
Measuring Security Without Fooling Ourselves: Why Benchmarking Agents Is Hard
May 21, 2026Sahar Abdelnabi, Chris Hicks, Konrad Rieck +1AI Agent Evaluation AI Agent Security
ELLIS Institute Tübingen & MPI-IS & Tübingen AI Center, Germany · The Alan Turing Institute, London, UK · BIFOLD & Technische Universität Berlin, Germany +1
Benchmarking Autonomous Agents against Temporal, Spatial, and Semantic Evasions
May 21, 2026Jianan Ma, Xiaohu Du, Ruixiao Lin +8LLM Agent Evaluation LLM Agent Security
Hangzhou Dianzi University · Ant Group · Zhejiang University +1
Measuring Safety Alignment Effects in Autonomous Security Agents
May 19, 2026Isaac David, Arthur GervaisLanguage Model Safety Evaluation LLM Safety Alignment
University College London
RoboJailBench: Benchmarking Adversarial Attacks and Defenses in Embodied Robotic Agents
May 19, 2026Doguhuan Yeke, Yanming Zhou, Leo Y. Lin +3Jailbreak Attacks AI Agent Security Benchmarks
Purdue University
POLAR-Bench: A Diagnostic Benchmark for Privacy-Utility Trade-offs in LLM Agents
May 18, 2026Qiaoyuan Zheng, Yiqu Yang, Qi Gao +1LLM Agent Evaluation LLM Agent Security
ETH Zurich Zurich, Switzerland · ETH AI Center Zurich, Switzerland
Overeager Coding Agents: Measuring Out-of-Scope Actions on Benign Tasks
May 18, 2026Yubin Qu, Ying Zhang, Yanjun Zhang +4Tool Access Control for LLM Agents Computer-Use Agent Benchmarks
Griffith University · Wake Forest University · Nanyang Technological University +2
LivePI: More Realistic Benchmarking of Agents Against Indirect Prompt Injection
May 18, 2026Lei Zhao, Abhay Bhaskar, Edgar DobribanAI Agent Security AI Agent Safety
University of Pennsylvania
Trust No Tool: Evaluating and Defending LLM Agents under Untrusted Tool Feedback
May 17, 2026Lecheng Yan, Ruizhe Li, Xicheng Han +5AI Agent Reliability LLM Agent Evaluation
Southern University of Science and Technology · University of Science and Technology of China · Zhejiang University +2
ADR: An Agentic Detection System for Enterprise Agentic AI Security
May 17, 2026Chenning Li, Pan Hu, Justin Xu +9MCP Security AI Agent Security
Uber Technologies, Inc., USA · Massachusetts Institute of Technology, USA · University of Oxford, United Kingdom
ASPI: Seeking Ambiguity Clarification Amplifies Prompt Injection Vulnerability in LLM Agents
May 17, 2026Udari Madhushani Sehwag, Zhengyang Shan, Heming Liu +3LLM Agent Security AI Agent Security Benchmarks
Scale AI · 2Boston University · 4Human Frontier Collective +3
SLEIGHT-Bench: A Benchmark of Evasion Attacks Against Agent Monitors
May 15, 2026Elle Najt, Colin Toft, Tyler Tracy +2Software Engineering Agents Adversarial Attacks
1Anthropic Fellows Program · University of Waterloo · 3Redwood Research +1
Talk is (Not) Cheap: A Taxonomy and Benchmark Coverage Audit for LLM Attacks
May 14, 2026Karthik Raghu Iyer, Yazdan Jamshidi, Nicholas Bray +1LLM Safety Benchmarks Benchmark Auditing
Palo Alto Networks
Do Coding Agents Understand Least-Privilege Authorization?
May 14, 2026Zheng Yan, Jingxiang Weng, Charles Chen +9Software Engineering Agents AI Agent Security
Evolvent AI Research Team
Auditing Agent Harness Safety
May 14, 2026Chengzhi Liu, Yichen Guo, Yepeng Liu +8AI Agent Auditing LLM Auditing
University of California, Santa Barbara · University of California, Berkeley · Microsoft Research +2
ExploitBench: A Capability Ladder Benchmark for LLM Cybersecurity Agents
May 13, 2026Seunghyun Lee, David BrumleyAI Agent Security Benchmarks LLMs for Cybersecurity
Carnegie Mellon University · Pittsburgh, PA 15213 · Carnegie Mellon University and Bugcrowd +1
AgentTrap: Measuring Runtime Trust Failures in Third-Party Agent Skills
May 13, 2026Haomin Zhuang, Hanwen Xing, Yujun Zhou +5AI Agent Security LLM Agent Security
University of Notre Dame · University of Southern California · LMU Munich +1
The Misattribution Gap: When Memory Poisoning Looks Like Model Failure in Agentic AI Systems
May 12, 2026Tanzim Ahad, Ismail Hossain, Md Jahangir Alam +3AI Agent Security Benchmarks Agent Memory Poisoning
Department of Computer Science, University of Texas at El Paso, TX, USA 79902 · School of Computing, Southern Illinois University Carbondale, IL, USA 62901 · University of Illinois Urbana-Champaign, IL, USA
Do Androids Dream of Breaking the Game? Systematically Auditing AI Agent Benchmarks with BenchJack
May 12, 2026Hao Wang, Hanchen Li, Qiuyang Mang +3Computer-Use Agent Benchmarks Reward Hacking
UC Berkeley
SkillSafetyBench: Evaluating Agent Safety under Skill-Facing Attack Surfaces
May 12, 2026Chang Jin, An Wang, Zeming Wei +7LLM Agent Security AI Agent Security Benchmarks
1Shanghai AI Laboratory · 2Peking University · 3East China Normal University
IPI-proxy: An Intercepting Proxy for Red-Teaming Web-Browsing AI Agents Against Indirect Prompt Injection
May 12, 2026Chia-Pei, Chen, Kentaroh Toyoda +2AI Agent Security AI Agent Security Benchmarks
Janet · Vulcan Research, AIFT
Cochise: A Reference Harness for Autonomous Penetration Testing
May 12, 2026Andreas Happe, Jürgen CitoLLM Agent Orchestration LLM Agent Evaluation
TU Wien · Vienna, Austria
CTFusion: A CTF-based Benchmark for LLM Agent Evaluation
May 12, 2026Dongjun Lee, Ga-eun Bae, Insu YunBenchmark Contamination LLM Agent Evaluation
School of Electrical Engineering, KAIST, Daejeon, Republic of Korea
ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks?
May 11, 2026Zhun Wang, Nico Schiller, Hongwei Li +13AI Agent Security AI Agent Security Benchmarks
1UC Berkeley · 2Max Planck Institute for Security and Privacy · 3UC Santa Barbara +4
From Controlled to the Wild: Evaluation of Pentesting Agents for the Real-World
May 11, 2026Pedro Conde, Henrique Branquinho, Valerio Mazzone +3Software Vulnerability Detection AI Agent Evaluation
Ethiack Coimbra, Portugal · Ethiack Porto, Portugal · University of Notre Dame Notre Dame, Indiana, USA
LITMUS: Benchmarking Behavioral Jailbreaks of LLM Agents in Real OS Environments
May 11, 2026Chiyu Zhang, Huiqin Yang, Bendong Jiang +8AI Agent Security Benchmarks LLM Jailbreak Attacks
Nanjing University of Aeronautics and Astronautics · Collaborative lnnovation Center of Novel Software Technology and Industrialization · Zhejiang University
CrackMeBench: Binary Reverse Engineering for Agents
May 11, 2026Isaac David, Arthur GervaisSoftware Reverse Engineering LLM Agent Evaluation
University College London
Red-Teaming Agent Execution Contexts: Open-World Security Evaluation on OpenClaw
May 11, 2026Hongwei Yao, Yiming Liu, Yiling He +1AI Agent Security LLM Agent Security
Department of Computer Science, City University of Hong Kong, Hong Kong · School of Computer Science and Technology, Beijing Institute of Technology, Beijing, China · University College London, London, United Kingdom +1
MonitoringBench: Semi-Automated Red-Teaming for Agent Monitoring
May 10, 2026Monika Jotautaitė, Maria Angelica Martinez, Ollie Matthews +1Software Engineering Agents AI Agent Security Benchmarks
Independent · Redwood Research

May 22, 2026J/K move · Enter open · S save

When the Manual Lies: A Realistic Benchmark to Evaluate MCP Poisoning Attacks for LLM Agents

Shi Liu, Xuehai Tang, Xikang Yang, Liang Lin+3

Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China

MCP Security LLM Agent Security AI Agent Security Benchmarks

Open paper AskPDF arXiv

The rise of tool-using Large Language Model (LLM) agents, standardized by protocols like the Model Context Protocol (MCP), has unlocked unprecedented autonomous execution capabilities for LLM Agents by integrating external open-domain knowledge and tools. However, this interoperability introduces a covert attack surface targeting the agent's cognitive planning layer. This paper systematically investigates Tool Description Poisoning (TDP), a novel semantic attack. In TDP, malicious instructions are not embedded in a tool's executable code, but rather covertly injected into its descriptive metadata, the very "manual" an agent relies on for secure planning and decision-making. To rigorously and systematically evaluate this emerging threat, we introduce the MCP-TDP Security Benchmark. This high-fidelity sandbox environment comprises 32 realistic, real-world test cases spanning 6 distinct risk categories. Our evaluation of 8 mainstream LLMs reveals severe vulnerabilities, with leading models like GPT-4o exhibiting a nearly 100% Attack Success Rate (ASR) in six high-risk scenarios. Furthermore, our findings demonstrate that common prompt-guardrail defenses are largely ineffective and can, counterintuitively, even be counterproductive (a phenomenon which we term the "Firewall Fallacy"). Crucially, we also propose a defense mechanism: "Reactive Self-Correction," where an agent autonomously detects and reverts its own malicious actions post-execution. This work provides the first specialized security benchmark tailored for TDP, offering essential insights for securing the cognitive and planning layers of advanced agentic systems.