AI Agent Security Benchmarks

Follow this topic Ask about this topic

Momentum

21 papers in the last four weeks, up 91% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 200

All topics

VisualLeakBench: Reproducible Action-Boundary Propagation Failures in Vision-Language Agents
May 29, 2026Youting Wang, Yuan Tang, Yitian Qian +1Data Leakage Tool Use in VLMs
Northeastern University · Carnegie Mellon University · Boston University +1
PrivacyPeek: Auditing What LLM-Based Agents Acquire, Not Just What They Say
May 29, 2026Mingxuan Zhang, Jiahui Han, Dadi Guo +5Data Leakage Privacy Auditing
Shanghai Artificial Intelligence Laboratory · Southeast University
Send a SCOUT First: Pre-hoc Reasoning for Adaptive Detector Allocation in Prompt-Injection Defense
May 29, 2026Shuhao Zhang, Jiarui Li, Qi Cao +2AI Agent Security Benchmarks LLM Security
UC San Diego · University of Illinois Urbana-Champaign
MosaicLeaks:Privacy Risks in Querying-in-the-Open for Deep Research Agents
May 29, 2026Alexander Gurung, Spandana Gella, Alexandre Drouin +3Data Leakage Deep Research Agents
University of Edinburgh · ServiceNow AI Research · McGill University +2
The Surface You Test Is Not the Surface That Breaks
May 28, 2026Syed Nazmus Sakib, Nafiul Haque, Shahrear Bin Amin +1LLM Agent Security AI Agent Security Benchmarks
Department of Robotics and Mechatronics Engineering, University of Dhaka · Department of Computer Science and Engineering, University of Dhaka
Relevance as a Vulnerability: How Web Retrieval Degrades Safety Alignment in LLM Agents
May 28, 2026Aditya Nawal, Manit Baser, Mohan GurusamyAI Agent Security Benchmarks LLM Safety Evaluation
Department of Electrical and Computer Engineering · National University of Singapore
The Best-Laid SCHEMEs: Coordinated Sabotage and Monitoring in Multi-Agent Systems
May 27, 2026Nikolay Radev, Lennart Haas, Benjamin Arnav +1AI Agent Security Benchmarks AI Agent Monitoring
Plant, Persist, Trigger: Sleeper Attack on Large Language Model Agents
May 27, 2026Yongxiang Li, Moxin Li, Zhixin Ma +2LLM Agent Security AI Agent Security Benchmarks
University of Science and Technology of China · National University of Singapore · Singapore Management University
SNARE: Adaptive Scenario Synthesis for Eliciting Overeager Behavior in Coding Agents
May 27, 2026Yubin Qu, Yi Liu, Gelei Deng +4Coding Agents AI Agent Security
1Griffith University · 2Quantstamp · 3Nanyang Technological University +2
MIRAGE: Context-Aware Prompt Injection against Mobile GUI Agents via User-Generated Content
May 27, 2026Ruoqi Guo, Yi Liu, Gelei Deng +7Mobile GUI Automation GUI Agents
1Griffith University · 2Quantstamp · 3Nanyang Technological University +4
SEC-bench Pro: Can Language Models Solve Long-Horizon Software Security Tasks?
May 26, 2026Hwiwon Lee, Jiawei Liu, Dongjun Kim +3Software Vulnerability Detection Software Security
University of Illinois Urbana-Champaign
When the Manual Lies: A Realistic Benchmark to Evaluate MCP Poisoning Attacks for LLM Agents
May 22, 2026Shi Liu, Xuehai Tang, Xikang Yang +4MCP Security LLM Agent Security
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China
Boiling the Frog: A Multi-Turn Benchmark for Agentic Safety
May 21, 2026Piercosma Bisconti, Matteo Prandi, Federico Pierucci +11LLM Safety Benchmarks Computer-Use Agent Benchmarks
Icaro Foundation · Sapienza University of Rome · Sant’Anna School of Advanced Studies +6
Measuring Security Without Fooling Ourselves: Why Benchmarking Agents Is Hard
May 21, 2026Sahar Abdelnabi, Chris Hicks, Konrad Rieck +1AI Agent Evaluation AI Agent Security
ELLIS Institute Tübingen & MPI-IS & Tübingen AI Center, Germany · The Alan Turing Institute, London, UK · BIFOLD & Technische Universität Berlin, Germany +1
Benchmarking Autonomous Agents against Temporal, Spatial, and Semantic Evasions
May 21, 2026Jianan Ma, Xiaohu Du, Ruixiao Lin +8LLM Agent Evaluation LLM Agent Security
Hangzhou Dianzi University · Ant Group · Zhejiang University +1
Measuring Safety Alignment Effects in Autonomous Security Agents
May 19, 2026Isaac David, Arthur GervaisLanguage Model Safety Evaluation LLM Safety Alignment
University College London
RoboJailBench: Benchmarking Adversarial Attacks and Defenses in Embodied Robotic Agents
May 19, 2026Doguhuan Yeke, Yanming Zhou, Leo Y. Lin +3Jailbreak Attacks AI Agent Security Benchmarks
Purdue University
POLAR-Bench: A Diagnostic Benchmark for Privacy-Utility Trade-offs in LLM Agents
May 18, 2026Qiaoyuan Zheng, Yiqu Yang, Qi Gao +1LLM Agent Evaluation LLM Agent Security
ETH Zurich Zurich, Switzerland · ETH AI Center Zurich, Switzerland
Overeager Coding Agents: Measuring Out-of-Scope Actions on Benign Tasks
May 18, 2026Yubin Qu, Ying Zhang, Yanjun Zhang +4Computer-Use Agent Benchmarks Coding Agents
Griffith University · Wake Forest University · Nanyang Technological University +2
LivePI: More Realistic Benchmarking of Agents Against Indirect Prompt Injection
May 18, 2026Lei Zhao, Abhay Bhaskar, Edgar DobribanAI Agent Security AI Agent Safety
University of Pennsylvania
Trust No Tool: Evaluating and Defending LLM Agents under Untrusted Tool Feedback
May 17, 2026Lecheng Yan, Ruizhe Li, Xicheng Han +5AI Agent Reliability LLM Agent Evaluation
Southern University of Science and Technology · University of Science and Technology of China · Zhejiang University +2
ADR: An Agentic Detection System for Enterprise Agentic AI Security
May 17, 2026Chenning Li, Pan Hu, Justin Xu +9MCP Security AI Agent Security
Uber Technologies, Inc., USA · Massachusetts Institute of Technology, USA · University of Oxford, United Kingdom
ASPI: Seeking Ambiguity Clarification Amplifies Prompt Injection Vulnerability in LLM Agents
May 17, 2026Udari Madhushani Sehwag, Zhengyang Shan, Heming Liu +3LLM Agent Security AI Agent Security Benchmarks
Scale AI · 2Boston University · 4Human Frontier Collective +3
SLEIGHT-Bench: A Benchmark of Evasion Attacks Against Agent Monitors
May 15, 2026Elle Najt, Colin Toft, Tyler Tracy +2Software Engineering Agents Adversarial Attacks
1Anthropic Fellows Program · University of Waterloo · 3Redwood Research +1
Talk is (Not) Cheap: A Taxonomy and Benchmark Coverage Audit for LLM Attacks
May 14, 2026Karthik Raghu Iyer, Yazdan Jamshidi, Nicholas Bray +1LLM Safety Benchmarks Benchmark Auditing
Palo Alto Networks
Do Coding Agents Understand Least-Privilege Authorization?
May 14, 2026Zheng Yan, Jingxiang Weng, Charles Chen +9Software Engineering Agents AI Agent Security
Evolvent AI Research Team
Auditing Agent Harness Safety
May 14, 2026Chengzhi Liu, Yichen Guo, Yepeng Liu +8AI Agent Auditing LLM Auditing
University of California, Santa Barbara · University of California, Berkeley · Microsoft Research +2
ExploitBench: A Capability Ladder Benchmark for LLM Cybersecurity Agents
May 13, 2026Seunghyun Lee, David BrumleyAI Agent Security Benchmarks LLMs for Cybersecurity
Carnegie Mellon University · Pittsburgh, PA 15213 · Carnegie Mellon University and Bugcrowd +1
AgentTrap: Measuring Runtime Trust Failures in Third-Party Agent Skills
May 13, 2026Haomin Zhuang, Hanwen Xing, Yujun Zhou +5AI Agent Security LLM Agent Security
University of Notre Dame · University of Southern California · LMU Munich +1
The Misattribution Gap: When Memory Poisoning Looks Like Model Failure in Agentic AI Systems
May 12, 2026Tanzim Ahad, Ismail Hossain, Md Jahangir Alam +3AI Agent Security Benchmarks Agent Memory Poisoning
Department of Computer Science, University of Texas at El Paso, TX, USA 79902 · School of Computing, Southern Illinois University Carbondale, IL, USA 62901 · University of Illinois Urbana-Champaign, IL, USA

May 29, 2026, cs.CVJ/K move · Enter open · S save

VisualLeakBench: Reproducible Action-Boundary Propagation Failures in Vision-Language Agents

Youting Wang, Yuan Tang, Yitian Qian, Chen Zhao

Northeastern University · Carnegie Mellon University · Boston University+1

Data Leakage Tool Use in VLMs AI Agent Security Privacy Leakage in Language Models AI Agent Security Benchmarks Tool-Using Vision-Language Agents

Open paper AskPDF arXiv

Vision-language agents increasingly consume screenshots, documents, and user interfaces before writing to memory, sending messages, or invoking external tools. We study a concrete failure mode in this setting: action-boundary propagation, where sensitive or unsafe visible text is copied from an image into downstream tool arguments. We present VisualLeakBench, a diversified 500-image benchmark spanning UI, chat, document, form, and dashboard scenes, and evaluate a stratified 100-image agent subset with four production VLM systems under two workflows: note capture and external handoff. At baseline, target strings are propagated into tool arguments in 78.8% of PII cases and 85.5% of rendered unsafe-text cases. Under a defensive system prompt, rendered unsafe-text propagation remains high at 52.6%, while PII tool propagation falls to 2.0%, largely by suppressing tool use rather than preserving utility. Rates are tool-surface dependent: search-like tools suppress PII propagation, but rendered unsafe text still crosses tool boundaries. We measure visual-to-tool propagation rather than downstream instruction execution. We additionally provide a labeled-target oracle upper-bound diagnostic that localizes most failures at the tool boundary while leaving response-side leakage as residual risk.

Momentum

Related topics

Latest papers 200