AI Agent Security

Follow this topic Ask about this topic

Momentum

43 papers in the last four weeks, up 258% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 311

All topics

ToolHazard: Scaling Adversarial Environments for Security Evaluation and Alignment of LLM-based Agents
Aug 12, 2026Yutao Mou, Pengfei Yang, Zhe Yin +6AI Agent Evaluation Adversarial Scenario Generation
National Engineering Research Center for Software Engineering, Peking University · Harbin Institute of Technology · Beijing University of Posts and Telecommunications +1
ColluSkill: Adversarial Cross-Skill Composition for Evading Agent Skill Scanners
Aug 10, 2026Puyu Zeng, Simeng Qin, Jingzhi Li +3AI Agent Security LLM Agent Security
College of Cryptology and Cyber Science, Nankai University, China · Northeastern University, China · University of Science and Technology Beijing, China +2
ActBench: Self-Evolving Benchmark of Behavioral Safety in Cowork Agents
Aug 10, 2026Hongwei Yao, Yiming Liu, Meihui Chen +6AI Agent Security AI Agent Safety
City University of Hong Kong, Hong Kong · Zhejiang University, China · Xiaomi Corporation, China +1
Context Is Not Authority: Structured Runtime Governance for Financial Market Agents
Aug 10, 2026Rui Tang, Qiangqiang Liu, Yichi Zhang +3Financial Services AI Agent Security
OpenAsk · Binance · New York University New York, New York, USA +2
Not an A11y: How Android Accessibility Exposes Mobile AI Agents to Indirect Prompt Injection
Aug 9, 2026Rahul Deivasigamani, Sayeda Faatin Alvi, Derqui Andrea +2AI Agent Security LLM Agent Security
Radboud University · Nijmegen, The Netherlands
HarnessSafe: Evaluating Safety Across Persistent Carriers in Agent Harnesses
Aug 7, 2026Xiao Zhang, Yusheng Wang, Yuhao Fei +5AI Agent Security AI Agent Safety Benchmarks
Beijing University of Posts and Telecommunications · China Telecom Group Co., Ltd. · Beijing Academy of Artificial Intelligence
Hardware Keystores for AI Agent Signing Workflows: A Zero-Trust MCP Enforcement Architecture
Aug 6, 2026Leo Sambrook, Sampo SovioMCP Security AI Agent Security
Hardware Systems Security Lab (HSSL) Huawei Technologies & EPITA Helsinki, Finland
The Vulnerability With No CVE: Managing Persistent Gaps Between Mandate and Authority in AI Coding Agents
Aug 6, 2026Shayell Aharon Salomon Amir Shaked Matan NogaAI Coding Agents AI Agent Security
Bluebear Security
When Experience Becomes Instruction: Trajectory Poisoning in Self-Evolving Agent Skill Systems
Aug 6, 2026Jialuo Chen, Lingqi Jiang, Xinhao Deng +7Adversarial Attacks AI Agent Security
Ant Group · Zhejiang University · Tsinghua University +3
Combating Knowledge Corruption in Agent Systems: A Byzantine-Tolerant Secure Collaborative RAG Framework
Aug 5, 2026Zhaoqi Wang, Daqing He, Zijian Zhang +13RAG Poisoning Attacks AI Agent Security
Beijing Institute of Technology, Beijing, China
Internalising the Identity Primitive: Cryptographic Individuality for an Autonomous Agent on a Public Blockchain
Aug 4, 2026Keisuke SuzukiAI Agent Security
Center for Human Nature, Artificial Intelligence and Neuroscience (CHAIN), Hokkaido University
Magnet: Detecting Cross-Session AI Misuse Through Capability Accumulation
Aug 3, 2026Natalie Isak, Matthew DressmanAI Agent Security AI Agent Monitoring
Securing Agentic AI: From Per-Action Checks to Trajectory Assurance
Aug 3, 2026Alireza Lotfi, Subangkar Karmaker Shanto, Imtiaz Karim +1AI Agent Security LLM Agent Security
Purdue University · Department of Computer Science · West Lafayette, IN, USA +2
Adversarial Attacks in Multi-Agent LLM Pipelines: Unveiling Structural Vulnerabilities in Agentic AI Architectures
Aug 1, 2026Faisal Haque Bappy, Tahrim Hossain, Tarannum Shaila Zaman +3Adversarial Attacks on LLMs AI Agent Security
University of Maryland Baltimore County, MD, USA · Kent State University, OH, USA · Tennessee State University, TN, USA
OpenART: Scaling Agent Red Teaming via Open-Ended Environment Evolution
Aug 1, 2026Yunhao Chen, Xin Wang, Yixu Wang +6Adversarial Attacks Long-Horizon Agent Evaluation
Fudan University · Shanghai Artiﬁcial Intelligence Laboratory · Work done during Yunhao Chen’s internship at Shanghai AI Lab. +1
Skillsets on the Chain: A Blockchain-based Zero-Trust Framework for Agentic AI Networking
Jul 31, 2026Yayu Gao, Yong Xiao, Hao Hu +5Cybersecurity AI Agent Security
School of Electronic Information and Communications, the Huazhong University of Science and Technology, Wuhan, China 430074 · Peng Cheng Laboratory, Shenzhen, China · Pazhou Laboratory (Huangpu), Guangzhou, China +4
Security of World-Model-Based Embodied AI: A Lifecycle of Threats, Defenses, and Evaluation
Jul 30, 2026Fazhong Liu, Zhuoyan Chen, Haozhen Tan +3AI Agent Security World Model Evaluation
AgentSnare: Learning to Delay, Divert, and Defuse Autonomous Penetration Agents
Jul 29, 2026Ruoyu Wang, Heng Zhao, Renjie Wu +4Autonomous Cyber Defense Cybersecurity
The State Key Laboratory of Blockchain and Data Security, Zhejiang University · 3Hainan University · 4Anhui University +1
What Does It Take to Detect an AI Agent? Minimal Feature Sets for Behavioral Detection under Browser Automation
Jul 29, 2026Vishisht Choudhary, Lukas Schmidt, Anne Zoë Kenntner +3AI Agent Security Feature Selection
Technical University of Munich Munich, Germany · Kontext Munich, Germany
StealthBench: Measuring Operational Stealth in Autonomous Offensive-Security Agents
Jul 28, 2026Ads Dawson, Adrian WoodAI Agent Security LLM Agent Security
Security Researcher, USA
Toward Standardized Cross-Vendor Agent Tool Trust Management in Autonomous Networks
Jul 28, 2026Ravi Kant Sharma, Ashutosh Uttam, Ajay KumarAI Agent Security AI Agent Monitoring
Ericsson
Distributing Security Controls Through Harness Engineering
Jul 28, 2026William Robert GoreAI Agent Security LLM Agent Security
Georgia Institute of Technology
Cyber-Capable AI Agents: Vulnerabilities, Evaluation Containment, and Defensive Response
Jul 28, 2026Abu Bakar SiddikAI Agent Evaluation Cybersecurity
Department of Computer Science and Engineering, Rajshahi University of Engineering & Technology, Rajshahi 6204, Bangladesh
Where Is the Cost of Third-Party API Routers in Agentic Software Development?
Jul 26, 2026Donghao Fu, Jingxin Li, Xue Jiang +1Coding Agents AI Agent Security
School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · School of Software, Beihang University, Beijing, China · School of Computer Science, Peking University, Beijing, China
False Prophets: On the Security of World Models in Agentic Systems
Jul 25, 2026Erik Imgrund, Anna Wimbauer, Klim Kireev +1Adversarial Attacks AI Agent Security
BIFOLD & TU Berlin Berlin, Germany
Agent Security Needs Redefinition through a Holistic Framework
Jul 24, 2026Vincent Siu, Jingxuan He, Kyle Montgomery +3AI Agent Security AI Agent Security Benchmarks
1UC Santa Cruz · 2UC Berkeley
ToolGuardian: Declarative Security for AI Agent-Tool Interactions
Jul 23, 2026Arun Ravindran, Saurabh DeochakeAI Agent Security LLM Agent Security
Department of Electrical and Computer Engineering University of North Carolina at Charlotte Charlotte, NC · SentinelOne Mountain View, CA
Cryptographically verifiable authorization for autonomous AI agents: a falsifiable hypothesis and proof of concept
Jul 23, 2026M. Llambí-Morillas, D. Fernández-FernándezAI Agent Security AI Agent Governance
Universidad Tecnológica Atlántico-Mediterráneo (UTAMED) · Universidad de Santiago de Compostela (USC)
IssueTrojanBench: Benchmarking AI Coding Agents Against Malicious Issue Requests
Jul 22, 2026Ankur Singh, Jinqiu Yang, Tse-Hsun ChenAI Coding Agents AI Agent Security
Concordia University Montr´eal, Canada
The Ethics of Autonomous AI Agents for Offensive Security
Jul 22, 2026Andreas Happe, Jürgen Cito, Jasmin WachterAI Accountability Cybersecurity
TU Wien, Vienna, Austria · University of Klagenfurt, Klagenfurt, Austria

Aug 12, 2026, cs.CRJ/K move · Enter open · S save

ToolHazard: Scaling Adversarial Environments for Security Evaluation and Alignment of LLM-based Agents

Yutao Mou, Pengfei Yang, Zhe Yin, Zhangchi Xue+5

National Engineering Research Center for Software Engineering, Peking University · Harbin Institute of Technology · Beijing University of Posts and Telecommunications+1

AI Agent Evaluation Adversarial Scenario Generation LLM Safety Alignment AI Agent Security AI Agent Security Benchmarks Prompt Injection Attacks on AI Agents

Open paper AskPDF arXiv

Large language model (LLM) agents integrated with external tools are vulnerable to indirect prompt injections embedded in environmental states. However, existing studies largely rely on manually implemented or reused environments, stochastic LLM-based tool simulation, and predefined injection locations, limiting scalable security research across broader domains. To bridge this gap, we propose ToolHazard, a scalable adversarial environment synthesis framework that reduces human engineering and supports expansion with additional seed domains and compute. Through an Environment Simulator, an Attacker Agent, and a User Simulator, ToolHazard synthesizes executable stateful environments, discovers viable injection points and generates environment-specific payloads, and constructs state-grounded long-horizon tasks. Based on ToolHazard, we build ToolHazard-Bench for stress-testing agents under complex workflows and diverse environmental attacks. Experiments reveal substantial agent vulnerabilities and show that injection timing and placement affect attack effectiveness. Moreover, ToolHazard-generated alignment data improves security on both ToolHazard-Bench and AgentDojo while preserving benign task utility.

Momentum

Related topics

Latest papers 311