AI Agent Security

Follow this topic Ask about this topic

AI agent security studies how tool use, memory, delegation, and interaction with people or environments create risks beyond those of a standalone language model, including prompt injection, privilege misuse, data leakage, and unsafe actions. Researchers test these risks with red-teaming and execution-grounded benchmarks for coding, browsing, GUI, embodied, and multi-agent tasks, measuring whether harmful effects occur across trajectories rather than judging text alone. Defenses range from input and memory provenance checks to sandboxing, least-privilege permissions, stateful runtime authorization, action verification, and cryptographic identity or audit mechanisms. An open challenge is making controls robust to adaptive, long-horizon attacks and interactions between tools or agents while preserving useful task completion; benchmark realism and comparable measures remain uneven.

Latest papers

517 papers74 papers in the last four weeks, up 32% on the four weeks before. 0.4% of all new papers.

The Authorization-Execution Gap Is a Major Safety and Security Problem in Open-World Agents
May 10, 2026Baoyuan Wu, Qingshan Liu, Adel Bibi +2Agent Failure Analysis AI Agent Security
Chinese University of Hong Kong, Shenzhen, China · Nanjing University of Posts and Telecommunications, China · University of Oxford, UK +2
Agent Collectives Should Not Detect Their Own Imposters: A Chess Case Study
May 9, 2026Alexandre Le Mercier, Chris Develder, Thomas DemeesterAI Agent Security AI Agent Security Benchmarks
IDLab–T2K, Ghent University–imec
PAAC: Privacy-Aware Agentic Device-Cloud Collaboration
May 9, 2026Liangqi Yuan, Wenzhi Fang, Shiqiang Wang +1Privacy-Preserving Language Models AI Agent Security
Purdue University · University of Exeter
Injection-Execution Dissociation: A Mechanistic Evaluation of Persistent Memory Attacks and Defenses in Stateful LLM Agents
May 8, 2026Jun Wen LeongLanguage Model Safety Evaluation AI Agent Security
OrchJail: Jailbreaking Tool-Calling Text-to-Image Agents by Orchestration-Guided Fuzzing
May 8, 2026Jianming Chen, Yawen Wang, Junjie Wang +3Adversarial Prompt Generation T2I Generation
Institute of Software, Chinese Academy of Sciences, Beijing, China · Science & Technology on Integrated Information System Laboratory, Beijing, China · State Key Laboratory of Complex System Modeling and Simulation Technology, Beijing, China +1
Securing Computer-Use Agents: A Unified Architecture-Lifecycle Framework for Deployment-Grounded Reliability
May 8, 2026Zejian Chen, Zhanyuan Liu, Chaozhuo Li +6Agent Reliability Computer-Use Agents
Beijing University of Posts and Telecommunications, Beijing, China · China Unicom, Beijing, China · China Academy of Information and Communications Technology, Artificial Intelligence Institute, Beijing, China
TeamBench: Evaluating Agent Coordination under Enforced Role Separation
May 8, 2026Yubin Kim, Chanwoo Park, Taehan Kim +9Multi-Agent Coordination LLM Agent Evaluation
MIT · Google Research · Independent Researcher +1
Agentic AI and the Industrialization of Cyber Offense: Forecast, Consequences, and Defensive Priorities for Enterprises and the Mittelstand
May 6, 2026Christopher KochAI Risk Management Cybersecurity
Independent Researcher
Securing the Agent: Vendor-Neutral, Multitenant Enterprise Retrieval and Tool Use
May 6, 2026Francisco Javier Arceo, Varsha Prasad NarsingLLM Agent Orchestration AI Agent Security
Red Hat AI · Boston, USA
DecodingTrust-Agent Platform (DTap): A Controllable and Interactive Red-Teaming Platform for AI Agents
May 6, 2026Zhaorun Chen, Xun Liu, Haibo Tong +14AI Agent Evaluation AI Agent Security
Virtue AI · University of Chicago · University of Illinois, Urbana-Champaign +4
AgentTrust: Runtime Safety Evaluation and Interception for AI Agent Tool Use
May 6, 2026Chenglin YangAI Agent Security AI Agent Security Benchmarks
Independent Researcher
Redefining AI Red Teaming in the Agentic Era: From Weeks to Hours
May 5, 2026Raja Sekhar Rao Dheekonda, Will Pearce, Nick LandersAdversarial Attacks on LLMs LLM Red Teaming
1Dreadnode, USA
TRAP: Tail-aware Ranking Attack for World-Model Planning
May 3, 2026Siyuan Duan, Ke Zhang, Xizhao LuoBackdoor Attacks World Model-Based Planning
Department of Computer Science and Technology, Soochow University Suzhou, China
Architectural Obsolescence of Unhardened Agentic-AI Runtimes
May 3, 2026Alfredo MetereAI Agent Security AI Agent Security Benchmarks
Metere Consulting, LLC.
A Low-Latency Fraud Detection Layer for Detecting Adversarial Interaction Patterns in LLM-Powered Agents
May 1, 2026Sheldon Yu, Yingcheng Sun, Hanqing Guo +1AI Agent Security LLM Agent Security
University of California, San Diego · UNC at Greensboro · Indiana University Bloomington
Skills as Verifiable Artifacts: A Trust Schema and a Biconditional Correctness Criterion for Human-in-the-Loop Agent Runtimes
May 1, 2026Alfredo MetereAI Agent Security LLM Agent Verification
Enclawed, LLC, California, USA
AgentReputation: A Decentralized Agentic AI Reputation Framework
Apr 30, 2026Mohd Sameen Chishti, Damilare Peter Oyinloye, Jingyue LiAI Accountability AI Agent Security
Department of Computer Science NTNU Trondheim, Norway
Security Attack and Defense Strategies for Autonomous Agent Frameworks: A Layered Review with OpenClaw as a Case Study
Apr 30, 2026Luyao Xu, Xiang ChenAI Agent Security LLM Agent Security
School of Artificial Intelligence and Computer Science, Nantong University, Nantong, China · State Key Lab. for Novel Software Technology, Nanjing University, Nanjing, China
Ambient Persuasion in a Deployed AI Agent: Unauthorized Escalation Following Routine Non-Adversarial Content Exposure
Apr 29, 2026Diego F. Cuadros, Abdoul-Aziz MaigaAgent Failure Analysis AI Agent Security
Digital Epidemiology Laboratory, Digital Futures, University of Cincinnati, Cincinnati, OH, USA · Center for Humanities and Technology (CHaT), University of Cincinnati, Cincinnati, OH, USA · Norwegian University of Science and Technology, Trondheim, Norway
Agent Name Service (ANS): A Proof-of-Concept Trust Layer for Secure AI Agent Discovery, Identity, and Governance in Kubernetes
Apr 29, 2026Akshay Mittal, Elyson De La CruzAI Agent Security AI Agent Governance
Department of Information Technology University of the Cumberlands Williamsburg, KY, USA
From CRUD to Autonomous Agents: Formal Validation and Zero-Trust Security for Semantic Gateways in AI-Native Enterprise Systems
Apr 28, 2026Ignacio PeyranoCybersecurity AI Agent Security
Universidad Austral Rosario, Argentina
Structured Security Auditing and Robustness Enhancement for Untrusted Agent Skills
Apr 28, 2026Lijia Lv, Xuehai Tang, Jie Wen +2AI Agent Auditing AI Agent Security
Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences
AgentWard: A Lifecycle Security Architecture for Autonomous AI Agents
Apr 27, 2026Yixiang Zhang, Xinhao Deng, Jiaqing Wu +3AI Agent Security LLM Agent Security
A Comparative Evaluation of AI Agent Security Guardrails
Apr 27, 2026Qi Li, Jiu Li, Pingtao Wei +8LLM Guardrails AI Agent Security
Beijing Caizhi Tech, Beijing, China
Poster: ClawdGo: Endogenous Security Awareness Training for Autonomous AI Agents
Apr 27, 2026Jiaqi Li, Yang Zhao, Bin Sun +3AI Agent Security LLM Agent Training
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China · Network Management Center, China Mobile Group Liaoning Company Limited, Liaoning, China +2
Proof of Execution: Runtime Verification for Governed AI Agent Actions
Apr 26, 2026James Rhodes, George KangAI Agent Security Formal Verification
AlphaBitCore, Inc.
Structural Enforcement of Goal Integrity in AI Agents via Separation-of-Powers Architecture
Apr 26, 2026Rong XiangAI Agent Security AI Agent Safety
A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents: A Layered Attack Surface Framework
Apr 25, 2026Kexin ChuAI Agent Security LLM Agent Security
School of Computing, University of Connecticut
AI Identity: Standards, Gaps, and Research Directions for AI Agents
Apr 25, 2026Takumi Otsuka, Kentaroh Toyoda, Alex LeungAI Accountability AI Agent Security
1AIFT, Singapore · 2Graduate School of Fundamental Science and Engineering, Department of Computer Science and Communications Engineering, Waseda University, Japan · 3Keio Global Research Institute (KGRI), Japan
Semantic Denial of Service in LLM-controlled robots
Apr 25, 2026Jonathan Steinberg, Oren GalLanguage Model Safety Evaluation Adversarial Attacks on LLMs
Swarms & AI Lab (SAIL) · University of Haifa

May 10, 2026J/K move · Enter open · S save

The Authorization-Execution Gap Is a Major Safety and Security Problem in Open-World Agents

Baoyuan Wu, Qingshan Liu, Adel Bibi, Irwin King+1

Chinese University of Hong Kong, Shenzhen, China · Nanjing University of Posts and Telecommunications, China · University of Oxford, UK+2

Agent Failure Analysis AI Agent Security AI Agent Safety Access Control

Open paper AskPDF arXiv

This position paper argues that the Authorization-Execution Gap (AEG) is a major safety and security problem in open-world agents. The AEG is the divergence between what a principal intends to authorize and what an open-world agent ultimately executes. Because such agents act autonomously across tools, persistent state, and multi-agent handoffs, even small instances of authorization divergence can cause harm that is difficult or impossible to undo. We argue that many observed agent failures can be traced to three structural sources of AEG: delegation-level incompleteness, channel-level corruption, and composition-level fragmentation. The same observed failure may arise from any of these sources. Without identifying the source, a defense targeting the symptom alone cannot address the underlying cause. Agent safety and security should therefore emphasize source-oriented diagnosis and defense. Because the structural sources of AEG arise dynamically during execution, this approach necessarily requires authorization integrity checks applied during execution, rather than relying solely on one-shot upfront filtering or post-hoc audit. For NeurIPS, the implication is that papers on open-world agents should report not only outcome-level metrics such as task success or attack resistance, but also process-level evidence showing where AEG was detected, constrained, and attributed to a structural source during execution.