AI Agent Security

Ask about this topic

Overview

AI agent security studies how tool use, memory, delegation, and interaction with people or environments create risks beyond those of a standalone language model, including prompt injection, privilege misuse, data leakage, and unsafe actions. Researchers test these risks with red-teaming and execution-grounded benchmarks for coding, browsing, GUI, embodied, and multi-agent tasks, measuring whether harmful effects occur across trajectories rather than judging text alone. Defenses range from input and memory provenance checks to sandboxing, least-privilege permissions, stateful runtime authorization, action verification, and cryptographic identity or audit mechanisms. An open challenge is making controls robust to adaptive, long-horizon attacks and interactions between tools or agents while preserving useful task completion; benchmark realism and comparable measures remain uneven.

Latest papers

CardsList

517 papers74 papers in the last four weeks, up 32% on the four weeks before. 0.4% of all new papers.

  1. InferD: A Honeypot Framework for AI Inference Infrastructure

    Oct 29, 2026Hicham Hammouchi, Gabriele LenziniAI Agent SecurityLLM Security

    SnT, University of Luxembourg Luxembourg, Luxembourg

  2. Soei: Structured Observability of Evidence and Intent for Multi-Agent AI Software Engineering

    Oct 29, 2026Mohammed Nayeem Khan, K S Sarun KumarAI Agent MonitoringAI Coding Agents

    Intel Corporation

  3. REALM: Identifying Minimal Task Assets for LLM Agents

    Oct 29, 2026Zihou Ren, Byungchul Tak, Long WangAI Agent SecurityTool Access Control for LLM Agents

    INSC, Tsinghua University · Kyungpook National University

  4. Appreciation Teaches Unauthorized Execution: Self-Evolution Degrades the Robustness of Personal Agents to Indirect Prompt Injection

    Oct 29, 2026Xinyan Wang, Xiaogeng Liu, Jun Shao +1AI Agent SecurityAI Agent Safety

    University of Wisconsin–Madison · Johns Hopkins University

  5. PCM: A Proof-Carrying Message Protocol for Verifiable Multi-Agent Communication

    Oct 29, 2026Bravish GhoshAgent Communication ProtocolsAI Agent Security

    Cygnux Labs

  6. Tracekit Box: Running AI Agents in Micro-VMs Where Every Way Out Is Decided and Signed

    Oct 28, 2026Bravish GhoshRuntime Enforcement for AI AgentsAI Agent Security

    Cygnux Labs

  7. Is it my preference? Role Grounding Attack against Agent's Memory via Social Interaction

    Oct 28, 2026Haoran Bu, Yiting Shen, Litian Zhang +2AI Agent SecurityLLM Agent Safety

    Beijing University of Posts and Telecommunications, Beijing, China

  8. Constraining the Commit Boundary: A DIO/ODI-Informed Authorization Guardrail for Agentic Entity Resolution

    Oct 27, 2026Daniel BeránekEntity ResolutionAI Agent Governance

    Independent Researcher, Brno, Czech Republic

  9. Blindfold: Keeping the Audit Coin Secret from LLM Agents

    Oct 27, 2026Wei Zhu, Meiyun Zuo, Huiping SunAI Agent AuditingAI Agent Security

    School of Information, Renmin University of China, Beijing, 100872, China · Big Data and Responsible Artificial Intelligence for National Governance, Renmin University of China, Beijing, 100872, China · School of software and microelectronics, Peking University, Beijing, 100871, China

  10. The Agent in the Middle: How AI-Mediated Access Reshapes Web Vulnerability Exploitation

    Oct 26, 2026Ruozhao Yang, Jiongchi Yu, Ming Hu +1AI Agent SecurityAI Agent Security Benchmarks

    Singapore Management University, Singapore · Nanyang Technological University, Singapore · East China Normal University, China

  11. SEEK: Skill Extraction with Evolving Knowledge against Frontier Science Agents

    Oct 25, 2026Cheng Gui, Xiang Zheng, Yutao Wu +5AI Agent SecurityAgent Skill Security Auditing

    City University of Hong Kong, Hong Kong · Hong Kong Institute of AI for Science, Hong Kong · Deakin University, Australia +2

  12. NameTrap: Steering AI Coding Agents Toward Security Vulnerabilities via Variable Names

    Oct 25, 2026Chieh-Yuan Chang, Guan-Wei Chen, Chia-Yi Hsu +3AI Agent SecurityData Poisoning Attacks

    National Yang Ming Chiao Tung University, Taiwan

  13. ProbeID: Verifying MCP Tool Identity for LLM Agents under Registry Drift

    Oct 24, 2026Guanxing Chen, Sining Zhang, Haohuai He +3LLM Agent VerificationAI Agent Security

    City University of Hong Kong (Dongguan), Dongguan, China · The Hong Kong Polytechnic University, Hong Kong SAR, China · Zhengzhou University, Zhengzhou, China +1

  14. Data Governance for Generative and Agentic AI : Technology, Law, Economics, and Security

    Oct 24, 2026Narayana Darapaneni, Padmaja K, Gnana Chaithra DAI GovernanceAI Agent Governance

  15. Agent-to-Agent Synthetic Social Engineering on the Web

    Oct 24, 2026Bryan C. BootsAI Agent SecurityWeb Agents

    University of Missouri-Kansas City, Kansas City, MO USA

  16. AgentTrace: Execution-Level Fingerprinting of Black-Box LLM Agents

    Oct 23, 2026Gejian Zhao, Hanzhou Wu, Xinpeng ZhangLanguage Model FingerprintingAI Agent Security

    Shanghai University

  17. Agentic AI Defense: Decode-Time Provenance Binding for MCP

    Oct 22, 2026Eric Yocam, Varghese Vaidyan, Paris Kalathas +3AI Agent SecurityPrompt Injection Attacks on AI Agents

    Beacom College of Computer and Cyber Sciences, Dakota State University, Madison, SD, USA. · Information Networking Institute, Carnegie Mellon University, Pittsburgh, PA, USA.

  18. AgentTriage: Replayable Security Diagnosis Across Heterogeneous Agent Runtimes

    Oct 22, 2026Jie Ying, Xutao Zhang, Ningyuan Chen +4AI Agent AuditingAI Agent Security

    College of Information and Intelligence Engineering, Zhejiang Wanli University, Ningbo, China · College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou, China

  19. Secure Agentic AI and Multimodal Intelligence in 6G Communication Systems

    Oct 21, 2026Satyam Gaba, Devanshu Motwani, Siva Sai +3AI Agent SecurityMultimodal Agents

    Qualcomm Wireless Research, San Diego, CA 92121, USA · Department of Computer Science and Engineering, BITS Pilani, Pilani Campus, Rajasthan 333031, India · Department of Electrical and Computer Engineering, National University of Singapore 117583, Singapore +1

  20. Agentic Architecture of Information Systems. The AAM/5C Model: A Reference Model for Distributed Autonomy

    Oct 21, 2026Laurent GoujonAI Agent GovernanceDistributed Control

    IBM France

  21. PerMIT: Securing Persistent Memory for LLM Agents via Machine-Checked Integrity Theorems and Scalable Community Trust

    Oct 20, 2026Tanqiu Jiang, Min Bai, Sadat Shahriar +2LLM Agent MemoryAI Agent Security

    Stony Brook University · AWS AI Labs

  22. MAS-Guard: Unified Guardrails from Individual Agents to Multi-Agent LLM Systems

    Oct 20, 2026Mohammad Kachuee, Phu Mon Htut, Sheng Li +5AI Agent SecurityLLM Agent Safety

    Oracle AI

  23. ClawMMU: Protection-Coherent Virtual Memory for Stateful LLM Agents

    Oct 20, 2026Jinhao Hu, Bardia Mohammadi, Mofasshara Binte Rafique +4AI Agent SecurityMemory Management for LLM Agents

    MPI-SWS · Unaffiliated · EPFL +2

  24. Securing Agentic Commerce Through Behavior-Aware Fraud Detection and Cybersecurity for Trusted AI-Agent Payment Authorization

    Oct 20, 2026Kayode L. OgunsusiAI Agent SecurityAgentic Commerce

    Credit and Fraud Risk, American Express, Phoenix, Arizona, USA

  25. AutoRedClaw: Automated Red-Teaming for Claw-Style Agents

    Oct 19, 2026Hao Li, Tongyan Hu, Weidi Luo +8AI Agent SecurityAutomated Red Teaming

    Washington University in St. Louis · Johns Hopkins University · National University of Singapore +3

  26. Honeypots for AI Agent Insider Threats

    Oct 19, 2026Jan Wehner, Jam KraprayoonAI Agent SecurityAI Agent Monitoring

  27. Fake News, Real Crashes: Information Injection Attacks on LLM Financial Trading Agents

    Oct 19, 2026Mengxiao Wang, Nitesh SaxenaAdversarial Attacks on LLMsAI Agent Security

    Texas A&M University

  28. Whose Credential Is It? How Owner Attribution in Agent MemoryShapes Cross-User Account Misuse

    Oct 19, 2026Hau-Sing So, Daixian Liu, Carl Hou +3AI Agent SecurityLLM Agent Memory

    University of Macau · Tsinghua University · Lanzhou University +2

  29. Peer Pressure: Execution-Verified Co-Evolution of Attacker and Defender Populations for Code Security

    Oct 17, 2026Khushbu Pahwa, Yubin Ge, Daniele Bonadiman +2AI Agent SecurityAdversarial Training

    AWS AI

  30. Is This Your Agent? Identity Watermarking for Agent Systems in Dynamic Environments

    Oct 17, 2026Yuwei Zhong, Yu Cheng, Jiuan Zhou +5AI Agent SecurityRobust Watermarking

    East China Normal University · Shanghai Innovation Institute