cs.CRMar 11, 2026
SaveThe Attack and Defense Landscape of Agentic AI: A Comprehensive Survey
Organizations: UC Berkeley / Seoul National University, USA · UC Berkeley, USA · University of Illinois Urbana-Champaign, USA · UC Santa Barbara, USA
Abstract
AI agents that combine large language models with non-AI system components are rapidly emerging in real-world applications, offering unprecedented automation and flexibility. However, this unprecedented flexibility introduces complex security challenges fundamentally different from those in traditional software systems. This paper presents the first systematic and comprehensive survey of AI agent security, including an analysis of the design space, attack landscape, and defense mechanisms for secure AI agent systems. We further conduct case studies to point out existing gaps in securing agentic AI systems and identify open challenges in this emerging domain. Our work also introduces the first systematic framework for understanding the security risks and defense strategies of AI agents, serving as a foundation for building both secure agentic systems and advancing research in this critical area.
Figures & tables
Figure 1. Overview of an AI Agent’s structure. A diagram illustrating the structural components of an AI agent, including the LLM model, memory, tool use, and planning modules, and how they interact with each other and with external environments.
| Dimension | Level 1 : Least flexible | Level 2 : Moderately flexible | Level 3 : Most flexible |
| Input Trust | No external data ( Achiam et al., 2023 ; Touvron et al., 2023 ; GeminiTeam, 2023 ) | Predefined external data ( Lewis et al., 2020 ) | Arbitrary external data ( Yang et al., 2023 ; Patil et al., 2024 ; modelcontextprotocol, 2025 ) |
| Access Sensitivity | No sensitive data ( Achiam et al., 2023 ; Touvron et al., 2023 ; GeminiTeam, 2023 ) | Predefined sensitive data ( OpenAI, 2025c ) | Arbitrary sensitive data ( Cursor, 2025 ; Perplexity, 2025 ) |
| Workflow | Simple chatbot ( Achiam et al., 2023 ; Touvron et al., 2023 ; GeminiTeam, 2023 ) | Fixed, Developer-defined ( Kit, 2025 ) | Dynamic, LLM-defined ( Wei et al., 2022 ; Yao et al., 2023 ; Yang et al., 2023 ) |
| Action | LLM response only ( Achiam et al., 2023 ; Touvron et al., 2023 ; GeminiTeam, 2023 ) | LLM response, Retrieval ( Lewis et al., 2020 ; OpenAI, 2025c ) | LLM response, Retrieval, Execution ( Cursor, 2025 ; Perplexity, 2025 ) |
| Memory | No memory ( Achiam et al., 2023 ; Touvron et al., 2023 ; GeminiTeam, 2023 ) | Transient session memory ( LangChain, 2025 ) | Persistent memory across sessions ( OpenAI, 2025d ; Lee et al., 2023 ) |
| Tool | No tool ( Achiam et al., 2023 ; Touvron et al., 2023 ; GeminiTeam, 2023 ) | Known tools ( Schick et al., 2023 ) | Arbitrary tools ( Patil et al., 2024 ) |
Table 1. Agent design dimensions and corresponding flexibility for each dimension. The agent design dimensions are conceptually orthogonal, but it practice, their functional implementation can introduce dependencies between them.
Figure 2. Demonstrations of attack vectors ( V1 - V6 ) and security risks ( R1 - R7 ) against AI agents. A diagram mapping six attack vectors (V1-V6) to seven security risks (R1-R7) in AI agent systems, organized by threat models including prompt injection, data injection, tool misuse, and memory poisoning.
Figure 3. Relationship between agent design dimensions and risks, and the connection across different risks. A matrix or graph showing how each agent design dimension (trust, sensitivity, workflow, action, memory, tool, interface) maps to and influences specific security risks R1 through R7, with connecting arrows indicating dependencies between risks.
Figure 4. The defense landscape of AI agents. We illustrate key defense mechanisms and where they can be applied within the agent system. A layered diagram of an AI agent system annotated with defense mechanisms at each layer, including input guardrails, output guardrails, access control, information flow control, monitoring, human-in-the-loop controls, privilege separation, and identity management.
| Category | Mechanism | Representative works | Covered risks |
| Runtime Protection | Input guardrail | Prompt Guard2 ( Chennabasappa et al., 2025 ) , PromptShield ( Jacob et al., 2024 ) , DataSentinel ( Liu et al., 2025 ) , PromptArmor ( Shi et al., 2025c ) , URL allowlist ( Google, 2025d ) | R1 . Heterogeneous Untrusted Interfaces |
| Output guardrail | CodeShield ( meta-llama, 2025 ) , Firewalled Agentic Networks ( Abdelnabi et al., 2025 ) , Task Shield ( Jia et al., 2025a ) , GuardAgent ( Xiang et al., 2024 ) , Agrail ( Luo et al., 2025b ) , AlignmentCheck ( Chennabasappa et al., 2025 ) , ControlValve ( Jha et al., 2025 ) , Conseca ( Tsai and Bagdasarian, 2025 ) , Progent ( Shi et al., 2025b ) , Maris ( Cui et al., 2025 ) , | R2 . Wrong Instruction Following R4 . Hallucination and Model Mistakes R5 . Private Data Leakage R6 . Unintended/Unauthorized Action and Data Corruption | |
| Information flow control and taint tracking | Permissive ( Siddiqui et al., 2024 ) , MELON ( Zhu et al., 2025 ) , RTBAS ( Zhong et al., 2025b ) , AgentArmor ( Wang et al., 2025a ) , PFI ( Kim et al., 2025 ) , CaMeL ( Debenedetti et al., 2025 ) , FIDES ( Costa et al., 2025 ) , ACE ( Li et al., 2026 ) , PrivacyChecker ( Wang et al., 2025b ) | R3 . Unconstrained/Unsafe Data Flow R5 . Private Data Leakage R6 . Unintended/Unauthorized Action and Data Corruption | |
| Monitoring | AgentAuditor ( Luo et al., 2025a ) , AgentMonitor ( Naihin et al., 2023 ) , SentinelAgent ( He et al., 2025 ) , Guardian ( Zhou et al., 2025 ) | R5 . Private Data Leakage R6 . Unintended/Unauthorized Action and Data Corruption R7 . Resource Drain and Denial-of-Service | |
| Human-in-the-loop | Wu et al. ( Wu et al., 2025b ) | ||
| Secure By Design | Privilege separation | AirGapAgent ( Bagdasarian et al., 2024 ) , f-secure ( Wu et al., 2024b ) , CaMeL ( Debenedetti et al., 2025 ) , FIDES ( Costa et al., 2025 ) , PFI ( Kim et al., 2025 ) , IsolateGPT ( Wu et al., 2025a ) , ACE ( Li et al., 2026 ) , IPIGuard ( An et al., 2025 ) , DRIFT ( Li et al., 2025a ) | R2 . Wrong Instruction Following R3 . Unconstrained/Unsafe Data Flow |
Table 2. Defense mechanisms for AI agents organized by category, as well as the risks each defense covers.
Table 3. Defense coverage in real-world agents. Defense techniques include input guardrail (Input), output guardrail (Output), access control (Access), information flow control and taint tracking (IFC), monitoring (Monitor), human-in-the-loop (HITL), privilege separation (Priv), formal verification (Formal), identity management (ID), and credential management (Cred). indicates full support and indicates partial support.
More in this topic
AI Agent Security
Artificial intelligence agent security studies attacks and safeguards for tool-using, persistent, and collaborative systems across digital and physical settings.
See all 508 papers