Prompt Injection Attacks on AI Agents

Overview

Prompt injection occurs when an agent treats instructions hidden in retrieved pages, tool outputs, files, skills, images, or memory as authoritative, potentially redirecting actions or exposing data. Researchers study these risks with agent benchmarks, realistic tool-use environments, adaptive red-teaming, and tests that track how attacks move through multi-step or multi-agent workflows. Defenses include instruction-hierarchy training, content screening, provenance and information-flow controls, isolated agents, and runtime checks that authorize each tool action against the user’s task. A central tension is that fixed benchmarks and pattern-based defenses can miss adaptive attacks, while stricter safeguards may block legitimate work; persistent memory, varied input surfaces, and long-horizon behavior remain difficult to secure and evaluate.

Latest papers

CardsList

218 papers32 papers in the last four weeks, down 30% on the four weeks before. 0.2% of all new papers.

  1. Evolution Under Fire: Evolving Agent Safety Harnesses with Adaptive Injection Stressors

    Nov 17, 2026Yujun Wang, Yilun Liu, Jinhe Bi +4AI Agent SecurityAgent Harness Self-Evolution

  2. Evaluating Authorization and Trust Boundaries in Agentic Orchestration Frameworks

    Nov 16, 2026Sahaya Jestus Lazer, Kshitiz Aryal, Maanak Gupta +1AI Agent SecurityTool Access Control for LLM Agents

  3. MobileProbe: Adaptive Security Evaluation of Mobile Agents via Trajectory-Aware Injection

    Nov 12, 2026Yingbin Jin, Ji'an Wang, HaoYang Chai +7AI Agent AuditingMobile GUI Agents

  4. From Attack Surface to Defense Architecture: Compositional Security for Agentic Coding Harnesses

    Nov 10, 2026Simon Hegelich, Bardh Prenkaj, Habiba Sarhan +5AI Agent SecurityTool Access Control for LLM Agents

  5. Harness Matters: Measuring and Dissecting Security Resilience in Web Agents

    Nov 9, 2026Xinyi Wu, Jiagui Chen, JiaYi Dong +5LLM Agent SecurityLLM Agent Harnesses

  6. Treating the Model as an Untrusted Component: Deterministic Policy Enforcement, Human Approval and Taint Tracking Against Prompt Injection in Tool-Using Agents, with Validation on Open-Weights Models

    Nov 8, 2026Mustafa S. Aljumaily, Nawar S. Alseelawi, Raghad AlshabandarPrompt Injection DefenseLLM Agent Safety

  7. When Coding Agents Get Arms: Safety Risks Across the Digital-Physical Boundary

    Nov 8, 2026Yiming Sun, Mi ZhangAI Agent SafetyAI Coding Agents

  8. ErosionBench: Probing Alignment Decay in Enterprise Multi-Agent Systems

    Nov 8, 2026Riya Bharathwaj, Akshara Prabhakar, Roshan Ram +4AI Agent Safety BenchmarksMulti-Agent LLM Systems

  9. I Think, Therefore I Attack: Evaluating Thought Injection Attacks on Reasoning Agents

    Nov 6, 2026Parth Illendula, Garik Livingston, Zain Shamsi +2Prompt Injection Attacks on LLMsPrompt Injection Attacks on AI Agents

  10. MetaPI: Constructing Prompt Injection Benchmarks from LLM Agent Benchmark

    Nov 5, 2026Peiran Wang, Chong Xiang, Wenjie Qu +4Prompt Injection Attacks on AI AgentsAI Agent Security Benchmarks

  11. SecureGARD: Dynamic Ontology-Guided Governance for Secure Tool-Using LLM Agents

    Nov 4, 2026Guoqin Xiong, Shengtao Wen, Xiang Chen +5AI Agent SecurityTool Access Control for LLM Agents

  12. Voluntary Servitude: Weaponized Safety Warning Mechanisms for Controlling AI Agents

    Nov 3, 2026Ilan Zendel, Matan Levy, Stav Cohen +2AI Agent SecurityLLM Agent Safety

  13. StateFence: Task-State Protection Against Evidence-Chain Contamination in Blue-Team LLM Agents

    Nov 2, 2026Haowen Dai, Alejandro Guerra-ManzanaresAI Agent SecurityPrompt Injection Defense

  14. MemHarmBench: Evaluating Cross-Session Prompt Injection in Memory-Persistent AI Agents

    Nov 1, 2026Zhihua Wang, Yunhao Feng, Yifeng Gao +4Prompt Injection AttacksPrompt Injection Attacks on AI Agents

  15. SecPO: Principled Adversarial Training for Prompt Injection Security

    Oct 30, 2026Sizhe Chen, Yibo Peng, Jaewon Chang +2Prompt Injection DefensePrompt Injection Attacks on AI Agents

  16. ANGLER: Verifier-Grounded Benchmarking with an Adaptive Attacker for Web Agents

    Oct 30, 2026Yini Huang, Aiwen LU, Zhen Sun +3AI Agent Security BenchmarksAutomated Red Teaming

  17. Understanding and Mitigating Role Spoofing in LLMs

    Oct 29, 2026Ezra Edelman, Benjamin Edelman, Surbhi Goel +1Adversarial Attacks on LLMsPrompt Injection Attacks on LLMs

  18. Where You Inject Matters: The Trust Region Effect in LLM Agents

    Oct 28, 2026Chengquan Guo, Xiaodong Liu, Weiwei Yang +7Prompt InjectionPrompt Injection Attacks on AI Agents

  19. Heads of Security: Learned Attention Gates Improve Prompt Injection Robustness

    Oct 27, 2026Igor Shilov, Xiaoxue Yang, Alexander Panfilov +6LLM SecurityPrompt Injection Defense

  20. Poisoned Skills: Propagating Jailbreaks Through Agent Markdown Files

    Oct 27, 2026Kashyap R Manjusha, Murali EmaniPrompt Injection Attacks on AI AgentsLLM Jailbreak Attacks

  21. Log-To-Leak: Prompt Injection Attacks on Tool-Using LLM Agents via the Model Context Protocol

    Oct 26, 2026Yuepeng Hu, Chongyu Fan, Sirat Samyoun +1Prompt Injection Attacks on AI AgentsLLM Agent Safety

  22. Scope-Gap Injection: Exploiting Evaluator Channel Blind Spots in the OpenClaw Agent Security Competition

    Oct 26, 2026Weihang Niu, Mengxing Ren, Che QuPrompt Injection Attacks on AI AgentsIndirect Prompt Injection

  23. Agentic AI Defense: Decode-Time Provenance Binding for MCP

    Oct 22, 2026Eric Yocam, Varghese Vaidyan, Paris Kalathas +3AI Agent SecurityPrompt Injection Attacks on AI Agents