Prompt Injection Attacks on AI Agents

Overview

Prompt injection occurs when an agent treats instructions hidden in retrieved pages, tool outputs, files, skills, images, or memory as authoritative, potentially redirecting actions or exposing data. Researchers study these risks with agent benchmarks, realistic tool-use environments, adaptive red-teaming, and tests that track how attacks move through multi-step or multi-agent workflows. Defenses include instruction-hierarchy training, content screening, provenance and information-flow controls, isolated agents, and runtime checks that authorize each tool action against the user’s task. A central tension is that fixed benchmarks and pattern-based defenses can miss adaptive attacks, while stricter safeguards may block legitimate work; persistent memory, varied input surfaces, and long-horizon behavior remain difficult to secure and evaluate.

Latest papers

CardsList

218 papers32 papers in the last four weeks, down 30% on the four weeks before. 0.2% of all new papers.

  1. A Trajectory-Based Safety Audit of Clawdbot (OpenClaw)

    Feb 16, 2026Tianyu Chen, Dongrui Liu, Xia Hu +2AI Agent SecurityAI Agent Safety

  2. It's a TRAP! Task-Redirecting Agent Persuasion Benchmark for Web Agents

    Dec 29, 2025Karolina Korgul, Yushi Yang, Arkadiusz Drohomirecki +7Web Agent BenchmarksIndirect Prompt Injection

  3. Cross-User Poisoning: User-Task Boundary Failures in Multi-User Collaborative Language Agents

    Nov 21, 2025Atharv Singh Patlan, Peiyao Sheng, S. Ashwin Hebbar +2Data Poisoning AttacksAgent Memory Poisoning

  4. AgentTypo: Adaptive Typographic Prompt Injection Attacks against Black-box Multimodal Agents

    Oct 5, 2025Yanjie Li, Yiming Cao, Dong Wang +1Visual Prompt Injection AttacksPrompt Injection Attacks

  5. Red-Teaming Coding Agents from a Tool-Invocation Perspective: An Empirical Security Assessment

    Sep 6, 2025Yuchong Xie, Mingyu Luo, Zesen Liu +7Prompt InjectionAI Coding Agents

  6. Meta-SecAlign: Training LLMs against Prompt Injection for Robust Agents

    Jul 3, 2025Sizhe Chen, Arman Zharmagambetov, David Wagner +1LLM Agent SecurityAI Agent Security Benchmarks

  7. LLMail-Inject: A Dataset from a Realistic Adaptive Prompt Injection Challenge

    Jun 11, 2025Sahar Abdelnabi, Aideen Fay, Ahmed Salem +22LLM SecurityIndirect Prompt Injection