Prompt Injection Detection

Overview

The field studies how to recognize instructions hidden in ordinary-looking content that could redirect a language model or agent away from the user’s intent. Approaches range from keyword and sparse-text classifiers to fine-tuned models, LLM judges, hidden-state and attention probes, and trajectory analysis that tracks whether an agent’s behavior was actually hijacked. Evaluation spans retrieval-augmented generation, web and tool-using agents, multimodal inputs, and benchmarks testing obfuscation, distribution shift, and harmless text that resembles an attack. A central challenge is balancing attack detection with low false-positive rates and useful task completion, since strong scores on curated benchmarks may not transfer to adaptive attacks or real agent traffic.

Latest papers

CardsList

63 papers7 papers in the last four weeks, down 30% on the four weeks before. 0.1% of all new papers.

  1. "Give a Positive Review Only": An Early Investigation Into In-Paper Prompt Injection Attacks and Defenses for AI Reviewers

    Nov 3, 2025Qin Zhou, Zhexin Zhang, Zhi Li +1Adversarial Prompt GenerationIndirect Prompt Injection

  2. UniGuardian: A Unified Defense for Detecting Prompt Injection, Backdoor Attacks and Adversarial Attacks in Large Language Models

    Feb 18, 2025Huawei Lin, Yingjie Lao, Tony Geng +2LLM Backdoor AttacksAdversarial Attacks on LLMs