Prompt Injection Detection
Overview
The field studies how to recognize instructions hidden in ordinary-looking content that could redirect a language model or agent away from the user’s intent. Approaches range from keyword and sparse-text classifiers to fine-tuned models, LLM judges, hidden-state and attention probes, and trajectory analysis that tracks whether an agent’s behavior was actually hijacked. Evaluation spans retrieval-augmented generation, web and tool-using agents, multimodal inputs, and benchmarks testing obfuscation, distribution shift, and harmless text that resembles an attack. A central challenge is balancing attack detection with low false-positive rates and useful task completion, since strong scores on curated benchmarks may not transfer to adaptive attacks or real agent traffic.
Latest papers
59 papers10 papers in the last four weeks, level with the four weeks before. 0.1% of all new papers.