Prompt Injection Attacks on LLMs
LLM: Large Language Model
Overview
Prompt injection studies how content supplied as data—such as retrieved documents, webpages, tool outputs, memories, images, and audio—can be treated as instructions and divert a model or agent from its intended task. Attack methods range from role and delimiter tricks to hidden or fragmented payloads, adaptive black-box search, gradient-based optimization, and reinforcement-learning red teaming; evaluations cover retrieval-augmented generation, web and computer-use agents, coding, and multi-turn tasks. Defenses include detection and sanitization, provenance-aware separation of trusted instructions from untrusted data, runtime action auditing, and model-level training or activation interventions. Results depend strongly on the model, input channel, and attacker adaptivity, leaving a central tension between blocking harmful actions and preserving useful content and authorized tasks; success on static benchmarks may not predict deployment robustness.
Latest papers
125 papers19 papers in the last four weeks, up 111% on the four weeks before. 0.1% of all new papers.