Prompt Injection Attacks

Ask about this topic

Overview

Prompt injection attacks exploit the fact that language models may treat instructions embedded in untrusted text, images, tool outputs, or stored memory as authoritative. Researchers test attacks in settings such as web browsing, coding, peer review, and computer use, using adaptive payloads, visual or other hidden carriers, and multi-turn scenarios. Defenses range from input detection and adversarial training to provenance tracking, restricted tool permissions, and deterministic checks outside the model. A central challenge is resisting attacks that change across tasks and interfaces without blocking legitimate work; model refusals alone are often not a dependable security boundary.

Latest papers

CardsList

89 papers13 papers in the last four weeks, up 44% on the four weeks before. 0.1% of all new papers.

  1. Agentic AI Defense: Decode-Time Provenance Binding for MCP

    Oct 22, 2026Eric Yocam, Varghese Vaidyan, Paris Kalathas +3AI Agent SecurityPrompt Injection Attacks on AI Agents

    Beacom College of Computer and Cyber Sciences, Dakota State University, Madison, SD, USA. · Information Networking Institute, Carnegie Mellon University, Pittsburgh, PA, USA.

  2. SoK: PIAtlas: Decomposing the Unbounded Prompt Injection Attack Space

    Oct 18, 2026Stavros Damianakis, Konstantinos PsounisPrompt Injection AttacksPrompt Injection Attacks on LLMs

    University of Southern California

  3. MobileInject: Evaluating Reasoning-to-Action Security of MLLM-Based Mobile Agents under Cross-Modal Consistency Injection

    Oct 16, 2026Songping Wang, Peiyang Li, Yi Huang +5LLM Agent SecurityAI Agent Security Benchmarks

  4. DecisionRobust: Benchmarking Robustness for JEV Style Decision Models Across Text and Vision

    Oct 13, 2026Yifan Lan, Zhongye Liu, Yurui Chang +4Adversarial RobustnessMultimodal Robustness

    The Pennsylvania State University

  5. Nudge the Judge: Uncovering Program Decision Hijacking in Real-World Jev Applications

    Oct 13, 2026Yang Luo, Tiantian Ji, Zifeng Kang +4Software SecurityPrompt Injection Attacks

    Key Laboratory of Trustworthy Distributed Computing and Service (MoE), Beijing University of Posts and Telecommunications · Beijing University of Posts and Telecommunications · Zhongguancun Laboratory +1

  6. Encrypted Skills Can Become New Security Exploits If AI Agents Know How to Decipher

    Oct 13, 2026Yan-Lun Chen, Pin-Yu Chen, Chia-Mu YuPrompt Injection Attacks on AI AgentsPrompt Injection Attacks

    National Yang Ming Chiao Tung University · IBM Research

  7. Can agents whose sole duty is to keep humans alive be hijacked to kill? Delayed multi-turn derived-action trap attacks in simulated high-stakes environments

    Oct 10, 2026Josef A. HabdankPrompt Injection Attacks on AI AgentsPrompt Injection Attacks

    DXC Technology

  8. When Scene Text Hijacks the Scene: Uncovering, Exploiting, and Mitigating Rendered-Text Semantic Leakage in Image Generation Models

    Oct 8, 2026Feifei Li, Runjie Wang, Xiaohan Zhang +3Safe T2I GenerationPrompt Injection Attacks

    Fudan University · Shanghai University of Electric Power

  9. Hidden in the Comments: A Context-Injection Attack Surface in Code LLMs

    Oct 4, 2026Noor Munir, Francesco Quinzan, Stephen RobertsLLM SecurityIndirect Prompt Injection

    Department of Engineering Science, University of Oxford, Oxford OX1 3PJ, UK

  10. Hidden Risks of Jev: An Empirical Study of Security, Privacy, and Dual Use

    Oct 4, 2026Shang Wang, Tianqing Zhu, Huajie Chen +3LLM Backdoor AttacksModel Extraction Attacks

    School of Computer Science, University of Technology Sydney, Sydney, Australia · Faculty of Data Science, City University of Macau, Macau, China

  11. ToolFence: Fine-Grained Authorization for Secure Tool-Using LLM Agents

    Sep 29, 2026Yanjie Li, Xiangyu He, Xuelong Dai +1Runtime Enforcement for AI AgentsPrompt Injection Defense

    Hong Kong Polytechnic University · Shandong University

  12. Divide and Inject: Can Agents Reconstruct an Indirect Prompt Injection from Fragments?

    Sep 29, 2026Michael Lee, Zhipeng Wei, Yue Dong +1Indirect Prompt InjectionPrompt Injection Attacks

    International Computer Science Institute · DSO National Laboratories · UC Riverside +1

  13. Same Bytes, Different Authority: Reserved-Token Representations in Chat-Template Prompt Injection

    Sep 28, 2026Yan Zhan, Yunze Song, Mengkai Hou +3Prompt InjectionPrompt Injection Defense

    Peking University · National University of Singapore · BYD Company Limited +1

  14. ENDOPROMPT: Victim-Side Pseudo-References for Utility Degradation

    Sep 24, 2026Qingyu Wu, Zeyu Feng, Yongda Yu +2Adversarial Prompt GenerationPrompt Injection Attacks

    Defense Innovation Institute, Academy of Military Science, Beijing, China · Nanjing University, Nanjing, China

  15. Prefilling the Reasoning Channel: Output-Prefix Attacks on Reasoning LLMs

    Sep 24, 2026Lukáš Brůna, Robert Bridges, Adam EkAdversarial Attacks on LLMsPrompt Injection Attacks

    Uppsala University · AI Sweden

  16. Decision Hijacking: Prompt Injection Attacks on Jev's Typed Probabilistic Decisions

    Sep 23, 2026Tiantong Wu, Wei Yang Bryan LimAdversarial Prompt GenerationPrompt Injection Attacks

    Nanyang Technological University Singapore

  17. Defusing Explosive Prompts: Understanding and Preventing Trigger-Based Prompt Injections in LLM Agents

    Sep 18, 2026Justin Szczepaniak, Elad Feldman, Naum Viner +1Prompt Injection AttacksPrompt Injection Attacks on AI Agents

    Tel Aviv University

  18. Nameless Tokenization: A Lossless Tokenizer-Level Defense Against Control-Token Forgery in Open-Weight LLMs

    Sep 15, 2026Kisu Yang, Yoonna Jang, Heuiseok LimAdversarial Attacks on LLMsLLM Security

    1VAIV Company · 2Hanwha Aerospace · 3Korea University

  19. An Experimental Evaluation of Multimodal Prompt Injection Attacks on Agentic AI Frameworks

    Sep 8, 2026Viet K. Nguyen, Mohammad I. HusainAI Agent SecurityAI Agent Security Benchmarks

    California State Polytechnic University, Pomona Pomona, CA 91768, USA

  20. AgentHijack: Visual Patch Attacks on Multimodal Computer-Use Agents

    Sep 6, 2026Zhihao Liu, Hongyu Sun, Zhiyuan Fu +7Adversarial Patch AttacksComputer-Use Agents

    Hainan University, Hainan 570228, China · University of Chinese Academy of Sciences National Computer Network Intrusion Protection Center, Beijing 101408, China · School of Computing and Communications, Lancaster University, Lancaster, United Kingdom

  21. Repeat-After-Me: Black-Box Adaptive Visual Prompt Injection

    Sep 3, 2026Sizhe Chen, Yu-Lin Tsai, Ivan Evtimov +4Prompt Injection AttacksPrompt Injection Attacks on AI Agents

    UC Berkeley, FAIR at Meta Berkeley / Menlo Park, USA · UC Berkeley Berkeley, USA · FAIR at Meta New York City, USA

  22. Beyond the Payload: How User Invocation Shapes Coding Agent Vulnerability to Repository Poisoning

    Aug 31, 2026Fukang Zhu, Binbin Zhao, Ruixiao Lin +3Prompt SensitivityAI Coding Agents

    1Zhejiang University · 2State Key Laboratory of Internet Architecture, Tsinghua University

  23. LongPIBench: A Long-Context Benchmark for Prompt Injection

    Aug 28, 2026Yupei Liu, Yuqi Jia, Neil Zhenqiang Gong +1Prompt InjectionPrompt Injection Attacks

    The Pennsylvania State University · Duke University

  24. Visible Reasoning and Indirect Prompt-Injection Monitorability Across English, Tamil, and Tanglish

    Aug 15, 2026Madhusudhanan GLanguage Model Safety EvaluationCoT Monitoring

    Vellore Institute of Technology, India

  25. The Anatomy of a Prompt Injection: A Component Model for Structured Analysis

    Aug 7, 2026Jeremy McHughLLM SecurityPrompt Injection Attacks

    Preamble, Inc.

  26. AgentAntibody: An Adaptive Immune System for Defending LLM Agents against Prompt Injection

    Aug 4, 2026Shihao Weng, Yang Feng, Xiaofei Xie +1Continual Learning for LLM AgentsLLM Agent Security

    Nanjing University · Singapore Management University · Nanyang Technological University

  27. Invisible Ink Threats: Adversarial Goals Behind Legitimate Tasks in Computer-Use Agents

    Aug 3, 2026Jia-Chen Zhang, Ze-Yu Zhang, Kai-Wei ZhangComputer-Use Agent SafetyAI Agent Security Benchmarks

    1East China Normal University · 2Shanghai Jiao Tong University · 3Shanghai AI Laboratory

  28. CPInj: Uncovering Prompt Injection Risks in Textual Collaborative Prompt Optimization

    Jul 21, 2026Xinting Liao, Behnoosh Zamanlooy, Masoumeh Shafieinejad +4LLM SecurityPrompt Injection Defense

    University of British Columbia · 2Vector Institute · 3McMaster University +1

  29. Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for LLM Agent Security

    Jul 20, 2026Devina Jain, David Hartmann, Chuan LiAdversarial Attacks on LLMsLLM Agent Evaluation

    Lambda

  30. Bad Memory: Evaluating Prompt Injection Risks from Memory in Agentic Systems

    Jul 16, 2026Soham Gadgil, David Alexander, Sai Sunku +1AI Agent SecurityAgent Memory Poisoning

    University of Washington Seattle, WA, USA