Prompt Injection Attacks

Follow this topic Ask about this topic

Momentum

11 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 70

All topics

Hidden in the Comments: A Context-Injection Attack Surface in Code LLMs
Oct 4, 2026Noor Munir, Francesco Quinzan, Stephen RobertsLLM Security Indirect Prompt Injection
Department of Engineering Science, University of Oxford, Oxford OX1 3PJ, UK
Hidden Risks of Jev: An Empirical Study of Security, Privacy, and Dual Use
Oct 4, 2026Shang Wang, Tianqing Zhu, Huajie Chen +3LLM Backdoor Attacks Model Extraction Attacks
School of Computer Science, University of Technology Sydney, Sydney, Australia · Faculty of Data Science, City University of Macau, Macau, China
ToolFence: Fine-Grained Authorization for Secure Tool-Using LLM Agents
Sep 29, 2026Yanjie Li, Xiangyu He, Xuelong Dai +1Runtime Enforcement for AI Agents Prompt Injection Defense
Hong Kong Polytechnic University · Shandong University
Divide and Inject: Can Agents Reconstruct an Indirect Prompt Injection from Fragments?
Sep 29, 2026Michael Lee, Zhipeng Wei, Yue Dong +1Indirect Prompt Injection Prompt Injection Attacks
International Computer Science Institute · DSO National Laboratories · UC Riverside +1
Same Bytes, Different Authority: Reserved-Token Representations in Chat-Template Prompt Injection
Sep 28, 2026Yan Zhan, Yunze Song, Mengkai Hou +3Prompt Injection Prompt Injection Defense
Peking University · National University of Singapore · BYD Company Limited +1
ENDOPROMPT: Victim-Side Pseudo-References for Utility Degradation
Sep 24, 2026Qingyu Wu, Zeyu Feng, Yongda Yu +2Adversarial Prompt Generation Prompt Injection Attacks
Defense Innovation Institute, Academy of Military Science, Beijing, China · Nanjing University, Nanjing, China
Prefilling the Reasoning Channel: Output-Prefix Attacks on Reasoning LLMs
Sep 24, 2026Lukáš Brůna, Robert Bridges, Adam EkAdversarial Attacks on LLMs Prompt Injection Attacks
Uppsala University · AI Sweden
Decision Hijacking: Prompt Injection Attacks on Jev's Typed Probabilistic Decisions
Sep 23, 2026Tiantong Wu, Wei Yang Bryan LimAdversarial Prompt Generation Prompt Injection Attacks
Nanyang Technological University Singapore
Nameless Tokenization: A Lossless Tokenizer-Level Defense Against Control-Token Forgery in Open-Weight LLMs
Sep 15, 2026Kisu Yang, Yoonna Jang, Heuiseok LimAdversarial Attacks on LLMs LLM Security
1VAIV Company · 2Hanwha Aerospace · 3Korea University
AgentHijack: Visual Patch Attacks on Multimodal Computer-Use Agents
Sep 10, 2026Zhihao Liu, Hongyu Sun, Zhiyuan Fu +7Adversarial Patch Attacks Computer-Use Agents
Hainan University, Hainan 570228, China · University of Chinese Academy of Sciences National Computer Network Intrusion Protection Center, Beijing 101408, China · School of Computing and Communications, Lancaster University, Lancaster, United Kingdom
An Experimental Evaluation of Multimodal Prompt Injection Attacks on Agentic AI Frameworks
Sep 10, 2026Viet K. Nguyen, Mohammad I. HusainAI Agent Security AI Agent Security Benchmarks
California State Polytechnic University, Pomona Pomona, CA 91768, USA
Beyond the Payload: How User Invocation Shapes Coding Agent Vulnerability to Repository Poisoning
Aug 31, 2026Fukang Zhu, Binbin Zhao, Ruixiao Lin +3Prompt Sensitivity AI Coding Agents
1Zhejiang University · 2State Key Laboratory of Internet Architecture, Tsinghua University
The Anatomy of a Prompt Injection: A Component Model for Structured Analysis
Aug 7, 2026Jeremy McHughLLM Security Prompt Injection Attacks
Preamble, Inc.
AgentAntibody: An Adaptive Immune System for Defending LLM Agents against Prompt Injection
Aug 4, 2026Shihao Weng, Yang Feng, Xiaofei Xie +1Continual Learning for LLM Agents LLM Agent Security
Nanjing University · Singapore Management University · Nanyang Technological University
Invisible Ink Threats: Adversarial Goals Behind Legitimate Tasks in Computer-Use Agents
Aug 3, 2026Jia-Chen Zhang, Ze-Yu Zhang, Kai-Wei ZhangAI Agent Security Benchmarks Prompt Injection Defense
1East China Normal University · 2Shanghai Jiao Tong University · 3Shanghai AI Laboratory
CPInj: Uncovering Prompt Injection Risks in Textual Collaborative Prompt Optimization
Jul 21, 2026Xinting Liao, Behnoosh Zamanlooy, Masoumeh Shafieinejad +4LLM Security Prompt Injection Defense
University of British Columbia · 2Vector Institute · 3McMaster University +1
Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for LLM Agent Security
Jul 20, 2026Devina Jain, David Hartmann, Chuan LiAdversarial Attacks on LLMs LLM Agent Evaluation
Lambda
Bad Memory: Evaluating Prompt Injection Risks from Memory in Agentic Systems
Jul 16, 2026Soham Gadgil, David Alexander, Sai Sunku +1AI Agent Security Agent Memory Poisoning
University of Washington Seattle, WA, USA
Prismata: Confining Cross-Site Prompt Injection in Web Agents
Jul 9, 2026Corban Villa, Alp Eren Ozdarendeli, Sijun Tan +1LLM Agent Security Information Flow Control
UC Berkeley
A Lifecycle and Application-Stack Survey of Large Language Model Vulnerabilities: Attacks, Risks, Defenses, and Open Problems
Jun 30, 2026Seyed Bagher Hashemi Natanzi, Bo TangAI Agent Security LLM Security
Department of Electrical and Computer Engineering, Worcester Polytechnic Institute, Worcester, MA, USA
RIPA: Sensory-Vector Prompt Injection Attacks on LLM-Controlled ROS 2 Robots
Jun 26, 2026Nima DorzhievLanguage Model Safety Evaluation Prompt Injection Defense
Pennsylvania State University, State College, USA
Prompt Injection in Automated Résumé Screening with Large Language Models: Single and Multi-Injection Settings
Jun 25, 2026Preet Baxi, Jiannan Xu, Jane Yi Jiang +1Algorithmic Fairness Prompt Injection Attacks
Department of Physics, University of Michigan · Robert H. Smith School of Business, University of Maryland · Fisher College of Business, The Ohio State University +1
Adaptive Evaluation of Out-of-Band Defenses Against Prompt Injection in LLM Agents
Jun 25, 2026Praneeth Narisetty, Shiva Nagendra Babu Kore, Uday Kumar Reddy Kattamanchi +1LLM Agent Evaluation LLM Agent Security
LaunchSafe Research
Confidently Wrong: Severity-Aware Calibration of Prompt-Injection Detectors under Attack Shift
Jun 21, 2026Md Anas BiswasIndirect Prompt Injection Model Calibration
School of Computing, University of Portsmouth, Portsmouth, United Kingdom
Evaluating Prompting-Based Defenses Against Domain-Camouflaged Injection Attacks
Jun 16, 2026Aaditya PaiAI Agent Security Benchmarks Indirect Prompt Injection
Data Science Institute Columbia University
SafeClawBench: Separating Semantic, Audit-Evidence, and Sandbox Harm in Tool-Using LLM Agents
Jun 16, 2026Yuchuan Tian, Mengyu Zheng, Haocheng Mei +5LLM Safety Benchmarks LLM Agent Security
Peking University · Beijing Jiaotong University · SUIBE +2
Seeing Is Not Screening: Multimodal Hidden Instruction Attacks on Agent Skill Scanners
Jun 16, 2026Xiaojun Jia, Jie Liao, Simeng Qin +5LLM Agent Security Prompt Injection Defense
Nanyang Technological University, Singapore · Chongqing University, China · Northeastern University, China +2
Structural Role Injection in Handlebars-Templated LLM Prompts: Triple-Brace Interpolation, Delimiter Family, and the Limits of HTML Auto-Escaping
Jun 16, 2026Mohammadreza RashidiLLM Security Prompt Injection Defense
Department of Computer Science AI and Media Analysis Lab Berlin, Germany
MIRAGE: Stealthy Visual Prompt Injection for Vulnerability Detection in Web Agents
Jun 16, 2026Xuelong Dai, Jianyu Ma, Boyang Ma +3Adversarial Attacks on VLMs Adversarial Examples
Rapid Poison: Practical Poisoning Attacks Against the Rapid Response Framework
Jun 15, 2026David Huang, Jaewon Chang, Avidan Shah +2LLM Backdoor Attacks Prompt Injection Attacks
Princeton University · Anthropic · UC Berkeley +2

Oct 4, 2026, cs.LGJ/K move · Enter open · S save

Hidden in the Comments: A Context-Injection Attack Surface in Code LLMs

Noor Munir, Francesco Quinzan, Stephen Roberts

Department of Engineering Science, University of Oxford, Oxford OX1 3PJ, UK

LLM Security Indirect Prompt Injection Prompt Injection Attacks Code Language Models LLM Code Generation

Open paper AskPDF arXiv

Code large language model (Code LLM) assistants generate code from heterogeneous development contexts, including open files, imported modules, pasted snippets, and comments, much of which may originate from untrusted sources. We investigate whether insecure instructions embedded in such contexts can steer Code LLMs toward vulnerable code without access to model weights or training data. We evaluate ten open-weight Code LLMs spanning 3B--13B parameters, including four base and six instruction-tuned models, across ten web-application weakness classes. We compare completion tasks containing insecure instructions embedded as code comments with benign tasks without malicious instructions. Attack-condition completions contained a medium-or-higher weakness in {\bf 77.4--92.3}% of cases, compared with {\bf 1.7--5.1}% in the benign condition. Base and instruction-tuned models averaged 86.5% and 84.5% vulnerable outputs, respectively; equivalence testing and three matched model pairs indicated reductions of at most 8.1% after instruction tuning. Susceptibility showed no clear association with model scale or specialization. Among vulnerable attack outputs, 86.2--91.0% were rated high or critical, and the effect persisted without the pattern-based detector. Post-generation screening reduced but did not eliminate the risk, the strongest screen leaving roughly one-third undetected. These findings identify inference-time context injection as a substantial attack surface and motivate provenance-aware training objectives.

Momentum

Related topics

Latest papers 70