Prompt Injection Attacks on AI Agents

Follow this topic Ask about this topic

Momentum

26 papers in the last four weeks, up 271% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 108

All topics

NetInjectBench: Benchmarking Indirect Prompt Injection in Tool-Using Large Language Model Agents for Network Operations
Jul 11, 2026Ruksat Khan Shayoni, Muhammad Faraz Shoaib, S M Asif Hossain +1AI Agent Security Benchmarks Indirect Prompt Injection
School of Computing, Wichita State University, Kansas, USA · Department of Computer Science, American International University-Bangladesh, Dhaka, Bangladesh
Agent Data Injection Attacks are Realistic Threats to AI Agents
Jul 6, 2026Woohyuk Choi, Juhee Kim, Taehyun Kang +3AI Agent Security Indirect Prompt Injection
Seoul National University · Largosoft · University of Illinois Urbana-Champaign
DualView: Preventing Indirect Prompt Injection in Personal AI Agents
Jul 4, 2026Juhee Kim, Woohyuk Choi, Taehyun Kang +2LLM Agent Security Indirect Prompt Injection
Seoul National University
RIPA: Sensory-Vector Prompt Injection Attacks on LLM-Controlled ROS 2 Robots
Jun 26, 2026Nima DorzhievLanguage Model Safety Evaluation Prompt Injection Defense
Pennsylvania State University, State College, USA
Adaptive Evaluation of Out-of-Band Defenses Against Prompt Injection in LLM Agents
Jun 25, 2026Praneeth Narisetty, Shiva Nagendra Babu Kore, Uday Kumar Reddy Kattamanchi +1LLM Agent Evaluation LLM Agent Security
LaunchSafe Research
When AUC 0.998 Is Not Enough: A Candidate Evaluation Protocol for Hidden-State Probes of Indirect Prompt Injection in Multimodal Computer-Use Agents
Jun 22, 2026Yanhang Li, Zhichao Fan, Zexin ZhuangAI Agent Evaluation Indirect Prompt Injection
Northeastern University Boston, Massachusetts, USA · University of Illinois Urbana-Champaign Urbana, Illinois, USA · Southern Methodist University Dallas, Texas, USA
Analyzing Defensive Misdirection Against Model-Guided Automated Attacks on Agentic AI Systems
Jun 18, 2026Reza Soosahabi, Vivek NamsaniAdversarial Robustness Jailbreak Attacks
Application & Threat Intelligence Research Center Keysight Technologies Inc., USA
A Layered Security Framework Against Prompt Injection in RAG-Based Chatbots
Jun 17, 2026Gulshan Saleem, Nisar Ahmed, Muhammad Imran Zaman +1LLM Guardrails Indirect Prompt Injection
Sparkverse AI Ltd, Bradford, 100190, West Yorkshire, England, UK
Evaluating Prompting-Based Defenses Against Domain-Camouflaged Injection Attacks
Jun 16, 2026Aaditya PaiAI Agent Security Benchmarks Indirect Prompt Injection
Data Science Institute Columbia University
SafeClawBench: Separating Semantic, Audit-Evidence, and Sandbox Harm in Tool-Using LLM Agents
Jun 16, 2026Yuchuan Tian, Mengyu Zheng, Haocheng Mei +5LLM Safety Benchmarks LLM Agent Security
Peking University · Beijing Jiaotong University · SUIBE +2
PARSE: Provenance-Aware Retrieval Sanitization for Professional Domain LLM Agents
Jun 16, 2026Aaditya PaiAI Agent Security Benchmarks Prompt Injection Defense
Data Science Institute Columbia University
Defending against Adaptive Prompt Injection Attacks via Reasoning-enabled Task Alignment
Jun 13, 2026Lipeng He, Yihan Wang, Jiawen Zhang +1Adversarial Training AI Agent Safety
University of Waterloo Waterloo, Canada · Zhejiang University Hangzhou, China · KTH Royal Institute of Technology Stockholm, Sweden
AutoDojo: A Generative Benchmark for Evaluating Prompt Injection Defenses in LLM Agents
Jun 13, 2026Xinhang Ma, Taoran Li, Chaowei Xiao +3AI Agent Security Benchmarks Indirect Prompt Injection
Washington University in St. Louis, St. Louis, MO, USA · Texas A&M University, College Station, TX, USA · Johns Hopkins University, Baltimore, MD, USA
Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agents
Jun 11, 2026Zihao Wang, Yiming Li, Yutong Wu +8AI Agent Evaluation LLM Agent Security
Nanyang Technological University, Singapore · ST Engineering, Singapore · IBM Research, USA +1
PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections
Jun 10, 2026Pengfei He, Lesly Miculicich, Vishesh Sharma +5AI Agent Auditing LLM Auditing
Google Cloud AI Research · Google · Michigan State University
Smarter Saboteurs, Better Fixers: Scaling & Security in Linear Multi-Agent Workflows
Jun 10, 2026Timothy McAllister, Sina Abdidizaji, Ivan Garibay +1AI Agent Security LLM Agent Security
Department of Computer Science, University of Central Florida, Orlando, FL, USA · Department of Industrial Engineering, University of Central Florida, Orlando, FL, USA
Assessing Automated Prompt Injection Attacks in Agentic Environments
Jun 9, 2026David Hofer, Edoardo Debenedetti, Florian TramèrAdversarial Prompt Generation LLM Agent Security
ETH Zurich Zurich, Switzerland
GitInject: Real-World Prompt Injection Attacks in AI-Powered CI/CD Pipelines
Jun 7, 2026Jafar Isbarov, Umid Suleymanov, Ilia Shumailov +1AI Agent Security AI Agent Security Benchmarks
Virginia Tech Blacksburg, VA, USA · AI Sequrity Company London, UK
What If Prompt Injection Never Left? Exploring Cross-Session Stored Prompt Injection in Agentic Systems
Jun 3, 2026Yuanbo Xie, Tianyun Liu, Yingjie Zhang +4Prompt Injection AI Agent Security
Institute of Information Engineering, Chinese Academy of Sciences, China · School of Cyber Security, University of Chinese Academy of Sciences, China · AI Sec Lab, Beijing Chaitin Technology Co.,Ltd
AgentRedBench: Dynamic Redteaming and Integration-Aware Defense for LLM Agents over SaaS Integrations
Jun 1, 2026Hiskias Dingeto, William LeeneyLLM Red Teaming LLM Agent Security
StackOne Technologies
Send a SCOUT First: Pre-hoc Reasoning for Adaptive Detector Allocation in Prompt-Injection Defense
May 29, 2026Shuhao Zhang, Jiarui Li, Qi Cao +2AI Agent Security Benchmarks LLM Security
UC San Diego · University of Illinois Urbana-Champaign
Depth-Dependent Indirect Prompt Injection in Tool-Calling ReAct Agents: Injection Depth, Payload Framing, and Turn-Budget Sensitivity
May 29, 2026Mohammadreza RashidiLLM Agent Evaluation AI Agent Security
Department of Computer Science AI and Media Analysis Lab Berlin, Germany
Investigating Detection and Obfuscation of Prompt Injection Attacks Against Software Reverse Engineering AI Agents
May 29, 2026Brian Crawford, Patrick McClureSoftware Reverse Engineering Indirect Prompt Injection
Dept. of Computer Science Naval Postgraduate School Monterey, CA, USA
The Surface You Test Is Not the Surface That Breaks
May 28, 2026Syed Nazmus Sakib, Nafiul Haque, Shahrear Bin Amin +1LLM Agent Security AI Agent Security Benchmarks
Department of Robotics and Mechatronics Engineering, University of Dhaka · Department of Computer Science and Engineering, University of Dhaka
MIRAGE: Context-Aware Prompt Injection against Mobile GUI Agents via User-Generated Content
May 27, 2026Ruoqi Guo, Yi Liu, Gelei Deng +7Mobile GUI Automation GUI Agents
1Griffith University · 2Quantstamp · 3Nanyang Technological University +4
IterInject: Indirect Prompt Injection Against LLM Agents via Feedback-Guided Iterative Optimization
May 23, 2026Zixuan Chen, Jiaxiang Chen, Li Luo +4LLM Agent Security Indirect Prompt Injection
1Shanghai Jiao Tong University · 2The University of Hong Kong
LivePI: More Realistic Benchmarking of Agents Against Indirect Prompt Injection
May 18, 2026Lei Zhao, Abhay Bhaskar, Edgar DobribanAI Agent Security AI Agent Safety
University of Pennsylvania
ESLD (External Surrogate Latent Defense): A Latent-Space Architecture for Faster, Stronger Prompt-Injection Defense
May 18, 2026Yash NarendraAI Agent Security Safety Filtering
Microsoft
AI Agents May Always Fall for Prompt Injections
May 17, 2026Sahar Abdelnabi, Eugene BagdasarianContextual Integrity AI Agent Security
ELLIS Institute Tübingen & MPI-IS & Tübingen AI Center · University of Massachusetts Amherst
ASPI: Seeking Ambiguity Clarification Amplifies Prompt Injection Vulnerability in LLM Agents
May 17, 2026Udari Madhushani Sehwag, Zhengyang Shan, Heming Liu +3LLM Agent Security AI Agent Security Benchmarks
Scale AI · 2Boston University · 4Human Frontier Collective +3

Jul 11, 2026, cs.CRJ/K move · Enter open · S save

NetInjectBench: Benchmarking Indirect Prompt Injection in Tool-Using Large Language Model Agents for Network Operations

Ruksat Khan Shayoni, Muhammad Faraz Shoaib, S M Asif Hossain, M. F. Mridha

School of Computing, Wichita State University, Kansas, USA · Department of Computer Science, American International University-Bangladesh, Dhaka, Bangladesh

AI Agent Security Benchmarks Indirect Prompt Injection Safety Filtering Prompt Injection Defense Prompt Injection Attacks on LLMs Prompt Injection Attacks on AI Agents+1

Open paper AskPDF arXiv

Tool-using large language model (LLM) agents are attractive for network operations, but tickets, alerts, logs, runbooks, and ChatOps messages can carry indirect prompt injections. We present NetInjectBench, a 130-scenario benchmark that separates untrusted artifact text, trusted policy metadata, and evaluation labels for network-operation tool use. The sample contains 40 benign, 40 weak-attack, 40 strong-attack, and 10 approved high-impact change scenarios; each is evaluated with Qwen2.5-7B, Llama3.1-8B, and Mistral-7B. Across 240 attack instances, naive execution reached an 82.50% unsafe tool-action rate. Prompt-only safety, Self-Reminder, Spotlighting, and a Two-Pass LLM Judge reduced this rate to 25.63%, 21.67%, 18.33%, and 10.00%, respectively. Static allowlisting reached 5.00% but blocked all approved changes, yielding 0.00% usefulness and 100.00% overblocking on approved cases. Under the stated metadata-integrity assumption, the metadata-aware policy gate produced 0/240 unsafe attack actions, with a 95% Wilson upper bound of 1.58%, while preserving 99.17% attack-scenario usefulness and 100.00% approved-change usefulness. The findings show that network-operation agents need execution-time authorization boundaries alongside prompt-level instruction hygiene.

Momentum

Related topics

Latest papers 108