Prompt Injection Defense

Follow this topic Ask about this topic

Latest papers 101

17 papers in the last four weeks, up 113% on the four weeks before. 0.1% of all new papers.

All topics

Send a SCOUT First: Pre-hoc Reasoning for Adaptive Detector Allocation in Prompt-Injection Defense
May 29, 2026Shuhao Zhang, Jiarui Li, Qi Cao +2AI Agent Security Benchmarks LLM Security
UC San Diego · University of Illinois Urbana-Champaign
Investigating Detection and Obfuscation of Prompt Injection Attacks Against Software Reverse Engineering AI Agents
May 29, 2026Brian Crawford, Patrick McClureSoftware Reverse Engineering Indirect Prompt Injection
Dept. of Computer Science Naval Postgraduate School Monterey, CA, USA
Evaluating using Mock Tool Calls to Quarantine Untrusted Prompt Inputs
May 28, 2026David Gros, Adam GleaveLanguage Model Safety Evaluation Adversarial Attacks on LLMs
The Surface You Test Is Not the Surface That Breaks
May 28, 2026Syed Nazmus Sakib, Nafiul Haque, Shahrear Bin Amin +1LLM Agent Security AI Agent Security Benchmarks
Department of Robotics and Mechatronics Engineering, University of Dhaka · Department of Computer Science and Engineering, University of Dhaka
Mitigating Adaptive Attacks against Reasoning Models with Activation Consistency Training
May 27, 2026Avidan Shah, Jannik Brinkmann, Rico AngellPrompt Injection Defense Adversarial Robustness of Language Models
New York University · Technical University Clausthal
Disentangling Adversarial Prompts: A Semantic-Graph Defense for Robust LLM Security
May 27, 2026Xiang Fang, Wanlong FangLLM Security Prompt Injection Defense
School of Software Engineering, Huazhong University of Science and Technology · Nanyang Technological University, Singapore
Poisoning the Watchtower: Prompt Injection Attacks Against LLM-Augmented Security Operations Through Adversarial Log Content
May 23, 2026Rohan Pandey, Archit BhujangIndirect Prompt Injection LLMs for Cybersecurity
DigitalOcean · Arizona State University
LivePI: More Realistic Benchmarking of Agents Against Indirect Prompt Injection
May 18, 2026Lei Zhao, Abhay Bhaskar, Edgar DobribanAI Agent Security AI Agent Safety
University of Pennsylvania
ESLD (External Surrogate Latent Defense): A Latent-Space Architecture for Faster, Stronger Prompt-Injection Defense
May 18, 2026Yash NarendraAI Agent Security Safety Filtering
Microsoft
AI Agents May Always Fall for Prompt Injections
May 17, 2026Sahar Abdelnabi, Eugene BagdasarianContextual Integrity AI Agent Security
ELLIS Institute Tübingen & MPI-IS & Tübingen AI Center · University of Massachusetts Amherst
WARD: Adversarially Robust Defense of Web Agents Against Prompt Injections
May 14, 2026Tri Cao, Yulin Chen, Hieu Cao +8Adversarial Training Prompt Injection
National University of Singapore · University of Science · Vietnam National University, Ho Chi Minh City
When Prompts Become Payloads: A Framework for Mitigating SQL Injection Attacks in Large Language Model-Driven Applications
May 11, 2026Farzad Nourmohammadzadeh Motlagh, Mehrdad Hajizadeh, Mehryar Majd +3LLM Security Text-to-SQL
Hasso Plattner Institute, University of Potsdam, Germany · Chemnitz University of Technology, Chemnitz, Germany
The Granularity Mismatch in Agent Security: Argument-Level Provenance Solves Enforcement and Isolates the LLM Reasoning Bottleneck
May 11, 2026Linfeng Fan, Ziwei Li, Yuan Tian +3LLM Agent Security Information Flow Control
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China · King Abdullah University of Science and Technology, Thuwal, Saudi Arabia · Dongbei University of Finance and Economics, Dalian, China +1
CALYREX: Cross-Attention LaYeR EXtended Transformers for System Prompt Anchoring
May 10, 2026Li LixingSelf-Attention Cross-Attention
Cornell University Ithaca, NY 14853
Defense effectiveness across architectural layers: a mechanistic evaluation of persistent memory attacks on stateful LLM agents
May 8, 2026Jun Wen LeongLanguage Model Safety Evaluation AI Agent Security
MIPIAD: Multilingual Indirect Prompt Injection Attack Defense with Qwen -- TF-IDF Hybrid and Meta-Ensemble Learning
May 8, 2026Al Muhit Muhtadi, Mostafa Rifat TazwarPrompt Injection Defense Prompt Injection Attacks on LLMs
Bangladesh University of Engineering and Technology
Shattering the Echo Chamber: Hidden Safeguards in Manuscripts Against the AI Takeover of Peer Review
May 6, 2026Oubo Ma, Ruixiao Lin, Jiahao Chen +3LLM Security Indirect Prompt Injection
Zhejiang University
PIIGuard: Mitigating PII Harvesting under Adversarial Sanitization
May 4, 2026Mingshuo Liu, Yiwei Zha, Min ChenData Leakage Cybersecurity
Vrije Universiteit Amsterdam
A Sentence Relation-Based Approach to Sanitizing Malicious Instructions
May 1, 2026Soumil Datta, Melissa Umble, Daniel S. Brown +1Prompt Injection Defense RAG Security
University of Utah
Evaluation of Prompt Injection Defenses in Large Language Models
Apr 26, 2026Priyal Deep, Shane Emmons, Amy Fox +4Data Leakage Privacy Leakage in Language Models
Swept AI[1] · University of Michigan
CASCADE: A Component Ablation and Corpus Audit of a Layered Local Defense for MCP-Based Systems
Apr 18, 2026İpek Abasıkeleş Turgut, Edip GümüşMCP Security LLM Security
Department of Computer Engineering, Faculty of Engineering and Natural Sciences, Iskenderun Technical University, Hatay, Türkiye · Department of Computer Engineering, Institute of Graduate Studies, Iskenderun Technical University, Hatay, Türkiye
ClawGuard: A Runtime Security Framework for Tool-Augmented LLM Agents Against Indirect Prompt Injection
Apr 13, 2026Wei Zhao, Zhe Li, Peixin Zhang +1AI Agent Security LLM Agent Security
Singapore Management University
PIArena: A Platform for Prompt Injection Evaluation
Apr 9, 2026Runpeng Geng, Chenlong Yin, Yanting Wang +2Prompt Injection Prompt Injection Attacks
The Pennsylvania State University
The Defense Trilemma: Why Prompt Injection Defense Wrappers Fail?
Apr 7, 2026Manish Bhatt, Sarthak Munshi, Vineeth Sai Narajala +6Prompt Injection Defense LLM Security
OWASP · Amazon Leo · UNO
Automating Cloud Security and Forensics Through a Secure-by-Design Generative AI Framework
Apr 5, 2026Dalal Alharthi, Ivan Roberto Kawaminami GarciaCybersecurity Digital Forensics
University of Arizona, Tucson, AZ, USA
Your Agent is More Brittle Than You Think: Uncovering Indirect Injection Vulnerabilities in Agentic LLMs
Apr 4, 2026Wenhui Zhu, Xuanzhao Dong, Xiwen Chen +7AI Agent Security Indirect Prompt Injection
Arizona State University · Morgan Stanley · UC Davis +4
Evaluation and Hardening of LLM System Instructions Against Extraction via Encoding Attacks
Apr 1, 2026Anubhab Sahu, Diptisha Samanta, Reza SoosahabiLLM Security Privacy Leakage in Language Models
Keysight Technologies, India · Keysight Technologies, USA
Architecting Secure AI Agents: Perspectives on System-Level Defenses Against Indirect Prompt Injection Attacks
Mar 31, 2026Chong Xiang, Drew Zagieboylo, Shaona Ghosh +5AI Agent Security Prompt Injection Defense
NVIDIA · Independent Researcher · Johns Hopkins University
Kill-Chain Canaries: Stage-Level Tracking of Prompt Injection Across Attack Surfaces and Five Production LLMs
Mar 30, 2026Haochuan Kevin Wang, Zechen ZhangMulti-Agent LLM Systems Language Model Safety Evaluation
Massachusetts Institute of Technology · University of Chicago
Evaluating Prompt Injection Defenses for Educational LLM Tutors: Security-Usability-Latency Trade-offs
Mar 29, 2026Alexandre Cristovão MaioranoPrompt Injection Defense LLM Guardrails

May 29, 2026J/K move · Enter open · S save

Send a SCOUT First: Pre-hoc Reasoning for Adaptive Detector Allocation in Prompt-Injection Defense

Shuhao Zhang, Jiarui Li, Qi Cao, Ruiyi Zhang+1

UC San Diego · University of Illinois Urbana-Champaign

AI Agent Security Benchmarks LLM Security Prompt Injection Defense Prompt Injection Attacks on LLMs Prompt Injection Attacks on AI Agents Prompt Injection Detection

Open paper AskPDF arXiv

Prompt-injection detectors are heterogeneous: each is strong on a different slice of attacks, and none is always reliable. Yet existing systems still treat detection as a fixed single-detector pipeline, committing every request to one detector's blind spots. We reframe defense as detector allocation: given a heterogeneous pool, decide per request which detectors to run and whether to escalate to an LLM judge. Our framework SCOUT (Scalable and Controllable Outcome-prediction for Uncertainty-aware Triage) makes this decision dynamic by predicting each detector's per-sample reliability and latency from how it behaved on similar past inputs, and exposes a single safety-utility threshold to the operator (where utility bundles benign-pass rate and wall-clock). To evaluate this setting, we build SCOUT-450, a benchmark that captures the structurally complex, agent-facing injections that older prompt-injection sets under-represent. On SCOUT-450, a safety-oriented operating point reduces attack-success rate by 46% and total wall-clock by 40% relative to an always-on GPT-4o judge, at a 5.1-point benign-utility drop. SCOUT also transfers to three external benchmarks (BIPIA, IPI, and IHEval), improving the safety-utility frontier.