Prompt Injection Detection

Follow this topic Ask about this topic

Momentum

4 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 39

All topics

RAG-PIBench: A Leakage-Aware Benchmark for Prompt-Injection Detection in Trustworthy RAG Systems
Oct 6, 2026Niveen O. Jaffal, Ahmet Yuksel, David MohaisenRAG Security Prompt Injection Attacks on LLMs
Birzeit University, Birzeit, Palestine · Independent Researcher · University of Central Florida, Orlando, FL, USA
Evaluating System One Models for Agent Security Decisions: Reliability, Calibration, and Selective Automation
Sep 27, 2026Yixuan LiuProbability Calibration Selective Prediction
TraceStone and Nanyang Technological University, Singapore
PIDS-Bench: Evaluating Prompt-Injection Detectors Under Over-Defense, Obfuscation, and Distribution Shift
Sep 14, 2026Yusuf Khalid Shire, Sang-Chul KimDistribution Shift Robustness Prompt Injection Detection
Department of Software Convergence, Kookmin University, Seoul, Republic of Korea
DriftNet: A Dual-Head Trajectory Transformer for Detecting and Localizing Prompt Injection in LLM Agents
Sep 12, 2026Asif Pinjari, Mithun Paul Saint-GermainAI Agent Security Benchmarks AI Agent Monitoring
School of Informatics, Computing, and Cyber Systems Northern Arizona University Flagstaff, AZ, USA
No-Box Vulnerability Analysis: Description-only Detection of Indirect Prompt Injection Vulnerabilities in MCP Servers
Sep 9, 2026Zehua Zhang, Jie Hu, Pratham Hegde +13MCP Security Software Vulnerability Detection
School of Computing and Augmented Intelligence, Arizona State University
HiveTraceGuard-Pro: A Compact Generative Guardrail for Prompt Injection, Jailbreaks, and Adversarial Obfuscation
Sep 1, 2026Nikita Oblakov, Sabrina Sadiekh, Evgeniy KokuykinLanguage Model Safety Evaluation LLM Guardrails
HiveTraceLab
The Latent Diagnostic Taxonomy: A Framework for Constructing Classifiers and Diagnosing Their Decisions, Applied to Prompt Injection Detection
Aug 26, 2026Jaturong Kongmanee, Smile ThanapattheerakulInterpretable ML Prompt Injection Defense
Research Incubation, TrendAI™Research
BASIS: Breach-Aware Selective Prompt Injection Shielding with Prefill Attention Probes
Aug 8, 2026Laiqiao Qin, Tianqing Zhu, Longxiang Gao +1Prompt Injection Defense Prompt Injection Attacks on LLMs
City University of Macau, Macau · Qilu University of Technology
Robust Context-Aware Detection of Malicious Instructions in Text
Aug 5, 2026Buzhao Liu, Xinhang Ma, Yevgeniy VorobeychikAdversarial Training Prompt Injection Defense
Washington University in St. Louis
Your Agentic LLMs Secretly Encode Latent Signals of Indirect Prompt-Injection Exposure
Aug 1, 2026Jianshuo Dong, Yiming Liu, Maosen Zhang +6Indirect Prompt Injection Prompt Injection Defense
Tsinghua University · MatrixOrigin · Nanyang Technological University +1
When AUC 0.998 Is Not Enough: A Candidate Evaluation Protocol for Hidden-State Probes of Indirect Prompt Injection in Multimodal Computer-Use Agents
Jun 22, 2026Yanhang Li, Zhichao Fan, Zexin ZhuangAI Agent Evaluation Indirect Prompt Injection
Northeastern University Boston, Massachusetts, USA · University of Illinois Urbana-Champaign Urbana, Illinois, USA · Southern Methodist University Dallas, Texas, USA
DE-FIVE: Detecting Malicious Image Prompts via Fourier Features and Image Vector Embeddings
Jun 22, 2026Xingwei Zhong, Varun Sharma, Kar Wai Fok +1VLM Robustness Indirect Prompt Injection
Confidently Wrong: Severity-Aware Calibration of Prompt-Injection Detectors under Attack Shift
Jun 21, 2026Md Anas BiswasIndirect Prompt Injection Model Calibration
School of Computing, University of Portsmouth, Portsmouth, United Kingdom
A Layered Security Framework Against Prompt Injection in RAG-Based Chatbots
Jun 17, 2026Gulshan Saleem, Nisar Ahmed, Muhammad Imran Zaman +1LLM Guardrails Indirect Prompt Injection
Sparkverse AI Ltd, Bradford, 100190, West Yorkshire, England, UK
Hiding in Plain Floats: Steganographic Carriers for Indirect Prompt and Content Injection
Jun 7, 2026Mudit Sinha, Sanika ChavanLLM Security Indirect Prompt Injection
1Lineaje Inc. · 2Arizona State University.
GuardNet: Ensemble Strategies of Shallow Neural Networks for Robust Prompt Injection and Jailbreak Detection
Jun 4, 2026Paulo Ricardo Ferreira Neves, Edson Rodrigues da Cruz Filho, Paulo Henrique Eleuterio Falsetti +7LLM Guardrails Neural Network Robustness
Quickium Technology Ltd. – Piracicaba, São Paulo · Federal Institute of Education, Science and Technology of São Paulo (IFSP) – Piracicaba Campus – São Paulo · Federal University of São Carlos (UFSCar) – Sorocaba Campus – São Paulo
Gate AI: LLM Security Benchmark Evaluation Methodology and Results
Jun 1, 2026Ryle Goehausen, Marcus SousaLLM Safety Benchmarks Language Model Safety Evaluation
AgentRedBench: Dynamic Redteaming and Integration-Aware Defense for LLM Agents over SaaS Integrations
Jun 1, 2026Hiskias Dingeto, William LeeneyLLM Red Teaming LLM Agent Security
StackOne Technologies
Send a SCOUT First: Pre-hoc Reasoning for Adaptive Detector Allocation in Prompt-Injection Defense
May 29, 2026Shuhao Zhang, Jiarui Li, Qi Cao +2AI Agent Security Benchmarks LLM Security
UC San Diego · University of Illinois Urbana-Champaign
Investigating Detection and Obfuscation of Prompt Injection Attacks Against Software Reverse Engineering AI Agents
May 29, 2026Brian Crawford, Patrick McClureSoftware Reverse Engineering Indirect Prompt Injection
Dept. of Computer Science Naval Postgraduate School Monterey, CA, USA
Measuring Real-World Prompt Injection Attacks in LLM-based Resume Screening
May 27, 2026Mohan Zhang, Yuqi Jia, Zhen Tan +4LLM Security Indirect Prompt Injection
University of North Carolina at Chapel Hill · Duke University · Arizona State University +2
Prompt Injection Detection is Regime-Dependent: A Deployment-Aware Evaluation with Interpretable Structural Signals
May 26, 2026Akindoyin Akinrele, Shreyank N GowdaLanguage Model Safety Evaluation Prompt Injection Attacks on LLMs
School of Computer Science, University of Nottingham, Jubilee Campus, Wollaton Road, Nottingham, NG8 1BB, United Kingdom
Blind Spots in the Guard: How Domain-Camouflaged Injection Attacks Evade Detection in Multi-Agent LLM Systems
May 21, 2026Aaditya PaiMulti-Agent LLM Systems AI Agent Security
Data Science Institute Columbia University
ESLD (External Surrogate Latent Defense): A Latent-Space Architecture for Faster, Stronger Prompt-Injection Defense
May 18, 2026Yash NarendraAI Agent Security Safety Filtering
Microsoft
When Prompts Become Payloads: A Framework for Mitigating SQL Injection Attacks in Large Language Model-Driven Applications
May 11, 2026Farzad Nourmohammadzadeh Motlagh, Mehrdad Hajizadeh, Mehryar Majd +3LLM Security Text-to-SQL
Hasso Plattner Institute, University of Potsdam, Germany · Chemnitz University of Technology, Chemnitz, Germany
AgentShield: Deception-based Compromise Detection for Tool-using LLM Agents
May 10, 2026Yassin H. Rassul, Tarik A. RashidLLM Agent Security AI Agent Monitoring
Computer Science and Engineering Department, School of Science and Engineering, University of Kurdistan Hewlêr, Erbil, Iraq · Artificial Intelligence and Innovation Centre, University of Kurdistan Hewlêr, Erbil, Iraq
MIPIAD: Multilingual Indirect Prompt Injection Attack Defense with Qwen -- TF-IDF Hybrid and Meta-Ensemble Learning
May 8, 2026Al Muhit Muhtadi, Mostafa Rifat TazwarPrompt Injection Defense Prompt Injection Attacks on LLMs
Bangladesh University of Engineering and Technology
A Sentence Relation-Based Approach to Sanitizing Malicious Instructions
May 1, 2026Soumil Datta, Melissa Umble, Daniel S. Brown +1Prompt Injection Defense RAG Security
University of Utah
CleanBase: Detecting Malicious Documents in RAG Knowledge Databases
May 1, 2026Weifei Jin, Xilong Wang, Wei Zou +2RAG Poisoning Attacks RAG Security
Duke University · The Pennsylvania State University
Latent Adversarial Detection: Adaptive Probing of LLM Activations for Multi-Turn Attack Detection
Apr 30, 2026Prashant KulkarniPrompt Injection Attacks on LLMs Adversarial Example Detection
Mountain View, CA

Oct 6, 2026, cs.CRJ/K move · Enter open · S save

RAG-PIBench: A Leakage-Aware Benchmark for Prompt-Injection Detection in Trustworthy RAG Systems

Niveen O. Jaffal, Ahmet Yuksel, David Mohaisen

Birzeit University, Birzeit, Palestine · Independent Researcher · University of Central Florida, Orlando, FL, USA

RAG Security Prompt Injection Attacks on LLMs Prompt Injection Detection

Open paper AskPDF arXiv

Retrieval-Augmented Generation (RAG) systems are vulnerable to prompt-injection attacks embedded in retrieved content. We introduce RAG-PIBench, a benchmark for RAG-style prompt-injection detection containing 4,876 contextual examples across frozen train, validation, and protected-test splits. Using a leakage-aware construction pipeline and strict evaluation protocol, we compare keyword-based, semantic-reference, TF-IDF, and transformer-based detectors. DistilBERT achieves the best protected-test performance (F1 = 0.896, PR-AUC = 0.968), while TF-IDF SVM and logistic regression remain competitive. Our results demonstrate the value of leakage-aware benchmark design and strong sparse baselines for reliable prompt-injection detection in RAG systems.

Momentum

Related topics

Latest papers 39