Jailbreak Detection

Follow this topic Ask about this topic

Momentum

1 paper in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 28

All topics

Does the Unsafe Gradient Survive a Conversation? On the Fragility of Gradient-Based Jailbreak Detection in Multi-Turn Dialogue
Sep 29, 2026Omar Sheta, Rinku Deuja, Hadi Masoudi +1Language Model Safety Evaluation Adversarial Attacks on LLMs
University of Louisville Louisville, Kentucky, USA
HiveTraceGuard-Pro: A Compact Generative Guardrail for Prompt Injection, Jailbreaks, and Adversarial Obfuscation
Sep 1, 2026Nikita Oblakov, Sabrina Sadiekh, Evgeniy KokuykinLanguage Model Safety Evaluation LLM Guardrails
HiveTraceLab
Reflex-Guard: A Low-Latency Guardrail for LLM Prompt Safety Using Dense Semantic Embeddings
Aug 18, 2026Istiaque Ahmed, Afia Anjum Borsha, Ranat Das Prangon +2LLM Guardrails Adversarial Attacks on LLMs
Graduate School of Informatics, Osaka Metropolitan University, Osaka 558-8585, Japan · Dept. of Computer Science and Engineering, BRAC University, Dhaka 1212, Bangladesh · Dept. of Chemical Engineering, Bangladesh University of Engineering and Technology (BUET), Dhaka 1000, Bangladesh +1
Robust Harmful Features Under Jailbreak Attacks: Mechanistic Evidence from Attention Head Specialization in Large Language Models
Jun 26, 2026Yanchen Yin, Dongqi Han, Linghui LiAttention Head Analysis Jailbreak Attacks
Beijing University of Posts and Telecommunications, Beijing, China.
What Intermediate Layers Know: Detecting Jailbreaks from Entropy Dynamics
Jun 23, 2026Sofiia Nikolenko, Michele Papucci, Mina Rezaei +1LLM Interpretability LLM Jailbreak Attacks
LMU Munich, Munich, Germany · relAI – Konrad Zuse School of Excellence in Reliable AI · University of Pisa, Pisa, Italy +2
Scalable Hierarchical Attention Transformers for Multi-Turn Jailbreak Detection in Long Conversations
Jun 19, 2026Chenhui Hu, Muhammed Salih, Sudipto Guha +1Transformer Multi-Turn Jailbreak Attacks
Zscaler, Inc.
Rapid Poison: Practical Poisoning Attacks Against the Rapid Response Framework
Jun 15, 2026David Huang, Jaewon Chang, Avidan Shah +2LLM Backdoor Attacks Prompt Injection Attacks
Princeton University · Anthropic · UC Berkeley +2
BELLS-O: Evaluating the Operational Trade-offs of LLM Supervision Systems
Jun 12, 2026Leonhard Waibl, Felix Michalak, Hadrien MariacciaLLM Safety Benchmarks LLM Guardrails
University of Graz, Graz, Austria · Supervised Program for Alignment Research (SPAR) · Centre pour la Sécurité de l’IA (CeSIA), Paris, France.
GuardNet: Ensemble Strategies of Shallow Neural Networks for Robust Prompt Injection and Jailbreak Detection
Jun 4, 2026Paulo Ricardo Ferreira Neves, Edson Rodrigues da Cruz Filho, Paulo Henrique Eleuterio Falsetti +7LLM Guardrails Neural Network Robustness
Quickium Technology Ltd. – Piracicaba, São Paulo · Federal Institute of Education, Science and Technology of São Paulo (IFSP) – Piracicaba Campus – São Paulo · Federal University of São Carlos (UFSCar) – Sorocaba Campus – São Paulo
Gate AI: LLM Security Benchmark Evaluation Methodology and Results
Jun 1, 2026Ryle Goehausen, Marcus SousaLLM Safety Benchmarks Language Model Safety Evaluation
Opir: Efficient Multi-Task Safety Classification for Toxicity, Jailbreaks, Hate Speech, and Harmful Content
May 28, 2026Ihor Stepanov, Aleksandr SmechovLanguage Model Safety Evaluation LLM Guardrails
1Knowledgator, Kyiv, Ukraine · 2Wordcab, New York, USA
Beyond Attack Success Rate: Temporal Logit Observability for LLM Safety Failures
May 28, 2026Junyoung Park, Sunghwan Park, Seongyong Ju +1LLM Refusal Behavior LLM Jailbreak Attacks
Chung-Ang University
Evolving and Detecting Multi-Turn Deception using Geometric Signatures
May 26, 2026Surender Suresh Kumar, Mary L. CummingsAdversarial Prompt Generation Deception in Language Models
George Mason University Fairfax, USA
Jailbreak susceptibility prediction and mitigation via the behavioral geometry of models
May 26, 2026Hayden Helm, Xiaodong Liu, Weiwei YangJailbreak Robustness Jailbreak Detection
Helivan · Microsoft Research
Reflect-Guard: Enhancing LLM Safeguards against Adversarial Prompts via Logical Self-Reflection
May 24, 2026Lixing Lin, Juli You, Yue Li +4Language Model Safety Evaluation LLM Guardrails
yaleYale University, New Haven, CT, USA · indIndependent Researcher · columbiaColumbia University, New York, NY, USA +2
Detecting Fluent Optimization-Based Adversarial Prompts via Sequential Entropy Changes
May 19, 2026Mohammed Alshaalan, Miguel R. D. RodriguesAdversarial Attacks on LLMs Jailbreak Detection
Department of Electronic and Electrical Engineering, University College London, London, United Kingdom.
Exploring and Developing a Pre-Model Safeguard with Draft Models
May 19, 2026Hongyu Cai, Arjun Arunasalam, Yiming Liang +2LLM Guardrails Safety Filtering
Purdue University West Lafayette, Indiana, USA · Florida International University Miami, Florida, USA
Before the Last Token: Diagnosing Final-Token Safety Probe Failures
May 12, 2026Shravan DodaLanguage Model Safety Evaluation Safety Filtering
Kipo AI
Re-Triggering Safeguards within LLMs for Jailbreak Detection
May 11, 2026Zheng Lin, Zhenxing Niu, Haoxuan Ji +2LLM Jailbreak Attacks Jailbreak Detection
1Xidian University · 2Xi’an Jiaotong University
GLiGuard: Schema-Conditioned Classification for LLM Safeguard
May 8, 2026Urchade Zaratiana, Mary Newhauser, George Hurn-Maloney +1LLM Guardrails Text Classification
Fastino Labs
Tracing the Dynamics of Refusal: Exploiting Latent Refusal Trajectories for Robust Jailbreak Detection
May 2, 2026Xulin Hu, Che Wang, Wei Yang Bryan Lim +2Adversarial Attacks on LLMs LLM Refusal Behavior
1Peking University · 2Nanyang Technological University · 3Beijing Jiaotong University
TwinGate: Stateful Defense against Decompositional Jailbreaks in Untraceable Traffic via Asymmetric Contrastive Learning
Apr 30, 2026Bowen Sun, Chaozhuo Li, Yaodong Yang +2Jailbreak Attacks LLM Security
Johns Hopkins University · Microsoft Research Asia · Peking University +1
Cross-Lingual Jailbreak Detection via Semantic Codebooks
Apr 28, 2026Shirin Alanova, Bogdan Minko, Sabrina Sadiekh +1Multilingual Language Model Evaluation LLM Security
AI Talent Hub, ITMO University · HiveTraceLab
Layerwise Convergence Fingerprints for Runtime Misbehavior Detection in Large Language Models
Apr 27, 2026Nay Myat Min, Long H. Pham, Jun SunLLM Backdoor Attacks Adversarial Attacks on LLMs
Singapore Management University
One Jailbreak, Many Tongues: Learning Language-Insensitive Intention Representations for Multilingual Jailbreak Detection
Apr 22, 2026Shuyu Jiang, Kaiyu Xu, Xingshu Chen +5Multilingual Language Models LLM Jailbreak Attacks
School of Cyber Science and Engineering, Sichuan University, Chengdu, 610065, China · School of Computer Science and Engineering, Nanyang Technological University · School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, 611731, China
An Empirical Study of Multi-Generation Sampling for Jailbreak Detection in Large Language Models
Apr 20, 2026Hanrui Luo, Shreyank N GowdaLanguage Model Safety Evaluation LLM Auditing
School of Computer Science, University of Nottingham, Nottingham, United Kingdom.
SafeDream: Safety World Model for Proactive Early Jailbreak Detection
Apr 18, 2026Bo Yan, Weikai Lin, Yada Zhu +1Language Model Safety Evaluation World Models
University of Central Florida · University of Rochester · IBM Research
Segment-Level Coherence for Robust Harmful Intent Probing in LLMs
Apr 16, 2026Xuanli He, Bilgehan Sel, Faizan Ali +3LLM Safety LLM Jailbreak Attacks
University College London · Virginia Tech · Anthropic

Sep 29, 2026, cs.CRJ/K move · Enter open · S save

Does the Unsafe Gradient Survive a Conversation? On the Fragility of Gradient-Based Jailbreak Detection in Multi-Turn Dialogue

Omar Sheta, Rinku Deuja, Hadi Masoudi, Minghong Fang

University of Louisville Louisville, Kentucky, USA

Language Model Safety Evaluation Adversarial Attacks on LLMs Multi-Turn Dialogue Evaluation Jailbreak Detection Jailbreak Defense

Open paper AskPDF arXiv

Safety-aligned language models are commonly deployed as multi-turn assistants, which lets adversaries spread unsafe intent across several user turns instead of a single prompt. Gradient-based jailbreak detectors such as GradSafe were developed for single prompts: they score an input by the alignment between its induced gradient and a fixed unsafe reference direction, and their effectiveness in multi-turn dialogue remains unclear. We conduct a controlled evaluation of gradient-based jailbreak detection in multi-turn settings. We extend GradSafe with a Context Window Scanner that applies the detector to fixed-size windows of user turns and uses the maximum window score as the conversation-level score. We evaluate different window sizes, attack families, benign conversation distributions, and target models. The results differ sharply between synthetic and realistic benign settings. Against synthetic benign conversations, the detector achieves an ROC-AUC of 0.98 on human-authored multi-turn jailbreaks. On WildChat benign conversations, ROC-AUC drops to 0.76, and a threshold calibrated on synthetic data flags more than 90% of benign conversations as unsafe. Under realistic benign distributions, single-turn windows give the highest separability, whereas longer windows and accumulated contexts reduce performance. The detector is also sensitive to the attack-generation method and target model: successful Crescendo attacks receive scores comparable to or lower than benign conversations, and Qwen2.5-7B-Instruct yields near-random separability with a different optimal window size. These findings show that gradient-based signals can support multi-turn jailbreak detection, but reliable deployment requires calibration on realistic benign conversations, short-window scoring, length-aware thresholds, and evaluation across attack types and model architectures.

Momentum

Related topics

Latest papers 28