Jailbreak Evaluations

Follow this field Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

2 new papers

A weekly snapshot of new work published in Jailbreak Evaluations.

Period ending 2026-09-14

6 new papers

A weekly snapshot of new work published in Jailbreak Evaluations.

Period ending 2026-09-07

6 new papers

A weekly snapshot of new work published in Jailbreak Evaluations.

Inside this field

Focused directions

Focused topic

Large Language Model Jailbreaks

Within Jailbreak Evaluations

64 papersExplore

Focused topic

Jailbreak Evaluations

Within Jailbreak Evaluations

61 papersExplore

Focused topic

Jailbreaks

Within Jailbreak Evaluations

57 papersExplore

Focused topic

Jailbreak Attacks

Within Jailbreak Evaluations

43 papersExplore

Focused topic

Jailbreak Detection

Within Jailbreak Evaluations

30 papersExplore

Focused topic

Novel Jailbreak Method

Within Jailbreak Evaluations

21 papersExplore

Focused topic

Adversarial Jailbreaks

Within Jailbreak Evaluations

20 papersExplore

184 papers

Latest in Jailbreak Evaluations

Browse all topics →

BELLS-O: Evaluating the Operational Trade-offs of LLM Supervision Systems
Jun 12, 2026Leonhard Waibl, Felix Michalak, Hadrien MariacciaJailbreak Detection First Comprehensive Benchmark
FreoStream:Enhancing Stream Guardrails via Future-Aware Reasoning and Safety-Aligned Optimization
Jun 11, 2026Jianwei Wang, Guoyang Shen, Yanhong Wu +5Streaming Guardrails Large Language Model Safety
Grammar-Constrained Decoding Can Jailbreak LLMs into Generating Malicious Code
Jun 10, 2026Yitong Zhang, Shiteng Lu, Jia LiLarge Language Model Jailbreaks Jailbreak Evaluations
JailbreakOPT: Tool-Assisted Iterative Jailbreak Prompt Optimization
Jun 9, 2026Ge Shi, Jun Yin, Donglin Xie +3Jailbreak Evaluations Novel Jailbreak Method
Context-Fractured Decomposition Attacks on Tool-Using LLM Agents: Exploiting Artifact Provenance Gaps
Jun 8, 2026Xiaofeng Lin, Yukai Yang, Daniel Guo +3Novel Jailbreak Method Jailbreak Detection
MLingualFC: Evaluating Jailbreak Vulnerabilities in Multilingual Vision-Language Models
Jun 5, 2026Rishabh Makwana, Mamta, Deeksha Varshney +1Large Language Model Jailbreaks Multilingual Safety
SpeechJBB: Probing Safety Alignment and Comprehension in Large Audio Language Models under Code-Switched Speech
Jun 4, 2026Virginia Ceccatelli, Yejin Jeon, David Ifeoluwa AdelaniLarge Language Model Jailbreaks Large Audio Language Models
Membrane: A Self-Evolving Contrastive Safety Memory for LLM Agent Defense
Jun 4, 2026Minseok Choi, Seungbin Yang, Dongjin Kim +5Large Language Model Safety Jailbreak Attacks
Safety Paradox: How Enhanced Safety Awareness Leaves LLMs Vulnerable to Posterior Attack
Jun 4, 2026Long P. Hoang, Hai V. Le, Shaoyang Xu +2Large Language Model Safety Safety Alignment
SlotGCG: Exploiting the Positional Vulnerability in LLMs for Jailbreak Attacks
Jun 4, 2026Seungwon Jeong, Jiwoo Jeong, Hyeonjin Kim +2Large Language Model Jailbreaks Jailbreak Attacks
GuardNet: Ensemble Strategies of Shallow Neural Networks for Robust Prompt Injection and Jailbreak Detection
Jun 4, 2026Paulo Ricardo Ferreira Neves, Edson Rodrigues da Cruz Filho, Paulo Henrique Eleuterio Falsetti +7Indirect Prompt Injection Jailbreak Detection
Voices Across Registers: Corpus-Conditioned Vernacular Jailbreaks against Aligned LLMs via Fanfiction Subgenres
Jun 3, 2026Zhongze Luo, Ruihe Shi, Zhenshuai Yin +3Jailbreak Evaluations Jailbreak Detection
Black-box, Adaptive, Efficient, Transferable, Harmful, Applicable... Attacks Are All You Need to Break LLMs
Jun 2, 2026Vincent Limbach, Jonas Dornbusch, David Lüdke +2Large Language Model Jailbreaks Black Box
PsychoPass: Geometric Profiling of Multi-Turn Adversarial LLM Conversations
Jun 2, 2026Muberra Ozmen, Subhabrata MajumdarLanguage Model Evasion Attacks Jailbreak Attacks
Patcher: Post-Hoc Patching of Backdoored Large Language Models
Jun 2, 2026Anjun Gao, Yueyang Quan, Yufei Xia +2Large Language Model Jailbreaks Adversarial Jailbreaks
MaskForge: Structure-Aware Adaptive Attacks for Jailbreaking Diffusion Large Language Models
Jun 1, 2026Yingzi Ma, Zhengyue Zhao, Xiaogeng Liu +3Large Language Model Jailbreaks Large Language Model Baselines
Jailbreaking Multimodal Large Language Models using Multi-Clip Video
Jun 1, 2026Choongwon Kang, Seungjong Sun, Hyunmin Jun +1Large Language Model Jailbreaks Video Multimodal Large Language Models
THRD: A Training-Free Multi-Turn Defense Framework for Jailbreak Attacks on Large Language Models
Jun 1, 2026Zhiqing Ma, Zhonghao Xu, Dong Yu +3Jailbreak Attacks Novel Jailbreak Method
TukaBench: A Culturally Grounded Jailbreak Benchmark for African Languages
May 31, 2026Victor Akinode, Senyu Li, Wassim Hamidouche +3Large Language Model Safety African Languages
D-Judge: Disrupting Multi-Turn Jailbreaks using Semantics-Preserving Output Rewriting
May 31, 2026Huanli Gong, Zhipeng Wei, Yu Fu +4Jailbreak Attacks Jailbreak Detection
MultiTurnPSB: Evaluating Multi-Turn Jailbreak Attacks an dClassifier-Based Defenses for Medical AI Safety
May 30, 2026Anushka Sheoran, Yiduo HaoJailbreak Attacks
Persona Attack: Incremental Memory Injection Jailbreak Attack against Large Language Models
May 29, 2026Junyoung Park, Seongyong Ju, Sunghwan Park +1Large Language Model Jailbreaks Adversarial Jailbreaks
Audio Jailbreaks in Large Audio-Language Models: Taxonomy, Attack-Defense Analysis, and Cost-Aware Evaluation
May 28, 2026Bo-Han Feng, Yu-Hsuan Li Liang, Chien-Feng Liu +2Large Language Model Jailbreaks Large Audio Language Models
Opir: Efficient Multi-Task Safety Classification for Toxicity, Jailbreaks, Hate Speech, and Harmful Content
May 28, 2026Ihor Stepanov, Aleksandr SmechovLarge Language Model Safety Guardrail
Beyond Attack Success Rate: Temporal Logit Observability for LLM Safety Failures
May 28, 2026Junyoung Park, Sunghwan Park, Seongyong Ju +1Higher Attack Success Rates Large Language Model Safety
Mitigating Adaptive Attacks against Reasoning Models with Activation Consistency Training
May 27, 2026Avidan Shah, Jannik Brinkmann, Rico AngellLarge Reasoning Models Adversarial Jailbreaks
When Think-with-Image Meets Safety: What Determines Multimodal Jailbreak Robustness?
May 27, 2026Yuan Tian, Bing Hu, Fang Wu +3Jailbreak Evaluations Jailbreaks
PAST2HARM: A Simple Adaptive Past Tense Attack for Jailbreaking Multimodal AI
May 26, 2026Snehasis MukhopadhyayLarge Language Model Jailbreaks Harms
BAIT: Boundary-Guided Disclosure Escalation LLM Jailbreaking via Self-Conditioned Reasoning
May 26, 2026Xuan Luo, Yue Wang, Geng Tu +2Jailbreak Evaluations Higher Attack Success Rates
Open-Weight LLM Fine-Tuning Defenses are Susceptible to Simple Attacks
May 26, 2026Kevin Kuo, Chhavi Yadav, Virginia SmithLanguage Model Evasion Attacks Adversarial Jailbreaks
Jailbreak susceptibility prediction and mitigation via the behavioral geometry of models
May 26, 2026Hayden Helm, Xiaodong Liu, Weiwei YangJailbreak Detection Jailbreak Attacks
Curriculum Learning for Safety Alignment
May 25, 2026Sandeep Kumar, Virginia Smith, Chhavi YadavSafety Alignment Large Language Model Alignment
Localization then Neutralization: Gradient-guided Token Suppression against Visual Prompt Injection Attack
May 24, 2026Dongpeng Zhang, Ke Ma, Yangbangyan Jiang +4Adversarial Examples Jailbreak Attacks
Reflect-Guard: Enhancing LLM Safeguards against Adversarial Prompts via Logical Self-Reflection
May 24, 2026Lixing Lin, Juli You, Yue Li +4Large Language Model Safety Guard
Jailbreak to Protect: Buffering and Reinforcing via Temporary Jailbreaking for Safe Fine-Tuning in Large Language Models
May 23, 2026Seokil Ham, Jaehyuk Jang, Wonjun Lee +1Harmful Fine-Tuning Jailbreak Evaluations
Steering Beyond the Support: Adversarial Training on Unsupervised Jailbroken Activation Simulation
May 23, 2026Luoyu Chen, Weiqi Wang, Zhiyi Tian +5Novel Jailbreak Method Jailbreak Detection
Reasoning as an Attack Surface: Adaptive Evolutionary CoT Jailbreaks for LLMs
May 23, 2026Jianan Li, Simeng Qin, Xiaojun Jia +5Large Language Model Jailbreaks Jailbreak Evaluations
Same Model, Different Weakness: How Language and Modality Reshape the Jailbreak Attack Surface in Frontier MLLMs
May 22, 2026Casey Ford, Madison Van Doren, Sicheng Jin +1Large Language Model Jailbreaks Multilingual Safety
On-Policy Consistency Training Improves LLM Safety with Minimal Capability Degradation
May 20, 2026Andy Han, Kristina Fujimoto, Avidan Shah +5Safety Alignment Large Language Model Safety
How Far Will They Go? Red-Teaming Online Influence with Large Language Models
May 20, 2026Daniel C. Ruiz, Anna Serbina, Ashwin Rao +2Social Influence Social Media
LASH: Adaptive Semantic Hybridization for Black-Box Jailbreaking of Large Language Models
May 20, 2026Abdullah Al Nomaan Nafi, Fnu Suya, Swarup Bhunia +1Large Language Model Jailbreaks Adversarial Jailbreaks
REFLECTOR: Internalizing Step-wise Reflection against Indirect Jailbreak
May 20, 2026Jiachen Ma, Jiawen Zhang, Xiangtian Li +3Jailbreak Attacks Novel Jailbreak Method
Attention-Guided Reward for Reinforcement Learning-based Jailbreak against Large Reasoning Models
May 19, 2026Zheng Lin, Zhenxing Niu, Haoxuan Ji +2Large Language Model Jailbreaks Large Reasoning Models
Adaptive Probe-based Steering for Robust LLM Jailbreaking
May 19, 2026Junxi Chen, Junhao Dong, Xiaohua XieLarge Language Model Jailbreaks Activation Steering
The Evaluation Game: Beyond Static LLM Benchmarking
May 19, 2026Paul Wang, Jade Garcia-Bourrée, Anne-Marie Kermarrec +1Adversarial Evaluation Benchmarking Large Language Models
Multilingual jailbreaking of LLMs using low-resource languages
May 18, 2026Dylan Marx, Marcel DunaiskiLarge Language Model Jailbreaks Multilingual Safety
Acoustic Interference: A New Paradigm Weaponizing Acoustic Latent Semantic for Universal Jailbreak against Large Audio Language Models
May 18, 2026Yanyun Wang, Yu Huang, Zi Liang +2Large Audio Language Models Large Language Model Safety
Babel: Jailbreaking Safety Attention via Obfuscation Distribution Optimized Sampling
May 18, 2026Ziwei Wang, Jing Chen, Ruichao Liang +6Large Language Model Jailbreaks Jailbreak Evaluations
DMN: A Compositional Framework for Jailbreaking Multimodal LLMs with Multi-Image Inputs
May 18, 2026Wenzhuo Xu, Zhipeng Wei, Zonghao Ying +4Large Language Model Jailbreaks Jailbreak Evaluations
Why Do Safety Guardrails Degrade Across Languages?
May 16, 2026Max Zhang, Ameen Patel, Sang T. Truong +1Multilingual Safety Large Language Model Safety
New Wide-Net-Casting Jailbreak Attacks Risk Large Models
May 16, 2026Qiuchi Xiang, Haoxuan Qu, Hossein Rahmani +1Adversarial Jailbreaks Jailbreak Attacks
EVA: Editing for Versatile Alignment against Jailbreaks
May 14, 2026Yi Wang, Hongye Qiu, Yue Xu +4Large Language Model Jailbreaks Jailbreak Attacks
The Great Pretender: A Stochasticity Problem in LLM Jailbreak
May 14, 2026Jean-Philippe Monteuuis, Cong Chen, Jonathan PetitLarge Language Model Jailbreaks Code Evaluation
Quantifying LLM Safety Degradation Under Repeated Attacks Using Survival Analysis
May 13, 2026Zvi TopolLarge Language Model Safety Large Language Model Jailbreaks
Before the Last Token: Diagnosing Final-Token Safety Probe Failures
May 12, 2026Shravan DodaJailbreak Evaluations Safety
Metaphor Is Not All Attention Needs
May 12, 2026Olga Sorokoletova, Francesco Giarrusso, Giacomo De Luca +6Large Language Model Jailbreaks Jailbreaks
Break the Brake, Not the Wheel: Untargeted Jailbreak via Entropy Maximization
May 11, 2026Mengqi He, Xinyu Tian, Xin Shen +6Jailbreak Evaluations Jailbreak Detection
Re-Triggering Safeguards within LLMs for Jailbreak Detection
May 11, 2026Zheng Lin, Zhenxing Niu, Haoxuan Ji +2Jailbreak Detection Jailbreaks
Guaranteed Jailbreaking Defense via Disrupt-and-Rectify Smoothing
May 11, 2026Zheng Lin, Zhenxing Niu, Haoxuan Ji +1Large Language Model Jailbreaks Adversarial Jailbreaks
Metis: Learning to Jailbreak LLMs via Self-Evolving Metacognitive Policy Optimization
May 11, 2026Huilin Zhou, Jian Zhao, Yilu Zhong +7Large Language Model Jailbreaks Metacognition

Jun 12, 2026 · cs.CRJ/K move · Enter open · S save

BELLS-O: Evaluating the Operational Trade-offs of LLM Supervision Systems

Leonhard Waibl, Felix Michalak, Hadrien Mariaccia

University of Graz, Graz, Austria · Supervised Program for Alignment Research (SPAR) · Centre pour la Sécurité de l’IA (CeSIA), Paris, France.

LLM supervision systems, namely input/output moderation filters and jailbreak detectors, are the primary safeguard against misuse in deployed AI applications, yet existing benchmarks are often vendor-biased, omit cost and latency, and rarely compare specialized guardrails against repurposed generalist LLMs. We present BELLS-O (Benchmark for the Evaluation of LLM Supervision Systems, Operational), the first independent operational benchmark of LLM supervision systems. BELLS-O evaluates 28 systems from 17 providers: every major specialized guardrail (e.g., LlamaGuard-4, ShieldGemma-2, Lakera Guard) and frontier generalists repurposed as supervisors (e.g., GPT-5.4, Claude Sonnet 4.6, Grok-4.1), jointly on detection rate, false-positive rate, latency, and monetary cost. We cover input/output moderation across 11 harm categories and jailbreak detection across 13 attack techniques, using in-house datasets built from handcrafted prompts, expert-curated samples, and quality-controlled synthetic generation. To suppress latent generator fingerprints in synthetic data, every generated sample is paraphrased. Mapping the Pareto frontier reveals use-case-dependent tradeoffs. On content moderation, specialized supervisors are operationally dominant: top systems match frontier LLMs on detection (~95% vs. 94%) at comparably low false-positive rates (<=2%), while running 5-10x faster and ~10x cheaper. On jailbreak detection, the tradeoff shifts: frontier LLMs achieve higher detection and lower false-positive rates but at 10-50x higher cost and 5-10x higher latency. We release the benchmark, framework, leaderboard, and datasets as the first vendor-neutral basis for selecting safeguards under real deployment constraints.

Jailbreak Detection First Comprehensive Benchmark Trade-Off Fingerprint

Open paper Ask about it PDF