Jailbreak Robustness

Follow this topic Ask about this topic

Jailbreak robustness examines whether language models and agents can resist adversarial prompts, including adaptive attacks, transfer attacks, and best-of-N sampling, without losing helpful capabilities. Defenses range from refusal training and prompt smoothing to representation-based interventions, while evaluations use attack benchmarks, paired baseline tests, and checks that harmful answers are valid and operationally useful. Work also studies robustness across model families, system-prompt states, merged models, and agent pipeline components, since a safe-looking response or default configuration may hide vulnerabilities elsewhere. A central tension is how to measure and improve resistance across changing attacks and operating conditions without sacrificing legitimate performance, and whether results from static tests predict real-world safety.

Latest papers

11 papers1 paper in the last four weeks, down 80% on the four weeks before. 0.0% of all new papers.

MLCommons Jailbreak Benchmark v1.0
Oct 2, 2026Carsten Maple, Cagatay Yucel, Isaac Holeman +35Jailbreak Attack Evaluation LLM Safety Benchmarks
Victor · MLCommons · University of Warwick +18
Escaping Alignment: A Physical Trap Model of Best-of-N Jailbreaking
Sep 26, 2026Marco BiroliJailbreak Attacks LLM Jailbreak Attacks
Physical Science Division Chicago, IL 60637, USA
SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Consideration
Sep 11, 2026Md Jueal Mia, Yanzhao Wu, Selcuk Uluagac +1Language Model Safety Evaluation Jailbreak Robustness
Knight Foundation School of Computing and Information Sciences, Florida International University, USA · Security, Optimization, and Learning for InterDependent Networks (solid) Laboratory, Miami, Florida, USA
FUSE: An Evaluating Framework for Dangerous Capabilities of LLMs
Sep 2, 2026Zhengyi Jin, Ru Zhang, Xiao Chen +5LLM Safety Benchmarks Jailbreak Robustness
School of Cyberspace Security, Beijing University of Posts and Telecommunications, China
Validity-Aware Jailbreak Evaluation for Large Language Models
Aug 31, 2026Qilong Wu, Sahil Wadhwa, Pranab Mohanty +2Language Model Safety Evaluation LLM Auditing
University of Illinois Urbana-Champaign · Capital One
The Fragility of Jailbreak Robustness Across Operational States
Aug 31, 2026Yuna Park, Hwang Youn Kim, Yujin Kim +3Prompt Sensitivity Language Model Safety Evaluation
Yonsei University · Korea Institute of Science and Technology (KIST) · Korea University +1
When Skills Meet Safety: Benchmarking and Characterizing the Adaptive Jailbreak Robustness of Skill-Merged LLMs
Aug 9, 2026Yu Ma, Hongli Shi, Jing Li +2LLM Safety Benchmarks Language Model Safety Evaluation
Google · University of New South Wales · University of Technology Sydney +2
Jailbreak susceptibility prediction and mitigation via the behavioral geometry of models
May 26, 2026Hayden Helm, Xiaodong Liu, Weiwei YangJailbreak Robustness Jailbreak Detection
Helivan · Microsoft Research
Guaranteed Jailbreaking Defense via Disrupt-and-Rectify Smoothing
May 11, 2026Zheng Lin, Zhenxing Niu, Haoxuan Ji +1Randomized Smoothing Jailbreak Robustness
Xidian University · Xi’an Jiaotong University
Jailbroken Frontier Models Retain Their Capabilities
Apr 30, 2026Daniel Zhu, Zihan Wang, Xuchan Bao +1Language Model Safety Evaluation Jailbreak Attacks
Anthropic
Towards Understanding the Robustness of Sparse Autoencoders
Apr 20, 2026Ahson Saiyed, Sabrina Sadiekh, Chirag AgarwalAdversarial Attacks on LLMs Jailbreak Robustness
University of Virginia · Independent Researcher

Oct 2, 2026J/K move · Enter open · S save

MLCommons Jailbreak Benchmark v1.0

Carsten Maple, Cagatay Yucel, Isaac Holeman, Chris Knotz+34

Victor · MLCommons · University of Warwick+18

Jailbreak Attack Evaluation LLM Safety Benchmarks Language Model Safety Evaluation Jailbreak Attacks Jailbreak Robustness LLM Jailbreak Attacks

Open paper AskPDF arXiv

Modern AI systems are designed to refuse hazardous requests. A jailbreak is a prompt crafted to bypass those safeguards and elicit outputs that the system would normally refuse to provide. The MLCommons Jailbreak Benchmark v1.0 provides an end-to-end methodology for evaluating the robustness of large language models to single-turn, text-based jailbreak attacks. It combines criteria-driven system and attack selection, paired baseline and adversarial evaluation, human annotation, automated evaluator calibration, scoring, grading, and risk-calibrated disclosure within a single benchmarking pipeline. The benchmark evaluates eight open-weight systems using 264 seed prompts spanning eleven hazard categories and representative attacks drawn from the MLCommons Jailbreak Taxonomy. Responses are assessed using the AILuminate Assessment Standard v1.4, and robustness is measured through the Resilience Gap: the change in safety performance between baseline and adversarial conditions. Across all evaluated systems and attacks, the unsafe-response rate increased from 11.08% under baseline conditions to 18.65% under jailbreak conditions, producing an average Resilience Gap of 7.57%. Accessible systems showed a larger mean gap, while attack effectiveness varied substantially across attack categories and hazards. The benchmark also examines evaluator reliability and sources of measurement error. Beyond reporting results, Jailbreak Benchmark v1.0 establishes a reproducible methodological foundation for comparative jailbreak evaluation and for future expansion across systems, attacks, hazards, and evaluation methods.