Jailbreak Defense

Follow this topic Ask about this topic

Jailbreak defense studies attacks that disguise, encode, or gradually reveal harmful intent, and how to protect language and multimodal models without blocking legitimate requests. Common approaches include guard classifiers and input rewriting, safety fine-tuning or unlearning, and inference-time methods such as refusal steering, hidden-state monitoring, and selective regeneration. Recent work increasingly tracks intent across conversation turns and addresses multilingual, image, and audio inputs, using benchmarks such as HarmBench, JailbreakBench, and XSTest. No single defense works across all attacks: the central challenge is balancing robustness against adaptive and shifted attacks with false refusals, capability loss, latency, and reliable evaluation on realistic benign inputs.

Latest papers

60 papers5 papers in the last four weeks, down 17% on the four weeks before. 0.0% of all new papers.

PatchBench: Measuring Collateral Damage in Activation Patching
Oct 7, 2026Alexi Canesse, Mathis Le Bail, Maël Jenny +3Jailbreak Attack Evaluation Language Model Safety Evaluation
LIX (École Polytechnique, IP Paris, CNRS), France · AMIAD (Agence Ministérielle pour l’IA de Défense), France
Reactivating Alignment: Defending LLMs from Jailbreaks via Intention-Aware Input-Output Matching
Oct 3, 2026Luoyu Chen, Weiqi Wang, Chenhan Zhang +2Jailbreak Defense LLM Safety Alignment
University of Technology Sydney, Sydney, NSW, Australia · Xi’an Jiaotong University, Xi’an, Shaanxi, China · Southeast University, Nanjing, Jiangsu, China
TRACE: Trajectory Return Attribution and Contrastive Erasure for Multi-Turn Safety
Oct 1, 2026Fengpeng Li, Kemou Li, Qizhou Wang +3LLM Fine-Tuning Adversarial Attacks on LLMs
PRADA Lab, King Abdullah University of Science and Technology · State Key Laboratory of Internet of Things for Smart City, University of Macau · Imperfect Information Learning Team, RIKEN Center for Advanced Intelligence Project +1
MOMAT: Mixture of Multiple Atlases for Low-Power Jailbreak Defense of Quantized LLMs
Oct 1, 2026Boyang Li, Bingyu Shen, Weihao Hong +6Retrieval-Augmented Generation LLM Safety
Department of Computer Science, Kean University, 1000 Morris Ave, Union, 07083, NJ, USA · Department of Computer Science and Engineering, University of Notre Dame, 257 Fitzpatrick Hall of Engineering, Notre Dame, 46556, IN, USA · McGill University, 845 Sherbrooke Street, Montréal, H3A 0G4, Québec, Canada +2
Does the Unsafe Gradient Survive a Conversation? On the Fragility of Gradient-Based Jailbreak Detection in Multi-Turn Dialogue
Sep 29, 2026Omar Sheta, Rinku Deuja, Hadi Masoudi +1Language Model Safety Evaluation Adversarial Attacks on LLMs
University of Louisville Louisville, Kentucky, USA
AEGIS: Audio Endogenous Guarding via Internal Signals Against Large Audio-Language Model Jailbreaks
Sep 24, 2026Yu-Ling Liao, Tzu-Chin Chiu, Zong-You Chen +2Audio-Language Models Jailbreak Defense
National Taiwan University
CASCADE Against Jailbreaks: Combination Across Stages with Controlled Attack-Defense Evaluation
Sep 18, 2026Jiale Luo, Eric HanJailbreak Defense LLM Jailbreak Attacks
School of Computing National University of Singapore
HE-Guardrail: A Homomorphic Guardrail Against Jailbreak Attacks for Encrypted Large Language Model Inference
Sep 18, 2026Byeongseo Min, Yongwoo Lee, Young-Sik Kim +1LLM Security Jailbreak Defense
Pohang University of Science and Technology (POSTECH) · Inha University · Daegu Gyeongbuk Institute of Science and Technology (DGIST)
Beyond Routine Compliance: Cunning Data Cultivates Safety Vigilance in Large Language Models
Sep 16, 2026Youjia Wang, Lin Xu, Yang Sun +3LLM Safety Alignment LLM Safety
Huawei Technology Inc
AlcaTRAz - Anchored Tree-Rule Defense Against Jailbreaks
Sep 3, 2026Jakub Reš, Petr Kaška, Martin Perešíni +2LLM Safety Jailbreak Defense
Brno University of Technology, Faculty of Information Technology, Czechia · Red Hat, Czechia
SEAL: Reinforcing Global Safety in Mixture-of-Experts through Shared Expert ALignment
Sep 2, 2026Qingyu Meng, Yiwei Zha, Jiahuan Pei +3LLM Safety Alignment Jailbreak Defense
Vrije Universiteit Amsterdam Amsterdam, Netherlands
A Self-Evolving Multi-Agent Framework Defense against LLM Jailbreak Attacks
Aug 26, 2026Tongyan Hu, Bryan HooiJailbreak Defense LLM Safety
National University of Singapore
Breaking the Assumptions: Auditing Input-Side Jailbreak Defenses Against Semantic Attacks
Aug 22, 2026Aaditya Pratap, Harsh Kasyap, Somanath TripathyJailbreak Defense LLM Safety Evaluation
NCE Chandi Nalanda, India · IIT (BHU) Varanasi, India · IIT Patna Patna, India
ReFrame: Evidence-Guided Test-Time Safety Alignment in Multimodal Large Language Models
Aug 21, 2026Wenzheng Jiang, Xuankun Rong, Yuanzhao Zhai +2MLLM Safety LLM Safety Alignment
College of Computer Science and Technology, National University of Defense Technology · State Key Laboratory of Complex & Critical Software Environment · School of Computer Science, Wuhan University
TRACE: Trajectory Aware Reasoning for Multi-Turn Adversarial Conversation Evaluation
Aug 16, 2026Md Messal Monem Miah, Adrita Anika, Zhiyuan Yu +1Adversarial Training LLM Jailbreak Attacks
Texas A&M University · Amazon
Tripwire: Triggering Aligned Refusal via Statistically Certified Safety Neurons
Aug 14, 2026Wei Zhao, Zhe Li, Peixin Zhang +1LLM Safety Jailbreak Defense
Singapore Management University
Dynamic Deep Prompt Optimization for Defending Against Jailbreak Attacks on LLMs
Aug 10, 2026Doniyorkhon Obidov, Honggang Yu, Xiaolong Guo +1Jailbreak Defense LLM Safety
Michigan Technological University · Miami University · Kansas State University
Depth, Not Breadth: Best-of-N Jailbreaking Beyond Surface Noise
Jul 29, 2026Haoyu Zhang, Hanwen Liu, Yang Chen +9Adversarial Attacks Jailbreak Attacks
1Northeastern University
When LLM Defenses Backfire: Characterizing Safety, Performance, and Cost Trade-offs
Jul 27, 2026Tong Zhang, Zexin Li, Simin Chen +1LLM Inference Efficiency Language Model Safety Evaluation
Fudan University · University of California, Riverside · Columbia University
Scalable Hierarchical Attention Transformers for Multi-Turn Jailbreak Detection in Long Conversations
Jun 19, 2026Chenhui Hu, Muhammed Salih, Sudipto Guha +1Transformer Multi-Turn Jailbreak Attacks
Zscaler, Inc.
SafeSpec: Fast and Safe LLM via Dynamic Reflective Sampling
Jun 18, 2026Haotian Xu, Zeyang Zhang, Linbao Li +3Speculative Decoding LLM Inference Acceleration
Zhejiang University, Hangzhou, China · Harbin Institute of Technology, Shenzhen, China · Huawei
Adaptive and Explicit safe: Triggering Latent Safety Awareness in Large Reasoning Models
Jun 15, 2026Ke Miao, Jiaxin Li, Hongliang Chen +2LLM Safety Alignment LLM Safety
The State Key Laboratory of Blockchain and Data Security, Zhejiang University · Hangzhou HighTech Zone (Binjiang) Blockchain and Data Security Research Institute, China · Li Auto Inc. +2
DoubtProbe: Black-Box Jailbreak Defense via Structural Verification and Semantic Auditing
Jun 15, 2026Xuanyu Yin, Yilin Jiang, Jun Zhou +3LLM Auditing LLM Jailbreak Attacks
East China Normal University Shanghai, China · The Hong Kong University of Science and Technology (Guangzhou) Guangzhou, China · Institute of Information Engineering, Chinese Academy of Sciences Beijing, China
NeuroArmor: Safe-Variant-Guided Representation Consistency for Selective Re-Anchoring in Jailbreak Defense
Jun 2, 2026Zhongyang Lin, Ziran Zhao, Feifei Zhai +1LLM Jailbreak Attacks Jailbreak Defense
PsychoPass: Geometric Profiling of Multi-Turn Adversarial LLM Conversations
Jun 2, 2026Muberra Ozmen, Subhabrata MajumdarAdversarial Attacks on LLMs LLM Safety
Coveo · Indian Institute of Management Bangalore
THRD: A Training-Free Multi-Turn Defense Framework for Jailbreak Attacks on Large Language Models
Jun 1, 2026Zhiqing Ma, Zhonghao Xu, Dong Yu +3LLM Safety Multi-Turn Jailbreak Attacks
1Beijing Language and Culture University · †Corresponding authors
EvoDefense: Co-Evolving Black-Box Defense with Large Language Models
May 29, 2026Yu Li, Yuenan Hou, Yingmei Wei +2Evolutionary Optimization Adversarial Attacks on LLMs
National University of Defense Technology · Shanghai AI Laboratory
Audio Jailbreaks in Large Audio-Language Models: Taxonomy, Attack-Defense Analysis, and Cost-Aware Evaluation
May 28, 2026Bo-Han Feng, Yu-Hsuan Li Liang, Chien-Feng Liu +2Audio-Language Model Evaluation Audio-Language Models
National Taiwan University
Opir: Efficient Multi-Task Safety Classification for Toxicity, Jailbreaks, Hate Speech, and Harmful Content
May 28, 2026Ihor Stepanov, Aleksandr SmechovLanguage Model Safety Evaluation LLM Guardrails
1Knowledgator, Kyiv, Ukraine · 2Wordcab, New York, USA
Disentangling Adversarial Prompts: A Semantic-Graph Defense for Robust LLM Security
May 27, 2026Xiang Fang, Wanlong FangLLM Security Prompt Injection Defense
School of Software Engineering, Huazhong University of Science and Technology · Nanyang Technological University, Singapore

Oct 7, 2026J/K move · Enter open · S save

PatchBench: Measuring Collateral Damage in Activation Patching

Alexi Canesse, Mathis Le Bail, Maël Jenny, Clément Elliker+2

LIX (École Polytechnique, IP Paris, CNRS), France · AMIAD (Agence Ministérielle pour l’IA de Défense), France

Jailbreak Attack Evaluation Language Model Safety Evaluation Jailbreak Defense Jailbreak Attacks LLM Jailbreak Attacks Activation Steering

Open paper AskPDF arXiv

An LLM safety patch can pass a benchmark while still being a poor repair. This risk is especially acute for jailbreak repairs, where the goal is to correct a specific unsafe behaviour without changing unrelated behaviours. A patch may block exact evaluation prompts yet fail on close harmful variants, or suppress harmful behaviour by over-refusing benign prompts that share its wording or structure. Existing protocols primarily test whether models can be broken, while aggregate metrics (attack success, refusal rates, global capability) cannot distinguish selective repairs from broader local suppression. To address this gap, we introduce PatchBench, a benchmark of empirically observed model-specific jailbreak failures inducing actionable harmful answers. Starting from 27,870 prompts from 37 public datasets, we curate 15,314 English prompts and query 8 open-source instruction-tuned models. Combining WildGuard filtering, pairwise Elo ranking, and manual verification, we retain a curated bank of 400 high-confidence jailbreak failures. We further introduce PatchBench-Local, an evaluation protocol testing whether a patch is behaviourally precise. For each harmful source prompt, PatchBench-Local generates three families of local neighbours: harmful variants preserving malicious intent, benign prompts with matched structure, and benign prompts reusing key harmful terms. It evaluates harmful-neighbour correction and benign-neighbour preservation, distinguishing selective repair from broader local suppression. Evaluating four activation steering methods with PatchBench-Local and MMLU shows that global capability can remain nearly unchanged while local benign regressions are severe, confirming aggregate metrics miss important collateral damage. PatchBench-Local provides a more precise basis for developing and comparing jailbreak repair methods.