Jailbreak Defense

Follow this topic Ask about this topic

Momentum

5 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 43

All topics

PatchBench: Measuring Collateral Damage in Activation Patching
Oct 7, 2026Alexi Canesse, Mathis Le Bail, Maël Jenny +3Language Model Safety Evaluation Jailbreak Defense
LIX (École Polytechnique, IP Paris, CNRS), France · AMIAD (Agence Ministérielle pour l’IA de Défense), France
TRACE: Trajectory Return Attribution and Contrastive Erasure for Multi-Turn Safety
Oct 1, 2026Fengpeng Li, Kemou Li, Qizhou Wang +3LLM Fine-Tuning Adversarial Attacks on LLMs
PRADA Lab, King Abdullah University of Science and Technology · State Key Laboratory of Internet of Things for Smart City, University of Macau · Imperfect Information Learning Team, RIKEN Center for Advanced Intelligence Project +1
MOMAT: Mixture of Multiple Atlases for Low-Power Jailbreak Defense of Quantized LLMs
Oct 1, 2026Boyang Li, Bingyu Shen, Weihao Hong +6Retrieval-Augmented Generation LLM Safety
Department of Computer Science, Kean University, 1000 Morris Ave, Union, 07083, NJ, USA · Department of Computer Science and Engineering, University of Notre Dame, 257 Fitzpatrick Hall of Engineering, Notre Dame, 46556, IN, USA · McGill University, 845 Sherbrooke Street, Montréal, H3A 0G4, Québec, Canada +2
Does the Unsafe Gradient Survive a Conversation? On the Fragility of Gradient-Based Jailbreak Detection in Multi-Turn Dialogue
Sep 29, 2026Omar Sheta, Rinku Deuja, Hadi Masoudi +1Language Model Safety Evaluation Adversarial Attacks on LLMs
University of Louisville Louisville, Kentucky, USA
AEGIS: Audio Endogenous Guarding via Internal Signals Against Large Audio-Language Model Jailbreaks
Sep 24, 2026Yu-Ling Liao, Tzu-Chin Chiu, Zong-You Chen +2Audio-Language Models Jailbreak Defense
National Taiwan University
Beyond Routine Compliance: Cunning Data Cultivates Safety Vigilance in Large Language Models
Sep 16, 2026Youjia Wang, Lin Xu, Yang Sun +3LLM Safety Alignment LLM Safety
Huawei Technology Inc
SEAL: Reinforcing Global Safety in Mixture-of-Experts through Shared Expert ALignment
Sep 2, 2026Qingyu Meng, Yiwei Zha, Jiahuan Pei +3LLM Safety Alignment Jailbreak Defense
Vrije Universiteit Amsterdam Amsterdam, Netherlands
TRACE: Trajectory Aware Reasoning for Multi-Turn Adversarial Conversation Evaluation
Aug 16, 2026Md Messal Monem Miah, Adrita Anika, Zhiyuan Yu +1Adversarial Training LLM Jailbreak Attacks
Texas A&M University · Amazon
Borrowed Strength: Best-of-N Search over a Code EncodingBreaks Self-Check Jailbreak Defenses
Jul 29, 2026Haoyu Zhang, Shibo Zheng, Xiangchen Guan +4Adversarial Attacks Jailbreak Attacks
1Northeastern University
When LLM Defenses Backfire: Characterizing Safety, Performance, and Cost Trade-offs
Jul 27, 2026Tong Zhang, Zexin Li, Simin Chen +1LLM Inference Efficiency Language Model Safety Evaluation
Fudan University · University of California, Riverside · Columbia University
Scalable Hierarchical Attention Transformers for Multi-Turn Jailbreak Detection in Long Conversations
Jun 19, 2026Chenhui Hu, Muhammed Salih, Sudipto Guha +1Transformer Multi-Turn Jailbreak Attacks
Zscaler, Inc.
SafeSpec: Fast and Safe LLM via Dynamic Reflective Sampling
Jun 18, 2026Haotian Xu, Zeyang Zhang, Linbao Li +3Speculative Decoding LLM Inference Acceleration
Zhejiang University, Hangzhou, China · Harbin Institute of Technology, Shenzhen, China · Huawei
Adaptive and Explicit safe: Triggering Latent Safety Awareness in Large Reasoning Models
Jun 15, 2026Ke Miao, Jiaxin Li, Hongliang Chen +2LLM Safety Alignment LLM Safety
The State Key Laboratory of Blockchain and Data Security, Zhejiang University · Hangzhou HighTech Zone (Binjiang) Blockchain and Data Security Research Institute, China · Li Auto Inc. +2
DoubtProbe: Black-Box Jailbreak Defense via Structural Verification and Semantic Auditing
Jun 15, 2026Xuanyu Yin, Yilin Jiang, Jun Zhou +3LLM Auditing LLM Jailbreak Attacks
East China Normal University Shanghai, China · The Hong Kong University of Science and Technology (Guangzhou) Guangzhou, China · Institute of Information Engineering, Chinese Academy of Sciences Beijing, China
NeuroArmor: Safe-Variant-Guided Representation Consistency for Selective Re-Anchoring in Jailbreak Defense
Jun 2, 2026Zhongyang Lin, Ziran Zhao, Feifei Zhai +1LLM Jailbreak Attacks Jailbreak Defense
PsychoPass: Geometric Profiling of Multi-Turn Adversarial LLM Conversations
Jun 2, 2026Muberra Ozmen, Subhabrata MajumdarAdversarial Attacks on LLMs LLM Safety
Coveo · Indian Institute of Management Bangalore
THRD: A Training-Free Multi-Turn Defense Framework for Jailbreak Attacks on Large Language Models
Jun 1, 2026Zhiqing Ma, Zhonghao Xu, Dong Yu +3LLM Safety Multi-Turn Jailbreak Attacks
1Beijing Language and Culture University · †Corresponding authors
EvoDefense: Co-Evolving Black-Box Defense with Large Language Models
May 29, 2026Yu Li, Yuenan Hou, Yingmei Wei +2Evolutionary Optimization Adversarial Attacks on LLMs
National University of Defense Technology · Shanghai AI Laboratory
Audio Jailbreaks in Large Audio-Language Models: Taxonomy, Attack-Defense Analysis, and Cost-Aware Evaluation
May 28, 2026Bo-Han Feng, Yu-Hsuan Li Liang, Chien-Feng Liu +2Audio-Language Model Evaluation Audio-Language Models
National Taiwan University
Opir: Efficient Multi-Task Safety Classification for Toxicity, Jailbreaks, Hate Speech, and Harmful Content
May 28, 2026Ihor Stepanov, Aleksandr SmechovLanguage Model Safety Evaluation LLM Guardrails
1Knowledgator, Kyiv, Ukraine · 2Wordcab, New York, USA
Disentangling Adversarial Prompts: A Semantic-Graph Defense for Robust LLM Security
May 27, 2026Xiang Fang, Wanlong FangLLM Security Prompt Injection Defense
School of Software Engineering, Huazhong University of Science and Technology · Nanyang Technological University, Singapore
Evolving and Detecting Multi-Turn Deception using Geometric Signatures
May 26, 2026Surender Suresh Kumar, Mary L. CummingsAdversarial Prompt Generation Deception in Language Models
George Mason University Fairfax, USA
Jailbreak susceptibility prediction and mitigation via the behavioral geometry of models
May 26, 2026Hayden Helm, Xiaodong Liu, Weiwei YangJailbreak Robustness Jailbreak Detection
Helivan · Microsoft Research
Reflect-Guard: Enhancing LLM Safeguards against Adversarial Prompts via Logical Self-Reflection
May 24, 2026Lixing Lin, Juli You, Yue Li +4Language Model Safety Evaluation LLM Guardrails
yaleYale University, New Haven, CT, USA · indIndependent Researcher · columbiaColumbia University, New York, NY, USA +2
Steering Beyond the Support: Adversarial Training on Unsupervised Jailbroken Activation Simulation
May 23, 2026Luoyu Chen, Weiqi Wang, Zhiyi Tian +5Adversarial Training Adversarial Robustness of Language Models
University of Technology Sydney, Sydney, Australia · Xi’an Jiaotong University, Xi’an, China · School of Cyber Science and Engineering, Southeast University, Nanjing, China.
Exploring and Developing a Pre-Model Safeguard with Draft Models
May 19, 2026Hongyu Cai, Arjun Arunasalam, Yiming Liang +2LLM Guardrails Safety Filtering
Purdue University West Lafayette, Indiana, USA · Florida International University Miami, Florida, USA
Adaptive Steering and Remasking for Safe Generation in Diffusion Language Models
May 13, 2026Yejin Lee, Ungsik Kim, Yo-Sub HanAdversarial Attacks on LLMs Diffusion Language Model Inference
Department of Computer Science Yonsei University Seoul, Republic of Korea
Re-Triggering Safeguards within LLMs for Jailbreak Detection
May 11, 2026Zheng Lin, Zhenxing Niu, Haoxuan Ji +2LLM Jailbreak Attacks Jailbreak Detection
1Xidian University · 2Xi’an Jiaotong University
GLiGuard: Schema-Conditioned Classification for LLM Safeguard
May 8, 2026Urchade Zaratiana, Mary Newhauser, George Hurn-Maloney +1LLM Guardrails Text Classification
Fastino Labs
Mitigating Many-shot Jailbreak Attacks with One Single Demonstration
May 8, 2026Kejia Chen, Jiawen Zhang, Boheng Li +6Jailbreak Attacks In-Context Learning
Zhejiang University · Nanyang Technological University · University of Science and Technology of China +2

Oct 7, 2026, cs.LGJ/K move · Enter open · S save

PatchBench: Measuring Collateral Damage in Activation Patching

Alexi Canesse, Mathis Le Bail, Maël Jenny, Clément Elliker+2

LIX (École Polytechnique, IP Paris, CNRS), France · AMIAD (Agence Ministérielle pour l’IA de Défense), France

Language Model Safety Evaluation Jailbreak Defense Jailbreak Attacks LLM Jailbreak Attacks Activation Steering

Open paper AskPDF arXiv

An LLM safety patch can pass a benchmark while still being a poor repair. This risk is especially acute for jailbreak repairs, where the goal is to correct a specific unsafe behaviour without changing unrelated behaviours. A patch may block exact evaluation prompts yet fail on close harmful variants, or suppress harmful behaviour by over-refusing benign prompts that share its wording or structure. Existing protocols primarily test whether models can be broken, while aggregate metrics (attack success, refusal rates, global capability) cannot distinguish selective repairs from broader local suppression. To address this gap, we introduce PatchBench, a benchmark of empirically observed model-specific jailbreak failures inducing actionable harmful answers. Starting from 27,870 prompts from 37 public datasets, we curate 15,314 English prompts and query 8 open-source instruction-tuned models. Combining WildGuard filtering, pairwise Elo ranking, and manual verification, we retain a curated bank of 400 high-confidence jailbreak failures. We further introduce PatchBench-Local, an evaluation protocol testing whether a patch is behaviourally precise. For each harmful source prompt, PatchBench-Local generates three families of local neighbours: harmful variants preserving malicious intent, benign prompts with matched structure, and benign prompts reusing key harmful terms. It evaluates harmful-neighbour correction and benign-neighbour preservation, distinguishing selective repair from broader local suppression. Evaluating four activation steering methods with PatchBench-Local and MMLU shows that global capability can remain nearly unchanged while local benign regressions are severe, confirming aggregate metrics miss important collateral damage. PatchBench-Local provides a more precise basis for developing and comparing jailbreak repair methods.

Momentum

Related topics

Latest papers 43