Depth Charge: Jailbreak Large Language Models from Deep Safety Attention Heads

Jinman Wu; Yi Xie; Shiqian Zhao; Xiaofeng Chen

cs.CRMar 6, 2026

Depth Charge: Jailbreak Large Language Models from Deep Safety Attention Heads

Authors: Jinman Wu, Yi Xie, Shiqian Zhao, Xiaofeng Chen

Organizations: Xidian University, Xi’an, China · Tsinghua University, Shenzhen, China · Nanyang Technological University, Singapore

Abstract

Currently, open-sourced large language models (OSLLMs) have demonstrated remarkable generative performance. However, as their structure and weights are made public, they are exposed to jailbreak attacks even after alignment. Existing attacks operate primarily at shallow levels, such as the prompt or embedding level, and often fail to expose vulnerabilities rooted in deeper model components, which creates a false sense of security for successful defense. In this paper, we propose Safety Attention Head Attack (SAHA), an attention-head-level jailbreak framework that explores the vulnerability in deeper but insufficiently aligned attention heads. SAHA contains two novel designs. Firstly, we reveal that deeper attention layers introduce more vulnerability against jailbreak attacks. Based on this finding, SAHA introduces Ablation-Impact Ranking head selection strategy to effectively locate the most vital layer for unsafe output. Secondly, we introduce a boundary-aware perturbation method, i.e. Layer-Wise Perturbation, to probe the generation of unsafe content with minimal perturbation to the attention. This constrained perturbation guarantees higher semantic relevance with the target intent while ensuring evasion. Extensive experiments show the superiority of our method: SAHA improves ASR by 14% over SOTA baselines, revealing the vulnerability of the attack surface on the attention head. Our code is available at https://anonymous.4open.science/r/SAHA.

Figures & tables

Figure 1: Overview of SAHA. Given a victim LLM, we derive a steering vector that captures the linear separability between benign and malicious samples. In the selection stage, AIR identifies the attention heads whose ablation most degrades safety; in the perturbation stage, LWP allocates a layer-wise budget and ranks heads within each layer. The steering vector for each chosen head is scaled by its assigned perturbation magnitude and injected into the model, yielding the final perturbed embeddings that drive the jailbreak.

Methods	Llama3.1-8B-Instruct		Qwen1.5-7B-Chat		Deepseek-llm-7B-Chat
Methods	JailbreakBench ASR $\uparrow$ / BERTScore $\uparrow$	MaliciousInstruct ASR $\uparrow$ / BERTScore $\uparrow$	JailbreakBench ASR $\uparrow$ / BERTScore $\uparrow$	MaliciousInstruct ASR $\uparrow$ /BertScore $\uparrow$	JailbreakBench ASR $\uparrow$ / BertScore $\uparrow$	MaliciousInstruct ASR $\uparrow$ / BertScore $\uparrow$
Based	0.06 / 0.62	0.01 / 0.84	0.06 / 0.74	0.03 / 0.66	0.29 / 0.64	0.10 / 0.79
Prompt-level attacks
PAIR	0.57 / 0.07	0.6 / 0.06	0.1 / 0.23	0.03 / 0.4	0.3 / 0.59	0.3 / 0.51
GCG	0.10 / 0.29	0.03 / 0.69	0.47 / 0.57	0.60 / 0.66	0.37 / 0.64	0.37 / 0.67
AutoDAN	0.40 / 0.58	0.67 / 0.66	0.57 / 0.62	0.83 / 0.64	0.60 / 0.55	0.63 / 0.70

Table 1: Comparison with baselines on JailbreakBench and MaliciousInstruct. ASR is evaluated by Llama-Guard-3-8B, and BERTScore is evaluated by SentenceBert. The best results are in bold and the second best are underlined .

Panel of Figure 2 — Figure 2: Layer-averaged perturbation magnitude $\varepsilon(\alpha)$ plotted across layers for representative $\alpha$ values.

Strategies	Llama3.1-8B-Instruct		Qwen1.5-7B-Chat		Deepseek-llm-7B-Chat
Strategies	JailbreakBench ASR $\uparrow$ / BertScore $\uparrow$	MaliciousInstruct ASR $\uparrow$ / BertScore $\uparrow$	JailbreakBench ASR $\uparrow$ / BertScore $\uparrow$	MaliciousInstruct ASR $\uparrow$ / BertScore $\uparrow$	JailbreakBench ASR $\uparrow$ / BertScore $\uparrow$	MaliciousInstruct ASR $\uparrow$ / BertScore $\uparrow$
APR+LWP	0.82 / 0.75	0.83 / 0.80	0.75 / 0.74	0.83 / 0.81	0.88 / 0.71	0.82 / 0.78
APR+GWP	0.81 / 0.75	0.83 / 0.79	0.76 / 0.74	0.83 / 0.81	0.88 / 0.71	0.77 / 0.78
AIR+GWP	0.76 / 0.69	0.62 / 0.75	0.75 / 0.69	0.85 / 0.79	0.88 / 0.70	0.80 / 0.78
AIR+LWP (SAHA)	0.85 / 0.76	0.87 / 0.84	0.82 / 0.75	0.86 / 0.82	0.91 / 0.72	0.81 / 0.79

Table 2: Ablation Study of SAHA Strategy Combinations on JailbreakBench and MaliciousInstruct.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 3 — Figure 3: The ASR tendency with $\alpha$ of AIR+LWP.

Panel of Figure 4 — Figure 4: The ASR tendency with $\alpha$ .

Panel of Figure 5 — Figure 5: Layer-wise average perturbation magnitude $\varepsilon_{\ell}(\alpha)$ plotted across layers for representative $\alpha$ values.

Panel of Figure 6 — Figure 6: Layer $\times$ $\alpha$ heatmap of average perturbation magnitudes $R_{l}(\alpha)$ on JailbreakBench. The model is Deepseek-llm-7B-Chat.

Panel of Figure 7 — Figure 7: Layer $\times$ $\alpha$ heatmap of average perturbation magnitudes $R_{l}(\alpha)$ on MaliciousInstruct. The model is Deepseek-llm-7B-Chat.

Panel of Figure 8 — Figure 8: Layer $\times$ $\alpha$ heatmap of average perturbation magnitudes $R_{l}(\alpha)$ on JailbreakBench. The model is Llama3.1-8B-Instruct.

Panel of Figure 9 — Figure 9: Layer $\times$ $\alpha$ heatmap of average perturbation magnitudes $R_{l}(\alpha)$ on MaliciousInstruct. The model is Llama3.1-8B-Instruct.

Panel of Figure 10 — Figure 10: Layer $\times$ $\alpha$ heatmap of average perturbation magnitudes $R_{l}(\alpha)$ on JailbreakBench. The model is Qwen1.5-7B-Chat.

Panel of Figure 11 — Figure 11: Layer $\times$ $\alpha$ heatmap of average perturbation magnitudes $R_{l}(\alpha)$ on MaliciousInstruct. The model is Qwen1.5-7B-Chat.

Panel of Figure 12 — Figure 12: Spatial distribution of attention head selection frequencies across transformer layers for the AIR+GWP configuration. Color intensity indicates the selection frequency across all $\alpha$ values, while red circles mark the top-10 most frequently selected safety-critical heads.

Panel of Figure 13 — Figure 13: Jaccard similarity comparison of AIR/APR.

Panel of Figure 14 — Figure 14: Jaccard similarity comparison of LWP/GWP. Minor phase-dependent differences.

Explore similar work

Jun 26, 2026

Robust Harmful Features Under Jailbreak Attacks: Mechanistic Evidence from Attention Head Specialization in Large Language Models

Jailbreak attacks bypass LLM safety alignment, yet their mechanisms remain poorly understood. We provide evidence that attacks do not comprehensively eliminate safety features, but instead selectively suppress specific attention heads. We identify two functionally differentiated types: Adversarially Compromised Heads (ACHs) concentrated in early layers, which are suppressed under attacks, and Safety-Aligned Heads (SAHs) in mid-layers, which maintain robust activations even when attacks succeed. Ablation studies support the causal role of ACHs and the contribution of SAHs to robust activations: suppressing a small number of ACHs is sufficient to induce jailbreak-like behavior on normally refused inputs, while removing SAHs substantially weakens mid-layer safety activations. Token-level attribution further shows that ACH suppression is driven specifically by attack-template tokens, providing a mechanistic account of why attacks can bypass refusal decisions through ACH suppression while leaving internal safety signals sustained by SAHs -- a phenomenon we term Robust Harmful Features. To validate the practical significance of this robustness, we show that simply reading these persistent activations -- without any training -- yields competitive aggregate detection performance with strong adversarial robustness.

Yanchen Yin, Dongqi Han, Linghui Li

Beijing University of Posts and Telecommunications, Beijing, China.

Attention Head Analysis Jailbreak Attacks LLM Safety Alignment Mechanistic Interpretability Jailbreak Detection

Ask PDF

Apr 11, 2026

Jailbreaking the Matrix: Nullspace Steering for Controlled Model Subversion

Large language models remain vulnerable to jailbreak attacks -- inputs designed to bypass safety mechanisms and elicit harmful responses -- despite advances in alignment and instruction tuning. We propose Head-Masked Nullspace Steering (HMNS), a circuit-level intervention that (i) identifies attention heads most causally responsible for a model's default behavior, (ii) suppresses their write paths via targeted column masking, and (iii) injects a perturbation constrained to the orthogonal complement of the muted subspace. HMNS operates in a closed-loop detection-intervention cycle, re-identifying causal heads and reapplying interventions across multiple decoding attempts. Across multiple jailbreak benchmarks, strong safety defenses, and widely used language models, HMNS attains state-of-the-art attack success rates with fewer queries than prior methods. Ablations confirm that nullspace-constrained injection, residual norm scaling, and iterative re-identification are key to its effectiveness. To our knowledge, this is the first jailbreak method to leverage geometry-aware, interpretability-informed interventions, highlighting a new paradigm for controlled model steering and adversarial safety circumvention.

Vishal Pramanik, Maisha Maliha, Susmit Jha +1

Department of Computer & Information Science & Engineering, University of Florida, Gainesville, FL 32611, USA · School of Computer Science, University of Oklahoma, Norman, OK 73019, USA · Computer Science Laboratory, SRI International, Menlo Park, CA 94025, USA

Adversarial Attacks on LLMs LLM Jailbreak Attacks LLM Safety Activation Steering Circuits in Language Models

Ask PDF

May 18, 2026

Babel: Jailbreaking Safety Attention via Obfuscation Distribution Optimized Sampling

Despite rigorous safety alignment, Large Language Models (LLMs) remain vulnerable to jailbreak attacks. Existing black-box methods often rely on heuristic templates or exhaustive trials, lacking mechanistic interpretability and query efficiency. In this study, we investigate an intrinsic vulnerability in the safety mechanisms of LLMs, where safety alignment relies on a small set of sparsely distributed attention heads, leaving much of the representational space weakly monitored. We formalize this phenomenon with a mathematical jailbreaking model that characterizes the delicate boundary of effective text obfuscation and analytically explains observed jailbreak behaviors. Guided by this model, we propose Babel, an efficient black-box attack framework that exploits the identified safety gap through systematic obfuscation sampling with iterative, feedback-driven distribution refinement, enabling reliable and high-success jailbreak attacks without access to model internals. Comprehensive evaluations on frontier commercial models demonstrate that Babel achieves state-of-the-art attack success rates and superior query efficiency. Specifically, compared to state-of-the-art methods, Babel increases the attack success rate on GPT-4o from 41.33% to 82.67% and on Claude-3-5-haiku from 38.33% to 78.33% within an average of 40 queries, providing a robust red-teaming methodology for LLMs safety research.

Ziwei Wang, Jing Chen, Ruichao Liang +6

Wuhan University · Nanyang Technological University · Southeast University +1

LLM Safety LLM Jailbreak Attacks Attention Mechanisms Black-Box Adversarial Attacks Adversarial Text Attacks

Ask PDF