Depth Charge: Jailbreak Large Language Models from Deep Safety Attention Heads

Jinman Wu; Yi Xie; Shiqian Zhao; Xiaofeng Chen

cs.CRMar 6, 2026

Depth Charge: Jailbreak Large Language Models from Deep Safety Attention Heads

Authors: Jinman Wu, Yi Xie, Shiqian Zhao, Xiaofeng Chen

Organizations: Xidian University, Xi’an, China · Tsinghua University, Shenzhen, China · Nanyang Technological University, Singapore

Abstract

Currently, open-sourced large language models (OSLLMs) have demonstrated remarkable generative performance. However, as their structure and weights are made public, they are exposed to jailbreak attacks even after alignment. Existing attacks operate primarily at shallow levels, such as the prompt or embedding level, and often fail to expose vulnerabilities rooted in deeper model components, which creates a false sense of security for successful defense. In this paper, we propose Safety Attention Head Attack (SAHA), an attention-head-level jailbreak framework that explores the vulnerability in deeper but insufficiently aligned attention heads. SAHA contains two novel designs. Firstly, we reveal that deeper attention layers introduce more vulnerability against jailbreak attacks. Based on this finding, SAHA introduces Ablation-Impact Ranking head selection strategy to effectively locate the most vital layer for unsafe output. Secondly, we introduce a boundary-aware perturbation method, i.e. Layer-Wise Perturbation, to probe the generation of unsafe content with minimal perturbation to the attention. This constrained perturbation guarantees higher semantic relevance with the target intent while ensuring evasion. Extensive experiments show the superiority of our method: SAHA improves ASR by 14% over SOTA baselines, revealing the vulnerability of the attack surface on the attention head. Our code is available at https://anonymous.4open.science/r/SAHA.

Figures & tables

Figure 1: Overview of SAHA. Given a victim LLM, we derive a steering vector that captures the linear separability between benign and malicious samples. In the selection stage, AIR identifies the attention heads whose ablation most degrades safety; in the perturbation stage, LWP allocates a layer-wise budget and ranks heads within each layer. The steering vector for each chosen head is scaled by its assigned perturbation magnitude and injected into the model, yielding the final perturbed embeddings that drive the jailbreak.

Methods	Llama3.1-8B-Instruct		Qwen1.5-7B-Chat		Deepseek-llm-7B-Chat
Methods	JailbreakBench ASR $\uparrow$ / BERTScore $\uparrow$	MaliciousInstruct ASR $\uparrow$ / BERTScore $\uparrow$	JailbreakBench ASR $\uparrow$ / BERTScore $\uparrow$	MaliciousInstruct ASR $\uparrow$ /BertScore $\uparrow$	JailbreakBench ASR $\uparrow$ / BertScore $\uparrow$	MaliciousInstruct ASR $\uparrow$ / BertScore $\uparrow$
Based	0.06 / 0.62	0.01 / 0.84	0.06 / 0.74	0.03 / 0.66	0.29 / 0.64	0.10 / 0.79
Prompt-level attacks
PAIR	0.57 / 0.07	0.6 / 0.06	0.1 / 0.23	0.03 / 0.4	0.3 / 0.59	0.3 / 0.51
GCG	0.10 / 0.29	0.03 / 0.69	0.47 / 0.57	0.60 / 0.66	0.37 / 0.64	0.37 / 0.67
AutoDAN	0.40 / 0.58	0.67 / 0.66	0.57 / 0.62	0.83 / 0.64	0.60 / 0.55	0.63 / 0.70

Table 1: Comparison with baselines on JailbreakBench and MaliciousInstruct. ASR is evaluated by Llama-Guard-3-8B, and BERTScore is evaluated by SentenceBert. The best results are in bold and the second best are underlined .

Panel of Figure 2 — Figure 2: Layer-averaged perturbation magnitude $\varepsilon(\alpha)$ plotted across layers for representative $\alpha$ values.

Strategies	Llama3.1-8B-Instruct		Qwen1.5-7B-Chat		Deepseek-llm-7B-Chat
Strategies	JailbreakBench ASR $\uparrow$ / BertScore $\uparrow$	MaliciousInstruct ASR $\uparrow$ / BertScore $\uparrow$	JailbreakBench ASR $\uparrow$ / BertScore $\uparrow$	MaliciousInstruct ASR $\uparrow$ / BertScore $\uparrow$	JailbreakBench ASR $\uparrow$ / BertScore $\uparrow$	MaliciousInstruct ASR $\uparrow$ / BertScore $\uparrow$
APR+LWP	0.82 / 0.75	0.83 / 0.80	0.75 / 0.74	0.83 / 0.81	0.88 / 0.71	0.82 / 0.78
APR+GWP	0.81 / 0.75	0.83 / 0.79	0.76 / 0.74	0.83 / 0.81	0.88 / 0.71	0.77 / 0.78
AIR+GWP	0.76 / 0.69	0.62 / 0.75	0.75 / 0.69	0.85 / 0.79	0.88 / 0.70	0.80 / 0.78
AIR+LWP (SAHA)	0.85 / 0.76	0.87 / 0.84	0.82 / 0.75	0.86 / 0.82	0.91 / 0.72	0.81 / 0.79

Table 2: Ablation Study of SAHA Strategy Combinations on JailbreakBench and MaliciousInstruct.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 3 — Figure 3: The ASR tendency with $\alpha$ of AIR+LWP.

Panel of Figure 4 — Figure 4: The ASR tendency with $\alpha$ .

Panel of Figure 5 — Figure 5: Layer-wise average perturbation magnitude $\varepsilon_{\ell}(\alpha)$ plotted across layers for representative $\alpha$ values.

Panel of Figure 6 — Figure 6: Layer $\times$ $\alpha$ heatmap of average perturbation magnitudes $R_{l}(\alpha)$ on JailbreakBench. The model is Deepseek-llm-7B-Chat.

Panel of Figure 7 — Figure 7: Layer $\times$ $\alpha$ heatmap of average perturbation magnitudes $R_{l}(\alpha)$ on MaliciousInstruct. The model is Deepseek-llm-7B-Chat.

Panel of Figure 8 — Figure 8: Layer $\times$ $\alpha$ heatmap of average perturbation magnitudes $R_{l}(\alpha)$ on JailbreakBench. The model is Llama3.1-8B-Instruct.

Panel of Figure 9 — Figure 9: Layer $\times$ $\alpha$ heatmap of average perturbation magnitudes $R_{l}(\alpha)$ on MaliciousInstruct. The model is Llama3.1-8B-Instruct.

Panel of Figure 10 — Figure 10: Layer $\times$ $\alpha$ heatmap of average perturbation magnitudes $R_{l}(\alpha)$ on JailbreakBench. The model is Qwen1.5-7B-Chat.

Panel of Figure 11 — Figure 11: Layer $\times$ $\alpha$ heatmap of average perturbation magnitudes $R_{l}(\alpha)$ on MaliciousInstruct. The model is Qwen1.5-7B-Chat.

Panel of Figure 12 — Figure 12: Spatial distribution of attention head selection frequencies across transformer layers for the AIR+GWP configuration. Color intensity indicates the selection frequency across all $\alpha$ values, while red circles mark the top-10 most frequently selected safety-critical heads.

Panel of Figure 13 — Figure 13: Jaccard similarity comparison of AIR/APR.

Panel of Figure 14 — Figure 14: Jaccard similarity comparison of LWP/GWP. Minor phase-dependent differences.

Explore similar work

Robust Harmful Features Under Jailbreak Attacks: Mechanistic Evidence from Attention Head Specialization in Large Language Models
Jun 26, 2026Yanchen Yin, Dongqi Han, Linghui LiAttention Head Analysis Jailbreak Attacks
Beijing University of Posts and Telecommunications, Beijing, China.
Jailbreaking the Matrix: Nullspace Steering for Controlled Model Subversion
Apr 11, 2026Vishal Pramanik, Maisha Maliha, Susmit Jha +1Adversarial Attacks on LLMs LLM Jailbreak Attacks
Department of Computer & Information Science & Engineering, University of Florida, Gainesville, FL 32611, USA · School of Computer Science, University of Oklahoma, Norman, OK 73019, USA · Computer Science Laboratory, SRI International, Menlo Park, CA 94025, USA
Babel: Jailbreaking Safety Attention via Obfuscation Distribution Optimized Sampling
May 18, 2026Ziwei Wang, Jing Chen, Ruichao Liang +6LLM Safety LLM Jailbreak Attacks
Wuhan University · Nanyang Technological University · Southeast University +1

Jun 26, 2026J/K move · Enter open · S save

Robust Harmful Features Under Jailbreak Attacks: Mechanistic Evidence from Attention Head Specialization in Large Language Models

Yanchen Yin, Dongqi Han, Linghui Li

Beijing University of Posts and Telecommunications, Beijing, China.

Attention Head Analysis Jailbreak Attacks LLM Safety Alignment Mechanistic Interpretability Jailbreak Detection

Open paper AskPDF arXiv

Jailbreak attacks bypass LLM safety alignment, yet their mechanisms remain poorly understood. We provide evidence that attacks do not comprehensively eliminate safety features, but instead selectively suppress specific attention heads. We identify two functionally differentiated types: Adversarially Compromised Heads (ACHs) concentrated in early layers, which are suppressed under attacks, and Safety-Aligned Heads (SAHs) in mid-layers, which maintain robust activations even when attacks succeed. Ablation studies support the causal role of ACHs and the contribution of SAHs to robust activations: suppressing a small number of ACHs is sufficient to induce jailbreak-like behavior on normally refused inputs, while removing SAHs substantially weakens mid-layer safety activations. Token-level attribution further shows that ACH suppression is driven specifically by attack-template tokens, providing a mechanistic account of why attacks can bypass refusal decisions through ACH suppression while leaving internal safety signals sustained by SAHs -- a phenomenon we term Robust Harmful Features. To validate the practical significance of this robustness, we show that simply reading these persistent activations -- without any training -- yields competitive aggregate detection performance with strong adversarial robustness.

Abstract

Explore similar work

Robust Harmful Features Under Jailbreak Attacks: Mechanistic Evidence from Attention Head Specialization in Large Language Models

Jailbreaking the Matrix: Nullspace Steering for Controlled Model Subversion

Babel: Jailbreaking Safety Attention via Obfuscation Distribution Optimized Sampling