Organizations: School of Computer Science and Technology, Beijing Institute of Technology, Beijing, China · Beike Language and Intelligence, Beijing, China
With the rapid advancement of large language models (LLMs), the safety of LLMs has become a critical concern. Despite significant efforts in safety alignment, current LLMs remain vulnerable to jailbreaking attacks. However, the root causes of such vulnerabilities are still poorly understood, necessitating a rigorous investigation into jailbreak mechanisms across both academic and industrial communities. In this work, we focus on a continuation-triggered jailbreak phenomenon, whereby simply relocating a continuation-triggered instruction suffix can substantially increase jailbreak success rates. To uncover the intrinsic mechanisms of this phenomenon, we conduct a comprehensive mechanistic interpretability analysis at the level of attention heads. Through causal interventions and activation scaling, we show that this jailbreak behavior primarily arises from an inherent competition between the model's intrinsic continuation drive and the safety defenses acquired through alignment training. Furthermore, we perform a detailed behavioral analysis of the identified safety-critical attention heads, revealing notable differences in the behaviors of safety heads across different model architectures. Grounded in these mechanistic findings, we propose Head Competition Steering (HCS), a mechanistically grounded inference-time strategy that explicitly leverages the competition between safety heads and continuation heads to suppress harmful generation, and further distill its behavioral signal into a student model via knowledge distillation, achieving inference-time safety improvements without additional computational overhead.
Figures & tables
Figure 1: The procedure of the continuation-triggered jailbreak. We observe that placing the continuation-triggered instruction suffix outside the prompt boundary can trigger jailbreak behavior.
Figure 2: Changes in ASR induced by the clean prompt and jailbreak prompt across different models.
Figure 3: The framework of HCS.
Figure 4: Path patching analysis of attention heads contributing to the jailbreak behavior in LLaMA-2-7B-Chat.
Figure 5: Effects of the key attention head activation zeroing on ASR in LLaMA-2-7B-Chat.
Figure 6: Attack success rate under safety head and continuation head activation scaling in LLaMA-2-7B-Chat across three datasets (AdvBench, JailbreakBench and MaliciousInstruct), and w=1 corresponds to the baseline model without activation scaling.
Dataset
Baseline
Safety Heads
Continuation Heads
1 Head
2 Heads
3 Heads
1 Head
2 Heads
3 Heads
AdvBench
16.0
11.7
4.8
4.4
53.9
64.4
64.2
JailbreakBench
26.0
21.0
14.0
16.0
53.0
62.0
63.0
MaliciousInstruct
58.0
41.0
25.0
16.0
88.0
90.0
92.0
Table 1: The results of scaling multiple safety heads and continuation heads on ASR across three datasets on LLaMA-2-7B-Chat ( w=5 ).
Model
Dataset
w=0
w=1
w=2
w=3
w=4
w=5
LLaMA-2-7B-Chat
AdvBench (Harmful)
51.5
89.0
93.1
94.6
95.2
96.2
Alpaca (Harmless)
0.2
0.2
0.4
1.0
1.6
2.0
Qwen2.5-7B-Instruct
AdvBench (Harmful)
97.1
97.1
96.9
92.9
57.1
44.6
Alpaca (Harmless)
0.0
0.0
0.0
0.0
0.0
0.0
Table 2: Impacts of Activation Scaling coefficient ( w ) on HDR for LLaMA-2-7B-Chat and Qwen2.5-7B-Instruct under AdvBench (harmful instruction dataset) and Alpaca (harmless instruction dataset), and w=1 corresponds to the baseline model without activation scaling.
Jailbreak Resistance
General Capability
JB ↓
MI ↓
MMLU ↑
IFEval ↑
Overall ↑
Model
ASR
Δ
ASR
Δ
Acc.
Δ
Acc.
Δ
Score
Δ
LLaMA-2-7B-Chat
26.0
-
58.0
-
47.1
-
31.6
-
48.7
-
+ SFT
8.0
− 18.0
10.0
− 48.0
44.2
− 2.9
26.3
− 5.3
63.1
+ 14.4
+ HCS random
24.0
− 2.0
53.0
− 5.0
47.1
± 0.0
28.8
− 2.8
49.7
+ 1.0
+ HCS
11.0
− 15.0
10.0
− 48.0
48.1
+ 1.0
29.8
− 1.8
64.2
+ 15.5
Table 3: Results of SFT, HCS random , HCS, and KD HCS on jailbreak resistance and general capability benchmarks. JB and MI denote the JailbreakBench and MaliciousInstruct datasets. ASR metrics are lower-is-better ( ↓ ); capability metrics are higher-is-better ( ↑ ). Deltas are relative to each model’s baseline.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
α=1
α=2
α=3
α=4
JB
20.0
14.0
11.0
15.0
MI
28.0
15.0
10.0
9.0
Appendix
Table 4: Effects of different contrastive coefficients α on ASR for HCS on LLaMA-2-7B-Chat.
Dataset
α=1
α=2
α=3
α=4
JB
10.0
9.0
16.0
18.0
MI
9.0
7.0
10.0
18.0
Appendix
Table 5: Effects of different contrastive coefficients α on ASR for HCS on Qwen2.5-7B-Instruct.
Figure 7: Path patching analysis of attention heads contributing to jailbreak behavior in Qwen2.5-7B-Instruct.
Figure 8: Effects of the key attention head activation zeroing on ASR in Qwen2.5-7B-Instruct.
Figure 9: Attack success rate under safety head and continuation head activation scaling in Qwen2.5-7B-Instruct across three datasets (AdvBench, JailbreakBench and MaliciousInstruct), and w=1 corresponds to the baseline model without activation scaling.
Dataset
Baseline
Head Type
1 Head
2 Heads
3 Heads
AdvBench
16.0
Safety Heads
20.8
27.1
34.0
Continuation Heads
14.8
13.5
11.2
Random Heads
15.6
15.6
15.6
JailbreakBench
26.0
Safety Heads
28.0
33.0
38.0
Continuation Heads
22.0
21.0
20.0
Random Heads
26.0
25.0
25.0
Appendix
Table 6: Ablation results of safety heads, continuation heads and random heads on LLaMA-2-7B-Chat.
While Large Language Models (LLMs) have achieved remarkable performance, they remain vulnerable to jailbreak attacks that circumvent safety constraints. Existing strategies, ranging from heuristic prompt engineering to computationally intensive optimization, often face significant trade-offs between effectiveness and efficiency. In this work, we propose Contextual Representation Ablation (CRA), a novel inference-time intervention framework designed to dynamically silence model guardrails. Predicated on the geometric insight that refusal behaviors are mediated by specific low-rank subspaces within the model's hidden states, CRA identifies and suppresses these refusal-inducing activation patterns during decoding without requiring expensive parameter updates or training. Empirical evaluation across multiple safety-aligned open-source LLMs demonstrates that CRA significantly outperforms baselines. These results expose the intrinsic fragility of current alignment mechanisms, revealing that safety constraints can be surgically ablated from internal representations, and underscore the urgent need for more robust defenses that secure the model's latent space.
Wenpeng Xing, Moran Fang, Guangtai Wang +2
Zhejiang University · Binjiang Institute of Zhejiang University
Jailbreak attacks bypass LLM safety alignment, yet their mechanisms remain poorly understood. We provide evidence that attacks do not comprehensively eliminate safety features, but instead selectively suppress specific attention heads. We identify two functionally differentiated types: Adversarially Compromised Heads (ACHs) concentrated in early layers, which are suppressed under attacks, and Safety-Aligned Heads (SAHs) in mid-layers, which maintain robust activations even when attacks succeed. Ablation studies support the causal role of ACHs and the contribution of SAHs to robust activations: suppressing a small number of ACHs is sufficient to induce jailbreak-like behavior on normally refused inputs, while removing SAHs substantially weakens mid-layer safety activations. Token-level attribution further shows that ACH suppression is driven specifically by attack-template tokens, providing a mechanistic account of why attacks can bypass refusal decisions through ACH suppression while leaving internal safety signals sustained by SAHs -- a phenomenon we term Robust Harmful Features. To validate the practical significance of this robustness, we show that simply reading these persistent activations -- without any training -- yields competitive aggregate detection performance with strong adversarial robustness.
Yanchen Yin, Dongqi Han, Linghui Li
Beijing University of Posts and Telecommunications, Beijing, China.
Large Language Models (LLMs) have been extensively used across diverse domains, including virtual assistants, automated code generation, and scientific research. However, they remain vulnerable to jailbreak attacks, which manipulate the models into generating harmful responses despite safety alignment. Recent studies have shown that current safety-aligned LLMs undergo shallow safety alignment. In this work, we conduct an in-depth investigation into the underlying mechanism of this phenomenon and reveal that it manifests through learned ''safety trigger tokens'' that activate the model's safety patterns when paired with the specific input. Through both analysis and empirical verification, we further demonstrate the high similarity of the safety trigger tokens across different harmful inputs. Accordingly, we propose D-STT, a simple yet effective defense algorithm that identifies and explicitly decodes safety trigger tokens of the given safety-aligned LLM to activate the model's learned safety patterns. In this process, the safety trigger is constrained to a single token, which effectively preserves model usability by introducing minimum intervention in the decoding process. Extensive experiments across diverse jailbreak attacks and benign prompts demonstrate that D-STT significantly reduces output harmfulness while preserving model usability and incurring negligible response time overhead, outperforming ten baseline methods.
Haoran Gu, Handing Wang, Yi Mei +2
Xidian University · Victoria University of Wellington · Westlake University