The Struggle Between Continuation and Refusal: A Mechanistic Analysis of the Continuation-Triggered Jailbreak in LLMs

Yonghong Deng; Zhen Yang; Ping Jian; Xinyue Zhang; Zhongbin Guo; Chengzhi Li; Junxi Yin

cs.AIMar 9, 2026

The Struggle Between Continuation and Refusal: A Mechanistic Analysis of the Continuation-Triggered Jailbreak in LLMs

Authors: Yonghong Deng, Zhen Yang, Ping Jian, Xinyue Zhang, Zhongbin Guo, Chengzhi Li, Junxi Yin

Organizations: School of Computer Science and Technology, Beijing Institute of Technology, Beijing, China · Beike Language and Intelligence, Beijing, China

Abstract

With the rapid advancement of large language models (LLMs), the safety of LLMs has become a critical concern. Despite significant efforts in safety alignment, current LLMs remain vulnerable to jailbreaking attacks. However, the root causes of such vulnerabilities are still poorly understood, necessitating a rigorous investigation into jailbreak mechanisms across both academic and industrial communities. In this work, we focus on a continuation-triggered jailbreak phenomenon, whereby simply relocating a continuation-triggered instruction suffix can substantially increase jailbreak success rates. To uncover the intrinsic mechanisms of this phenomenon, we conduct a comprehensive mechanistic interpretability analysis at the level of attention heads. Through causal interventions and activation scaling, we show that this jailbreak behavior primarily arises from an inherent competition between the model's intrinsic continuation drive and the safety defenses acquired through alignment training. Furthermore, we perform a detailed behavioral analysis of the identified safety-critical attention heads, revealing notable differences in the behaviors of safety heads across different model architectures. Grounded in these mechanistic findings, we propose Head Competition Steering (HCS), a mechanistically grounded inference-time strategy that explicitly leverages the competition between safety heads and continuation heads to suppress harmful generation, and further distill its behavioral signal into a student model via knowledge distillation, achieving inference-time safety improvements without additional computational overhead.

Figures & tables

Figure 1: The procedure of the continuation-triggered jailbreak. We observe that placing the continuation-triggered instruction suffix outside the prompt boundary can trigger jailbreak behavior.

Figure 2: Changes in ASR induced by the clean prompt and jailbreak prompt across different models.

Figure 4: Path patching analysis of attention heads contributing to the jailbreak behavior in LLaMA-2-7B-Chat.

Figure 5: Effects of the key attention head activation zeroing on ASR in LLaMA-2-7B-Chat.

Panel of Figure 6 — Figure 6: Attack success rate under safety head and continuation head activation scaling in LLaMA-2-7B-Chat across three datasets (AdvBench, JailbreakBench and MaliciousInstruct), and $w=1$ corresponds to the baseline model without activation scaling.

Dataset	Baseline	Safety Heads			Continuation Heads
Dataset	Baseline	1 Head	2 Heads	3 Heads	1 Head	2 Heads	3 Heads
AdvBench	16.0	11.7	4.8	4.4	53.9	64.4	64.2
JailbreakBench	26.0	21.0	14.0	16.0	53.0	62.0	63.0
MaliciousInstruct	58.0	41.0	25.0	16.0	88.0	90.0	92.0

Table 1: The results of scaling multiple safety heads and continuation heads on ASR across three datasets on LLaMA-2-7B-Chat (

w=5

Model	Dataset	$w=0$	$w=1$	$w=2$	$w=3$	$w=4$	$w=5$
LLaMA-2-7B-Chat	AdvBench (Harmful)	51.5	89.0	93.1	94.6	95.2	96.2
LLaMA-2-7B-Chat	Alpaca (Harmless)	0.2	0.2	0.4	1.0	1.6	2.0
Qwen2.5-7B-Instruct	AdvBench (Harmful)	97.1	97.1	96.9	92.9	57.1	44.6
Qwen2.5-7B-Instruct	Alpaca (Harmless)	0.0	0.0	0.0	0.0	0.0	0.0

Table 2: Impacts of Activation Scaling coefficient (

w

) on HDR for LLaMA-2-7B-Chat and Qwen2.5-7B-Instruct under AdvBench (harmful instruction dataset) and Alpaca (harmless instruction dataset), and

w=1

corresponds to the baseline model without activation scaling.

	Jailbreak Resistance				General Capability
	JB $\downarrow$		MI $\downarrow$		MMLU $\uparrow$		IFEval $\uparrow$		Overall $\uparrow$
Model	ASR	$\mathbf{\Delta}$	ASR	$\mathbf{\Delta}$	Acc.	$\mathbf{\Delta}$	Acc.	$\mathbf{\Delta}$	Score	$\mathbf{\Delta}$
LLaMA-2-7B-Chat	26.0	-	58.0	-	47.1	-	31.6	-	48.7	-
+ SFT	8.0	$-$ 18.0	10.0	$-$ 48.0	44.2	$-$ 2.9	26.3	$-$ 5.3	63.1	$+$ 14.4
+ HCS ${}_{\text{random}}$	24.0	$-$ 2.0	53.0	$-$ 5.0	47.1	$\pm$ 0.0	28.8	$-$ 2.8	49.7	$+$ 1.0
+ HCS	11.0	$-$ 15.0	10.0	$-$ 48.0	48.1	$+$ 1.0	29.8	$-$ 1.8	64.2	$+$ 15.5

Table 3: Results of SFT, HCS

{}_{\text{random}}

, HCS, and KD

{}_{\text{HCS}}

on jailbreak resistance and general capability benchmarks. JB and MI denote the JailbreakBench and MaliciousInstruct datasets. ASR metrics are lower-is-better (

\downarrow

); capability metrics are higher-is-better (

\uparrow

). Deltas are relative to each model’s baseline.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	$\alpha=1$	$\alpha=2$	$\alpha=3$	$\alpha=4$
JB	20.0	14.0	11.0	15.0
MI	28.0	15.0	10.0	9.0

Appendix

Table 4: Effects of different contrastive coefficients

\alpha

on ASR for HCS on LLaMA-2-7B-Chat.

Dataset	$\alpha=1$	$\alpha=2$	$\alpha=3$	$\alpha=4$
JB	10.0	9.0	16.0	18.0
MI	9.0	7.0	10.0	18.0

Appendix

Table 5: Effects of different contrastive coefficients

\alpha

on ASR for HCS on Qwen2.5-7B-Instruct.

Figure 7: Path patching analysis of attention heads contributing to jailbreak behavior in Qwen2.5-7B-Instruct.

Figure 8: Effects of the key attention head activation zeroing on ASR in Qwen2.5-7B-Instruct.

Panel of Figure 9 — Figure 9: Attack success rate under safety head and continuation head activation scaling in Qwen2.5-7B-Instruct across three datasets (AdvBench, JailbreakBench and MaliciousInstruct), and $w=1$ corresponds to the baseline model without activation scaling.

Dataset	Baseline	Head Type	1 Head	2 Heads	3 Heads
AdvBench	16.0	Safety Heads	20.8	27.1	34.0
		Continuation Heads	14.8	13.5	11.2
		Random Heads	15.6	15.6	15.6
JailbreakBench	26.0	Safety Heads	28.0	33.0	38.0
		Continuation Heads	22.0	21.0	20.0
		Random Heads	26.0	25.0	25.0

Appendix

Table 6: Ablation results of safety heads, continuation heads and random heads on LLaMA-2-7B-Chat.

LLM Jailbreak Attacks

Large language model (LLM) jailbreak attacks study how adversarial inputs bypass safety rules and elicit disallowed outputs.

See all 203 papers

Silencing the Guardrails: Inference-Time Jailbreaking via Dynamic Contextual Representation Ablation
Apr 9, 2026Wenpeng Xing, Moran Fang, Guangtai Wang +2Adversarial Attacks on LLMs Jailbreak Attacks
Zhejiang University · Binjiang Institute of Zhejiang University
Robust Harmful Features Under Jailbreak Attacks: Mechanistic Evidence from Attention Head Specialization in Large Language Models
Jun 26, 2026Yanchen Yin, Dongqi Han, Linghui LiAttention Head Analysis Jailbreak Attacks
Beijing University of Posts and Telecommunications, Beijing, China.
Decoding One Safety Trigger Token for Balancing Safety and Usability in Large Language Models
May 12, 2025Haoran Gu, Handing Wang, Yi Mei +2LLM Safety Alignment LLM Safety
Xidian University · Victoria University of Wellington · Westlake University

Apr 9, 2026J/K move · Enter open · S save

Silencing the Guardrails: Inference-Time Jailbreaking via Dynamic Contextual Representation Ablation

Wenpeng Xing, Moran Fang, Guangtai Wang, Changting Lin+1

Zhejiang University · Binjiang Institute of Zhejiang University

Adversarial Attacks on LLMs Jailbreak Attacks LLM Guardrails LLM Safety Causal Interventions in Language Models

Open paper AskPDF arXiv

While Large Language Models (LLMs) have achieved remarkable performance, they remain vulnerable to jailbreak attacks that circumvent safety constraints. Existing strategies, ranging from heuristic prompt engineering to computationally intensive optimization, often face significant trade-offs between effectiveness and efficiency. In this work, we propose Contextual Representation Ablation (CRA), a novel inference-time intervention framework designed to dynamically silence model guardrails. Predicated on the geometric insight that refusal behaviors are mediated by specific low-rank subspaces within the model's hidden states, CRA identifies and suppresses these refusal-inducing activation patterns during decoding without requiring expensive parameter updates or training. Empirical evaluation across multiple safety-aligned open-source LLMs demonstrates that CRA significantly outperforms baselines. These results expose the intrinsic fragility of current alignment mechanisms, revealing that safety constraints can be surgically ablated from internal representations, and underscore the urgent need for more robust defenses that secure the model's latent space.

Abstract

LLM Jailbreak Attacks

Explore similar work

Silencing the Guardrails: Inference-Time Jailbreaking via Dynamic Contextual Representation Ablation

Robust Harmful Features Under Jailbreak Attacks: Mechanistic Evidence from Attention Head Specialization in Large Language Models

Decoding One Safety Trigger Token for Balancing Safety and Usability in Large Language Models