Conflicts Make Large Reasoning Models Vulnerable to Attacks

Honghao Liu; Chengjin Xu; Xuhui Jiang; Cehao Yang; Shengming Yin; Zhengwu Ma; Lionel Ni; Jian Guo

cs.CRApr 10, 2026

Conflicts Make Large Reasoning Models Vulnerable to Attacks

Authors: Honghao Liu, Chengjin Xu, Xuhui Jiang, Cehao Yang, Shengming Yin, Zhengwu Ma, Lionel Ni, Jian Guo

Organizations: International Digital Economy Academy · The Hong Kong University of Science and Technology (Guangzhou) · DataArc Tech · The City University of Hong Kong

Abstract

Large Reasoning Models (LRMs) have achieved remarkable performance across diverse domains, yet their decision-making under conflicting objectives remains insufficiently understood. This work investigates how LRMs respond to harmful queries when confronted with two categories of conflicts: internal conflicts that pit alignment values against each other and dilemmas, which impose mutually contradictory choices, including sacrificial, duress, agent-centered, and social forms. Using over 1,300 prompts across five benchmarks, we evaluate three representative LRMs - Llama-3.1-Nemotron-8B, QwQ-32B, and DeepSeek R1 - and find that conflicts significantly increase attack success rates, even under single-round non-narrative queries without sophisticated auto-attack techniques. Our findings reveal through layerwise and neuron-level analyses that safety-related and functional representations shift and overlap under conflict, interfering with safety-aligned behavior. This study highlights the need for deeper alignment strategies to ensure the robustness and trustworthiness of next-generation reasoning models. Our code is available at https://github.com/DataArcTech/ConflictHarm. Warning: This paper contains inappropriate, offensive and harmful content.

Figures & tables

Figure 1: The illustration of conflict injection for investigation the vulnerability of LRMs.

Figure 2: Overview of our approach: Left , the overall framework for conflict injection to jailbreak language models. Right , layerwise and neuron-level analyses of model internal states.

Model	Conflict	AdvBench		HarmBench		HarmfulQ		JBBench		StrongReject
Model	Conflict	ASR	$\Delta$	ASR	$\Delta$	ASR	$\Delta$	ASR	$\Delta$	ASR	$\Delta$
QwQ	direct_q	0.04		0.235		0.015		0.13		0.06
	inner	0.492	0.452	0.42	0.185	0.33	0.315	0.41	0.28	0.305	0.245
	dilemma	0.417	0.377	0.365	0.13	0.395	0.38	0.42	0.29	0.44	0.38
Llama-N	direct_q	0.375		0.545		0.025		0.45		0.396
	inner	0.442	0.067	0.59	0.045	0.065	0.04	0.45	0	0.469	0.073

Table 1: Attack success rates (ASR) of QwQ and Llama-Nemotron under direct queries, internal conflicts, and dilemmas across five safety benchmarks.

\Delta

indicates the increase in ASR relative to direct query.

Conflict	AdvBench		HarmBench		HarmfulQ		JailBreakBench		StrongReject
Conflict	ASR	$\Delta$	ASR	$\Delta$	ASR	$\Delta$	ASR	$\Delta$	ASR	$\Delta$
direct_q	0.04		0.235		0.015		0.13		0.06
agent-centered	0.455	0.415	0.465	0.230	0.403	0.388	0.438	0.308	0.491	0.431
duress	0.345	0.305	0.335	0.100	0.210	0.195	0.293	0.163	0.375	0.315
sacrificial	0.520	0.480	0.458	0.223	0.390	0.375	0.427	0.297	0.498	0.438
social	0.390	0.350	0.388	0.153	0.278	0.263	0.348	0.218	0.397	0.337

Table 2: The effect of single conflict on QwQ (average of ASR on 10 samples). The bold values are the highest ASR, the textit is the lowest ASR, and the underlined values are the second highest ASR.

Model	Conflict	HarmfulQ
		ASR	$\Delta$
DS-R1	direct_q	0
	inner	0.12	0.12
	dilemma	0.18	0.18

Table 3: ASR of DeepSeek-R1 under direct queries, internal conflicts and dilemmas on HarmfulQ.

Type	AB	HB	HQ	JBB	SR
Remove	0.042	0.19	0.015	0.14	0.1
Dilemma	0.417	0.42	0.395	0.42	0.44

Table 4: Controlled ablation on prompt formatting.

Figure 3: Layerwise cosine similarity between malicious-only prompts (M–M pairs) and conflict-augmented malicious prompts (M–D pairs). Larger gaps indicate stronger representational shifts.

Figure 4: PCA projections of neuron activation patterns across representative layer groups in QwQ-32B.

Layer ID	3	33	43	53	63
FDR	16.1	0.13	0.07	0.0059	1.41
ED	7.58	7.50	7.15	0.73	204

Table 5: Quantitative measures on PCA interpretation.

Model	direct_q	inner	dilemma
STAR1-8B	$7\cdot 10^{-4}$	$5\cdot 10^{-3}$	$2\cdot 10^{-3}$
STAR1-1.5B	$2\cdot 10^{-2}$	$3\cdot 10^{-2}$	$1.3\cdot 10^{-2}$
RealSafe	$7\cdot 10^{-4}$	$1\cdot 10^{-2}$	$1.5\cdot 10^{-3}$

Table 6: Weighted average ASRs on more rigorous safety aligned models across five benchmarks.

Figure 5: Laywerwise comparison between STAR1-R1 and Llama-Nemotron-8B.

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

Name	Value
Max new tokens	32,769
Temperature	0.6
Top-p	0.95
Padding	True
Truncation	True

Appendix

Table 7: Inference parameter settings.

Benchmark	#Query	Subset
AdvBench	520	N/A
HarmBench	200	Standard
HarmfulQ	200	N/A
JailBreakBench	100	behaviors/harmful
StrongReject	313	N/A

Appendix

Table 8: Benchmark information.

Model	Conflict	AdvBench		HarmBench		HarmfulQ		JBBench		StrongReject
Model	Conflict	ASR	$\Delta$	ASR	$\Delta$	ASR	$\Delta$	ASR	$\Delta$	ASR	$\Delta$
Llama-N	direct_q	0.375		0.545		0.025		0.45		0.396
	inner	0.382	0.007	0.54	-0.005	0.105	0.08	0.41	-0.04	0.424	0.028
	dilemma	0.569	0.194	0.58	0.035	0.335	0.31	0.49	0.04	0.517	0.121

Appendix

Table 9: Attack success rates (ASR) of Llama-Nemotron-8B under direct queries, internal conflicts and dilemmas across five safety benchmarks with new prompts.

Model	direct_q	inner	dilemma
QwQ	0.0769	0.406	0.411
Llama-N	0.358	0.414	0.484

Appendix

Table 10: Weighted average across five benchmarks.

	Agent-centered	Duress	Sacrificial	Social	AvN	HvH	HvP	SvC
Average	0.477	0.326	0.479	0.371	0.32	0.481	0.426	0.432

Appendix

Table 11: Weighted average across five benchmarks in single conflict effect experiment.

Conflict	AdvBench	HarmBench	HarmfulQ	JailBreakBench	StrongReject
Conflict	Variance
agent-centered	$4.22\cdot 10^{-4}$	$1.24\cdot 10^{-3}$	$7.63\cdot 10^{-4}$	$1.07\cdot 10^{-4}$	$1.03\cdot 10^{-3}$
duress	$1.3\cdot 10^{-4}$	$5.12\cdot 10^{-4}$	$3.22\cdot 10^{-4}$	$9.8\cdot 10^{-4}$	$1.98\cdot 10^{-4}$
sacrificial	$7.23\cdot 10^{-4}$	$2.08\cdot 10^{-4}$	$8.24\cdot 10^{-4}$	$1.55\cdot 10^{-3}$	$4.02\cdot 10^{-4}$
social	$3.17\cdot 10^{-4}$	$6.95\cdot 10^{-4}$	$3.7\cdot 10^{-4}$	$1.34\cdot 10^{-3}$	$4.78\cdot 10^{-4}$
avn	$1.35\cdot 10^{-4}$	$3.13\cdot 10^{-4}$	$9.4\cdot 10^{-4}$	$7.8\cdot 10^{-4}$	$3.25\cdot 10^{-4}$

Appendix

Table 12: The variance of single conflict on QwQ with 10 samples for each query.

Figure 6: The error bars of single conflict on QwQ with 10 samples for each query.

Figure 7: Cumulative effect of conflicts on QWQ.

Figure 8: Neuron-level activation patterns for dilemma-augmented and direct malicious queries across early, middle, and late layers.

Figure 9: Neuron-level PCA projections with alternative layer samples across groups.

Figure 10: Neuron-level activation patterns visualized using t-SNE.

Model	direct_q	all conflicts
DeepSeek-R1	0.02	0.19

Appendix

Table 13: The ASR on DeepSeek-R1 evaluated using Qwen3Guard as the safety judge.

Panel of Figure 11 — Figure 11: Prompt template V1.

Panel of Figure 12 — Figure 12: Prompt template V2.

Panel of Figure 13 — Figure 13: Ablation prompt for disentangling the effect of coercive formatting.

Panel of Figure 14 — Figure 14: Prompts of internal conflict items.

Panel of Figure 15 — Figure 15: Prompts of dilemma items.

Figure 16: An example of harmful response on QwQ-32B by injecting internal conflicts (harmful content is redacted).

Figure 17: An example of harmful response on QwQ-32B by injecting dilemmas (harmful content is redacted).

Figure 18: An example of harmful response on QwQ 32B by injecting single sacrificial dilemma (harmful content is redacted).

Figure 19: An example of harmful response on Llama-Nemotron 8B by injecting dilemmas (harmful content is redacted).

Figure 20: An example of DeepSeek R1 with conflicts.

Explore similar work

First Token Matters: Understanding Safety Collapse in Large Reasoning Models
Sep 16, 2026Yizheng Yang, Haining Yu, Yuechen Wang +4LLM Safety Alignment Inference-Time Intervention
Harbin Institute of Technology, Harbin, China · Guangzhou University, Guangzhou, China · City University of Macau, Macau, China
Towards Mitigating Deceptive Safety Alignment in Large Reasoning Models
Sep 28, 2026Xiangyu Zhou, Saleh Zare Zade, Rafi Ibn Sultan +2Language Model Safety Evaluation Deception in Language Models
Wayne State University
Reasoning Structure Matters for Safety Alignment of Reasoning Models
Apr 21, 2026Yeonjun In, Wonjoong Kim, Sangwu Park +1LLM Safety Alignment Large Reasoning Models
KAIST

Sep 16, 2026J/K move · Enter open · S save

First Token Matters: Understanding Safety Collapse in Large Reasoning Models

Yizheng Yang, Haining Yu, Yuechen Wang, Yikai Hou+3

Harbin Institute of Technology, Harbin, China · Guangzhou University, Guangzhou, China · City University of Macau, Macau, China

LLM Safety Alignment Inference-Time Intervention Refusal Behavior in Language Models Large Reasoning Models

Open paper AskPDF arXiv

Large Reasoning Models (LRMs) exhibit strong problem-solving abilities, yet their safety alignment often degrades when handling harmful queries. Existing approaches to improving safety largely rely on additional training or preference optimization, while offering limited understanding of the internal mechanisms behind safety failures. In this work, we investigate this failure through a token-level positional analysis of refusal dynamics and identify a localized vulnerability at the onset of reasoning, which we term Onset Refusal Collapse (ORC). We find that the refusal-related signal of LRMs drops sharply at the first generated token under harmful queries, which is associated with unsafe response generation. Motivated by this finding, we propose SafeToken, a lightweight inference-time intervention that injects a learned continuous safety anchor precisely at reasoning onset. Despite updating only a single token embedding, SafeToken effectively mitigates ORC, improves safety on harmful-query benchmarks, and largely preserves reasoning utility. These results suggest that safety failures in LRMs can arise from a transient breakdown at the critical transition from understanding to generation.

Abstract

Explore similar work

First Token Matters: Understanding Safety Collapse in Large Reasoning Models

Towards Mitigating Deceptive Safety Alignment in Large Reasoning Models

Reasoning Structure Matters for Safety Alignment of Reasoning Models