Deactivating Refusal Triggers: Understanding and Mitigating Overrefusal in Safety Alignment

Zhiyu Xue; Zimo Qi; Guangliang Liu; Bocheng Chen; Ramtin Pedarsani

cs.AIMar 12, 2026

Deactivating Refusal Triggers: Understanding and Mitigating Overrefusal in Safety Alignment

Authors: Zhiyu Xue, Zimo Qi, Guangliang Liu, Bocheng Chen, Ramtin Pedarsani

Organizations: University of California, Santa Barbara · Johns Hopkins University · Michigan State University · University of Mississippi

Abstract

Safety alignment aims to ensure that large language models (LLMs) refuse harmful requests by post-training on harmful queries paired with refusal answers. Although safety alignment is widely adopted in industry, the overrefusal problem where aligned LLMs also reject benign queries after safety alignment post-training, remains insufficiently studied. Such an issue degrades the usability of safety alignment in real-world applications. In this paper, we examine how overrefusal arises under safety alignment, and propose a mitigation strategy inspired by our findings. We define refusal triggers as linguistic cues in the training data that elicit refusal responses, safety alignment encourages LLMs to associate refusal triggers within a training sample with refusal responses, leading aligned LLMs to refuse harmful queries. However, the refusal triggers include not only harmful linguistic cues but also non-harmful cues, therefore causing overrefusal to benign queries. Building on this mechanistic analysis, we propose a method that explicitly considers refusal triggers in the safety alignment fine-tuning. Empirical results demonstrate that our approach achieves a more favorable trade-off between defense against jailbreak attacks and responsiveness to benign queries, outperforming prior methods. Warning: this paper contains harmful and biased sentences.

Figures & tables

Figure 1: How safety alignment can induce overrefusal. Top: During training, harmful intent is aligned with refusal, but harmless events (e.g., create a video ) and generic help-seeking wording (e.g., Can you help me ) can also become associated with refusal. Bottom: At test time, benign queries containing these cues may be rejected.

Panel of Figure 2 — Figure 2: Examples of rephrased refusal triggers across Levels 1-3.

Figure 3: Similarity scores in the hidden state space between refusal triggers and test benign queries. For each testing benign query, we retrieve the top-k most similar refusal triggers and compute the mean similarity scores separately for rejected and accepted queries. It is obvious that rejected test queries are more similar to the extracted refusal triggers than that of the accepted queries.

		Baseline	$\alpha=1$	$\alpha=0.2$
SFT	RT	3.63	100.00	71.37
	-Level1 $\downarrow$	3.23	100.00	70.56
	-Level2 $\downarrow$	3.21	99.60	64.26
	-Level3 $\downarrow$	2.01	96.39	45.38
P-SFT	RT	22.18	94.73	66.94
	-Level1 $\downarrow$	16.63	94.35	66.13

Table 2: Refusal rate (%) of extracted refusal triggers (RT) on Llama3-Uncensored after SFT and Llama2 after P-SFT. From Level 1 to Level 3, the rephrased testing benign queries become progressively less similar to the original RT.

Figure 4: Overview of the proposed method. Refusal triggers are first extracted from the harmful training dataset $\mathcal{D}_{h}$ . These triggers are then repurposed to generate benign training samples $\mathcal{D}_{b}$ that match the trigger distribution, enabling the model to differentiate between harmful intent and benign queries containing refusal triggers.

		RR $\downarrow$					ASR $\downarrow$			Avg. $\downarrow$
Model	Method	Koala	JBench-B	GSM-8k	SQL-1k	OrBench-H	SorryBench	JBench-H	HEX-PHI
Llama3-U	Baseline	5	10	0	0.8	16.83	85.71	80	84.55	89.95
	$D_{b}$ as Alpaca	57.22	96	95.75	99.1	99.85	1.36	0	0.00	90.04
	$D_{b}$ as Our Data	21.11	56	0.99	5.4	59.21	7.95	3	2.12	32.90
Qwen2.5-U	Baseline	3.33	6	0	0	10.39	90.67	78	87.88	89.46
	$D_{b}$ as Alpaca	43.33	90	30.4	74.1	99.55	0.45	0	0.30	67.73

Table 3: Comparison of RR

\downarrow

and ASR

\downarrow

after SFT on Llama3-U and Qwen2.5-U under different choices of benign training data

\mathcal{D}_{b}

(Baseline/no-SFT, Alpaca, and Our Data). RR is reported on Koala, JBench-B, GSM-8k, SQL-1k, and OrBench-H; ASR is reported on SorryBench, JBench-H, and HEx-PHI. Avg.

\downarrow

summarizes the overall safety–utility trade-off across harmful and benign benchmarks (lower is better).

		RR $\downarrow$					ASR $\downarrow$			Avg
Setting / Model	Method	Koala	JBench-B	GSM-8k	SQL-1k	OrBench-H	SorryBench	JBench-H	HEX-PHI
P-SFT, Llama2	Baseline	7.77	56	0.23	0.3	4.93	47.56	36	62.42	53.80
	$D_{b}$ as Alpaca	33.33	92	51.18	10.2	99.85	18.22	0	3.33	77.03
	$D_{b}$ as Our Data	12.22	39	0.99	3.4	55.34	25.11	5	5.76	36.71
RLVR, Llama3-U	Baseline	5	10	0	0.8	16.83	85.71	80	84.55	70.72
	$D_{b}$ as Alpaca	6.67	67	0	2.1	98.48	8.22	0.00	0.30	45.69

Table 4: Comparison of RR

\downarrow

and ASR

\downarrow

under two safety-alignment settings: P-SFT on Llama2 and RLVR on Llama3-U. For each setting, we report three training conditions (Baseline, via Alpaca, and via Our Data), where RR is evaluated on Koala, JBench-B, GSM-8k, SQL-1k, and OrBench-H, and ASR is evaluated on SorryBench, JBench-H, and HEx-PHI. Avg summarizes the overall safety–utility trade-off across these benchmarks (lower is better). For P-SFT, ASR is evaluated using prefill attacks following Qi et al. (2024) .

General	Koala (RR)	SorryBench (ASR)
Ours	21.11	7.95
-level2	54.56	3.46
-level3	60.34	4.23

Table 5: Trade-off between RR and ASR. Compared to the original refusal triggers, using less similar data (level 2 and level 3) relative to the finetuning discourse can noticeably increase the refusal rate (RR) while reducing the attack success rate (ASR).

LLM Safety Alignment

Large language model safety alignment trains and evaluates systems to follow human policies while resisting harmful requests and misuse.

See all 344 papers

Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness
Sep 3, 2026Hoang Cuong Nguyen, Mark Dras, Usman NaseemMechanistic Interpretability Circuits in Language Models
Macquarie University, Sydney, Australia
Please refuse to answer me! Mitigating Over-Refusal in Large Language Models via Adaptive Contrastive Decoding
Apr 18, 2026Yupeng Qi, Ziyu Lyu, Lixin Cui +2Language Model Safety Evaluation LLM Refusal Behavior
School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University · School of Information, Central University of Finance and Economics · School of Artificial Intelligence, Beijing Normal University +1
Refuse without Refusal: A Structural Analysis of Safety-Tuning Responses for Reducing False Refusals in Language Models
Sep 4, 2026Minji Kim, Hyounghun KimRefusal Behavior in Language Models Refusal Mechanisms in Language Models
Graduate School of Artificial Intelligence, POSTECH · Department of Computer Science and Engineering, POSTECH

Sep 3, 2026J/K move · Enter open · S save

Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness

Hoang Cuong Nguyen, Mark Dras, Usman Naseem

Macquarie University, Sydney, Australia

Mechanistic Interpretability Circuits in Language Models LLM Post-Training Language Model Robustness Refusal Behavior in Language Models Refusal Mechanisms in Language Models

Open paper AskPDF arXiv

How do the methods used to train language models to refuse harmful requests shape how that refusal actually works inside the model? We compare three post-training methods - supervised fine-tuning, reasoning-augmented fine-tuning (training on reasoning chains that justify a safety decision), and preference optimization (ORPO) - across three architecturally distinct models (Llama-3.1-8B, Gemma-2-9B, Qwen3-8B). We find that training method, not just data, reshapes how refusal is computed internally: reasoning-augmented training consistently produces a distinct kind of refusal computation, visible across all three models, while architecture independently shapes internal structure and how reliably refusal can be steered. Most importantly, no method we study achieves all three properties we would want from safe alignment at once: refusal that isn't concentrated in a few fragile components, safety gains that don't cost general capability, and safety behavior correctable through small, targeted edits. We caution against treating current post-training methods as a solved, reliable defense, especially for security-critical use. Code and models are available in https://github.com/hoangcuongnguyen2001/Beyond-Shallow-Alignment.

Abstract

LLM Safety Alignment

Explore similar work

Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness

Please refuse to answer me! Mitigating Over-Refusal in Large Language Models via Adaptive Contrastive Decoding

Refuse without Refusal: A Structural Analysis of Safety-Tuning Responses for Reducing False Refusals in Language Models