Revisiting Over-refusal in Safety-aligned Large Language Models: A Representation and Optimization Perspective

Chao Chen; Pengyang Shao; Wei Qin; Shijie Hao; Fei Shen; Richang Hong

cs.AIOct 29, 2026

Revisiting Over-refusal in Safety-aligned Large Language Models: A Representation and Optimization Perspective

Authors: Chao Chen, Pengyang Shao, Wei Qin, Shijie Hao, Fei Shen, Richang Hong

Organizations: Hefei University of Technology Hefei, China · National University of Singapore Singapore, Singapore

Abstract

Over-refusal is a critical side effect of safety alignment in large language models (LLMs), causing harmless but superficially risky queries to be mistakenly rejected. Despite the effectiveness of existing mitigation methods, why safety-aligned LLMs may still reject harmless queries under correct safety supervision remains insufficiently understood. To address this gap, we analyze over-refusal through the lens of representation and optimization. We make two observations: 1) the representations of harmful and over-refusal samples are visually distinguishable at certain intermediate layers but increasingly overlap in deeper layers; and 2) harmful-sample gradients exhibit high intra-class similarity and concentrate in a low-rank subspace. Moreover, they are more strongly aligned with refusal-target gradients than with correct helpful-target gradients on over-refusal prompts. This imbalanced gradient coupling can bias over-refusal samples toward refusal, while the reduced apparent representational separation in deeper layers may make it more difficult for the model to maintain a clear boundary between harmful and over-refusal samples. To this end, we propose GOAL (Gradient Orthogonalization with contrAstive Learning), a safety-alignment framework for mitigating over-refusal. Specifically, GOAL consists of two components: 1) to prevent imbalanced gradient coupling from biasing the optimization of over-refusal samples toward refusal, we orthogonalize over-refusal gradients against the low-rank refusal subspace extracted from harmful gradients; and 2) to explicitly enhance the separation between harmful and over-refusal representations, we introduce a contrastive learning objective. Extensive experiments across multiple models and benchmarks demonstrate that GOAL effectively mitigates over-refusal while preserving safety performance and general capabilities.

Figures & tables

Figure 1. Layer-wise PCA visualization of last-token representations for benign, harmful, and over-refusal samples in Llama-2-7B-Chat. Notably, harmful and over-refusal samples are visually distinguishable at layer 13 but show increasing overlap in deeper layers. Due to space limitations, representation visualization for an additional model is provided in Appendix E .

Figure 2. Gradient analyses on Llama-2-7B-Chat. (a) Cumulative Explained Ratio (CER) of the top- $k$ singular values for gradient subspaces of different sample types. (b) Pairwise gradient similarity across different sample types during safety alignment.

Figure 3. Overview of the proposed GOAL methodology.

Method	Over-refusal C.R. $\uparrow$					Safety S.S. $\uparrow$		General Capability Acc $\uparrow$
Method	XST	ORH	PHT	OKT	CoCo	JBB	STL	MMLU	OBQA	PIQA	ARC-C
Gemma-7B-it
Base	0.6977	0.6225	0.8596	0.6400	0.7652	0.90	0.9748	0.5063	0.368	0.7731	0.4778
SFT	0.8089	0.5808	0.8528	0.7467	0.8100	0.96	0.9767	0.5024	0.366	0.7693	0.4650
SCANS	0.5600	0.6998	0.8197	0.5633	0.8391	0.89	0.9612	0.5056	0.364	0.7737	0.4761
Surgical	0.7511	0.6745	0.8899	0.7800	0.7795	0.87	0.9399	0.5044	0.362	0.7644	0.4710

Table 1. Performance comparison of GOAL and baselines across over-refusal, safety, and general-capability benchmarks. Benchmark abbreviations: XST (XSTest), ORH (OR-Bench-Hard), PHT (PHTest), OKT (OKTest), CoCo (CoCoNot), MMLU, OBQA (OpenBookQA), PIQA, and ARC-C (ARC-Challenge). The best results within each model block are shown in bold , and the second-best results are underlined .

Method	Over-refusal C.R. $\uparrow$					Safety S.S. $\uparrow$		General Capability Acc $\uparrow$
Method	XST	ORH	PHT	OKT	CoCo	JBB	STL	MMLU	OBQA	PIQA	ARC-C
Gemma-7B-it
Ortho only	0.8978	0.6596	0.8699	0.8533	0.8602	0.96	0.9845	0.5092	0.3720	0.7666	0.4710
Contrast only	0.8489	0.6399	0.8801	0.8200	0.8391	0.95	0.9787	0.4979	0.3520	0.7742	0.4599
GOAL	0.9200	0.7006	0.9094	0.8733	0.8813	0.97	0.9884	0.5059	0.3680	0.7753	0.4727
Llama-2-7B-Chat

Table 2. Ablation of GOAL’s components on over-refusal, safety, and general-capability benchmarks across three base models.

Configuration	Over-refusal C.R. $\uparrow$					Safety S.S. $\uparrow$		General Capability Acc $\uparrow$
Configuration	XST	ORH	PHT	OKT	CoCo	JBB	STL	MMLU	OBQA	PIQA	ARC-C
Early (layers 1–11)	0.8267	0.6194	0.9196	0.9000	0.7995	0.96	0.9923	0.4657	0.3120	0.7519	0.4053
Late (layers 23–32)	0.8089	0.5595	0.8996	0.8133	0.7493	0.97	0.9826	0.4679	0.3060	0.7622	0.4121
Full (layers 1–32)	0.9556	0.7699	0.9552	0.9733	0.8892	0.97	0.9942	0.4675	0.3180	0.7595	0.4104
GOAL (layers 12–22)	0.9511	0.7699	0.9498	0.9700	0.9024	0.96	0.9864	0.4682	0.3180	0.7557	0.4138

Table 3. Ablation of target layer selection on Llama-2-7B-Chat (32 transformer layers).

Figure 4. Robustness to harmful sample size

N

. Impact of the number of harmful prompts

N

used for subspace extraction on model performance.

Figure 5. Sensitivity heatmaps of projection rank $k$ and contrastive weight $\lambda$ .

Appendix figures & tables23 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 6 — Figure 6. Per-layer cosine similarity analysis across Llama-2-7B-Chat, Llama-2-13B-Chat, and Gemma-7B-it to identify safety-relevant layers for orthogonal gradient projection. The $\rho^{\text{NN}}$ , $\rho^{\text{MM}}$ , and $\rho^{\text{NM}}$ curves show the similarity trends for each model, where solid lines represent the mean, shaded bands indicate the $\pm 1$ standard deviation, and gray dashed lines mark the selected safety band $\mathcal{L}^{*}$ .

Figure 7. PCA-based layer-wise visualization of the last-token hidden representations of benign, harmful, and over-refusal samples on Llama-2-13B-Chat. At layer 14, harmful and over-refusal samples are separable in the representation space; however, the two types of samples show increasing visual overlap in deeper layers.

Method	Compliance Rate (C.R.) $\uparrow$					Safety Score (S.S.) $\uparrow$
Method	XST	ORH	PHT	OKT	CoCo	JBB	STL
GOAL (Hex-Phi)	0.9467	0.7604	0.9522	0.9667	0.8865	0.96	0.9845
GOAL (HH-RLHF)	0.9511	0.7526	0.9352	0.9700	0.8945	0.96	0.9884
GOAL (Ours)	0.9511	0.7699	0.9498	0.9700	0.9024	0.96	0.9864

Appendix

Table 4. Robustness of refusal subspaces constructed from different harmful prompt sources. We compare subspaces built using the default harmful instruction set, Hex-Phi, and HH-RLHF.

Figure 8. Robustness to harmful sample size

N

. Impact of the number of harmful prompts

N

used for subspace extraction on model performance.

Figure 9. Sensitivity heatmaps of projection rank $k$ and contrastive weight $\lambda$ .

Figure 10. Layer-wise representation visualization across middle-to-late layers for benign, harmful, and over-refusal samples on Llama-2-7B-Chat.

\mathbf{g}_{\mathrm{GOAL}}=\mathbf{g}_{\mathrm{SFT}}^{\mathrm{total}}+\lambda\nabla_{\theta}\mathcal{L}_{\mathrm{cont}}.

Appendix

Algorithm 1 GOAL Training

Target-layer choice	Selected layer(s)	XST C.R. $\uparrow$	ORH C.R. $\uparrow$	PHT C.R. $\uparrow$	OKT C.R. $\uparrow$	CoCo C.R. $\uparrow$
H–O Separable Layer	13	0.7867	0.5343	0.8601	0.7867	0.7124
Overlap Layers	14–24	0.8400	0.6241	0.9230	0.8533	0.8047
Safety-relevant Layers (Ours)	12–22	0.9511	0.7699	0.9498	0.9700	0.9024

Appendix

Table 5. Comparison of different target-layer choices on Llama-2-7B-Chat. H–O Separable Layer uses only the intermediate layer where harmful and over-refusal representations are distinguishable. Overlap Layers uses a subsequent contiguous block with the same number of layers as the safety-relevant block used by GOAL. All other training and optimization configurations are held fixed. Higher C.R. indicates better over-refusal mitigation.

Figure 11. GPT-5.1 Safety Judge Template.

Figure 12. GPT-5.1 Over-Refusal Judge Template.

Model	$E^{\mathrm{harm}}$ $\uparrow$	$S_{1}$ (%)	$S_{2}$ (%)	$S_{3}$ (%)	$S_{4}$ (%)	$S_{5}$ (%)	Median $q$	Spearman $\rho$
Gemma-7B-it	0.89	78.5	76.5	77.5	79.5	78.5	1.25	0.37
Llama-2-7B-Chat	0.82	72.1	70.3	72.6	74.3	72.4	1.16	0.24
Llama-2-13B-Chat	0.85	81.2	83.7	81.3	79.6	81.5	1.24	0.35

Appendix

Table 6. Empirical verification of Proposition 1 .

E^{\mathrm{harm}}

denotes the fraction of aggregated harmful-gradient energy captured by the refusal subspace.

S_{e}

is the percentage of over-refusal samples satisfying

A_{i,t}>|R_{i,t}|

at the checkpoint after epoch

e

. Median

q

is computed over all evaluated sample–checkpoint pairs.

\rho

is the Spearman correlation between the predicted and actual finite-step margin changes. Results are averaged across five random seeds.

	Refusal Preference Margin $\downarrow$				AdamW Robustness
Model	Base	SFT	Ortho only	$\Delta M$ $\uparrow$	$L^{\mathrm{Adam}}$ $\downarrow$	$Q^{\mathrm{Adam}}$ $\uparrow$
Gemma-7B-it	0.187	-0.086	-0.141	0.055	0.036	0.885
Llama-2-7B-Chat	0.215	-0.093	-0.118	0.025	0.041	0.841
Llama-2-13B-Chat	0.159	-0.081	-0.113	0.032	0.047	0.858

Appendix

Table 7. Refusal preference margins and robustness under AdamW preconditioning. “Base”, “SFT”, and “Ortho only” report the average refusal preference margins of the base model, vanilla SFT model, and refusal-orthogonally trained model, respectively. A lower margin indicates a weaker relative preference for refusal.

\Delta M

denotes the margin reduction from SFT to Ortho only, thereby isolating the additional effect of refusal-orthogonal projection under the same training data and configuration.

L^{\mathrm{Adam}}

denotes the residual refusal-subspace leakage after AdamW preconditioning, and

Q^{\mathrm{Adam}}

denotes the coupling reduction relative to the unprojected AdamW-preconditioned gradient. Results for SFT, Ortho only, and the AdamW diagnostics are averaged over five runs with different random seeds; the base models are evaluated once because they involve no additional training.

Figure 13. Qualitative comparison on a harmful instruction. GOAL effectively preserves safety guardrails and decisively rejects the malicious request, maintaining strong safety alignment.

Model	$S_{1}$ (%)	$S_{2}$ (%)	$S_{3}$ (%)	$S_{4}$ (%)	$S_{5}$ (%)	Median $r$	Spearman $\rho$
Gemma-7B-it	75.6	73.8	72.9	75.1	74.4	1.18	0.29
Llama-2-7B-Chat	70.9	69.5	71.1	71.3	70.2	1.12	0.24
Llama-2-13B-Chat	74.2	74.8	75.9	76.1	75.4	1.21	0.33

Appendix

Table 8. Empirical verification of the sufficient condition in Proposition 1 .

S_{e}

denotes the percentage of held-out over-refusal samples satisfying

r_{i,e}>1

at the end of epoch

e

. Median

r

is computed over all held-out samples and five epoch checkpoints. Spearman

\rho

measures the association between the sample-level condition ratio averaged across epochs and the final refusal-margin reduction. Results are averaged over five random seeds.

Figure 14. Qualitative comparison on an over-refusal query containing sensitive keywords. While the baseline triggers a false refusal, GOAL successfully discerns the benign intent and delivers a safe, informative response.

Gradient group	$\mathrm{CER}(10)$ $\uparrow$	$\widetilde{\mathrm{CER}}(10)$ $\uparrow$
Harmful+Refusal	0.78	0.59
Benign+Refusal	0.57	0.32
Benign+Helpful	0.29	0.24

Appendix

Table 9. Matched-target control for the effect of refusal-targets on gradient low-rankness on Llama-2-7B-Chat. We report the cumulative explained ratio of the top-

10

singular directions before and after within-group gradient centering. Higher values indicate stronger concentration of gradient energy in a low-dimensional subspace. Harmful+Refusal and Benign+Refusal use the same refusal-response pool, while Benign+Refusal and Benign+Helpful use the same benign prompts. Results are averaged over five runs.

Evaluator	Method	Refusal Rate $\downarrow$	Helpfulness $\uparrow$	Correctness $\uparrow$	Completeness $\uparrow$	Useful Compliance $\uparrow$
GPT-5.1	SFT	0.195	3.83	3.88	3.76	68.5%
	ACTOR	0.145	4.01	4.03	3.84	72.5%
	GOAL	0.090	4.27	4.23	4.18	78.5%
Human	SFT	0.215	3.76	3.92	3.69	66.5%
	ACTOR	0.150	3.84	3.88	3.78	70.0%
	GOAL	0.100	4.12	4.17	4.03	76.0%

Appendix

Table 10. GPT-5.1 and human evaluation of response quality on the same

200

held-out over-refusal prompts using Llama-2-7B-Chat. Helpfulness, correctness, and completeness are rated on a five-point scale. Useful Compliance is the percentage of responses that do not exhibit unwarranted refusal and receive both helpfulness and correctness scores of at least

4

. Agreement denotes Fleiss’

\kappa

for refusal and ordinal Krippendorff’s

\alpha

for quality ratings.

Backbone	Method	Over-refusal C.R. $\uparrow$					Safety S.S. $\uparrow$
Backbone	Method	XST	ORH	PHT	OKT	CoCo	JBB	STL
Gemma-7B-it	SFT	0.8098 $\pm$ 0.0058	0.5820 $\pm$ 0.0045	0.8491 $\pm$ 0.0055	0.7420 $\pm$ 0.0090	0.8153 $\pm$ 0.0083	0.9640 $\pm$ 0.0055	0.9771 $\pm$ 0.0046
	ACTOR	0.7876 $\pm$ 0.0085	0.7046 $\pm$ 0.0053	0.8933 $\pm$ 0.0076	0.7760 $\pm$ 0.0055	0.8169 $\pm$ 0.0071	0.8800 $\pm$ 0.0071	0.9593 $\pm$ 0.0049
	GOAL	0.9218 $\pm$ 0.0074 ∗	0.7004 $\pm$ 0.0046	0.9047 $\pm$ 0.0072 ∗	0.8767 $\pm$ 0.0062 ∗	0.8828 $\pm$ 0.0051 ∗	0.9740 $\pm$ 0.0055	0.9891 $\pm$ 0.0040
Llama-2-7B-Chat	SFT	0.8978 $\pm$ 0.0070	0.6600 $\pm$ 0.0073	0.9105 $\pm$ 0.0063	0.9260 $\pm$ 0.0086	0.8834 $\pm$ 0.0051	0.9620 $\pm$ 0.0045	0.9791 $\pm$ 0.0046
	ACTOR	0.9289 $\pm$ 0.0083	0.7494 $\pm$ 0.0044	0.9565 $\pm$ 0.0063	0.9353 $\pm$ 0.0096	0.8464 $\pm$ 0.0082	0.9500 $\pm$ 0.0071	0.9771 $\pm$ 0.0065

Appendix

Table 11. Multi-seed over-refusal and safety results on three backbone models. Results are reported as mean

\pm

sample standard deviation across five random seeds. C.R. denotes compliance rate, and S.S. denotes safety score. Higher values indicate better performance. ∗ indicates that GOAL significantly outperforms ACTOR on the corresponding over-refusal benchmark under hierarchical paired bootstrap resampling with

10{,}000

replicates (

p<0.05

after Holm–Bonferroni correction). Statistical significance is not assessed for the two safety benchmarks.

Method	XST C.R. $\uparrow$	ORH C.R. $\uparrow$	PHT C.R. $\uparrow$	OKT C.R. $\uparrow$	CoCo C.R. $\uparrow$	JBB S.S. $\uparrow$	STL S.S. $\uparrow$
PCGrad	0.8933	0.6698	0.8904	0.9300	0.8786	0.96	0.9767
Harmful Mean Ortho	0.9111	0.6990	0.8899	0.9267	0.8813	0.95	0.9806
Ortho only	0.9200	0.7195	0.9001	0.9400	0.8892	0.95	0.9806

Appendix

Table 12. Comparison with alternative gradient intervention strategies on Llama-2-7B-Chat. PCGrad applies generic conflict-triggered gradient surgery across the harmful, benign, and over-refusal objectives. Harmful Mean Ortho projects over-refusal gradients against the rank-one mean harmful-gradient direction, while Ortho only projects them against the low-rank subspace extracted from harmful per-sample gradients. None of the variants uses the contrastive objective. All methods use identical training data, trainable parameters, target layers, and optimization settings.

Method	XST C.R. $\uparrow$	ORH C.R. $\uparrow$	PHT C.R. $\uparrow$	OKT C.R. $\uparrow$	CoCo C.R. $\uparrow$	JBB S.S. $\uparrow$	STL S.S. $\uparrow$
SFT ( $\lambda_{\mathrm{harm}}=1.00$ )	0.8978	0.6604	0.9098	0.9233	0.8813	0.96	0.9787
Harmful DW ( $\lambda_{\mathrm{harm}}=0.75$ )	0.9022	0.6761	0.9103	0.9600	0.8945	0.91	0.9360
Harmful DW ( $\lambda_{\mathrm{harm}}=0.50$ )	0.8978	0.6856	0.9211	0.9667	0.9050	0.86	0.8779
Ortho only	0.9200	0.7195	0.9001	0.9400	0.8892	0.95	0.9806

Appendix

Table 13. Effect of harmful-gradient downweighting on Llama-2-7B-Chat.

\lambda_{\mathrm{harm}}

denotes the weight assigned to the harmful-sample training objective, with

\lambda_{\mathrm{harm}}=1

corresponding to standard SFT. Ortho only preserves the original harmful objective and projects over-refusal gradients against the low-rank refusal subspace. None of the variants uses the contrastive objective. All other training and evaluation configurations are held fixed.

	Over-refusal C.R. $\uparrow$					Safety S.S. $\uparrow$		General Capability Acc $\uparrow$
Method	XST	ORH	PHT	OKT	CoCo	JBB	STL	MMLU	OBQA	PIQA	ARC-C
Base	0.9600	0.8437	0.9303	0.9600	0.9393	0.97	0.9957	0.7683	0.572	0.8123	0.5119
SFT	0.9689	0.8597	0.9459	0.9667	0.9472	0.98	0.9961	0.7675	0.567	0.8116	0.5102
SCANS	0.9778	0.8512	0.9421	0.9633	0.9499	0.97	0.9845	0.7657	0.569	0.8108	0.5108
Surgical	0.9644	0.8575	0.9359	0.9667	0.9472	0.97	0.9849	0.7589	0.565	0.8102	0.5102
DCR	0.9644	0.8463	0.9365	0.9667	0.9409	0.95	0.9709	0.7442	0.544	0.8002	0.4915

Appendix

Table 14. Performance comparison on Qwen3-8B across over-refusal, safety, and general-capability benchmarks. C.R. denotes Compliance Rate, S.S. denotes Safety Score, and Acc denotes accuracy. All results are averaged across five independent runs. The best mean result in each column is shown in bold, and the second-best is underlined.

Layer	Probe AUC $\uparrow$	$R^{(\ell)}$ $\uparrow$
1	0.612 [0.574, 0.650]	0.052 [0.041, 0.065]
5	0.684 [0.649, 0.718]	0.087 [0.071, 0.105]
9	0.761 [0.742, 0.791]	0.219 [0.193, 0.248]
13	0.858 [0.837, 0.871]	0.357 [0.345, 0.372]
17	0.737 [0.712, 0.761]	0.311 [0.294, 0.321]
21	0.716 [0.683, 0.718]	0.288 [0.277, 0.296]

Appendix

Table 15. Quantitative separation between harmful and over-refusal representations in Llama-2-7B-Chat. Values in brackets denote

95\%

bootstrap confidence intervals.

Method	XST	ORH	PHT	OKT	CoCo
SFT	0.8978	0.6604	0.9098	0.9233	0.8813
Random Ortho	0.8489	0.5965	0.8679	0.8833	0.8575
Benign Ortho	0.7956	0.5122	0.7861	0.7867	0.7731
Harmful Ortho	0.9200	0.7195	0.9001	0.9400	0.8892

Appendix

Table 16. Rank-matched subspace controls on Llama-2-7B-Chat. C.R. denotes Compliance Rate. The best result is shown in bold and the second-best result is underlined.

Over-Refusal

Over-refusal studies why language models reject benign requests and how to improve refusal calibration without weakening safety.

See all 27 papers

Deactivating Refusal Triggers: Understanding and Mitigating Overrefusal in Safety Alignment
Mar 12, 2026Zhiyu Xue, Zimo Qi, Guangliang Liu +2LLM Safety Alignment Over-Refusal
University of California, Santa Barbara · Johns Hopkins University · Michigan State University +1
Please refuse to answer me! Mitigating Over-Refusal in Large Language Models via Adaptive Contrastive Decoding
Apr 18, 2026Yupeng Qi, Ziyu Lyu, Lixin Cui +2Language Model Safety Evaluation LLM Refusal Behavior
School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University · School of Information, Central University of Finance and Economics · School of Artificial Intelligence, Beijing Normal University +1
Over-Refusal and Representation Subspaces: A Mechanistic Analysis of Task-Conditioned Refusal in Aligned LLMs
Mar 29, 2026Utsav Maskey, Mark Dras, Usman NaseemLLM Interpretability LLM Refusal Behavior
Macquarie University

Mar 12, 2026J/K move · Enter open · S save

Deactivating Refusal Triggers: Understanding and Mitigating Overrefusal in Safety Alignment

Zhiyu Xue, Zimo Qi, Guangliang Liu, Bocheng Chen+1

University of California, Santa Barbara · Johns Hopkins University · Michigan State University+1

LLM Safety Alignment Over-Refusal Refusal Behavior in Language Models Adversarial Robustness of Language Models

Open paper AskPDF arXiv

Safety alignment aims to ensure that large language models (LLMs) refuse harmful requests by post-training on harmful queries paired with refusal answers. Although safety alignment is widely adopted in industry, the overrefusal problem where aligned LLMs also reject benign queries after safety alignment post-training, remains insufficiently studied. Such an issue degrades the usability of safety alignment in real-world applications. In this paper, we examine how overrefusal arises under safety alignment, and propose a mitigation strategy inspired by our findings. We define refusal triggers as linguistic cues in the training data that elicit refusal responses, safety alignment encourages LLMs to associate refusal triggers within a training sample with refusal responses, leading aligned LLMs to refuse harmful queries. However, the refusal triggers include not only harmful linguistic cues but also non-harmful cues, therefore causing overrefusal to benign queries. Building on this mechanistic analysis, we propose a method that explicitly considers refusal triggers in the safety alignment fine-tuning. Empirical results demonstrate that our approach achieves a more favorable trade-off between defense against jailbreak attacks and responsiveness to benign queries, outperforming prior methods. Warning: this paper contains harmful and biased sentences.

Abstract

Over-Refusal

Explore similar work

Deactivating Refusal Triggers: Understanding and Mitigating Overrefusal in Safety Alignment

Please refuse to answer me! Mitigating Over-Refusal in Large Language Models via Adaptive Contrastive Decoding

Over-Refusal and Representation Subspaces: A Mechanistic Analysis of Task-Conditioned Refusal in Aligned LLMs