Addition by Subtraction: Toward Mitigating Fragile Safety Alignment in Large Language Models

Hamid Dashtbani; Alexandra Dmitrienko

cs.AINov 4, 2026

Addition by Subtraction: Toward Mitigating Fragile Safety Alignment in Large Language Models

Authors: Hamid Dashtbani, Alexandra Dmitrienko

Organizations: Department of Computer Science, University of Duisburg-Essen

Abstract

Large language models (LLMs) undergo safety alignment to prevent harmful generations, yet they remain vulnerable to jailbreak attacks. Recent work reveals a deeper structural weakness: safety alignment can be highly sparse, such that pruning a small number of safety-critical neurons or ablating a single refusal direction can substantially undermine aligned behavior. In this work, we take the first steps toward strengthening safety alignment such that it is less dependent on sparse model components. We propose Addition by Subtraction, which gradually removes identified safety-critical neurons during finetuning and uses preference optimization to re-establish safety-aligned behavior through other parts of the model. Our results show that the resulting models are substantially more robust to pruning-based attacks. Remarkably, these benefits generalize beyond the white-box attacks motivating our design: Addition by Subtraction improves robustness against other sparsity-based interventions, including refusal-direction ablation, as well as a broad range of black-box jailbreak attacks. The finetuned models also exhibit improved truthfulness while largely preserving general capabilities. Comprehensive experiments across four models from three vendors demonstrate consistent improvements in safety robustness, reducing attack success rates by up to 90% for some attacks.

Figures & tables

Panel of Figure 1 — Fig. 1 : Overview of the proposed method. The depicted new and original safety neurons are conceptual.

Model	Method	Per Iteration ASR $\downarrow$					Cumulative $\downarrow$
Model	Method	1	2	3	4	5	Cumulative $\downarrow$
Gemma-2-2B-IT	No Defense	0.55	0.58	0.59	0.55	0.53	0.71
Gemma-2-2B-IT	ABS-Align	0.07	0.15	0.15	0.15	0.13	0.23
Llama-3-8B-Instruct	No Defense	0.64	0.66	0.68	0.69	0.68	0.81
Llama-3-8B-Instruct	ABS-Align	0.04	0.04	0.07	0.06	0.03	0.11
Llama-3.2-3B-Instruct	No Defense	0.59	0.54	0.36	0.4	0.35	0.74

TABLE I : Performance of ABS-Align aganist TwinBreak [ 23 ] Attack.

Model	Method	NS [ 22 ]	Rf-Dir [ 25 ]	Average
Gemma-2-2B-IT	No Defense	0.50	0.80	0.65
Gemma-2-2B-IT	ABS-Align	0.05	0.08	0.06
Llama-3-8B-Instruct	No Defense	0.84	0.75	0.79
Llama-3-8B-Instruct	ABS-Align	0.03	0.05	0.04
Llama-3.2-3B-Instruct	No Defense	0.79	0.57	0.68
Llama-3.2-3B-Instruct	ABS-Align	0.37	0.30	0.33

TABLE II : Performance of ABS-Align against other Sparsity-based Attacks.

		Attack Success Rate $\downarrow$
Model	Method	Role Play	Evidence-based	Expert	Authority	Misrepresentation	Average over
Model	Method	Role Play	Persuasion	Endorsement	Endorsement	Misrepresentation	all 10 attacks
Gemma-2-2B-IT	No Defense	20.68	55.68	66.81	65.45	72.95	41.58
Gemma-2-2B-IT	ABS-Align	6.81	11.81	16.36	20.45	17.27	13.24
Llama-3-8B-Instruct	No Defense	36.36	56.59	60.68	60.90	55.45	48.34
Llama-3-8B-Instruct	ABS-Align	15.90	22.04	18.63	22.50	13.63	17.79

TABLE III : Performance of ABS-Align across black-box attacks.

	Defense Method	Attack Success Rate $\downarrow$				Over-Refusal $\downarrow$	Utility $\uparrow$
	Defense Method	TB [ 23 ]	NS [ 22 ]	SB [ 21 ]	Ref-Dir [ 25 ]	XSTest	Arc	TQA	WG	MMLU	HS
Gemma-2-	No Defense	0.71	0.50	41.58	0.80	2.00	42.06	55.15	65.43	29.12	60.25
2B-IT	W-DOOR [ 30 ]	0.62	0.19	35.36	0.55	9.00	44.37	52.71	65.98	36.44	64.94
	ABS-Align (Ours)	0.23	0.05	13.24	0.08	4.00	42.06	64.29	63.93	27.34	57.99
	No Defense	0.81	0.84	48.34	0.75	5.00	50.34	52.79	64.33	57.76	64.92
Llama-3-	Deep Refusal [ 42 ]	0.25	0.06	21.81	0.07	48.00	52.47	51.33	69.38	63.12	72.59

TABLE IV: Comparison of attack robustness, over-refusal, and utility performance under different defense methods.

Model	Baseline	Fine-tuned
Gemma-2-2B-IT	2.00	4.00
Llama-3-8B-Instruct	5.00	1.00
Llama-3.2-3B-Instruct	4.00	4.00
Qwen-3-4B	3.00	1.00

TABLE V: Change in over-refusal performance for each model and fine-tuning using Addition by Subtraction.

Model	ARC	TQA	WG	MMLU	HS
Gemma-2-2B-IT	0.00	+9.14	-1.50	-1.77	-2.25
Llama-3-8B-Instruct	-3.67	+14.26	+3.00	-0.37	+2.32
Llama-3.2-3B-Instruct	+0.60	+13.78	+1.03	-3.15	-2.02
Qwen-3-4B	-0.09	+11.75	-0.79	+0.69	-1.18

TABLE VI: Change in utility performance for each model.

	Setting	ASR $\downarrow$	Over-refusal $\downarrow$	Utility Difference
	Setting	TwinBreak [ 23 ]	XSTest [ 40 ]	Arc Challenge [ 45 ]	WinoGrande [ 46 ]
1	No Pruning	0.52	7.0%	+0.09%	-0.39%
2	Prune Random Neurons instead of Safety Neurons	0.50	6.0%	+0.09%	-0.24%
3	Non-gradual pruning	0.38	3.0%	+0.43%	-0.63%
4	Decremental pruning	0.31	3.0%	+0.09%	-1.50%
5	Prune Randomly Sampled Safety Neurons ( $r=0.2$ )	0.41	2.0%	+0.26%	-0.32%

TABLE VII: Ablation study on the design choices of our method using Gemma-2-2B-IT. ASR is reported on TwinBreak; over-refusal on XSTest; utility on two representative tasks.

Model	ASR@5	ASR@10	ASR@15	No Def.
Gemma-2-2B-IT	0.23	0.26	0.27	0.71
Llama-3-8B-Instruct	0.11	0.25	0.25	0.81
Llama-3.2-3B-Instruct	0.33	0.38	0.39	0.74
Qwen-3-4B	0.13	0.22	0.29	0.79

TABLE VIII: Attack success rate (ASR) against adaptive attackers with increasing iteration budgets, comparing no defense vs. our method.

Attack Type	Description
Question	Harmful instruction rephrased as question
Slang	Rewrites the harmful prompt using colloquial language and slang.
Uncommon Dialects	Expresses the harmful request using less common dialects.
Technical Terms	Rephrases the request using technical terminology.
Role Play	Frames the harmful request within an assigned role or hypothetical persona.
Misspellings	Introduces spelling or grammatical errors into the request.

TABLE IX: SORRY-Bench black-box attack transformations used in our evaluation.

Attack	No Defense	ABS-Align
Question	19.77	12.50
Slang	31.59	13.86
Uncommon Dialects	27.95	12.50
Technical Terms	24.09	8.86
Role Play	20.68	6.81
Misspellings	30.90	12.04

TABLE X: Attack success rate (%) of Gemma-2-2B-IT across all ten SORRY-Bench black-box attacks used in our evaluation. Lower is better.

Attack	No Defense	ABS-Align
Question	41.13	18.86
Slang	41.36	16.59
Uncommon Dialects	41.36	11.36
Technical Terms	46.36	20.00
Role Play	36.36	15.90
Misspellings	43.18	18.40

TABLE XI: Attack success rate (%) of Llama-3-8B-Instruct across all ten SORRY-Bench black-box attacks used in our evaluation. Lower is better.

Attack	No Defense	ABS-Align
Question	36.82	27.27
Slang	36.36	22.73
Uncommon Dialects	35.23	20.91
Technical Terms	43.18	27.95
Role Play	34.55	22.05
Misspellings	34.77	20.00

TABLE XII: Attack success rate (%) of Llama-3.2-3B across all ten SORRY-Bench black-box attacks used in our evaluation. Lower is better.

Attack	No Defense	ABS-Align
Question	44.09	6.81
Slang	55.90	9.54
Uncommon Dialects	58.18	13.86
Technical Terms	52.50	13.40
Role Play	52.04	7.04
Misspellings	50.45	8.40

TABLE XIII: Attack success rate (%) of Qwen-3-4B across all ten SORRY-Bench black-box attacks used in our evaluation. Lower is better.

LLM Safety Alignment

Large language model safety alignment trains and evaluates systems to follow human policies while resisting harmful requests and misuse.

See all 355 papers

Fail-Closed Alignment for Large Language Models
Feb 19, 2026Zachary Coalson, Beth Sohler, Aiden Gabriel +1LLM Alignment Refusal Behavior in Language Models
Oregon State University, Corvallis, OR USA.
NeuronGuard: Robust LLM Safety Alignment via Ablation-Aware Safety Signal Redistribution
Aug 25, 2026Anjun Gao, Yueyang Quan, Yufei Xia +2LLM Safety Alignment Adversarial Robustness of Language Models
University of Louisville · University of North Texas
Beyond Refusal Patterns: Safe-Role Internalization for Robust and Generalizable LLM Safety Alignment
Oct 4, 2026Jinghao Pang, Jitai Hao, Qiang Huang +2LLM Alignment LLM Safety Alignment
Harbin Institute of Technology (Shenzhen) · Leiden University

Feb 19, 2026J/K move · Enter open · S save

Fail-Closed Alignment for Large Language Models

Zachary Coalson, Beth Sohler, Aiden Gabriel, Sanghyun Hong

Oregon State University, Corvallis, OR USA.

LLM Alignment Refusal Behavior in Language Models Adversarial Robustness of Language Models Adversarial Attacks on LLMs

Open paper AskPDF arXiv

We identify a structural weakness in current large language model (LLM) alignment: modern refusal mechanisms are fail-open. While existing approaches encode refusal behaviors across multiple latent features, suppressing a single dominant feature

-

via prompt-based jailbreaks

-

can cause alignment to collapse, leading to unsafe generation. Motivated by this, we propose fail-closed alignment as a design principle for robust LLM safety: refusal mechanisms should remain effective even under partial failures via redundant, independent causal pathways. We present a concrete instantiation of this principle: a progressive alignment framework that iteratively identifies and ablates previously learned refusal directions, forcing the model to reconstruct safety along new, independent subspaces. Across four jailbreak attacks, we achieve the strongest overall robustness while mitigating over-refusal and preserving generation quality, with small computational overhead. Our mechanistic analyses confirm that models trained with our method encode multiple, causally independent refusal directions that prompt-based jailbreaks cannot suppress simultaneously, providing empirical support for fail-closed alignment as a principled foundation for robust LLM safety.

Abstract

LLM Safety Alignment

Explore similar work

Fail-Closed Alignment for Large Language Models

NeuronGuard: Robust LLM Safety Alignment via Ablation-Aware Safety Signal Redistribution

Beyond Refusal Patterns: Safe-Role Internalization for Robust and Generalizable LLM Safety Alignment