Restoring General and Improving Domain-Specific Safety in Fine-Tuned LLMs Without Retraining

Vishnu Kabir Chhabra; Ding Zhu; Zhihui Zhu; Xueru Zhang; Mohammad Mahdi Khalili

cs.AINov 6, 2026

Restoring General and Improving Domain-Specific Safety in Fine-Tuned LLMs Without Retraining

Authors: Vishnu Kabir Chhabra, Ding Zhu, Zhihui Zhu, Xueru Zhang, Mohammad Mahdi Khalili

Organizations: The Ohio State University, Columbus, OH

Abstract

Safety-aligned language models are known to suffer degradation in refusal behavior following domain-specific fine-tuning, even when the fine-tuning data is entirely benign. The predominant mitigation strategy, preference optimization on the fine-tuned model, is computationally expensive and requires domain-specific preference data that is often unavailable in practice. We propose SPECTRA, a training-free, weight-level method that restores refusal behavior by transferring the base model's refusal direction into the fine-tuned model's weights via an activation-aware, low-rank update, requiring no gradient-based optimization or domain-specific safety data. The low-rank design restricts the intervention to the subspace most causally active on harmful inputs, leaving benign behavior largely undisturbed by construction. Empirically, SPECTRA substantially improves refusal behavior across a comprehensive safety evaluation, while preserving general coherence, domain-specific knowledge, and producing negligible change in false refusal rates.

Figures & tables

Direction	Layer	ASR ${}_{\textrm{Adv-Dec.}}^{I}$	ASR ${}_{\textrm{Vanilla}}$	ASR ${}_{\textrm{Adv-Dec.}}^{\times}$	GCG
No steering (base FT)		1.4	0.0	18.4	31
$\mathbf{r}_{1}$	$l_{2}$ (default)	2.8	0	10.8	4
$\mathbf{r}_{1}$	$l_{1}$	3.2	1.0	11.6	20
$\mathbf{r}_{2}$	$l_{2}$	2.0	0.0	16.4	25
$\mathbf{r}_{2}$	$l_{1}$	1.8	1.0	18.6	27
$\mathbf{r}_{2}-\mathbf{r}_{1}$	$l_{2}$	8.0	3.0	13.6	4

Table 1: Full-matrix injection (Eq. 5 ) of each refusal direction into the attention and MLP projections of a given layer, on medical Llama-2-7b-chat.

\alpha{=}0.015

(attention) and

0.01

(MLP). Lower is better; bold marks the best value per column among steered variants.

Figure 1: Separation (

d^{\prime}

) of harmful from harmless prompts at layer

l_{2}

and the last prompt token, projected onto

\mathbf{r}_{1}

\mathbf{r}_{2}

, and DiM, for

M_{1}

and

M_{2}

Figure 2: (a) Write along each direction at its extraction position, over harmful prompts. (b) Share of the

M_{1}

–

M_{2}

drop in (a) due to the weight difference

W_{1}{-}W_{2}

l_{2}

; the remainder comes from differing inputs.

Figure 3: Gate selectivity of the SPECTRA update at layer

l_{2}

: ratio of the mean

|\mathbf{r}^{\top}\Pi Wx|

on harmful to harmless prompts, for attention and MLP.

Model	Domain	Method	GCG	AdvDecoding	Vanilla	GPTFuzz	TAP	In-Domain
Llama2	Medical	BASE	$31.0$	$18.40$	$0.0$	$8.8$	$22.0$	$28.78$
	Medical	SPECTRA	$\mathbf{6.0}$	$\mathbf{9.6}$	$0.0$	$\mathbf{0.0}$	$\mathbf{0.0}$	$\mathbf{11.56}$
	Law	BASE	$66$	$29.6$	$0.0$	$15.0$	$58.0$	$17.0$
	Law	SPECTRA	$\mathbf{43.0}$	$\mathbf{23.2}$	$0.0$	$\mathbf{13.0}$	$\mathbf{0.0}$	$\mathbf{6.5}$
	Finance	BASE	$62.0$	$29.0$	$0.0$	$7.8$	$28.0$	$8.45$
	Finance	SPECTRA	$\mathbf{47.0}$	$29.0$	$0.0$	$7.8$	$\mathbf{0.0}$	$\mathbf{7.05}$

Table 2: ASR (%) of base fine-tuned models vs. SPECTRA across models and domains. Lower is better; best results per domain block are bolded . SPECTRA consistently reduces ASR across attack types, with the most pronounced improvements for Gemma2.

Model	Domain	Medical		Law		Finance
		PubMedQA	MedMCQA	Abercrombie	Hearsay	FinanceBench
Llama2	Medical	$77.5$ / $77.5$ $(+0.0)$	$41.7$ / $41.7$ $(+0.0)$	–	–	–
	Law	–	–	$29.4$ / $28.4$ $(+1.0)$	$63.0$ / $65.2$ $(-2.0)$	–
	Finance	–	–	–	–	$70.0$ / $66.7$ $(+3.3)$
Gemma2	Medical	$95.5$ / $95.5$ $(+0.0)$	$52.5$ / $52.5$ $(+0.0)$	–	–	–
Gemma2	Finance	–	–	–	–	$66.7$ / $68.0$ $(-1.3)$

Table 3: In-domain performance of base fine-tuned models and SPECTRA across models and domains. Medical domains are evaluated on PubMedQA and MedMCQA, Law on Abercrombie and Hearsay from LegalBench, and Finance on FinanceBench.

Model	Domain	RTE	ARC	BoolQ	Winogrande	HellaSwag
Llama2	Medical	$68.5$ / $67.5$ $(+1.0)$	$45.0$ / $45.0$ $(+0.0)$	$78.0$ / $78.5$ $(-0.5)$	$69.0$ / $68.5$ $(+0.5)$	$60.0$ / $58.5$ $(+1.5)$
	Law	$69.5$ / $69.0$ $(+0.5)$	$44.5$ / $44.0$ $(+0.5)$	$78.0$ / $80.0$ $(-2.0)$	$65.5$ / $68.0$ $(-2.5)$	$57.0$ / $57.5$ $(-0.5)$
	Finance	$70.0$ / $70.5$ $(-0.5)$	$41.5$ / $40.5$ $(+1.0)$	$81.0$ / $79.5$ $(+1.5)$	$73.5$ / $70.0$ $(+3.5)$	$52.0$ / $52.0$ $(+0.0)$
Gemma2	Medical	$72.5$ / $72.0$ $(+0.5)$	$55.5$ / $57.5$ $(-2.0)$	$88.0$ / $88.0$ $(+0.0)$	$72.5$ / $71.0$ $(+1.5)$	$52.5$ / $53.0$ $(-0.5)$
Gemma2	Finance	$77.0$ / $80.5$ $(-3.5)$	$53.0$ / $52.5$ $(+0.5)$	$86.0$ / $91.0$ $(-5.0)$	$70.5$ / $72.5$ $(-2.0)$	$53.5$ / $53.5$ $(+0.0)$

Table 4: General coherence benchmark scores across models and domains. For each entry, black denotes the SPECTRA result and gray denotes the base fine-tuned model result, with green indicating improvement and red indicating degradation relative to the base model.

Model	Domain	ActAdd	ROSI	CAST	Steer2Edit	SafeMERGE	SPECTRA
Gemma2	Finance	$91.2$	$7.3$	$55.8$	$18.8$	$16.4$	$\mathbf{1.3}$
Gemma2	Medical	$88.9$	$5.7$	$65.9$	$17.4$	$11.2$	$\mathbf{0.3}$
Llama2	Finance	$93.4$	$9.5$	$61.8$	$22.5$	$13.7$	$\mathbf{0.1}$
	Law	$97.9$	$6.9$	$62.4$	$21.6$	$19.1$	$\mathbf{-0.8}$
	Medical	$98.5$	$8.1$	$63.8$	$20.4$	$14.4$	$\mathbf{0.2}$
Average		$94.0$	$7.5$	$61.9$	$20.1$	$15.0$	$\mathbf{0.2}$

Table 5: Change in false refusal rate (Post

-

Pre, in %) after steering. Lower is better; best per row in bold.

Domain	Model	Method	Inference	Self	Instruction	Reliable
			Memory	Coherence	Retention	Editing
Finance	Gemma2	Base	2.15	9.09	29.41	0.00
	Gemma2	SPECTRA	4.42	10.00	13.04	5.00
	Llama2	Base	4.42	2.04	14.49	14.63
	Llama2	SPECTRA	5.31	10.00	5.80	9.76
Medical	Gemma2	Base	8.85	6.00	10.14	4.88

Table 6: Model performance on MultiChallenge ( Sirdeshmukh et al., 2025 ) across domains.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Model family	Domain	Attention α	MLP α
Llama-2 Chat	Medical	0.07	0.05
Llama-2 Chat	Law	0.1	0.03
Llama-2 Chat	Finance	0.1	0.05
Gemma2-9b-it	Medical	0.01	0.00
Gemma2-9b-it	Finance	0.01	0.00

Appendix

Table 7: Values of the steering hyperparameter

\alpha

used for each model variant and domain in all experiments.

Figure 4: ASR of GCG attacks vs $\alpha$ for attention heads, with fixed $\alpha$ for MLPs.

Figure 5: ASR of GCG attacks vs $\alpha$ for MLP, with fixed $\alpha$ for attention.

Model family	Domain	Attention	MLP
Llama-2 Chat	Medical	2096/4096	2096/4096
Llama-2 Chat	Law	3096/4096	3096/4096
Llama-2 Chat	Finance	2096/4096	2096/4096
Gemma2-9b-it	Medical	1048/2048	2048/3048
Gemma2-9b-it	Finance	1048/2048	2048/3048

Appendix

Table 8: Low-rank projection ranks in SPECTRA, reported as selected rank

\rho

/full rank.

Model	Domain	$l_{2}/l_{1}$	$tp_{2}/tp_{1}$
Llama2	Medical	$13/14$	$-5/-1$
	Law	$12/14$	$-2/-1$
	Finance	$12/14$	$-5/-1$
Gemma2	Medical	$21/31$	$-1/-1$
	Finance	$31/31$	$-5/-1$

Appendix

Table 9: Layer and token position of the refusal direction in the base model (

l_{1}

tp_{1}

) and fine-tuned model (

l_{2}

tp_{2}

) for each variant.

LLM Safety Alignment

Large language model safety alignment trains and evaluates systems to follow human policies while resisting harmful requests and misuse.

See all 355 papers

RefusalGuard: Geometry-Preserving Fine-Tuning for Safety in LLMs
May 3, 2026Sadia Asif, Mohammad Mohammadi AmiriFine-Tuning LLM Fine-Tuning
Department of Computer Science Rensselaer Polytechnic Institute Troy, NY, USA
Safety Offset Learning: When Benign Fine-Tuning Misaligns LLMs
Oct 19, 2026Jesson Wang, Zhanhao Hu, Minghao Lin +3LLM Fine-Tuning LLM Safety Alignment
University of Southern California · University of California, Berkeley · University of Washington
HyperSafe: Inference-Time Safety Recovery for Fine-Tuned Language Models
Jul 13, 2026Aznaur Aliev, Carlos Hinojosa, Abdelrahman Eldesokey +3LLM Fine-Tuning LLM Safety Alignment
King Abdullah University of Science and Technology, Saudi Arabia

May 3, 2026J/K move · Enter open · S save

RefusalGuard: Geometry-Preserving Fine-Tuning for Safety in LLMs

Sadia Asif, Mohammad Mohammadi Amiri

Department of Computer Science Rensselaer Polytechnic Institute Troy, NY, USA

Fine-Tuning LLM Fine-Tuning LLM Safety Alignment Adversarial Robustness of Language Models Safety-Preserving Fine-Tuning of Language Models

Open paper AskPDF arXiv

Fine-tuning safety-aligned language models for downstream tasks often leads to substantial degradation of refusal behavior, making models vulnerable to adversarial misuse. While prior work has shown that safety-relevant features are encoded in structured representations within the model's activation space, how these representations change during fine-tuning and why alignment degrades remains poorly understood. In this work, we investigate the representation-level mechanisms underlying alignment degradation. Our analysis shows that standard fine-tuning induces systematic drift in safety-relevant representations, distorts their geometric structure, and introduces interference between task optimization and safety features. These effects collectively lead to increased harmful compliance. Motivated by these findings, we introduce REFUSALGUARD, a representation-level fine-tuning framework that preserves safety-relevant structure during model adaptation. Our approach constrains updates in hidden representation space, ensuring that safety-mediating components remain stable while allowing task-specific learning in complementary directions. We evaluate REFUSALGUARD across multiple model families, including LLaMA, Gemma, and Qwen, on adversarial safety benchmarks such as AdvBench, DirectHarm4, and JailbreakBench, as well as downstream utility tasks. Our approach achieves attack success rates comparable to base safety-aligned models while maintaining competitive task performance, significantly outperforming baselines.

Abstract

LLM Safety Alignment

Explore similar work

RefusalGuard: Geometry-Preserving Fine-Tuning for Safety in LLMs

Safety Offset Learning: When Benign Fine-Tuning Misaligns LLMs

HyperSafe: Inference-Time Safety Recovery for Fine-Tuned Language Models