Enhancing Safety of Large Language Models via Embedding Space Separation

Xu Zhao; Xiting Wang; Weiran Shen

cs.CLMar 1, 2026

Enhancing Safety of Large Language Models via Embedding Space Separation

Authors: Xu Zhao, Xiting Wang, Weiran Shen

Organizations: Gaoling School of Artificial Intelligence, Renmin University of China.

Abstract

Large language models (LLMs) have achieved impressive capabilities, yet ensuring their safety against harmful prompts remains a critical challenge. Recent work has revealed that the latent representations (embeddings) of harmful and safe queries in LLMs typically exhibit linear separability, a property that has been exploited to construct attacks by perturbing the embeddings of harmful queries towards the safe subspace. Motivated by this observation, we propose a representation-level fine-tuning approach, named Embedding Space Separation (ES2), which improves LLM safety by explicitly enlarging the distance between harmful and safe representations in the embedding space. To prevent degradation of model's general capabilities, we introduce a Kullback-Leibler (KL) divergence regularization term into the loss function, which constrains the logits of the fine-tuned model to align with those of the original base model on harmless inputs. We evaluate our method on several open-source LLMs using standard safety benchmarks. Extensive experimental results demonstrate that our approach substantially improves model safety while maintaining comparable general capabilities.

Figures & tables

Figure 1: Intuition of ES2. (a) In the original base model, the embedding space exhibits a small margin between harmful (blue) and harmless (orange) prompts, making the safety boundary easily traversable. (b) ES2 explicitly increasing this margin, creating a large and safety margin. (c) For the base model, a small adversarial perturbation (

\epsilon_{small}

) is sufficient to cross the linear separation hyperplane, leading to a successful attack. (d) In the fine-tuned model, the increased margin forces the attacker to apply a large perturbation (

\epsilon_{large}

) to breach the safety guardrail, which alters or destroys the semantics of the prompt, effectively neutralizing the attack.

Figure 2: Classification accuracy with increasing layer depth of different LLMs.

		RepE Attack, DSR, $\uparrow$			Soft Prompt Attack, DSR, $\uparrow$			SCAV Attack, DSR, $\uparrow$
LLMs	Methods	Keyword	Answer	Useful	Keyword	Answer	Useful	Keyword	Answer	Useful
*LLama-2 (7B-Chat-hf)	Base Model	50%	46%	42%	84%	77%	71%	10%	6%	6%
	STL	90%	88%	79%	94%	91%	86%	49%	37%	34%
	DPL	90%	87%	76%	93%	93%	90%	54%	44%	39%
	ES2 (ours)	100%	98%	94%	100%	98%	97%	80%	72%	70%
	$\Delta$	+10%	+10%	+15%	+6%	+5%	+7%	+26%	+28%	+31%

Table 1: Defense robustness against embedding-level attacks. We report the Defense Success Rate (DSR) across four different LLMs under three representative embedding attacks: RepE, Soft Prompt, and SCAV. The metrics (Keyword, Answer, Useful) evaluate the refusal quality from lexical to semantic levels. Bold indicates the best performance and underline represents the second best. “*” indicates that the metrics are evaluated by human judgment.

\Delta\text{ = ES2 - Best baseline}

LLMs	Methods	IFEval	BBH	GPQA	MATH	MuSR	MMLU-Pro	Average
Llama-2 (7B-Chat-hf)	Base Model	0.423	0.298	0.250	0.010	0.368	0.108	0.243
	STL	0.445	0.298	0.245	0.012	0.362	0.105	0.245
	DPL	0.444	0.298	0.245	0.015	0.365	0.106	0.246
	ES2 (ours)	0.433	0.297	0.262	0.019	0.355	0.107	0.246
Llama-3 (8B-Instruct)	Base Model	0.763	0.420	0.257	0.084	0.357	0.113	0.332
	STL	0.758	0.417	0.257	0.085	0.358	0.113	0.331

Table 2: Preserved performance on general capabilities. We evaluate the general capabilities of fine-tuned LLMs using the Open LLM Leaderboard tasks. We report the accuracy (%) on IFEval, BBH, GPQA, MATH, MuSR, and MMLU-Pro, along with the average performance. Bold indicates the best performance and underline indicates the second best.

		RepE Attack		Soft Prompt Attack		SCAV Attack
LLMs	Methods	Inc.	Gib.	Inc.	Gib.	Inc.	Gib.
Llama-2 (7B-Chat-hf)	Base Model	0.0238%	0.0000%	0.0141%	0.0000%	0.0000%	0.0000%
	STL	0.0253%	0.0000%	0.0349%	0.0000%	0.0000%	0.0000%
	DPL	0.0132%	0.0000%	0.0222%	0.0000%	0.0256%	0.0000%
	ES2 (ours)	0.0426%	0.0745%	0.0408%	0.0515%	0.2286%	0.2857%
Llama-3 (8B-Instruct)	Base Model	0.0208%	0.0000%	0.0000%	0.0000%	0.0000%	0.0869%

Table 3: Analysis of output integrity We report the Incoherent Rate (Inc.) and Gibberish Rate (Gib.) . Higher values indicate that the defense successfully forced the attack into semantic collapse (producing garbled or irrelevant text) rather than generating harmful content.

Figure 3: Average perturbation distance required by the SCAV attack across different defense methods and LLMs.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Refusal Keywords
I’m sorry	Sorry
As an assistant	I cannot
unethical	not ethical
I am sorry	I apologize
I’m really sorry	It is not
As an AI	I’m an

Appendix

Table 4: List of keywords used for DSR-Keyword evaluation.

Hyperparameter	Value
Optimizer	AdamW
Learning Rate	$1e^{-6}$
Batch Size	60
Safe Prompt Number per Batch	50
Unsafe Prompt Number per Batch	10
Gradient Accumulation Steps	4

Appendix

Table 5: Hyperparameter settings for ES2 training.

	Llama-2-7B-Chat-hf			Llama-3-8B-Instruct			Qwen-2.5-7B-Instruct
Method / Configuration	Keyword	Answer	Useful	Keyword	Answer	Useful	Keyword	Answer	Useful
Baselines & Proposed Method
Base Model	10%	6%	6%	34%	29%	23%	29%	28%	25%
STL	49%	37%	34%	48%	43%	37%	30%	28%	26%
DPL	54%	44%	39%	51%	46%	40%	31%	28%	26%
ES2 (Ours, Two-Layer)	80%	72%	70%	77%	71%	69%	55%	48%	46%

Appendix

Table 6: Experiments on one layer selection. We report the Defense Success Rate (DSR,

\uparrow

) under SCAV attacks across three different LLMs. The study compares our method (ES2) with two-layer against single-layer fine-tuning configurations, ranging from the semantic emergence layer (Layer 10) to the final layers. The results demonstrate that single-layer constraints provide minimal safety improvements compared to the two-layer. Note: Qwen-2.5-7B has fewer total layers, hence the omission of results for Layer 29 and 32.

	Llama-2-7B-Chat-hf			Llama-3-8B-Instruct			Qwen-2.5-7B-Instruct
Method / Configuration	Keyword	Answer	Useful	Keyword	Answer	Useful	Keyword	Answer	Useful
ES2 (Ours, $l_{early}+l_{final}$ )	80%	72%	70%	77%	71%	69%	55%	48%	46%
No Final Layer
Layer $l_{early}+l_{11}$	20%	13%	11%	39%	31%	27%	30%	27%	27%
Layer $l_{early}+l_{14}$	19%	11%	9%	38%	29%	29%	28%	28%	26%
Layer $l_{early}+l_{17}$	22%	16%	13%	40%	30%	26%	30%	28%	27%

Appendix

Table 7: Experiments on two-layer . We report the Defense Success Rate (DSR,

\uparrow

) across three LLMs. The study compares our proposed ES2 configuration (combining the semantic emergence layer

l_{early}

and the terminal layer

l_{final}

) against two ablation settings: “No Final Layer” (pairing

l_{early}

with intermediate layers) and “Late Intervention” (pairing intermediate layers with

l_{final}

). Results demonstrate that ES2 achieves superior safety performance, confirming the necessity of simultaneously constraining both the early semantic formation and the final layer.

	Llama-2-7B-Chat-hf	Llama-3-8B-Instruct	Qwen-2.5-7B-Instruct
Method / Configuration	Avg. Perturbation Dist.	Avg. Perturbation Dist.	Avg. Perturbation Dist.
Base Model	1.5043	0.5407	0.6854
STL	1.8496	0.6684	0.8955
DPL	1.9256	0.7985	0.8795
ES2 (Ours, $l_{early}+l_{final}$ )	3.9986	1.9798	1.8865
No Final Layer

Appendix

Table 8: Adversarial cost analysis (perturbation distance) across layer pairs. We report the average perturbation distance required by the SCAV attack. Higher distances indicate a more robust safe barrier. The results show that removing the final layer constraint (No Final Layer) or delaying the first constraint (Late Intervention) significantly reduces the attack cost compared to our proposed ES2 configuration.

		Vanilla Attack, DSR, $\uparrow$			AutoDAN Attack, DSR, $\uparrow$			GCG Attack, DSR, $\uparrow$
LLMs	Methods	Keyword	Answer	Useful	Keyword	Answer	Useful	Keyword	Answer	Useful
*Llama-2 (7B-Chat)	Base Model	99%	98%	98%	80%	77%	72%	89%	88%	78%
	STL	100%	99%	99%	91%	87%	78%	90%	88%	73%
	DPL	100%	100%	99%	90%	87%	76%	99%	99%	88%
	ES2 (ours)	100%	100%	100%	100%	100%	94%	100%	100%	92%
	$\Delta$	+0%	+0%	+1%	+9%	+13%	+16%	+1%	+1%	+4%

Appendix

Table 9: Defense robustness against prompt-level attacks. We evaluate safety using the DSR metrics (Keyword, Answer, Useful) under Direct Inquiry (Vanilla), AutoDAN, and GCG attacks. Bold indicates the best performance and underline indicates the second best. “*” indicates that the metrics are evaluated by human judgment.

LLM Safety

Large language model (LLM) safety studies how to prevent harmful, unreliable, or misaligned behavior while preserving useful responses.

See all 438 papers

Safety Anchor: Defending Harmful Fine-tuning via Geometric Bottlenecks
May 7, 2026Guoxin Lu, Letian Sha, Qing Wang +4LLM Fine-Tuning LLM Safety Alignment
Nanjing University of Posts and Telecommunications, Nanjing, China.
Finding and Reactivating Post-Trained LLMs' Hidden Safety Mechanisms
Mar 10, 2026Mingjie Li, Wai Man Si, Michael Backes +2LLM Safety LLM Safety Alignment
CISPA Helmholtz Center for Information Security · State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · Institute for Artificial Intelligence, Peking University
HyperSafe: Inference-Time Safety Recovery for Fine-Tuned Language Models
Jul 13, 2026Aznaur Aliev, Carlos Hinojosa, Abdelrahman Eldesokey +3LLM Fine-Tuning LLM Safety Alignment
King Abdullah University of Science and Technology, Saudi Arabia

May 7, 2026J/K move · Enter open · S save

Safety Anchor: Defending Harmful Fine-tuning via Geometric Bottlenecks

Guoxin Lu, Letian Sha, Qing Wang, Peijie Sun+3

Nanjing University of Posts and Telecommunications, Nanjing, China.

LLM Fine-Tuning LLM Safety Alignment LLM Safety Adversarial Robustness of Language Models

Open paper AskPDF arXiv

The safety alignment of Large Language Models (LLMs) remains vulnerable to Harmful Fine-tuning (HFT). While existing defenses impose constraints on parameters, gradients, or internal representations, we observe that they can be effectively circumvented under persistent HFT. Our analysis traces this failure to the inherent redundancy of the high-dimensional parameter space: attackers exploit optimization trajectories that are orthogonal to defense constraints to restore harmful capabilities while deceptively adhering to safety restrictions. To address this, we propose Safety Bottleneck Regularization (SBR). SBR shifts the defensive focus from the redundant parameter space to the unembedding layer, which serves as a geometric bottleneck. By anchoring the final hidden states of harmful queries to those of the safety-aligned model, SBR enables the model to maintain safe responses even under persistent HFT. Extensive experiments confirm SBR's effectiveness, demonstrating that utilizing just a single safety anchor is sufficient to reduce the Harmful Score to

<

10 while preserving competitive performance on benign downstream tasks.

Abstract

LLM Safety

Explore similar work

Safety Anchor: Defending Harmful Fine-tuning via Geometric Bottlenecks

Finding and Reactivating Post-Trained LLMs' Hidden Safety Mechanisms

HyperSafe: Inference-Time Safety Recovery for Fine-Tuned Language Models