CLEAR: Continuous Latent Adapter Routing for Utility-Preserving LLM Safety Alignment

Chengxiao Wang; Enyi Jiang; Xiaojing Liao; Sanmi Koyejo

cs.AIAug 21, 2026

CLEAR: Continuous Latent Adapter Routing for Utility-Preserving LLM Safety Alignment

Authors: Chengxiao Wang, Enyi Jiang, Xiaojing Liao, Sanmi Koyejo

Organizations: Siebel School of Computing and Data Science, University of Illinois at Urbana-Champaign · Computer Science, Stanford University

Abstract

Improving the safety of large language models (LLMs) often comes at the expense of utility, as globally applied safety tuning may affect model responses to both harmful and benign inputs. We propose Continuous LatEnt Adapter Routing (CLEAR), a conditional safety adaptation framework that uses a lightweight hidden-state gate to continuously control the activation strength of a safety low-rank adapter. CLEAR aims to reduce harmful completions while avoiding unnecessary changes to the frozen backbone that could degrade performance on benign prompts. Experiments on widely used safety and utility benchmarks show that CLEAR improves robustness on HarmBench while reducing the utility degradation observed with globally applied safety tuning such as SFT or standard low-rank adaptation (LoRA). On Llama-3-8B-Instruct, CLEAR reduces HarmBench ASR from 32.3% to 0.5%, while retaining most of the base model's utility and achieving up to 7.1 percentage points higher GSM8K accuracy than globally applied SFT or LoRA. These results suggest that CLEAR is a promising mechanism for improving the safety--utility trade-off in LLM alignment.

Figures & tables

Figure 1: Overview of CLEAR, a continuous latent adapter routing framework for utility-preserving LLM safety alignment. Given an input prompt, the frozen base LLM first produces hidden representations, from which selected latent states are extracted and passed to a lightweight safety gate. The gate predicts a continuous routing score $g(x)\in[0,1]$ , which controls the strength of a safety LoRA adapter. For benign prompts, the routing score remains low and the model behaves close to the frozen backbone; for harmful or adversarial prompts, the score increases and the safety adapter is activated to steer generation toward aligned refusal behavior.

Model	Method	HB ASR $\downarrow$	Unsafe Ref. $\uparrow$	Safe OR $\downarrow$	GSM8K $\uparrow$	MMLU $\uparrow$	TQA MC1 $\uparrow$	TQA MC2 $\uparrow$
Llama-3-8B-Instruct	Base	32.25	88.50	2.80	75.06	66.76	37.33	52.47
	SFT	2.00	94.00	10.80	66.34	63.86	32.93	48.28
	LoRA	0.00	82.50	6.40	66.72	64.18	34.88	50.46
	CLEAR	0.50	92.50	4.80	73.46	63.62	36.96	52.63
Gemma-2-2B-it	Base	9.50	48.50	4.80	46.47	56.68	37.09	53.13
	SFT	0.00	90.00	5.20	38.06	55.48	32.31	47.99

Table 1: Main safety and utility results. All numbers are percentages. Metric abbreviations are defined in the experimental setting. Bold indicates the best raw value within each backbone and metric; it does not indicate the best overall safety–utility trade-off.

Family	Model	HB ASR $\downarrow$	Unsafe Ref. $\uparrow$	Safe OR $\downarrow$	GSM8K $\uparrow$	MMLU $\uparrow$	TQA MC1 $\uparrow$	TQA MC2 $\uparrow$
Llama-3	Llama-3-8B ( Grattafiori et al. 2024 )	32.25	88.50	2.80	75.06	66.76	37.33	52.47
	Llama-3-8B-RR ( Zou et al. 2023 )	15.50	88.00	2.40	76.35	66.21	37.33	52.49
	CLEAR	0.50	92.50	4.80	73.46	63.62	36.96	52.63
Alpaca	Alpaca-7B ( PKU-Alignment 2023 )	42.00	9.00	1.20	6.37	39.92	25.83	39.56
	SACPO ( Wachi et al. 2024 )	21.75	21.50	4.80	5.61	39.32	38.31	54.65
	P-SACPO ( Wachi et al. 2024 )	22.50	26.00	2.00	5.99	40.08	39.05	54.56

Table 2: Comparison with safety-aligned model families. All numbers are percentages, and metric abbreviations follow the experimental setting. The Alpaca-family CLEAR variant is initialized from P-SACPO and is therefore labeled P-SACPO + CLEAR.

Figure 2: Routing quality on XSTest. The x-axis measures safe prompts incorrectly routed to the adapter, and the y-axis measures unsafe prompts routed to the adapter. Lower x and higher y are better. These are routing-level metrics, not generated refusal rates.

Guard / Detector	Detector Parameters
CLEAR internal gate	664,129
PromptGuard ( Meta 2024 )	278,811,651
Llama Guard 3 8B ( Inan et al. 2023 )	8,030,261,248

Table 3: Parameter cost of the routing detectors compared in Figure 2 . CLEAR uses a lightweight internal gate, while PromptGuard and Llama Guard 3 are external guard models used off the shelf.

Figure 3: Latent adaptive attack results on Gemma under perturbation budget $\epsilon=0.001$ . The x-axis reports the latent $L_{2}$ perturbation distance, while the y-axis reports the harmless reward score ( Yang et al. 2025 ) difference between attacked and baseline outputs. CLEAR-based defense achieves lower ASR than Full SFT and standard LoRA, indicating stronger robustness to latent perturbations while preserving safe behavior.

		Safety and Utility Evaluation							Gate Separability Diagnostics
Model	Method	HB ASR $\downarrow$	Unsafe Ref. $\uparrow$	Safe OR $\downarrow$	GSM8K $\uparrow$	MMLU $\uparrow$	TQA MC1 $\uparrow$	TQA MC2 $\uparrow$	ROC-AUC $\uparrow$	PR-AUC $\uparrow$	Margin $\uparrow$
Qwen-0.5B	Base	22.50	86.50	46.80	31.46	46.92	26.81	41.86	–	–	–
	CLEAR	0.25	95.00	51.60	32.60	45.81	27.29	42.02	0.7795	0.7475	0.0875
Qwen-1.5B	Base	6.25	95.50	48.80	51.40	60.26	31.21	46.70	–	–	–
	CLEAR	0.25	95.50	48.00	51.71	59.95	31.09	46.69	0.8800	0.8286	0.0749
Qwen-3B	Base	15.50	78.00	16.40	63.23	66.35	42.23	58.83	–	–	–

Table 4: Scaling results on Qwen2.5-Instruct models. All numbers are percentages, and metric abbreviations follow the experimental setting. Gate diagnostics are reported only for CLEAR: ROC-AUC and PR-AUC measure safe/unsafe separability, and Margin is the mean unsafe-safe gate score difference.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	SFT	LoRA	CLEAR
Training data	Same	Same	Same
Epochs	1	1	1
Micro-batch size	4	16	4
Gradient accumulation	16	4	16
Effective batch size	64	64	64
Learning rate	$2\times 10^{-5}$	$3\times 10^{-4}$	$3\times 10^{-4}$

Appendix

Table 5: Training and adapter hyperparameters for SFT, standard LoRA, and CLEAR. All methods use the same WildJailbreak training data and one training epoch; SFT and LoRA differ from CLEAR only where specified.

Backbone	Gate Input Layers
Gemma-2-2B-it	$[12,16,20]$
Llama-3-8B-Instruct	$[15,20,25]$
Qwen2.5-0.5B-Instruct	$[11,15,18]$
Qwen2.5-1.5B-Instruct	$[13,17,22]$
Qwen2.5-3B-Instruct	$[17,22,28]$
Qwen2.5-7B-Instruct	$[13,17,22]$

Appendix

Table 6: Transformer layers used as gate inputs for each backbone. We select three intermediate and later layers for each model.

Family	Model	Method	Training Time	Memory Usage (bytes)	Trainable Params	GPU
Llama-3	Llama-3-8B-Instruct	SFT	5:08:14	$9.23\times 10^{10}$	$8.03\times 10^{9}$	1 $\times$ NVIDIA GH200 120GB
Llama-3	Llama-3-8B-Instruct	LoRA	3:18:25	$3.43\times 10^{10}$	$3.41\times 10^{6}$	1 $\times$ NVIDIA GH200 120GB
Llama-3	Llama-3-8B-Instruct	CLEAR	5:08:55	$4.05\times 10^{10}$	$5.51\times 10^{6}$	1 $\times$ NVIDIA GH200 120GB
Gemma	Gemma-2-2B-it	SFT	2:49:47	$4.38\times 10^{10}$	$2.61\times 10^{9}$	1 $\times$ NVIDIA GH200 120GB
Gemma	Gemma-2-2B-it	LoRA	1:48:26	$2.59\times 10^{10}$	$1.60\times 10^{6}$	1 $\times$ NVIDIA GH200 120GB
Gemma	Gemma-2-2B-it	CLEAR	3:54:34	$2.35\times 10^{10}$	$2.26\times 10^{6}$	1 $\times$ NVIDIA GH200 120GB

Appendix

Table 7: Compute and cost analysis across different alignment methods. CLEAR introduces only a modest increase in trainable parameters compared to standard LoRA while remaining substantially more memory-efficient than SFT.

Gate	HB ASR $\downarrow$	Unsafe Ref. $\uparrow$	Safe OR $\downarrow$	GSM8K $\uparrow$	MMLU $\uparrow$	TQA MC1 $\uparrow$	TQA MC2 $\uparrow$
MLP-16	0.25	94.00	11.60	34.42	55.48	37.21	53.10
MLP-12/16/20	0.00	93.50	14.40	41.93	55.58	37.21	53.14
Transformer-16	1.50	83.00	6.00	33.13	55.51	37.21	53.14
Transformer-12/16/20	1.00	94.00	9.60	39.88	55.62	36.84	53.04

Appendix

Table 8: Full gate architecture and layer selection ablation. All numbers are percentages. HB ASR denotes HarmBench attack success rate. Unsafe Ref. denotes refusal rate on unsafe XSTest prompts, and Safe OR denotes over-refusal rate on safe XSTest prompts. TQA MC1 and TQA MC2 denote TruthfulQA MC1 and MC2. MLP-16 and Transformer-16 use only layer 16, while MLP-12/16/20 and Transformer-12/16/20 use layers 12, 16, and 20. Bold indicates the best result for each metric.

Variant	HB ASR $\downarrow$	Unsafe Ref. $\uparrow$	Safe OR $\downarrow$	GSM8K $\uparrow$	MMLU $\uparrow$	TQA MC1 $\uparrow$	TQA MC2 $\uparrow$
LoRA	1.00	88.50	4.40	38.21	54.52	31.21	47.16
BCE-only	3.25	83.50	14.40	36.85	55.57	36.96	53.04
CLEAR	0.00	96.00	9.60	41.62	55.40	36.84	53.19
CLEAR w/o LoRA $L_{2}$	0.50	96.00	10.40	40.26	55.31	36.96	53.08

Appendix

Table 9: Training objective ablation on Gemma-2-2B-it. All numbers are percentages. HB ASR denotes HarmBench attack success rate. Unsafe Ref. denotes refusal rate on unsafe XSTest prompts, and Safe OR denotes over-refusal rate on safe XSTest prompts. CLEAR denotes the full objective with subtype-weighted BCE, hard pairwise margin loss, and LoRA

L_{2}

regularization. Bold indicates the best result for each metric.

Figure 4: Training dynamics of CLEAR on Gemma-2-2B-it and Llama-3-8B-Instruct. We plot the LoRA language modeling loss, gate BCE loss, hard-pair margin loss, and mean gate scores for safe and unsafe prompts over normalized training progress. Across both models, the optimization losses decrease, while the safe and unsafe gate scores become increasingly separated, indicating improved routing behavior during training.

Safety-Utility Trade-Off in Language Models

See all 17 papers

HiRoute: Hierarchical Routed Prompt Tuning for Safety Alignment of Large Language Models
Aug 13, 2026Fangzhou Chen, Shiji Zhao, Mengyang Wang +4LLM Safety Alignment Prompt Tuning
Institute of Artificial Intelligence, Beihang University
SafeAnchor: Preventing Cumulative Safety Erosion in Continual Domain Adaptation of Large Language Models
Apr 20, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuContinual Learning for LLMs LLM Fine-Tuning
The University of Hong Kong, Hong Kong, China · Brain Investing Limited, Hong Kong, China
DNAlign: Dynamic Null-Space Safe Alignment for LLMs
Oct 2, 2026Jisheng Dang, Yushuo Zhao, Dewei Liu +6LLM Alignment LLM Safety Alignment
School of Information Science and Engineering, Lanzhou University, Lanzhou 730000, China · National University of Singapore, Singapore 119077 · School of Medical Technology, Beijing Institute of Technology, Beijing 100081, China

Aug 13, 2026J/K move · Enter open · S save

HiRoute: Hierarchical Routed Prompt Tuning for Safety Alignment of Large Language Models

Fangzhou Chen, Shiji Zhao, Mengyang Wang, Qihui Zhu+3

Institute of Artificial Intelligence, Beihang University

LLM Safety Alignment Prompt Tuning Over-Refusal

Open paper AskPDF arXiv

Large language models (LLMs) remain vulnerable to harmful requests and jailbreak attacks. Parameter-efficient safety alignment methods based on prompt tuning typically rely on a single global prompt or externally selected prompt modules. Such static designs struggle to maintain a cross-category safety boundary while generating constructive responses tailored to specific risks and avoiding over-refusal of benign inputs. To address these limitations, we propose HiRoute, an input-adaptive hierarchical prompt-tuning framework that separates category-agnostic safety control from category-specific response guidance. HiRoute first trains a lightweight hierarchical router on representations extracted from a frozen LLM to jointly detect harmful intent and predict multi-label risk scores. It then freezes both the backbone model and the router and uses preference optimization with alternating gradient updates to learn a shared coarse-grained prompt and a set of fine-grained prompt experts as continuous embeddings. At inference time, benign inputs bypass the safety branch, whereas risky inputs are processed using the shared prompt together with a router-weighted mixture of risk-specific prompt experts. Experiments across three instruction-tuned models show that HiRoute achieves high safety rates across multiple safety benchmarks while preserving safe-response helpfulness, reducing over-refusal, and maintaining competitive performance on general-purpose tasks.

Abstract

Safety-Utility Trade-Off in Language Models

Explore similar work

HiRoute: Hierarchical Routed Prompt Tuning for Safety Alignment of Large Language Models

SafeAnchor: Preventing Cumulative Safety Erosion in Continual Domain Adaptation of Large Language Models

DNAlign: Dynamic Null-Space Safe Alignment for LLMs