STR: Supervised Transcoder Replacement for Reducing Steering Side Effects

Haonan Yu; Junhao Liu; Zhenyu Yan; Haoran Lin; Xin Zhang

cs.AISep 26, 2026

STR: Supervised Transcoder Replacement for Reducing Steering Side Effects

Authors: Haonan Yu, Junhao Liu, Zhenyu Yan, Haoran Lin, Xin Zhang

Abstract

Model steering can strengthen a target behavior while degrading other useful behaviors. We introduce Supervised Transcoder Replacement (STR) to reduce these side effects for existing steering methods, including those fitted without a protection objective. STR learns a replacement for the multilayer perceptron (MLP) computation at the steering layer through supervision for target control, non-target preservation, and fidelity without steering. Selected steering methods then fit directions on the frozen replacement while retaining their own fitting objectives. We evaluate three steering methods across Gemma and Llama models using Corrigibility preferences and four harmful-request safety datasets. SALAD-Bench supplies protection training data and a separate in-distribution evaluation split; HarmBench, AdvBench, and StrongREJECT are reserved for out-of-distribution testing. STR substantially reduces steering side effects on the in-distribution evaluation and extends this protection to the unseen safety datasets while retaining effective target control. For target-only supervised steering vectors, pooled out-of-distribution attack success rate falls from 42.46% to 14.42% on Gemma-3-4B and from 34.97% to 12.91% on Gemma-3-12B. These results show that replacement training can benefit steering methods fitted without protection objectives.

Figures & tables

Figure 1: Overview of STR. (A) Train the replacement. A transcoder replaces one MLP mapping and is trained with an auxiliary vector $u$ using three objectives: improve target behavior under steering, preserve protected behavior under the same steering, and preserve unsteered outputs. The original model $M_{0}$ provides the teacher distributions; the surrounding language model remains frozen. (B) Fit and apply steering. After freezing the transcoder and setting aside $u$ , a selected steering method fits a latent direction $v_{j}$ under its original objective and applies it with strength $\alpha$ . Both JumpReLU and TopK follow this workflow, using their respective intervention rules in Equation 3 .

	Gemma-3-4B		Llama-3.1-8B		Gemma-3-12B
Condition	$U\uparrow$	ASR $\downarrow$	$U\uparrow$	ASR $\downarrow$	$U\uparrow$	ASR $\downarrow$
$M_{0}$ -off	78.96	4.60	79.60	2.44	77.98	3.07
$M_{\theta}$ -off	80.02	4.30	82.55	2.74	81.44	2.98
$M_{0}$ -CAA	81.46	32.22	86.06	6.97	84.13	26.71
$M_{\theta}$ -CAA	81.09	4.95	86.10	2.37	87.49	4.07
$M_{0}$ -SSV	80.69	36.12	77.39	4.97	84.74	29.29

Table 1: In-distribution evaluation on the SALAD-Bench development split.

	Gemma-3-4B		Llama-3.1-8B		Gemma-3-12B
Condition	$U\uparrow$	ASR $\downarrow$	$U\uparrow$	ASR $\downarrow$	$U\uparrow$	ASR $\downarrow$
$M_{0}$ -off	77.11	11.44	76.71	17.16	73.15	10.92
$M_{\theta}$ -off	78.50	10.85	79.25	16.29	78.19	11.09
$M_{0}$ -CAA	80.31	45.85	83.15	25.82	81.46	32.77
$M_{\theta}$ -CAA	79.53	14.49	82.23	14.41	84.13	12.29
$M_{0}$ -SSV	79.74	42.46	74.10	20.37	82.02	34.97

Table 2: Out-of-distribution evaluation.

Model	Method	$\Delta U_{\mathrm{STR}}-\Delta U_{\mathrm{original}}$	$\Delta\operatorname{ASR}_{\mathrm{STR}}-\Delta\operatorname{ASR}_{\mathrm{original}}$
Gemma-3-4B	CAA	$-0.77\;[-2.14,+0.61]$	$-31.36\;[-35.81,-26.91]$
	SSV	$-0.16\;[-1.50,+1.17]$	$-28.04\;[-31.93,-24.21]$
	OPIUM	$-1.32\;[-2.69,+0.11]$	$-25.02\;[-28.71,-21.45]$
Llama-3.1-8B	CAA	$-0.91\;[-1.52,-0.30]$	$-11.41\;[-14.30,-8.67]$
	SSV	$+10.64\;[+9.15,+12.13]$	$-5.00\;[-7.20,-2.91]$
	OPIUM	$-0.57\;[-1.07,-0.05]$	$-0.86\;[-2.79,+1.02]$

Table 3: Paired test differences between STR and original-model steering with the same method.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Condition	HarmBench 240 requests	AdvBench 520 requests	StrongREJECT 313 requests
Gemma-3-4B-it
$M_{0}$ -off	25.42	0.96	1.28
$M_{\theta}$ -off	22.64	1.54	2.34
$M_{0}$ -CAA	57.08	35.58	37.06
$M_{\theta}$ -CAA	29.58	1.79	2.98
$M_{0}$ -SSV	53.89	33.27	34.08

Appendix

Table 4: Request-averaged attack success rates on unseen safety benchmarks.

Condition	less-HHH 252 items	neutral-HHH 138 items	more-HHH 302 items	no-goal-change 24 items
Gemma-3-4B-it
$M_{0}$ -off	54.16	91.61	92.61	39.47
$M_{\theta}$ -off	56.40	92.24	93.72	40.08
$M_{0}$ -CAA	61.62	92.35	93.29	43.86
$M_{\theta}$ -CAA	65.21	89.65	90.34	35.74
$M_{0}$ -SSV	61.31	91.27	92.59	45.10

Appendix

Table 5: Target utility by Corrigibility test-item source.

Language Model Steering

Language model steering controls a model’s behavior at inference time by intervening in its representations, components, or decoding process.

See all 283 papers

Aug 9, 2026

Safety Cost of Steering Vectors Is Separable and Reducible

Steering vectors are a lightweight tool for controlling LLM behavior. However, emerging evidence shows that steering vectors can unintentionally compromise a model's safety mechanisms and increase compliance with harmful requests, while no effective mitigation yet exists. In this work, we show that this safety degradation arises from a separable component in the vector that disrupts the model's safety mechanisms but contributes little to the steering objective. We identify and remove this safety-degrading component, formulating the task as a constrained optimization problem solved through primal-dual updates, subject to preserving the intended steering effect and bounding false refusal. The resulting solution is both interpretable and surgical: the optimization recovers a single direction whose ablation from the steering vector restores model safety with minimal utility cost. Across models, steering behaviors, and attack suites, including unseen attacks types, our method substantially reduces steering-induced safety degradation while preserving the original steering effect with minimal impact on false refusal. Our method offers a post-hoc correction to steering vectors that mitigates their safety cost, and more broadly, it provides a general recipe for applying activation-level model interventions without paying a safety tax.

Yuxiao Li, Gjergji Kasneci

Technical University of Munich Munich Center for Machine Learning (MCML)

Language Model Steering LLM Safety Alignment LLM Safety Evaluation

Ask PDF

Oct 6, 2026

Does Steering Break Your Model? A Multi-Dimensional Evaluation Suite for LLM Steering Methods

Activation steering provides a lightweight and flexible way to control large language model (LLM) behavior. However, effective steering requires more than inducing the intended behavior: it should also limit unintended changes and remain robust across inputs and training data. Existing evaluations cover these dimensions only in fragments. As a result, the trade-offs between efficacy and side effects have not been systematically characterized. We introduce SteerScope, a two-axis, multi-dimensional evaluation suite that jointly characterizes steering outcomes and method properties through 15 metrics. We score target efficacy and side effects on language quality, task capabilities, and safety and reliability, and further assess generalization and data dependence through steering-specific metrics for sample efficiency and sample sensitivity. Rather than comparing methods at a single operating point, we characterize the trade-offs between efficacy and side effects. Under matched models, tasks, and evaluation protocols, we benchmark 23 methods spanning 4 families, including prompting, LoRA, and SFT as baseline methods, and release the suite as an extensible codebase. We find that current activation steering methods do not yet surpass the Prompt Steering baseline in their overall balance between steering efficacy and side effects: across both model scales, no evaluated activation steering method achieves higher efficacy without incurring greater composite side effects. We further uncover a consistent coupling between steering efficacy and side effects. Under OOD prompts, target efficacy is often preserved, whereas side effects tend to become more pronounced, particularly through declines in instruction relevance and fluency. Methods also exhibit sharply different sample-efficiency profiles.

Haotian Yang, Huikang Jiang, Yucheng Wu +4

State Key Laboratory of Multimedia Information Processing, Peking University · School of Computer Science, Peking University · Columbia University +2

LLM Evaluation Language Model Steering Language Model Safety Evaluation Activation Steering

Ask PDF

Jul 22, 2026

OPIUM: Mitigating Steering Externalities and Over-Refusal via Dual Objective Latent Optimization

Activation steering provides a lightweight mechanism for controlling large language models at inference time, but steering vectors can have unintended externalities: utility vectors may weaken safety behavior, while refusal vectors may induce over-refusal on benign prompts. We introduce OPIUM (Optimizing Protected Injections via Utility Manifolds), a training-free method for sanitizing steering vectors through representation matching. Given reference behaviors on two prompt sets, OPIUM optimizes a new steering vector that preserves the downstream representations induced by the desired intervention while matching a safer reference behavior on prompts where the original vector fails. Across steering-externality and over-refusal settings, OPIUM improves the safety--utility tradeoff relative to vanilla steering and directional ablation, suggesting that harmful side effects of activation steering can often be mitigated directly in activation space.

Kavin Aravindan, Arihant Rastogi, Krishak Aneja +4

IIIT Hyderabad, India.

LLM Safety Alignment Over-Refusal Activation Steering

Ask PDF

Abstract

Language Model Steering

Explore similar work

Safety Cost of Steering Vectors Is Separable and Reducible

Does Steering Break Your Model? A Multi-Dimensional Evaluation Suite for LLM Steering Methods

OPIUM: Mitigating Steering Externalities and Over-Refusal via Dual Objective Latent Optimization