Activation steering can be embedded directly into a language model's weights, shaping behaviour without inference-time intervention and offering a way to encode alignment prior to release. However, models are routinely fine-tuned after deployment, and it is unknown whether embedded interventions survive this. We study the stability of embedded steering for refusal ablation and brevity amplification across five instruction-tuned models (3B-14B) under non-adversarial SFT and RLHF. We find that steering degrades more under stronger optimisation pressure against the steered behaviour, which depends on both the training data and the training algorithm. Refusal ablation loses 69% of its effect on average under our SFT setup, but only 2% under our RLHF setup. The weight-edit, by contrast, remains almost unchanged (on average only 0.4% of it is restored), and the fine-tuning update along the steering direction is close to orthogonal to the pre-edit weight pattern (mean cosine similarity 0.071). Fine-tuning that restores the behaviour does so without reversing the edit. As the steered behaviour is not fully durable, embedded steering should be re-validated after downstream training.
Figures & tables
Figure 1: We embed activation steering into model weights and then fine-tune to test whether the steered behaviour survives. In the example (top), refusal ablation makes the model comply with a harmful prompt, and after SFT it refuses again, although only 1.5% of the weight-edit has been undone. Across all runs, fine-tuning can undo much of steering’s effect on behaviour (left) but almost none of the weight-edit (right). Behaviour can recover without the edit being reversed.
Model ID
Size
Reference
Llama-3.2-3B-Instruct
3B
Meta AI (2024)
Llama-3.1-8B-Instruct
8B
Grattafiori et al. (2024)
Llama-3-8B-Instruct
8B
Grattafiori et al. (2024)
Qwen3-14B
14B
Yang et al. (2025)
SOLAR-10.7B-Instruct
10.7B
Kim et al. (2024)
Table 1: List of evaluated models.
Refusal Ablation ( R )
Brevity Amplification ( L )
Model
base
steer
sft
rlhf
Psft
Prlhf
base
steer
sft
rlhf
Psft
Prlhf
Llama-3.2-3B
0.97
0.08
0.63
0.08
0.38
1.00
82.2
53.2
53.2
54.6
1.00
0.95
Llama-3.1-8B
0.96
0.01
0.51
0.01
0.47
1.00
169.8
85.1
70.2
77.4
1.18
1.09
Llama-3-8B
0.98
0.04
0.45
0.03
0.56
1.01
129.6
76.0
64.3
75.1
1.22
1.02
Qwen3-14B
0.99
0.12
0.86
0.14
0.15
0.98
30.8
17.1
26.8
15.3
0.29
1.13
SOLAR-10.7B
0.60
0.04
0.62
0.10
−0.04
0.89
126.5
72.1
128.2
74.5
−0.03
0.96
Table 2: Results for Refusal Ablation ( R ), Brevity Amplification ( L ), and Preservation P . R↓ and L↓ indicate more successful steering; P↑ indicates better steering preservation post-training, with P<0 indicating reversal past the base behaviour. 95% bootstrap CIs are reported in Appendix C . RLHF preserves both interventions, while SFT substantially reverses refusal ablation and preserves brevity on average.
Figure 2: Behavioural effects of activation steering across models and fine-tuning stages. (a) Refusal ablation: steering reduces the mean refusal rate from 0.90 to 0.06 across all models. Under RLHF, refusal remains low (mean Rrlhf=0.07 , range 0.01 – 0.14 ), while under SFT refusal partially returns (mean Rsft=0.61 , range 0.45 – 0.86 ). (b) Brevity amplification: steering shortens responses from 107.8 to 60.7 tokens on average. Under RLHF the effect is well preserved across all models; under SFT preservation is less consistent across models.
Figure 3: Scatterplot of behavioural recovery ( 1−P ) vs. vector recovery ρ across all runs. Plotted as 1−P so the top-right quadrant denotes greater recovery on both axes. If behaviour reverted by undoing the edit, points would rise towards the top right. Instead, ρ stays below 0.02 even for runs with near-complete behavioural recovery, and the two are not significantly correlated ( r=+0.23 , p=0.32 ).
Condition
Mean
Max
Ablation (SFT)
0.0102
0.0186
Ablation (RLHF)
0.0053
0.0090
Amplification (SFT)
0.0004
0.0010
Amplification (RLHF)
0.0001
0.0003
Table 3: Vector recovery ratio ρ across conditions. Values near 0 indicate the steering modification remains intact.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Refusal ( R )
Brevity ( L , tokens)
Model
base
SFT
base
RLHF
Llama-3.2-3B
0.97
0.99
82.2
85.0
Llama-3.1-8B
0.96
0.99
169.8
131.3
Llama-3-8B
0.98
0.96
129.6
121.4
Qwen3-14B
0.99
0.98
30.8
30.0
SOLAR-10.7B
0.60
0.79
126.5
111.6
Appendix
Table 4: Baseline training on non-steered models. Refusal rate R is shown pre-training (base) and after SFT; mean brief-response length L (tokens) is shown pre-training (base) and after RLHF.
Model
Rbase
Rsteer
Rsft
Rrlhf
Psft
Prlhf
Llama-3.2-3B
0.97 [0.93, 1.00]
0.08 [0.03, 0.14]
0.63 [0.53, 0.72]
0.08 [0.03, 0.14]
0.38 [0.28, 0.49]
1.00 [0.95, 1.06]
Llama-3.1-8B
0.96 [0.92, 0.99]
0.01 [0.00, 0.03]
0.51 [0.41, 0.61]
0.01 [0.00, 0.03]
0.47 [0.38, 0.57]
1.00 [1.00, 1.00]
Llama-3-8B
0.98 [0.95, 1.00]
0.04 [0.01, 0.08]
0.45 [0.35, 0.55]
0.03 [0.00, 0.07]
0.56 [0.46, 0.66]
1.01 [1.00, 1.03]
Qwen3-14B
0.99 [0.97, 1.00]
0.12 [0.06, 0.19]
0.86 [0.79, 0.92]
0.14 [0.08, 0.21]
0.15 [0.08, 0.23]
0.98 [0.92, 1.03]
SOLAR-10.7B
0.60 [0.50, 0.70]
0.04 [0.01, 0.08]
0.62 [0.53, 0.71]
0.10 [0.04, 0.16]
− 0.04 [ − 0.20, 0.10]
0.89 [0.81, 0.97]
Appendix
Table 5: Refusal Ablation ( R ) and preservation P with 95% bootstrap CIs over n=100 harmful prompts.
Model
Lbase
Lsteer
Lsft
Lrlhf
Psft
Prlhf
Llama-3.2-3B
82.2 [75.9, 88.7]
53.2 [50.2, 56.4]
53.2 [49.7, 57.1]
54.6 [51.6, 57.9]
1.00 [0.92, 1.08]
0.95 [0.91, 1.00]
Llama-3.1-8B
169.8 [159.0, 181.2]
85.1 [78.9, 92.0]
70.2 [65.6, 75.1]
77.4 [72.8, 82.2]
1.18 [1.12, 1.24]
1.09 [1.05, 1.14]
Llama-3-8B
129.6 [121.8, 138.1]
76.0 [69.4, 84.8]
64.3 [60.0, 69.0]
75.1 [70.8, 79.6]
1.22 [1.11, 1.42]
1.02 [0.94, 1.18]
Qwen3-14B
30.8 [29.3, 32.5]
17.1 [16.4, 17.8]
26.8 [25.7, 28.0]
15.3 [14.7, 16.0]
0.29 [0.20, 0.36]
1.13 [1.09, 1.17]
SOLAR-10.7B
126.5 [119.2, 134.1]
72.1 [67.1, 77.2]
128.2 [117.4, 139.8]
74.5 [70.8, 78.3]
− 0.03 [ − 0.18, 0.10]
0.96 [0.90, 1.02]
Appendix
Table 6: Brevity Amplification ( L , tokens) and preservation P with 95% bootstrap CIs over n=300 long-answer prompts.
Condition
u(d)
u(d)/u(r)
rev
Ablation (SFT)
0.0213
2.52 ×
0.00268
Ablation (RLHF)
0.0071
4.06 ×
0.00042
Amplification (SFT)
0.0135
1.65 ×
0.00043
Amplification (RLHF)
0.0065
3.45 ×
0.00009
Appendix
Table 7: Per-condition means of the relative update projection along the steering direction u(d) , its ratio to the mean of the 1,000 -direction random null u(r) , and the reversal fraction rev (Eq. 8 ).
Post-training adaptation of language models is commonly achieved through parameter updates or input-based methods such as fine-tuning, parameter-efficient adaptation, and prompting. In parallel, a growing body of work modifies internal activations at inference time to influence model behavior, an approach known as steering. Despite increasing use, steering is rarely analyzed within the same conceptual framework as established adaptation methods. In this work, we argue that steering should be regarded as a form of model adaptation. We introduce a set of functional criteria for adaptation methods and use them to compare steering approaches with classical alternatives. This analysis positions steering as a distinct adaptation paradigm based on targeted interventions in activation space, enabling local and reversible behavioral change without parameter updates. The resulting framing clarifies how steering relates to existing methods, motivating a unified taxonomy for model adaptation.
Simon Ostermann, Daniil Gurgurov, Tanja Baeumel +4
Saarland University · German Research Center for Artificial Intelligence (DFKI) · Centre for European Research in Trusted AI (CERTAIN) +6
Activation steering (AS) has emerged as a popular inference-time technique for modulating the behavior of large language models (LLMs). AS modulates model behavior in a flexible and lightweight manner by injecting task-specific steering vectors (SVs) into intermediate activations. Meanwhile, recent work has identified emergent misalignment (EM) as an emerging safety concern, whereby finetuning on unsafe examples from a narrow task unexpectedly induces broadly unsafe behavior in unrelated domains. With the increasing popularity of AS, it is pressing yet under-explored to thoroughly investigate whether AS can induce EM. In this paper, we conduct a comprehensive study of activation-steering-induced emergent misalignment. Specifically, (1) we identify a steering vector construction to confirm that SVs derived from a narrow task can indeed induce misalignment across broad unrelated domains. Qualitative measurement further reveals that AS-induced EM may pose more severe safety risks than finetuning-induced EM, i.e., AS elicits more actionable harmful responses with greater semantic coherence than those generated by finetuned models. (2) We characterize the properties of AS-induced EM by analyzing key steering-specific factors, including steering magnitude, the low-rank structure of the steering subspace, and the number of fine-tuning epochs used during steering-vector construction. (3) We evaluate the robustness and sensitivity of AS-induced EM across diverse model families, model scales, target tasks, and intervention layers. Our findings reveal activation steering as a significant yet underexamined source of emergent misalignment and provide an activation-space perspective on emergent misalignment and its safety risks.
Qi Cao, Jian Lou, Meiting Liu +3
Nanyang Technological University · Sun Yat-sen University · University of Science and Technology of China
Language models are instruction-tuned to refuse harmful requests, but the mechanisms underlying this behavior remain poorly understood. Popular steering methods operate on the residual stream and degrade output coherence at high intervention strengths, limiting their practical use. We introduce contrastive neuron attribution (CNA), which identifies the 0.1% of MLP neurons whose activations most distinguish harmful from benign prompts, requiring only forward passes with no gradients or auxiliary training. In instruct models, ablating the discovered circuit reduces refusal rates by over 50% on a standard jailbreak benchmark while preserving fluency and non-degeneracy across all steering strengths. Applying CNA to matched base and instruct models across Llama and Qwen architectures (from 1B to 72B parameters), we find that base models contain similar late-layer discrimination structures but steering these neurons produces only content shifts, not behavioral change. These results demonstrate that neuron-level intervention enables reliable behavioral steering without the quality tradeoffs of residual-stream methods. More broadly, our findings suggest that alignment fine-tuning transforms pre-existing discrimination structure into a sparse, targetable refusal gate.