Existing activation steering methods often assume that a high-level concept can be mediated by a single steering direction. To support this, two complementary interventions should be achieved: additive steering should induce the target behavior, while directional ablation should suppress it. Yet behaviors may occupy richer activation geometries beyond a single direction, and semantically similar behaviors may be represented by distinct directions. In this work, we study how many directions can reliably control two different types of refusal behaviors: refusal triggered by the safety alignment and refusal in general contexts. Given the limited expressive capability of a single steering vector, we study the general setting of steering subspaces and introduce the notion of steering dimensionality as the minimum subspace dimensionality required to reliably control a behavior. We characterize sufficient steering subspaces that cover the full extent of the target behavior through both the (monotonic) improvement before the sufficient dimensionality, and the saturation beyond it. Empirically, we find that refusal triggered by safety alignment is 1-dim steerable, while multiple distinct steering directions can achieve comparable control. In contrast, refusal in general contexts exhibits substantially richer activation geometry where even 5-dim steering subspaces fail to reliably capture its full steerable variation. Our results reveal that the activation geometry underlying refusal is highly context-dependent and can be substantially more complex than a single linear steering direction.
Figures & tables
Figure 1: Qualitative examples of two types of refusal studied in this work: Refusal triggered by the safety alignment, and refusal in general contexts.
Figure 2: Steering subspaces for refusal triggered by safety alignment on Llama-3.1-8B-Instruct. Top: steerability (%), measuring behavioral control. Bottom: incoherence ratio (%), measuring response-quality degradation. Higher steerability and lower incoherence are desirable. Results demonstrate that refusal triggered by the safety alignment is 1-dim steerable, while such steering subspaces do not cover the semantics of refusal in general contexts.
Figure 3: Steering subspaces for refusal triggered by safety alignment on Qwen2.5-3B-Instruct. Top: steerability (%), measuring behavioral control. Bottom: incoherence ratio (%), measuring response-quality degradation. Results demonstrate that refusal triggered by the safety alignment is 1-dim steerable and multiple distinct steering directions can achieve comparable control, while such steering subspaces do not fully cover the semantics of refusal in general contexts. We mark k∈{3,5} in gray because the ablation steering starts to hurt the response coherence, resulting in a decline in the steerability score.
Figure 4: Steering subspaces for refusal in general contexts on Llama-3.1-8B-Instruct. Top: steerability (%), measuring behavioral control. Bottom: incoherence ratio (%), measuring response-quality degradation. Higher steerability and lower incoherence are desirable. Results demonstrate that 5-dim steering subspaces still fail to capture the entire semantics of refusal in general contexts.
Figure 5: Steering subspaces for refusal in general contexts on Qwen2.5-3B-Instruct. Top: steerability (%), measuring behavioral control. Bottom: incoherence ratio (%), measuring response-quality degradation. Higher steerability and lower incoherence are desirable. Results demonstrate that 5-dim steering subspaces still fail to capture the entire semantics of refusal in general contexts. We mark k∈{3,5} in gray because the ablation steering starts to hurt the response coherence, resulting in a decline in the steerability score.
Prior work argues that refusal in large language models is mediated by a single direction, enabling steering and abliteration. We show that this account is incomplete: across diverse refusal and non-compliance categories, refusal behaviors correspond to geometrically distinct directions in activation space. Yet activation steering along any refusal-related direction produces nearly identical refusal--over-refusal trade-offs, acting as a shared one-dimensional control knob. Thus, different directions primarily affect not whether the model refuses, but how it refuses. Using sparse autoencoders, we uncover a structured internal representation of refusal: a reusable core of shared refusal latents supplemented by style- and domain-specific latents. Linear interventions collapse this structure into uniform behavioral control, flattening mechanistic differences across refusal types. Our results reconcile the apparent simplicity of refusal steering with the diversity of refusal behaviors, and clarify the limits of linear interpretability for aligned model behavior.
Faaiz Joad, Majd Hawasly, Sabri Boughorbel +2
Qatar Computing Research Institute, HBKU, Doha, Qatar
Activation steering manipulates large language model behavior by intervening on internal activations, but the mechanistic basis of these interventions remains poorly understood. We decompose refusal steering into component-level interventions across four open-weight models, identifying the sparse subsets of attention and MLP components whose steering suffices to reproduce the full behavioral effect. We find that refusal directions concentrate in sparse component mechanisms comprising 28--48% of upstream components, retaining 88--101% of steering effectiveness. Within these mechanisms, effective steering further concentrates in approximately 50% of residual stream dimensions, retaining 85--98% of the component-mechanism baseline, consistent with a privileged basis structure. Sparsity thus operates at two levels: which components are steered, and which dimensions within those components carry the signal. Together these findings show that refusal is not diffusely encoded across a transformer but assembled by a structured, identifiable mechanism, providing a foundation for mechanistic understanding of how refusal behaviors are represented and steered. To facilitate reproducibility, we release all code and raw experimental results in https://github.com/wang-research-lab/Refusal_Mechanisms.
Vincent Siu, Glenn Grant-Richards, Vlad Pavlovich +3
Refusal training protects AI models from jailbreaks by training models to decline unsafe queries, reducing the risk of misuse. Recent work finds that refusal behavior in aligned language models can be mediated by a single activation direction or a low-dimensional refusal subspace shared across harmful prompts: ablating those directions suppresses refusals while largely preserves other model capabilities. Yet it remains unclear why safety-critical features in a wide range of models emerge in a concentrated, low-dimensional structure. In a case study of OLMo-2-0425-1B-Instruct we find that the refusal geometry reflects refusal training: activation updates resulting from refusal-completion first-token losses explain the resulting refusal direction and refusal subspace. We study refusal directions through the training dynamics across refusal datasets and reveal that their brittleness is associated with repetitive refusal starts, which in turn is linked to concentration of gradients and refusal features in a low-dimensional subspace. Across frozen-model analyses and controlled synthetic fine-tuning, we find evidence of a hardening lever: diverse refusal starts can raise stable ranks of gradients and activation changes, making refusals harder to remove with a vector ablation attack.