Component and Dimension Sparsity in Transformer Refusal Mechanisms

Vincent Siu; Glenn Grant-Richards; Vlad Pavlovich; Yizhou Sun; Dawn Song; Chenguang Wang

cs.CLSep 30, 2026

Component and Dimension Sparsity in Transformer Refusal Mechanisms

Authors: Vincent Siu, Glenn Grant-Richards, Vlad Pavlovich, Yizhou Sun, Dawn Song, Chenguang Wang

Organizations: UC Santa Cruz · UCLA · UC Berkeley

Abstract

Activation steering manipulates large language model behavior by intervening on internal activations, but the mechanistic basis of these interventions remains poorly understood. We decompose refusal steering into component-level interventions across four open-weight models, identifying the sparse subsets of attention and MLP components whose steering suffices to reproduce the full behavioral effect. We find that refusal directions concentrate in sparse component mechanisms comprising 28--48% of upstream components, retaining 88--101% of steering effectiveness. Within these mechanisms, effective steering further concentrates in approximately 50% of residual stream dimensions, retaining 85--98% of the component-mechanism baseline, consistent with a privileged basis structure. Sparsity thus operates at two levels: which components are steered, and which dimensions within those components carry the signal. Together these findings show that refusal is not diffusely encoded across a transformer but assembled by a structured, identifiable mechanism, providing a foundation for mechanistic understanding of how refusal behaviors are represented and steered. To facilitate reproducibility, we release all code and raw experimental results in https://github.com/wang-research-lab/Refusal_Mechanisms.

Figures & tables

Figure 1: Two-level sparsity in transformer refusal mechanisms. The residual stream passes through a sequence of attention and MLP components; a sparse subset (28–48% across models) forms a sufficient mechanism to steer refusal behavior, where steering a component via orthogonalization means removing the output’s projection onto the refusal direction. Zooming into each identified component, effective steering also requires intervening on only the same

\approx 50\%

of residual dimensions across all components, consistent with a privileged basis structure in the residual stream. Prior work steers all dimensions of all upstream components simultaneously by intervening on the residual stream; we instead steer individual component outputs, allowing the residual connection to proceed unaltered.

Model	Attention	MLPs	Components	Mechanism ASR	Sparse Dim. ASR
	Components		(% of Upstream)	(% of Full)	(% of Full, % of Dims)
Gemma2-2B-IT	{4,6,9,10,11}	{4,6–12}	48.1	88.3	85.7 (51.0%)
Qwen3-4B	{12,14,16,18}	{6,12,14–18}	28.2	90.9	87.8 (53.1%)
Phi4-Mini	{9,13}	{4,6–13}	37.9	96.5	93.2 (54.3%)
Mistral-3.2-Small	{8,11}	{9–16}	28.6	100.5	98.6 (54.3%)

Table 1: Sparse refusal mechanisms identified across four models. The layer indices of the selected attention and MLP components in the identified mechanism are provided above. Components is the fraction of upstream components comprising the mechanism. Mechanism ASR reports attack success rate as a percentage of the full residual-stream steering baseline: orthogonalizing the dense direction

\mathbf{r}^{*}

at the input of

l^{*}

, as in prior work, which is the peak value of the layer sweep in Appendix C . A value above 100% indicates the mechanism steers more effectively than this single-point intervention. Sparse Dim. ASR reports attack success rate when steering the mechanism using only a subset of hidden state dimensions, with the percentage of residual dimensions used given in parentheses. We exclude the embedding matrix as it is never retained by the elimination procedure across models.

Figure 2: Residual stream steering effectiveness as a function of dimension percentage. Each marker shows the ASR recovered relative to full-dimensional steering, plotted against the fraction of hidden dimensions used. Marker labels indicate the corresponding norm explanation threshold $t$ – the fraction of $\|\mathbf{r}^{*}\|^{2}$ explained by the retained dimensions. Across all four models, retaining $\sim$ 50% of dimensions at the 95% norm threshold recovers over 90% of full steering ASR, with some models exceeding the dense baseline. Performance degrades sharply only as the budget falls below the 50% norm threshold, confirming that the refusal direction is concentrated in a sparse set of high-magnitude coordinates.

Figure 3: Sparse mechanisms versus their complement sets. Each arrow points from the complement (excluded components, hollow diamond) to the identified mechanism (included components, filled circle). Mechanisms comprising 28–48% of upstream components reproduce 88–101% of the full steering effect, while their complements – the remaining 52–72% of components – largely fail to elicit refusal.

	MMLU (acc. %)			GSM8k (acc. %)			Pile (PPL)
Model	Base	$\Delta$ Full	$\Delta$ Mech	Base	$\Delta$ Full	$\Delta$ Mech	Base	$\Delta$ Full	$\Delta$ Mech
Gemma2-2B	58.07	$-$ 1.54	$\bm{-0.60}$	46.32	$\bm{-1.59}$	$-$ 2.96	10.83	$+$ 0.267	$\bm{+0.233}$
Phi4-Mini	72.36	$-$ 22.44	$\bm{-1.98}$	86.66	$-$ 28.13	$\bm{-1.36}$	9.64	$+$ 2.593	$\bm{+0.113}$
Qwen3-4B	75.45	$-$ 0.73	$\bm{-0.27}$	89.61	$+$ 0.08	$\bm{+0.38}$	10.50	$+$ 0.023	$\bm{-0.021}$
Mistral-3.2-Small	83.24	$-$ 1.24	$\bm{-0.99}$	95.68	$\bm{-0.76}$	$-$ 0.99	6.06	$+$ 0.027	$\bm{+0.017}$

Table 2: Capability change relative to the unmodified model. MMLU and GSM8k are in accuracy points, so negative

\Delta

is degradation; Pile is in perplexity, so positive

\Delta

is degradation.

\Delta

Full is the dense intervention applied to all upstream components;

\Delta

Mech is the identified sparse mechanism under the 95% norm-explained coordinate mask. Bold marks the smaller degradation within each pair. The sparse mechanism is equal or better in ten of twelve cells.

Figure 4: Per-component linear attribution to the refusal direction. For each component in the identified mechanism, bars show the scalar projection of its difference-in-mean vector onto $\mathbf{r}^{*}$ , measuring how much each component contributes to assembling the refusal direction at $l^{*}$ . Attention components (orange) and MLP components (blue) are shown by layer index. Negative values indicate components that write against the refusal direction. Each reconstruction scores (fraction of $\|\mathbf{r}^{*}\|^{2}$ recovered) is provided in the top left.

Figure 5: Component write profiles sorted by refusal importance rank. Top 10% of coordinates by write magnitude shown for each component, sorted by $|\mathbf{r}^{*}_{i}|$ descending so the most refusal-important coordinates appear on the left. Outputs are unit-normalised to control for LayerNorm-induced scale differences. Point size scales with write magnitude; color indicates relative layer depth (early: purple, late: gold). High-magnitude writes are distributed throughout the coordinate space rather than concentrated at high refusal importance ranks, indicating component-specific rather than shared sparse subspaces.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Exhaustive steering performance of the difference-in-means direction extracted from each layer’s input. Steering is performed in the residual stream at the layer input, replicating prior work. The highest performing steering direction is highlighted in red and is used to find the mechanism in our study. All directions are extracted from the last post-instruction token. The highest ASR value is also used as a reference value that the mechanisms in Table 1 compare against.

Figure 7: Distribution of refusal direction coefficients at $l^{*}$ . Coordinate values of $\mathbf{r}^{*}$ for each model. Blue bars represent typical dimensions within the central distribution; red bars with black outlines denote statistical outliers ( $>2.5\times$ IQR). The y-axis is log-scaled to make the sparse high-magnitude dimensions visible. Summary statistics ( $\mu,\sigma$ ) and outlier counts are provided for each model.

Model	Expl. Norm (%)	Dimensions Used (%)	Rel. ASR (%)
Gemma2-2B	90	59.9	1.99
Gemma2-2B	95	49.1	1.10
Mistral-3.2-Small	90	56.6	14.35
Mistral-3.2-Small	95	45.7	12.30
Phi4-Mini	90	56.6	2.46
Phi4-Mini	95	45.7	2.25

Appendix

Table 3: Steering effectiveness when using only bottom-magnitude dimensions of

\mathbf{r}^{*}

. Relative ASR is reported as a percentage of the full-direction baseline. Despite comparable dimension budgets to the top-magnitude condition, bottom-magnitude steering fails to elicit refusal across all models.

Model	Layer	$t$ (%)	$k$ (%)	Orig. Rel. ASR (%)	Rot. Rel. ASR (%)	Std	$\Delta$	$N$
Gemma2-2B-IT	13	95	50.9	101.5	0.6	0.003	$-$ 101.0	5
Mistral-3.2-Small	17	95	54.3	96.8	11.2	0.005	$-$ 85.6	5
Phi4-Mini	14	95	54.3	100.6	1.8	0.002	$-$ 98.8	5
Qwen3-4B	19	95	53.1	99.0	10.8	0.005	$-$ 88.2	5

Appendix

Table 4: Rotation control experiment results. For each model, the same number of dimensions

k

used in the 95% norm-threshold sparse steering condition is selected from a randomly rotated direction

\tilde{r}=Q\mathbf{r}^{*}

, where

Q\sim\mathrm{Haar}(O(d))

. Orig. Rel. ASR is the ASR of the original sparse steering as a percentage of the full-component dense baseline. Rot. Rel. ASR is the mean ASR across

N=5

random rotations; Std is the standard deviation across trials.

\Delta

is the difference. Near-zero rotated ASR confirms that the specific native-basis coordinates selected by the norm threshold are functionally privileged rather than interchangeable with an arbitrary coordinate system of the same size.

Figure 8: Component cosine similarity with $\mathbf{r}^{*}$ on harmful vs. harmless prompts. Colored bars show the IQR of cosine similarity between each component’s last-token output and $\mathbf{r}^{*}$ over harmful prompts (blue: MLP, orange: attention). Outlined boxes show the corresponding IQR over harmless prompts. Red lines indicate the selectivity shift – the difference in mean cosine similarity between the two distributions. A large upward red line indicates a component that aligns more strongly with the refusal direction on harmful content than harmless content.

Figure 9: Mechanism Sufficiency: Leave-One-Out ASR Drop. The absolute drop in Attack Success Rate (ASR) when each individual component is removed from the identified sparse mechanism. Higher bars indicate components that act as critical bottlenecks for successful refusal steering. Blue bars represent MLP layers, and orange bars represent Attention mechanisms.

Figure 10: First-Round Component Ablation: Initial Redundancy. The absolute drop in ASR when individual components are removed from the initial, unpruned candidate circuit. The flat profile across all models indicates that no single component acts as a bottleneck in the presence of the full redundant set.

Model	$D$	All-comp.	$\alpha{=}0.90$	$\alpha{=}0.95$	$\bm{\alpha{=}0.95}$	$\alpha{=}0.95$	$\alpha{=}0.99$
		ASR	$\tau{=}0.05$	$\tau{=}0.02$	$\bm{\tau{=}0.05}$	$\tau{=}0.10$	$\tau{=}0.05$
Gemma2-2B	27	0.834	12 (44.4) / 0.740	15 (55.6) / 0.807	13 (48.1) / 0.764	12 (44.4) / 0.738	12 (44.4) / 0.785
Phi4-Mini	29	0.959	7 (24.1) / 0.740	13 (44.8) / 0.947	11 (37.9) / 0.924	9 (31.0) / 0.908	13 (44.8) / 0.902
Qwen3-4B	39	0.949	9 (23.1) / 0.691	14 (35.9) / 0.893	12 (30.8) / 0.893	10 (25.6) / 0.854	13 (33.3) / 0.906
Mistral-3.2-Small	35	0.936	7 (20.0) / 0.727	14 (40.0) / 0.912	10 (28.6) / 0.859	8 (22.9) / 0.816	10 (28.6) / 0.891

Appendix

Table 5: Sensitivity of mechanism size and behavior to the Stage 1 and Stage 2 thresholds. Each cell reports mechanism size as a raw count, the percentage of the component budget

D=2l^{*}+1

in parentheses, and held-out test ASR (

n=512

, LlamaGuard-3). All-comp. ASR is the reference condition in which every one of the

D

upstream components is steered. The paper’s operating point is bolded. Test ASR remains within approximately five points of that operating point across every threshold setting.

Model	Relation	Smaller $\|\mathcal{M}\|$	Larger $\|\mathcal{M}\|$	Shared	Subset?
Gemma2-2B	$\tau{=}0.05\subseteq\tau{=}0.02$	13	15	13	✓
	$\tau{=}0.10\subseteq\tau{=}0.05$	12	13	12	✓
	$\tau{=}0.10\subseteq\tau{=}0.02$	12	15	12	✓
	$\alpha{=}0.90\subseteq\alpha{=}0.95$	12	13	12	✓
	$\alpha{=}0.95\subseteq\alpha{=}0.99$	13	12	11	$\times$
Phi4-Mini	$\tau{=}0.05\subseteq\tau{=}0.02$	11	13	11	✓

Appendix

Table 6: Membership containment across threshold settings. Shared is the number of components present in both mechanisms. Subset? indicates whether the smaller mechanism is a strict subset of the larger. Every

\tau

relation nests;

\alpha=0.90\subseteq\alpha=0.95

nests for three of four models;

\alpha=0.95\subseteq\alpha=0.99

nests for none, including the Mistral-3.2-Small case where both mechanisms have size 10 but differ by one component.

Model	Tie steps	Mean tie	Mean	$\|\mathcal{M}\|$	Test ASR	Core /
	(%)	set size	Jaccard	range	range	peripheral
Gemma2-2B	58	1.92	0.88	13–13	0.764–0.791	12 / 3
Phi4-Mini	55	2.45	0.66	10–13	0.896–0.924	8 / 9
Qwen3-4B	73	2.18	0.71	12–15	0.871–0.893	9 / 8
Mistral-3.2-Small	62	2.31	0.81	10–11	0.859–0.887	8 / 4

Appendix

Table 7: Stability of Stage 2 under random tie-breaking. Tie steps is the fraction of elimination steps on the deterministic trajectory at which at least two candidates fall within

\epsilon=0.005

of the minimum drop, and mean tie set size the average number of such candidates. Remaining columns aggregate over four realizations per model (three random seeds plus the deterministic baseline). Core counts components present in all four realizations; peripheral counts components present in some but not all.

Model	Post-Instruction Token	Target Layer	Norm Explained (%)	$\alpha$	$\tau$
Phi-4-Mini	-1	14	95.0	0.95	0.05
Gemma-2-2B-IT	-1	13	95.0	0.95	0.05
Mistral-Small-3.2	-1	17	95.0	0.95	0.05
Qwen3-4B	-1	19	97.5	0.95	0.05

Appendix

Table 8: Per-model hyperparameters. Target layer indicates the layer

\mathbf{l}^{*}

at which the refusal direction

\mathbf{r}^{*}

was extracted. The % Norm Explained controls the fraction of the direction’s squared norm explained by the retained dimensions, resulting in the chosen dimensions and ASRs reported in Table 1 .

\alpha

is the projection retention threshold used in Stage 1 greedy elimination and

\tau

the ASR-drop tolerance used as the Stage 2 stopping rule; sensitivity to both is reported in Appendix K .

Refusal Behavior in Language Models

Refusal behavior in language models concerns when systems decline requests, how they detect risk, and how to balance safety with helpfulness.

See all 148 papers

On the Steering Dimensionality of Refusal in Language Models
Oct 3, 2026Han Wang, Erik Miehling, Dennis Wei +2Language Model Steering Activation Steering
University of Illinois Urbana-Champaign · IBM Research
There Is More to Refusal in Large Language Models than a Single Direction
Feb 2, 2026Faaiz Joad, Majd Hawasly, Sabri Boughorbel +2LLM Interpretability LLM Refusal Behavior
Qatar Computing Research Institute, HBKU, Doha, Qatar
What Drives Representation Steering? A Mechanistic Case Study on Steering Refusal
Apr 9, 2026Stephen Cheng, Sarah Wiegreffe, Dinesh ManochaLanguage Model Steering LLM Interpretability
University of Maryland, College Park

Oct 3, 2026J/K move · Enter open · S save

On the Steering Dimensionality of Refusal in Language Models

Han Wang, Erik Miehling, Dennis Wei, Karthikeyan Natesan Ramamurthy+1

University of Illinois Urbana-Champaign · IBM Research

Language Model Steering Activation Steering Refusal Behavior in Language Models Representation Geometry in Language Models

Open paper AskPDF arXiv

Existing activation steering methods often assume that a high-level concept can be mediated by a single steering direction. To support this, two complementary interventions should be achieved: additive steering should induce the target behavior, while directional ablation should suppress it. Yet behaviors may occupy richer activation geometries beyond a single direction, and semantically similar behaviors may be represented by distinct directions. In this work, we study how many directions can reliably control two different types of refusal behaviors: refusal triggered by the safety alignment and refusal in general contexts. Given the limited expressive capability of a single steering vector, we study the general setting of steering subspaces and introduce the notion of steering dimensionality as the minimum subspace dimensionality required to reliably control a behavior. We characterize sufficient steering subspaces that cover the full extent of the target behavior through both the (monotonic) improvement before the sufficient dimensionality, and the saturation beyond it. Empirically, we find that refusal triggered by safety alignment is 1-dim steerable, while multiple distinct steering directions can achieve comparable control. In contrast, refusal in general contexts exhibits substantially richer activation geometry where even 5-dim steering subspaces fail to reliably capture its full steerable variation. Our results reveal that the activation geometry underlying refusal is highly context-dependent and can be substantially more complex than a single linear steering direction.

Abstract

Refusal Behavior in Language Models

Explore similar work

On the Steering Dimensionality of Refusal in Language Models

There Is More to Refusal in Large Language Models than a Single Direction

What Drives Representation Steering? A Mechanistic Case Study on Steering Refusal