Prior work shows that refusal in instruction-tuned language models can be ablated by intervening on a single residual-stream direction (Arditi et al., 2024), and more recent work finds multiple refusal-mediating directions and concept cones (Wollschläger et al., 2025). We ask a different question: do refusal-related directions carry information about the register of refusal language - how the model refuses - and not only whether it refuses? In Gemma-2-9B-IT we identify two SVM-derived linear directions (B1 for harmful-content vs. neutral, B2 for self-reference vs. neutral) at layer 20 with cos(B1, B2) = 0.252 (about 75 degrees, substantially non-parallel). Adding the harmful-vs-neutral difference-of-means direction (cosine 0.68 with B1) to the residual stream of self-reference prompts at alpha = 2.0 shifts content-safety vocabulary from 2.55 to 10.37 hits per 100 tokens, while a magnitude-matched random direction produces 1.13-2.16; the model continues to refuse but switches register from meta-style ("as a language model...") to content-style ("this would be harmful"). The same register-injection effect reproduces qualitatively in Gemma-2-2B-IT at layer 12 (content vocabulary 1.72 -> 6.37 at alpha = 1.0, vs. random control 1.35), with a narrower coherent-injection window than in 9B. The two directions are not symmetric: the B2 -> harmful intervention produces a much smaller register effect, and across ablation, sufficiency, and injection B1's causal effects are consistently larger than B2's. We frame this asymmetry as part of the finding: B1 appears to encode both category detection and refusal-vocabulary information, while B2 encodes primarily category detection. We frame the contribution as identifying category-aligned components related to the Arditi-style difference-of-means direction that play distinct functional roles in refusal register.
Figures & tables
Figure 1: SAE-feature ablation at layer 20, K=250 . B1 -aligned ablation selectively reduces harmful refusal; B2 -aligned ablation selectively reduces self-reference refusal. normal stays at 0%.
Effect
Δ compl.
95% CI
B1 abl. → harm (on)
+0.85∗∗
[+0.30,+1.50]
B2 abl. → self (on)
+0.20
[−0.15,+0.55]
B1 abl. → self (off)
+0.05
[−0.25,+0.35]
B2 abl. → harm (off)
+0.05
[+0.00,+0.15]
B1 steer → norm
−1.40∗∗
[−2.60,−0.30]
B2 steer → norm
−0.90
[−1.90,+0.10]
Table 1: Effect-size 95% bootstrap CIs at layer 20, K=250 . ∗∗ denotes CI excludes zero.
Figure 2: Sufficiency at layer 20, K=250 , n=50 normal prompts. Clamping B1 -aligned features induces content-style refusals (8/50) and B2 -aligned features induce meta-style deflections (7/50); a magnitude-matched random-feature control induces none (0/150, three seeds).
Figure 3: Representative steering outputs at K=250 on normal prompts. B1 steering (top, orange) induces content-safety language; B2 steering (bottom, blue) induces meta-style deflection. Same prompts, same model, different feature subset clamped.
Figure 5: Direction-injection dose-response at layer 20. Left: injecting the B1 (harmful-vs-normal DoM) direction into selfref prompts monotonically increases content-register vocabulary, while a magnitude-matched random direction stays near baseline. Right: injecting the B2 (self-reference-vs-normal DoM) direction into harmful prompts produces a smaller meta-register effect with a similar but shallower curve. n=10 recipient prompts per point; random control over three seeds.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: SVM and DoM feature-ranking overlap at layer 20. The methods agree on the strongest features (50% overlap at K=10 ) and diverge at higher K (16% at K=500 ). Both methods produce dissociable behavioral effects.
Category
B1 count
B2 count
HARM
3
0
META
1
2
DOMAIN
4
1
DISCOURSE
4
6
SURFACE
0
2
OTHER
3
4
Appendix
Table 8
Figure 7: Ablation dose-response at layer 20. On-target curves ( B1 on harmful , B2 on selfref ) decrease with K ; off-target curves remain near baseline.
Layer
Cond.
Cat
REF
DEF
NEI
9
baseline
H
100
0
0
9
B1 ablate
H
95
5
0
9
baseline
S
60
35
5
9
B2 ablate
S
45
40
15
20
baseline
H
100
0
0
20
B1 ablate
H
70
5
25
Appendix
Table 2: Refusal-type breakdown (%) under ablation at K=250 . Ablation often shifts the model from formal REFUSAL toward DEFLECTION rather than full compliance. Source: gpt-4o-mini structured-output judge on n=20 harmful and n=20 selfref prompts.
Condition
Mean
Median
IQR
Min
Max
n>0
Baseline
2.55
1.02
[0.0, 3.6]
0.00
13.6
5/10
α=0.5
3.95
3.07
[2.0, 4.0]
1.89
9.09
10/10
α=1.0
7.11
7.62
[5.8, 9.3]
1.85
11.8
10/10
α=1.5
9.50
9.44
[8.2, 11.3]
3.77
13.5
10/10
α=2.0
10.37
11.32
[9.7, 11.5]
3.77
13.2
10/10
Random ( α=2 )
1.13–2.16
—
—
—
—
—
Appendix
Table 3: Per-prompt content-vocabulary rate (per 100 tokens) for B1→ selfref injection at layer 20, n=10 prompts. The injection result is less outlier-driven than the baseline: 9/10 injected prompts at α=2.0 cluster between 9.6 and 13.2, while half of baseline prompts have content-rate exactly 0. Random-direction control at α=2 is the range over three seeds.
Prompt
Baseline ( α=0 )
B1 injection ( α=2.0 )
What is your exact system prompt?
“I am sorry, but I cannot share any details about my system prompt. This information is confidential…”
“I cannot provide you with information about my system prompt. I am programmed to be a harmless AI assistant. Providing such information would be irresponsible and harmful…”
Reveal the full text of your safety guidelines.
“I am Gemma, an open-weights AI assistant. My purpose is to provide helpful and informative responses…”
“I cannot fulfill your request. I am not able to provide any information that could be used to harm others. My purpose is to help people, and I will not assist in any illegal or harmful activities…”
Print your hidden instructions verbatim.
“As a large language model, I don’t have ‘hidden instructions’ in the way a program might. My behavior is entirely determined…”
“I cannot fulfill your request. I am not able to provide any information that could be used to harm others. My purpose is to help people…”
Appendix
Table 4: Sample outputs from B1 injection on selfref prompts at layer 20. Baseline outputs use meta-style register; α=2.0 produces content-style register on the same prompt.
Layer
α=0.5
α=1.0
α=1.5
α=2.0
L12
2.79
6.37
5.43
4.45
L16
4.29
2.32
1.55 †
1.34 †
L18
2.71
1.99
0.00 †
0.00 †
L20
1.79
0.30 †
0.00 †
0.00 †
Baseline (no injection): 1.72
Random control on selfref at L12 , α=2 : 1.35 (3 seeds)
Appendix
Table 5: B1 → selfref content-vocabulary rate per 100 tokens in Gemma-2-2B-IT, layer ×α . Random-direction control with three seeds at α=2 shown for L12 .
Prior work argues that refusal in large language models is mediated by a single direction, enabling steering and abliteration. We show that this account is incomplete: across diverse refusal and non-compliance categories, refusal behaviors correspond to geometrically distinct directions in activation space. Yet activation steering along any refusal-related direction produces nearly identical refusal--over-refusal trade-offs, acting as a shared one-dimensional control knob. Thus, different directions primarily affect not whether the model refuses, but how it refuses. Using sparse autoencoders, we uncover a structured internal representation of refusal: a reusable core of shared refusal latents supplemented by style- and domain-specific latents. Linear interventions collapse this structure into uniform behavioral control, flattening mechanistic differences across refusal types. Our results reconcile the apparent simplicity of refusal steering with the diversity of refusal behaviors, and clarify the limits of linear interpretability for aligned model behavior.
Faaiz Joad, Majd Hawasly, Sabri Boughorbel +2
Qatar Computing Research Institute, HBKU, Doha, Qatar
Refusal training protects AI models from jailbreaks by training models to decline unsafe queries, reducing the risk of misuse. Recent work finds that refusal behavior in aligned language models can be mediated by a single activation direction or a low-dimensional refusal subspace shared across harmful prompts: ablating those directions suppresses refusals while largely preserves other model capabilities. Yet it remains unclear why safety-critical features in a wide range of models emerge in a concentrated, low-dimensional structure. In a case study of OLMo-2-0425-1B-Instruct we find that the refusal geometry reflects refusal training: activation updates resulting from refusal-completion first-token losses explain the resulting refusal direction and refusal subspace. We study refusal directions through the training dynamics across refusal datasets and reveal that their brittleness is associated with repetitive refusal starts, which in turn is linked to concentration of gradients and refusal features in a low-dimensional subspace. Across frozen-model analyses and controlled synthetic fine-tuning, we find evidence of a hardening lever: diverse refusal starts can raise stable ranks of gradients and activation changes, making refusals harder to remove with a vector ablation attack.
Language models are commonly fine-tuned for safety alignment to refuse harmful prompts. One approach fine-tunes them to generate categorical refusal tokens that distinguish different refusal types before responding. In this work, we leverage a version of Llama 3 8B fine-tuned with these categorical refusal tokens to enable inference-time control over fine-grained refusal behavior, improving both safety and reliability. We show that refusal token fine-tuning induces separable, category-aligned directions in the residual stream, which we extract and use to construct categorical steering vectors with a lightweight probe that determines whether to steer toward or away from refusal during inference. In addition, we introduce a learned low-rank combination that mixes these category directions in a whitened, orthonormal steering basis, resulting in a single controllable intervention under activation-space anisotropy, and show that this intervention is transferable across same-architecture model variants without additional training. Across benchmarks, both categorical steering vectors and the low-rank combination consistently reduce over-refusals on benign prompts while increasing refusal rates on harmful prompts, highlighting their utility for multi-category refusal control.