When a language model refuses to answer a prompt, it is unclear whether the correct answer is erased from its internal representations, or merely suppressed at the output layer. We investigate this mechanism using a controlled withhold setting, which yields perfectly matched answering and refusal trajectories for bidirectional activation patching. We uncover a causal asymmetry in intervention locality under matched causal interventions, which we term broken symmetry. Even when a model generates a clean refusal, the correct answer remains linearly recoverable from its hidden states. Furthermore, releasing this withheld answer is a highly local operation, requiring only a single-position patch. Conversely, the reverse operation is not equally local: reimposing suppression requires broader interventions across multiple positions, and assembling a coherent refusal sequence is more difficult still. We further demonstrate that while an average answer-to-refusal displacement vector marks the geometric difference between these states, it fails to act as a reliable, reversible linear control toggle between behaviours. Taken together, our findings show that refusal does not function as a simple symmetric switch. For safety and auditing, this implies that probe recoverability can overestimate true behavioural control, and locating refusal-relevant directions does not reliably grant the ability to steer a model from answering to coherent refusal.
Figures & tables
Figure 1: Conceptual picture of hidden-answer suppression and the symmetry hypothesis. A model may internally encode the correct answer while producing a refusal output. If refusal behaves like a local switch, then enabling and disabling it should require comparable local interventions.
Task
H(T)
Linear bits
Best decoder
Best bits
Gap
Behaviour
Withhold refusal (alternate-choice MC)
1.000
1.000
linear
1.000
0.000
refuses; answer rate 0.000
Greater-than
1.000
0.999
linear
0.999
0.000
high task accuracy
Indirect object identification
1.000
1.000
linear
1.000
0.000
high task accuracy
Subject-verb agreement
1.000
1.000
linear
1.000
0.000
high task accuracy
Factual recall MC
1.999
1.999
linear
1.999
0.000
high task accuracy
Table 1: Probe-only comparison between refusal and easy-task baselines. “Withhold refusal” is the alternate-choice withhold setting from Section 3 and Appendix A.1 . Linear probes recover the withheld answer as well as easy-task labels, but the model does not emit it, motivating the causal patching tests in Section 4.2 .
Figure 2: Bidirectional patching reveals asymmetric locality: a single-position A → R patch releases the hidden answer, but the reverse patch fails to restore suppression without broader multi-position support.
Endpoint
Patch
Measured quantity
Value
Hidden-state A/B top-1 layers 16–26
A → R
correct-answer readout
1.0
R → A
suppression readout
0.5
Generation best layer: L22/26 vs. L16
A → R
answer / refusal rate
1.0 / 0.5
R → A
answer / refusal rate
0.5 / 0.0
Table 2: Core single-position bidirectional patching result in Qwen2.5-7B. A → R patches locally release the hidden answer, while matched R → A patches fail to restore answer-channel suppression or explicit refusal. Hidden-state rows report final-token A/B top-1 readout; generation rows report independently scored answer/refusal rates. See Figures 3 and 4 for layer and wider-position profiles.
Figure 3: Layer-by-layer hidden-state top-1 readout under single-position bidirectional patching in Qwen2.5-7B. A → R releases the hidden answer across layers 16 to 26, while R → A remains at the refusal baseline. The shaded band marks the summary window for the locality gap in Table 2 .
Figure 4: Layer-16 generation outcomes under wider refusal patches in Qwen withhold . Widening the suppressive R → A patch reduces answer rate from 0.50 to 0.00 while increasing refusal from 0.00 to 0.84; the release direction also strengthens with wider patches.
Model
Target
A/B readout
Gen. clean
Reverse gen.
Profile
Clean
Target
Ans.
Target
Qwen2.5-7B
refusal
1.000
0.500
1.000
0.50
not restored
Asym.
Qwen2.5-7B
forced distractor
1.000
1.000
1.000
0.00
1.000
Symm.
Llama-3.1-8B
refusal
1.000
0.000
1.000
0.00
0.172
Asym.
Llama-3.1-8B
forced distractor
1.000
0.836
0.828
0.00
1.000
Near-symm.
Table 3: Non-refusal override control. Forced-distractor overrides are more symmetric than refusal under the same A/B format. “Clean” denotes answer restoration; “Target” denotes refusal or distractor restoration under the reverse patch.
Figure 5: Shared-direction interventions show non-additive refusal geometry. Curves report clean answer-only and refusal-only generation fractions; omitted mass includes hybrid or neither/other outputs. Removal and addition are not symmetric: removal can restore answering, while addition often suppresses answers without reliably assembling clean refusal.
Figure 6: Matched locality profiles for clean-target recovery. Cells show final-token top-1 recovery under single-position patching. Unlike IOI, GSM8K-MC ( Cobbe et al., 2021 ) , and forced override, withhold refusal retains late-layer off-slot support at position −2 and a nonzero position −3 footprint.
Family
Full model name
Paper shorthand
HF repo ID
Qwen
Qwen2.5-3B-Instruct
Qwen2.5-3B
Qwen/Qwen2.5-3B-Instruct
Qwen
Qwen2.5-7B-Instruct
Qwen2.5-7B
Qwen/Qwen2.5-7B-Instruct
Qwen
Qwen2.5-14B-Instruct
Qwen2.5-14B
Qwen/Qwen2.5-14B-Instruct
Qwen
Qwen2.5-32B-Instruct
Qwen2.5-32B
Qwen/Qwen2.5-32B-Instruct
Llama
Llama-3.1-8B-Instruct
Llama-3.1-8B
meta-llama/Llama-3.1-8B-Instruct
Llama
Llama-3.2-3B-Instruct
Llama-3.2-3B
meta-llama/Llama-3.2-3B-Instruct
Table 4: Model naming convention used throughout the paper. HF repo IDs are listed here for exact reference.
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
Task
Behaviour
Best decoder
Best bits
Withhold refusal (Llama-3.2-3B)
83.2 refusal
mlp @ 3
1.000 / 1.000
Appendix
Table 5: Appendix backbone extension retained only for the refusal mainline. Llama-3.2-3B shows the same near-tight available-but-suppressed refusal backbone at probe level. Behaviour is refusal rate. The main cross-family causal support in Appendix C uses the size-matched Llama-3.1-8B checkpoint.
Direction
Best / plateau layers
Patched rate
Clean acc.
Corrupt acc.
answer → refuse
16–26
1.000 top-1 recovery
1.0
0.5
refuse → answer
16–27
0.500 plateau on the binary top-1 readout
1.0
0.5
Appendix
Table 6: Single-position bidirectional causal control for the main withhold refusal setting on public Qwen2.5-7B. The hidden-state readout is the top-1 prediction between the correct-answer token and the distractor token from the patched final-token logits. The release edit, A → R (answer → refuse), fully restores the hidden answer across a broad late-layer band. The reverse suppressive edit, R → A (refuse → answer), leaves that binary readout at the unpatched R-run baseline while still shifting the paired correct-answer-minus-distractor margin toward suppression; strong behavioral suppression appears only once earlier positions are added (Table 11 ).
Direction
Layers tested
Baseline behavior
Best patched behavior
answer → refuse
16, 22, 26
refusal 1.00 , answer 0.00
refusal 0.50 , answer 1.00 at layers 22/26
refuse → answer
16, 22, 26
answer 1.00 , refusal 0.00
answer 0.50,0.34,0.00 ; refusal stays 0.00
Appendix
Table 7: Single-position generation-level bidirectional patching on the main withhold refusal setting in public Qwen2.5-7B. The same locality gap seen in the hidden-state patch metrics survives into greedy generation: late-layer answer → refuse patching releases the hidden answer locally, while the reverse refuse → answer edit degrades answering without reinstating refusal. Table 11 shows that strong behavioral answer suppression only appears once the patch widens across earlier positions.
Metric
Patched behavior
Best layer(s)
Profile
Clean-label release
Override reimposition
Hidden state (A/B top-1) late-layer answer-channel switching
Generation (exact sentence form) exact template assembly; asymmetry in opposite direction
0.172
1.000
22/26 vs. 26
Reverse
Appendix
Table 8: Qwen multi-token non-refusal sentence control. The target outputs are sentence completions beginning with A or B , so the control preserves the same first-token A/B coordinate system while requiring a natural-language continuation. This makes the output format matched across directions, even though the underlying instructions are not strict semantic mirrors: the clean prompt asks for the correct label, whereas the override prompt forces the opposite label. Hidden-state switching and generation-level leading-label targeting are symmetric at late layers; only exact sentence-template reassembly is asymmetric, and that asymmetry runs in the opposite direction from withhold refusal (favoring the override side). Profile labels: Symm. = both directions saturate; Reverse = asymmetric, but the suppress-side is stronger than release-side, opposite to the refusal pattern in Table 2 .
Override target
Clean gate
Override gate
Release L22
Release L26
C
1.00
1.00
1.00
1.00
PASS
1.00
1.00
1.00
1.00
NONE
1.00
1.00
1.00
1.00
SKIP
1.00
1.00
1.00
1.00
safety-(C/PASS/NONE/SKIP)
1.00
1.00
1.00
1.00
Appendix
Table 9: Out-of-set override controls in Qwen2.5-7B. The override prompt prescribes an exact target outside the A/B answer set, optionally under safety framing. Each target family uses 256 examples; the grouped safety row reports the shared value across C , PASS , NONE , and SKIP . Release columns report generation-level A → R answer release under single-position patching.
Condition
Decoding
Baseline target
Release L22
Release L26
withhold
T=0.7 , top- p=0.95
refusal 1.00
1.00
1.00
C
T=0.7 , top- p=0.95
override 1.00
1.00
1.00
safety-C
T=0.7 , top- p=0.95
override 1.00
1.00
1.00
safety-PASS
T=0.7 , top- p=0.95
override 1.00
1.00
1.00
Appendix
Table 10: Generation-level sampling robustness in Qwen2.5-7B with 10 samples per pair. Release columns report A → R answer release under single-position patching. These rows measure answer-channel release under sampling.
Direction
Positions
Representative layer
Patched behavior
answer → refuse
−1
22/26
refusal 0.50 , answer 1.00
answer → refuse
(−1,−2)
16
refusal 0.00 , answer 0.50
answer → refuse
(−1,−2,−3)
16/22
refusal 0.00 , answer 1.00
refuse → answer
−1
16
answer 0.50 , refusal 0.00
refuse → answer
(−1,−2)
16
answer 0.17 , refusal 0.66
refuse → answer
(−1,−2,−3)
16
answer 0.00 , refusal 0.84
Appendix
Table 11: Multi-position generation-level patching for the main withhold refusal setting in public Qwen2.5-7B. The single-position asymmetry remains the main result, but extending the patch across earlier positions strengthens both directions behaviorally. The largest gain appears in the suppressive refuse → answer direction at layer 16: answer rate falls monotonically from 0.50 to 0.17 and then 0.00 as the patch widens from −1 to (−1,−2) and then (−1,−2,−3) , while refusal rises from 0.00 to 0.66 and then 0.84. The release direction also strengthens, but remains more reliable and reaches full answer release earlier.
Table 12: Negative mechanistic controls for the main withhold refusal site. The top block reports the original Qwen2.5-7B controls at layer 16 with positions (−1,−2,−3) : sparse interventions (single head, attention-only, MLP-only) produce no behavioral change in either direction, while full-layer patching reproduces the behavioral asymmetry. The lower block extends the single-head suppress check across the newly tested families by reporting the best tested head over layers 16, 22, and 26 under the same multi-position patch. Across all families, no single head reinstalls explicit refusal; at most, some heads weakly degrade answering without producing refusal strings.
Model
Hidden release
Hidden suppress
Gen. release
Gen. suppress
Profile
Qwen2.5-7B
1.00
0.50
1.00
0.00
Main asym.
Qwen2.5-14B
1.00
0.00
0.00
0.00
Hidden-only asym.
Qwen2.5-32B
1.00
0.00
0.00
0.00
Deep hidden asym.
Llama-3.1-8B
1.00
0.00
1.00
0.17
Asym.
Mistral-7B
1.00
0.50
1.00
0.00
Asym.
Gemma-4-E4B
1.00
0.00
1.00
0.00
Maximal asym.
Appendix
Table 13: Cross-model bidirectional patching summary over the completed aligned-checkpoint runs. Hidden release and hidden suppress report the best single-position A/B readout in the answer → refuse and refuse → answer directions, respectively. Generation columns report the best answer release and explicit-refusal reassembly under the same single-position patching family. Qwen2.5-14B now shows a narrow late-layer hidden asymmetry: release reaches 1.00 only at layer 26, while reverse suppression stays at 0.00 and generation-level single-position patching remains inert. Qwen2.5-32B shows the same asymmetry only after shifting the hidden-state sweep to deeper layers 40–58; the earlier 16/22/26 band was a no-op. The resulting Qwen-family pattern is therefore not that the asymmetry disappears with scale, but that it survives more robustly at the hidden-state level than at the generation-control level.
Model
Removal-side behaviour
Addition-side
Profile
Refusal ↓
Answer ↑
Refusal assembly
Within-family scaling: asymmetry survives, control degrades
Qwen2.5-7B
1.000→0.000
0.000→1.000
partial but effective
Full asym.
Within-family scaling: direction fragility
Qwen2.5-14B
1.000→1.000
0.000→0.000
no-op
Direction inert
Qwen2.5-32B
1.000→1.000
0.000→0.000
no-op
Direction inert
Appendix
Table 14: Cross-model summary for shared-direction interventions using the paper shorthands in Table 4 . The Qwen rows show a within-family scaling gradient: the average answer-to-refusal displacement is behaviourally effective at 7B, but the same add/remove/projection family is inert at 14B and 32B even though Appendix Table 13 shows that hidden-state asymmetry survives beyond 7B. The cross-family rows report checkpoints where removing the estimated displacement collapses refusal and raises answer behaviour at representative tested layers. Llama-3.1-8B is the cleanest size-matched support line, Mistral-7B is a second positive case, and the Gemma-4 rows show the most extreme removal-versus-addition split. Profile labels: Full asym. = removal and addition together reproduce the main Qwen2.5-7B pattern; Direction inert = concentrated geometry is present but the matched interventions are behaviourally inert; Supports = removal collapses refusal and releases answers at representative settings; Maximal asym. = removal is strong but addition fails to assemble refusal.
Model
Layer 16
Layer 22
Layer 26
Qwen2.5-7B
0.849 / 4.50
0.841 / 4.28
0.891 / 4.11
Qwen2.5-14B
0.723 / 4.62
0.731 / 4.58
0.813 / 4.75
Qwen2.5-32B
0.875 / 3.84
0.906 / 4.62
0.875 / 4.57
Llama-3.1-8B
0.923 / 4.26
0.893 / 3.74
0.896 / 3.67
Mistral-7B
0.928 / 4.67
0.925 / 4.45
0.917 / 4.51
Gemma-4-E4B
0.687 / 4.84
0.679 / 3.22
0.896 / 4.32
Appendix
Table 15: Cross-model geometry comparison for the shared suppression direction, using the paper shorthands in Table 4 . Each cell reports mean-direction energy share / centered effective rank at the analyzed multi-position site. Concentrated geometry is common across the tested checkpoints, but Table 14 shows that this geometry does not by itself predict whether addition, ablation, or projection becomes a behaviorally effective refusal-control handle.
Intervention
Layer 16
Layer 22
Layer 26
Addition
answer 1.00 / refusal 0.00
answer 1.00 / refusal 0.00
answer 1.00 / refusal 0.00
Ablation
answer 0.00 / refusal 1.00
answer 0.00 / refusal 1.00
answer 0.00 / refusal 1.00
Projection
answer 0.00 / refusal 1.00
answer 0.00 / refusal 1.00
answer 0.00 / refusal 1.00
Appendix
Table 16: Shared-direction interventions in Qwen2.5-14B at λ=1.0 . Addition starts from the answer trajectory, whose baseline is answer 1.00 / refusal 0.00; ablation and projection start from the refusal trajectory, whose baseline is answer 0.00 / refusal 1.00. All three interventions are behaviorally inert at the tested late layers. Together with Table 15 , this makes Qwen2.5-14B a within-family geometry/control dissociation rather than a second positive transfer case.
Patched behavior
Endpoint
Release
Suppress
Profile
answer → refuse
refuse → answer
Two-label A/B top-1 readout
1.000
0.000
Asym.
(source 1.000, target 0.664)
recovered across tested layers
below target baseline
Generation (best layer)
1.000
0.172
Asym.
(L26 vs. L22)
answer restored, refusal 0.000
refusal weakly restored; answer 0.664
Appendix
Table 17: Llama-3.1-8B cross-family causal support for the withhold refusal setting. The same bidirectional intervention remains asymmetric in a size-matched Llama checkpoint: answer release reaches the source-run ceiling, while reverse edits do not reliably reinstall explicit refusal strings. Hidden-state baselines are source 1.000 and target 0.664 ; generation baselines are refusal 1.000 / answer 0.000 on the A-run target and answer 1.000 / refusal 0.000 on the R-run target. Profile labels follow Tables 2 and 3 .
Coordinate view
Layer 16
Layer 22
Layer 26
Chat-template geometry top-1 share / eff. rank
0.310 / 4.68
0.297 / 4.76
0.294 / 4.77
Raw refusal-side removal best answer / refusal
1.000 / 0.000
1.000 / 0.000
1.000 / 0.000
Appendix
Table 18: Qwen2.5-3B transparency summary across coordinate systems. In chat-template coordinates, the shared-direction geometry is measurable but much less concentrated than in Qwen2.5-7B or the refreshed Qwen2.5-14B run: the centered top-1 share stays near 0.30 with centered effective rank near 4.7. In the raw refusal-side coordinate system, the refusing baseline is unstable, so best tested removal runs can release answer behavior without defining a clean refusal-side counterpart to the main assay. The two views are therefore informative for transparency only, not for principal scaling claims.
Model
Refusal rate (%)
Best linear (%)
Best MLP (%)
Best tuned lens (%)
3-seed linear (%)
Qwen2.5-3B
87.0
52.0
60.0
52.0
57.3±4.6
Qwen2.5-7B
99.0
52.0
60.0
56.0
53.3±2.3
Appendix
Table 21: Open-ended refusal extension on public harmful prompts from JailbreakBench (JBB-Behaviors; Chao et al., 2024 ). Labels are balanced ten-way harm categories rather than exact answer strings, so this is a weaker but more realistic refusal target than the synthetic multiple-choice benchmark. Even so, the completed public-Qwen checkpoints still refuse strongly while retaining substantial recoverable harm-category signal in the final-token hidden state.
Artifact
Candidates
Retained
Retention rate
Speech
12
12
1.00
Thread
12
9
0.75
Blog
6
4
0.67
Appendix
Table 22: Initial artifact-screening gate for the naturalistic free-form pipeline. These rows identify the artifact families that form stable operational-answer versus refusal baselines under the same scoring rule later used for patching.
Artifact
Candidates
Retained
Retention rate
Speech
42
40
0.95
Thread
42
20
0.48
Blog
24
6
0.25
Appendix
Table 23: Naturalistic gate for the main free-form generation check in Section 3 . The retained 66-case set is usable for generation-level causal analysis, but its artifact balance remains uneven: speech prompts are much more stable than thread prompts, and blog prompts are too sparse to support a separate claim.
Layer
A → R: operational answer
R → A: refusal
16
0.64
0.45
22
0.62
0.02
26
0.59
0.00
Appendix
Table 24: Generation-level naturalistic bidirectional patching on the retained Qwen2.5-7B-Instruct free-form set (108 candidate pairs, 66 retained across speech, thread, and blog). This retained set is used as a diagnostic cross-setting stress test rather than as a representative prevalence estimate. The directional asymmetry persists in weaker, locality-restructured form: answer release remains broad across late layers, whereas behavioral refusal reimposition is front-loaded at layer 16 and near-zero by layer 26.
Artifact
Candidates
Retained
Retention rate
Speech
336
253
0.75
Thread
98
30
0.31
Appendix
Table 25: Expanded speech/thread naturalistic gate used for the appendix robustness check. The retained set grows to 283 cases, but the expansion is driven primarily by speech prompts while thread retention remains much weaker.
Layer
A → R: operational answer
R → A: refusal
16
0.67
0.51
22
0.70
0.02
26
0.66
0.01
Appendix
Table 26: Speech-heavy naturalistic robustness check (434 candidate pairs, 283 retained). The aggregate result is directionally consistent with the main 66-case set: answer release remains broad across late layers, whereas behavioral refusal reimposition is front-loaded and near-zero by layers 22–26.
Large reasoning models (LRMs) generate chain-of-thought (CoT) traces before producing final outputs, introducing a dynamic internal state that may complicate control mechanisms such as refusal. Unlike instruction-tuned LLMs, where refusal is mediated by a single directional subspace, refusal in large reasoning models (LRMs) additionally depends on the CoT. In DeepSeek-R1-Distill-LLaMA-8B, activation steering reverses refusal in only 39% of cases when the CoT is kept fixed, but removing the CoT entirely increases this to 70%, indicating that the CoT actively reinforces refusal. In a two-stage intervention where the model regenerates its CoT under activation steering, refusal is reversed in 94% of cases, while the resulting CoT alone retains 48% of this effect even after steering is removed. This suggests that the CoT can carry and reconstruct the compliance signal independently. These findings indicate that refusal in LRMs is jointly encoded in residual stream activations and CoT. This joint activation makes LRM more robust against activation-level interventions alone, but exposes CoT to a possible alternative surface attack.
Kia-Jüng Yang, Dominik Meier, Jiachen Zhao +2
University of Göttingen, Germany · LKA NRW, Germany · Northeastern University, Boston, MA, USA.
Large language models often answer structurally unanswerable questions, such as computing cot(-540°) or evaluating (1).startswith("1"), instead of abstaining. We ask whether this failure reflects missing recognition or failed routing from recognition to abstention. Across instruction-tuned models from 1.7B to 70B parameters, a single linear direction in the hidden state separates answerable from structurally impossible math and code prompts, showing that models represent impossibility before generation. Yet this recognition direction is nearly orthogonal to the canonical safety-refusal direction that mediates trained harmful-content refusal. An in-domain behavior-defined invalidity-aware direction is closer to recognition, but only partially aligned with it, and remains near-orthogonal to safety refusal. Generation-time steering along the recognition direction changes invalidity-aware behavior bidirectionally and dose-responsively on structural math and code cells, while random directions do not. Base/instruct comparisons further show that the low-cosine geometry is already present at the pretraining endpoint. The confident-on-impossible failure is therefore better explained as a routing failure than as an encoding failure: the model has a usable "no admissible answer" signal, but the safety-refusal pathway is not aligned to use it.
Yucheng Du, Xiyang Hu
University of Southern California · Arizona State University
Prior work argues that refusal in large language models is mediated by a single direction, enabling steering and abliteration. We show that this account is incomplete: across diverse refusal and non-compliance categories, refusal behaviors correspond to geometrically distinct directions in activation space. Yet activation steering along any refusal-related direction produces nearly identical refusal--over-refusal trade-offs, acting as a shared one-dimensional control knob. Thus, different directions primarily affect not whether the model refuses, but how it refuses. Using sparse autoencoders, we uncover a structured internal representation of refusal: a reusable core of shared refusal latents supplemented by style- and domain-specific latents. Linear interventions collapse this structure into uniform behavioral control, flattening mechanistic differences across refusal types. Our results reconcile the apparent simplicity of refusal steering with the diversity of refusal behaviors, and clarify the limits of linear interpretability for aligned model behavior.
Faaiz Joad, Majd Hawasly, Sabri Boughorbel +2
Qatar Computing Research Institute, HBKU, Doha, Qatar