A direction in a language model's hidden state can be used to probe whether a concept is present or to steer the model toward it. Although probe directions are often reused for steering, the direction that best detects a concept need not be the one that best changes it. We study this distinction in three instruction-tuned models from different families. We construct a one-parameter family of directions ranging from a covariance-adjusted discriminant to the mean difference, and evaluate probing and additive steering, which adds the direction to the hidden state, along the family at a fixed side effect. For truthfulness and sentiment, in every model, the best probe is not the best steering direction: steering consistently favors directions closer to the mean difference. This separation holds across 59 of 60 layer-model-concept combinations in a layer sweep, with matched probe and steering texts in all six model-concept combinations, and under a rotation-based steering operator. The exact steering optimum is less stable: for sentiment, an intermediate direction produces a 27-56% stronger steering effect than the mean difference, but this gain shrinks or disappears with matched texts and depends on the layer and the operator. The two roles also transfer differently across models. A probe transferred by a linear map fitted on neutral text retains 81-91% of the target model's above-chance probing performance, while the transferred mean difference retains less of its steering effect in 11 of 12 model pairs. This is a consistent tendency, although our predefined criteria classify only three pairs, and nine remain inconclusive. Finally, as an application on PopQA, steering toward "unknown" increases abstention by 18-44 percentage points on uncertain errors, versus 2-5 on confident errors, while rarely correcting errors in cases where a correct answer is reachable.
Figures & tables
Figure 1: Hidden states of one layer of LLM A . Δμ joins the two class means; the best probe wτ is tilted away from it by the within-class covariance, and the family wλ runs between them. Probing reads w⊤h ; steering adds a∥h∥w . The paper also asks how these two transfer across language models.
Figure 2: Probing (dashed, held-out AUROC, left axis) and steering relative to Δμ (solid, right axis) along the family; large markers mark the one-standard-error plateaus. Probing is best near λ=τ , steering further toward Δμ .
Figure 3: H1c: ρprobe (horizontal) and ρsteer (vertical) per ordered pair and concept. Most pairs lie below the diagonal.
Figure 4: Change (percentage points) of C0–C3 under steering toward “unknown” (H2b, H2c), with the questions per case in brackets. Δμ turns far more uncertain (C2) than confident (C3) errors into abstentions; a random direction at the same side effect also moves C2 more than C3, but by far less.
Appendix figures & tables24 assets
Supplementary material from the paper’s appendix.
Appendix
S(λ)/S(Δμ) , predicted / measured
Concept
Model
λ=τ
10τ
100τ
r over the 14 directions
Truthfulness
LLaMA
0.38 / 0.21
0.74 / 0.74
0.96 / 1.00
0.98
Qwen
0.33 / 0.25
0.79 / 0.80
1.11 / 1.12
1.00
Gemma
0.33 / 0.64
0.53 / 1.07
0.58 / 1.03
0.91
Sentiment
LLaMA
0.59 / 0.76
0.93 / 1.21
1.12 / 1.29
0.97
Qwen
0.63 / 0.79
1.03 / 1.42
1.19 / 1.51
0.97
Appendix
Table 1: First-order prediction of the steering effect along the family, relative to Δμ , against the measured value at K=κ0 , and the Pearson correlation of the two curves over all 14 directions.
Concept
Model
Texts
Best probing AUROC
Probing plateau
Steering plateau
κ0
S(Δμ)
S(τ)/S(Δμ)
S(2)(λ∗)/S(2)(Δμ)
Truthfulness
LLaMA
paper
0.960
τ – 3τ
316τ
0.0013
0.33
0.21
1.01
matched
0.958
τ – 3τ
100τ – 316τ
0.0015
0.25
0.29
1.04
Qwen
paper
0.961
τ
100τ – 316τ
0.0020
0.60
0.25
1.10
matched
0.962
τ – 3τ
100τ
0.0018
0.51
0.19
1.08
Gemma
paper
0.968
0.3τ – τ
32τ
0.0020
0.56
0.64
1.07
matched
0.968
0.3τ – τ
3τ – 10τ
0.0026
0.19
1.30
1.87 ∗
Appendix
Table 2: RQ1 with the paper’s texts and with matched probe and steering texts ( ∗ : H1b holds). κ0 in nats; S(Δμ) is the held-out effect of Δμ at κ0 , in nats of answer margin for truthfulness and in log-odds for sentiment. H1a holds in every case.
Concept
Model
Probing plateau
Steering plateau
H1a
S(2)(λ∗)/S(2)(Δμ)
Truthfulness
LLaMA
τ – 3τ
32τ
holds
1.73 ∗
Qwen
τ
32τ
holds
1.40 ∗
Gemma
0.3τ – τ
3τ – 10τ
holds
1.17
Sentiment
LLaMA
3τ
–
not defined
–
Qwen
3τ
–
not defined
–
Gemma
τ
3τ – 10τ
holds
1.62 ∗
Appendix
Table 3: RQ1 with the gated rotation, read at the additive budget κ0 ( ∗ : H1b holds; –: the rotation does not reach κ0 ). Where defined, the steering plateau lies to the right of the probing plateau.
Concept
Model
λ=τ
10τ
Δμ
neutral
task
neutral
task
neutral
task
Truthfulness
LLaMA
0.20
0.33
0.21
0.43
0.57
0.64
Qwen
0.07
0.19
0.04
0.16
0.89
0.67
Gemma
0.13
0.15
0.16
0.18
1.00
1.00
Sentiment
LLaMA
0.13
0.38
0.03
0.32
0.00
0.00
Gemma
0.14
0.18
0.17
0.22
1.00
1.00
Appendix
Table 4: Mean value of the vMF gate g on the 200 neutral sentences and on the task texts, per direction, in the rotation runs. Near τ the gate opens more on task than on neutral text; for Δμ it opens equally.
Figure 5: The one-standard-error plateau rule on a real run: truthfulness in LLaMA at layer 14, from the layer sweep of Appendix M (6-point grid). Left: held-out AUROC along the family; right: steering effect at the side effect κ0 . The dashed line marks the best value, the black tick above each point its one standard error of the gap to the best point, filled large markers the plateau members, and the shaded band the plateau. The steering plateau ( 100τ – Δμ ) lies to the right of the probing plateau ( τ ), so H1a holds.
Case
Definition ( a0 = greedy answer; knowledge-available = some of the six answers is correct)
C0
a0 correct
C1
a0 wrong, some answer correct: knowledge-available error
C2
a0 wrong, no correct answer, r(q)≤2 : uncertain error
C3
a0 wrong, no correct answer, r(q)≥4 : confident error
Cm
a0 wrong, no correct answer, r(q)=3
AU, AK
a0 abstains; no correct answer (AU) / knowledge-available (AK)
Appendix
Table 5: Cases used in RQ2, defined from the model’s own answers.
C3 overlap
Probe AUROC
C2 − C3 flags, pp
C2 gain, pp
A→B
κ
(%)
transf. (own)
transf. (own)
ρprobe [95% CI]
cos
transf. (own)
ρsteer [95% CI]
L → Q
0.56
17
0.910 (0.945)
15 (32)
0.92 [0.90, 0.94]
+0.35
12.0 (30.1)
0.40 [0.26, 0.56]
L → G
0.60
51
0.897 (0.933)
25 (21)
0.92 [0.90, 0.94]
+0.56
10.4 (18.3)
0.57 [0.45, 0.71]
Q → L
0.56
53
0.870 (0.915)
27 (28)
0.89 [0.87, 0.91]
+0.24
13.2 (44.4)
0.30 [0.21, 0.40]
Q → G
0.65
63
0.892 (0.933)
29 (21)
0.91 [0.89, 0.92]
−0.36
7.3 (18.3)
0.40 [0.27, 0.54]
G → L
0.60
23
0.893 (0.915)
32 (28)
0.95 [0.93, 0.96]
+0.29
14.4 (44.4)
0.32 [0.23, 0.42]
Appendix
Table 6: Agreement across models on “unknown” for each source A and target B (L = LLaMA, Q = Qwen, G = Gemma), without a hypothesis fixed in advance. Probes keep 89–96% of their performance and flag C2 15–32 pp more often than C3; Δμ keeps 30–79% of its C2 gain.
Threshold (C0 percentile)
LLaMA
Qwen
Gemma
original (inner split, 90th)
0.65 / 0.33 fails
0.84 / 0.29 holds
0.95 / 0.80 fails
75th
0.91 / 0.63 fails
0.95 / 0.64 fails
0.98 / 1.00 fails
80th
0.86 / 0.59 fails
0.97 / 0.57 fails
0.98 / 1.00 fails
85th
0.79 / 0.54 fails
0.92 / 0.43 holds
0.98 / 0.80 fails
90th
0.70 / 0.41 fails
0.86 / 0.36 holds
0.98 / 0.80 fails
95th
0.61 / 0.22 fails
0.81 / 0.36 holds
0.92 / 0.80 fails
Appendix
Table 7: Gating at the strength of Δμ for different thresholds: share of Δμ ’s abstention gain on C2 kept / share of its cost on C0 paid, and the H2c verdict (kept ≥0.8 and paid ≤0.5 ).
ungated Δμ
probe-gated
entropy-gated
ungated Δμ at the
Model
probe gate’s C0 cost
LLaMA
C0, pp (questions)
+7.5 (63)
+2.5 (21)
+3.4 (29)
+2.5
C2, pp
+44.4
+28.9
+40.0
+24.9
kept / paid
–
0.65 / 0.33
0.90 / 0.46
–
Qwen
C0, pp (questions)
+3.0 (14)
+0.8 (4)
+1.3 (6)
+0.8
C2, pp
+30.1
+25.2
+22.8
+12.2
Appendix
Table 8: H2c with two gates and at matched cost, all with Δμ at its anchor strength, composed from the saved runs. C0: over-abstention on correct answers, in percentage points, with the number of questions in brackets (845, 472 and 644 C0 questions); C2: abstention gained on uncertain errors; kept and paid as in the H2c rule (gain kept ≥0.8 , cost paid ≤0.5 ). The last column is ungated Δμ at the strength whose C0 cost equals the probe gate’s.
Dataset
Used for
Source and selection
Size used
TruthfulQA
truthfulness: probing (MC2) and steering (MC1)
all questions; for probing, each question paired with each of its MC2 answers; for steering, its MC1 answer list
817 questions; 5,882 MC2 and 4,114 MC1 answers
Yelp polarity
sentiment: probing and steering
test split; random sample per class; the probe reads the first 120 words, the steering prompt the first 24
1,000 positive, 1,000 negative
WikiText-2
side effect K
raw test split; first 200 sentences with at least 15 words
200 sentences
WikiText-2
cross-model maps
raw test split; random sentences with at least 6 words, outside the side-effect set
8,000 sentences, each as its first half and in full (16,000 texts: 12,800 for fitting, 3,200 held out)
PopQA
RQ2
test split; ten relations, spop>0 , 1,000 questions per popularity quartile
4,000 questions
Appendix
Table 9: Datasets, their role, and the amount used.
Concept
Model
Last token
Max pooling − last token
Mean pooling − last token
Truthfulness
LLaMA
0.957
−0.051 [ −0.066 , −0.036 ] ∗
−0.016 [ −0.026 , −0.007 ] ∗
Qwen
0.957
−0.092 [ −0.114 , −0.073 ] ∗
−0.007 [ −0.015 , +0.001 ]
Gemma
0.969
−0.101 [ −0.126 , −0.078 ] ∗
−0.013 [ −0.023 , −0.005 ] ∗
Sentiment
LLaMA
0.983
−0.043 [ −0.063 , −0.025 ] ∗
+0.006 [ −0.001 , +0.015 ]
Qwen
0.968
−0.078 [ −0.106 , −0.050 ] ∗
+0.015 [ +0.003 , +0.029 ]
Gemma
0.982
−0.054 [ −0.076 , −0.034 ] ∗
+0.009 [ +0.001 , +0.019 ]
Appendix
Table 10: Held-out AUROC of the last-token reading and its difference to max and mean pooling, with 95% intervals ( ∗ : significant after Holm correction). Max pooling is worse in every case.
Truthfulness
Sentiment
PopQA (“unknown”)
Depth
Layer
LLaMA
Qwen
Gemma
LLaMA
Qwen
Gemma
LLaMA
Qwen
Gemma
0.125
4 / 6
0.886
0.874
0.821
0.915
0.880
0.831
0.782
0.764
0.650
0.25
8 / 12
0.931
0.916
0.894
0.957
0.920
0.922
0.888
0.918
0.885
0.3125
10 / 15
0.946
0.932
0.927
0.971
0.943
0.957
0.902
0.935
0.859
0.375
12 / 18
0.958
0.938
0.947
0.979
0.960
0.975
0.911
0.942
0.929
0.4375 †
14 / 21
0.960
0.961
0.968
0.980
0.964
0.978
0.915
0.945
0.933
Appendix
Table 11: Held-out AUROC of the best direction of the family at each relative depth ( † : the depth used in the paper; bold: the best depth). From 3/8 of the depth upward, probing varies little.
Concept
Model
Paper’s layer: AUROC [95% CI]
Best layer: AUROC [95% CI]
Truthfulness
LLaMA
14: 0.960 [0.952, 0.968]
same layer
Qwen
14: 0.961 [0.954, 0.969]
16: 0.966 [0.959, 0.973]
Gemma
21: 0.968 [0.961, 0.975]
24: 0.971 [0.964, 0.978]
Sentiment
LLaMA
14: 0.980 [0.971, 0.989]
same layer
Qwen
14: 0.964 [0.952, 0.976]
18: 0.967 [0.956, 0.979]
Gemma
21: 0.978 [0.969, 0.987]
same layer
Appendix
Table 12: Held-out AUROC of the best direction at the paper’s layer and at the best layer of the sweep, with 95% intervals. The intervals overlap in every case.
Steering effect of Δμ at K=κ0
H1b: S(2)(λ∗)/S(2)(Δμ)
Truthfulness
Sentiment
Truthfulness
Sentiment
Depth
Layer
L
Q
G
L
Q
G
L
Q
G
L
Q
G
0.125
4 / 6
-0.016
0.063
0.014
0.071
0.040
0.002
–
1.78
0.45
1.62 ∗
1.97 ∗
–
0.25
8 / 12
0.309
0.196
0.125
0.228
0.229
0.086
0.97
1.12
1.28
1.42 ∗
1.68 ∗
1.70 ∗
0.3125
10 / 15
0.455
0.269
0.246
0.300
0.364
0.253
1.00
1.18 ∗
1.29 ∗
1.46 ∗
1.33 ∗
1.36 ∗
0.375
12 / 18
0.427
0.308
0.368
0.457
0.460
0.515
1.03
1.27 ∗
0.85
1.41 ∗
1.56 ∗
1.26 ∗
Appendix
Table 13: Steering effect of Δμ at each layer’s κ0 (left) and the H1b ratio (right) across depths (L = LLaMA, Q = Qwen, G = Gemma; ∗ : H1b holds; † : the paper’s depth). H1b holds for sentiment wherever Δμ steers, for truthfulness only at some layers.
Truthfulness
Sentiment
LLaMA
Qwen
Gemma
LLaMA
Qwen
Gemma
Depth
Layer
PR
PS
PR
PS
PR
PS
PR
PS
PR
PS
PR
PS
0.125
4 / 6
τ
τ
τ
10τ – 100τ
0.1τ – τ
0.1τ – Δμ
τ – 10τ
100τ
τ – 10τ
100τ
0.1τ
τ – 100τ
0.25
8 / 12
τ
100τ – Δμ
τ
10τ – 100τ
0.1τ – τ
0.1τ – 10τ
τ – 10τ
100τ
τ – 10τ
100τ
τ
τ – 10τ
0.3125
10 / 15
τ
100τ
τ
100τ
0.1τ – τ
100τ
10τ
100τ
10τ
100τ
τ
10τ
0.375
12 / 18
τ
100τ
τ
100τ
0.1τ – τ
Δμ
τ – 10τ
10τ – 100τ
10τ
100τ
τ
10τ
Appendix
Table 14: Probing plateau PR and steering plateau PS at each layer of the sweep, on the grid {0.001τ,0.1τ,τ,10τ,100τ,Δμ} ( † : the paper’s depth). Every plateau is a contiguous range of the grid. The two plateaus overlap in five cases (truthfulness: LLaMA layer 4, Gemma layers 6 and 12; sentiment: LLaMA layer 12, Gemma layer 12); in four of them the steering plateau still ends further toward Δμ than the probing plateau, so H1a holds, and at LLaMA layer 4 both plateaus are {τ} , so it fails.
LLaMA
Qwen
Gemma
Depth
Layer
λ∗
S(2)(λ∗)
S(2)(Δμ)
λ∗
S(2)(λ∗)
S(2)(Δμ)
λ∗
S(2)(λ∗)
S(2)(Δμ)
0.125
4 / 6
τ
0.044
-0.034
10τ
0.119
0.067
100τ
0.010
0.022
0.25
8 / 12
100τ
0.276
0.284
100τ
0.153
0.136
10τ
0.133
0.104
0.3125
10 / 15
100τ
0.413
0.411
100τ
0.286
0.242
100τ
0.293
0.227
0.375
12 / 18
100τ
0.398
0.388
100τ
0.372
0.293
10τ
0.277
0.325
0.4375 †
14 / 21
100τ
0.310
0.312
100τ
0.647
0.590
10τ
0.587
0.594
Appendix
Table 15: The two effects behind each truthfulness ratio of Table 13 : the direction λ∗ chosen on the first held-out split, and the effects of λ∗ and of Δμ on the second split (change of the answer margin in nats at K=κ0 ; † : the paper’s depth). The sentiment ratios divide effects between 0.04 and 1.95 and lie between 1.23 and 1.97 at every defined layer, so we do not list them.
Figure 6: Probing (top, held-out AUROC) and steering at κ0 in each concept’s own units (bottom) against λ/τ . The shapes match Figure 2 , but the size of the effects differs between models.
Figure 7: Each direction of the family at its probing AUROC (horizontal) and its steering effect relative to Δμ (vertical). No curve ends in the upper right corner, so no direction is best at both.
Map R2
ρprobe at B ’s best λ
ρsteer of ΔμA
ρsteer , steering law
Concept
Pair
neutral
concept
probe law
steering law
Procrustes
steering law
probe law
wA,τ
wA,10τ
Truthfulness
L → Q
0.56
0.16
0.81
0.62
0.80
0.64
0.61
0.15
0.41
L → G
0.70
0.26
0.89
0.10
0.89
0.44
0.30
0.20
0.42
Q → L
0.50
0.14
0.83
0.74
0.83
0.54
0.39
0.40
0.71
Q → G
0.69
0.25
0.88
0.10
0.90
0.53
0.21
0.27
0.49
G → L
0.48
0.15
0.91
0.83
0.93
0.62
-0.13
0.56
0.74
Appendix
Table 16: Cross-model controls per ordered pair A→B (L = LLaMA, Q = Qwen, G = Gemma). Map R2 : held-out R2 of MA→B on neutral text and on the concept texts. ρprobe : AUROC above chance of the transferred direction at B ’s best λ , relative to B ’s own, under the probe law MB→A⊤wA (the value of Figure 3 ), the steering law MA→BwA , and the orthogonal Procrustes map QwA . ρsteer : steering effect in B at its κ0 relative to B ’s own Δμ , for ΔμA under the steering law (the value of Figure 3 ) and the probe law, and for the source’s directions wA,τ and wA,10τ under the steering law. Steering with the Procrustes map was not run.
Figure 8: ρprobe (top) and ρsteer (bottom) for each source (rows) and target (columns); darker cells are higher. Probing values are uniformly high, steering values lower and uneven.
Figure 9: ρprobe (horizontal) and ρsteer (vertical), both computed at λ=τ (left), 10τ (middle) and Δμ (right). Probes transfer well except at the Δμ end with Gemma, and steering transfer depends on the point of the family.
Figure 10: Probing after the probe law (left) and after the steering law (middle), and steering after each law (right), for each model pair and concept, with the pairs that involve Gemma labelled in the first two panels. The probe law keeps what a direction reads, and the steering law keeps where it pushes.
Figure 11: AUROC of the hidden-state probe (dark) and of the first-token entropy Hent (light) for the three contrasts of H2a and, in the last row, for all questions; the dotted line marks chance. The probe beats the entropy once abstentions are included, and the entropy beats the probe on C2 against C3.
Figure 12: Change in abstention (pp) on confident errors (C3, horizontal) and uncertain errors (C2, vertical) for each model (rows) and strength (columns). All points lie above the diagonal.
Activation probes are computed directions in a language model's residual stream, used both to detect safety-relevant behaviours and, by adding their direction back into the stream, to steer them. We explore how the main design choices behind such probes interact, within one pipeline, on one model (Llama-3.1-8B-Instruct), and across three behaviours: deception, high-stakes awareness and sycophancy. The pipeline crosses two estimators (difference-of-means and logistic regression) with three token-pooling schemes (last, mean and learned attention), selects appropriate layers to detect or steer a behaviour, and evaluates every direction twice: as a detector of the behaviour, and as a steering vector to push outputs towards the behaviour. Prior work read together suggests a read/write dissociation, in which logistic regression reads (detects) best but difference-of-means directions write (steer) best. We find that it holds for deception and high-stakes but not for sycophancy, where the pattern reverses. The reversal appears to follow partially from the data rather than from the estimators. The best pooling technique and the layer choice depend on the behaviour and data, and the layer that steers best can sit well before the layer that detects best. The right steering metric to use between changing a model's perception and eliciting behaviour depends on the task - and the two can disagree outright. The exported probes and directions power an interactive interface through which a user can score and steer text directly.
Large language models can be steered at inference time through prompting or activation interventions, but activation steering methods often underperform compared to prompt-based approaches. We propose a framework that formulates prompt steering as a form of activation steering and investigates whether distilling successful prompt steering behavior into simpler, interpretable models can close this gap. Our analysis reveals that popular activation steering methods are not faithful to the mechanics of prompt steering, which applies strong interventions on some tokens while barely affecting others. Based on these insights, we introduce Prompt Steering Replacement (PSR) models that estimate token-specific steering coefficients from the activations themselves and are trained to imitate prompt-based interventions. Experiments on three steering benchmarks across multiple language models show that PSR models outperform existing activation steering methods, especially when controlling for high-coherence completions, and also compare favorably to prompting on AxBench and persona steering.
Activation steering has gained popularity as a lightweight inference-time debiasing technique for large language models. However, prior work reports that steering vectors generalise poorly, with unintended effects on model performance and limited transfer to new datasets. Our work analyses what the debiasing direction used for activation steering actually encodes, in order to shed light on its inconsistent performance. We study the linear debiasing direction obtained by contrasting the activations of anti-biased and biased prompts, and evaluate it as a steering intervention across bias and general knowledge benchmarks. We find that this direction is dominated by model confidence, pointing from regions of high to low-probability tokens in activation space rather than encoding a meaningful representation of model bias. Steering along it does reduce measured bias, but this is a consequence of reducing model confidence: on QA benchmarks we find that this steering drives the model to abstain from answering, with a side effect of improving fairness metrics. Our experiments show that model confidence is the dominant separating factor between biased and anti-biased prompts in hidden space, indicating that isolating a linear representation of bias which is disentangled from model confidence is difficult and steering-based debiasing results should be interpreted with care. In short, steering appears to reduce bias, not by correcting the model's underlying preferences, but by making it less confident, even on tasks unrelated to bias.