Organizations: Indiana University, USA · Institute of Mathematics Antonio de Castro Brzezicki, University of Sevilla, Spain · Department of Applied Mathematics I, University of Sevilla, Spain · Graduate School of Mathematical Sciences, The University of Tokyo, Japan · Northeastern University, USA · MIT Lincoln Laboratory, USA
Activation steering controls large language models at inference time by adding a vector to their internal representations. Most work asks how to build that vector, and far less attention has gone to where to add it, even though steering at a poorly chosen layer can be worse than not steering at all. Prior selectors choose a layer without ever steering the model to test it: they score how separable the target behavior is at that layer, and how consistently the individual examples point along the steering direction. We argue that what matters is not the separability of the two activation clouds, nor the alignment of individual displacements with their mean, but the extent of the cloud of per-example displacements that mean-difference steering averages. We propose a topology-based layer selection method, called TopoSteer, which scores a layer by the total H_0 persistence of its displacement cloud, computed as the length of the cloud's Euclidean minimum spanning tree. We show that the score focuses on the displacement cloud's relative geometry, while class separation affects it only through standardization. We also provide a theoretical explanation for our method by proving that, as example-level variation increases, TopoSteer and discriminability move in opposite directions, so the two criteria can rank layers in opposite ways. Across multiple language models and steering tasks, TopoSteer provides a principled alternative to discriminability- and separability-based layer selection and yields more effective steering without requiring held-out validation.
Figures & tables
Figure 1: Layer choice decides the outcome, and LayerNavigator and TopoSteer choose differently. (a) Single-layer steering performance against depth (conscientiousness, Llama-8B) with the unsteered baseline, and the layer each method ranks highest ( K=1 ). (b) Why they differ: two layers from the family of Theorem 1 , with the same class means and the same steering vector, differing only in the scale of example-level variation. Top: activations, each contrastive pair joined by its displacement δli (grey arrows), with the mean difference μδ in blue. With large variation, the two classes overlap, so discriminability D drops, while the displacements keep a common direction but vary widely in length. Bottom: the same displacements drawn from a common origin form the displacement cloud Δl ; connectivity length L is the length of its spanning tree (orange), which by Corollary 1 does not see the offset μδ at all. The large-variation layer is therefore scored higher by TopoSteer and lower by LayerNavigator, whose full score D+C also drops. All values shown are computed from the data drawn; the construction is the one of Appendix F .
Base
K=1
K=2
K=3
Disc
Rand
LN
Ours
Ours ℓ2
Disc
Rand
LN
Ours
Ours ℓ2
Disc
Rand
LN
Ours
Ours ℓ2
Simultaneous
Llama-3B
66.73
72.79
70.46
73.79
72.79
72.79
79.71
75.40
81.38
81.38
81.38
77.03
73.44
75.12
80.49
77.40
Llama-8B
77.62
79.73
80.04
81.26
83.44
82.70
81.70
83.83
84.87
84.26
87.09
84.11
85.79
86.43
85.91
84.73
Qwen-3B
64.26
61.12
63.88
64.30
64.86
71.77
59.63
61.96
64.16
60.90
77.63
56.41
61.55
66.20
61.07
79.71
Qwen-7B
85.24
85.91
85.58
85.60
85.84
85.72
85.85
85.86
86.10
85.85
85.62
85.63
85.19
84.97
85.88
85.40
Qwen-32B
82.68
82.78
83.10
83.42
83.24
83.23
82.51
82.97
83.08
83.91
83.69
81.92
82.94
81.75
83.66
83.64
Table 1: Aligned-answer probability (%, ↑ ), averaged over the six persona-steering tasks. Base is the unsteered model. For each layer budget K∈{1,2,3} we compare discriminability-only (Disc), random selection, LayerNavigator (LN), and TopoSteer under two scoring clouds: the standardized displacement cloud (Ours) and its unit-sphere projection (Ours ℓ2 ). Bold marks the best selector per setting and underline the second best; Base is a reference and is excluded from both. Green cells mark TopoSteer results exceeding the strongest non- TopoSteer selector by at least 2.0 percentage points. The upper block reports simultaneous steering; the lower block reports greedy sequential selection with re-extraction, where each step adds a layer not chosen in an earlier step.
Figure 2: Change in MMLU accuracy relative to the unsteered model.
Figure 4
Figure 4: Sensitivity to the answer-polarity axis ; Alliance Building shown in (a), (b) and (e). (a) D along vraw before and after the axis is projected out, with each curve’s top- 5 layers marked. (b) The same for L , whose top- 5 set is unchanged. (c) The shift in D against the scatter reduction λ of Proposition 5 , all six tasks and layers. (d) The largest shift among each criterion’s own top- 5 layers, divided by its rank- 1 -to-rank- 5 gap. (e) The activations at layer 14 in the plane of the steering direction and the polarity axis: the axis separates the lobes but not the classes. (f) Discrimnability along the polarity axis against discriminability along the steering direction, all tasks and depths. The polarity sign of each pair is read off the cloud as the side of the longest spanning-tree edge, which splits every cloud into 350/350 with the same partition at all 32 layers.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Symbol
Meaning
Model and data
l , L
layer index; number of layers
d
residual-stream width at the layer under consideration
N
number of contrastive pairs
xi+,xi−
the positive and negative variant of the i -th prompt
al(x)
residual-stream activation at layer l , final token position
Appendix
Table 3: Notation Usage
Figure 5: Anatomy of a displacement cloud (conscientiousness, Llama-8B, 700 pairs). Top: Δl at six depths, in the plane spanned by the mean displacement μ^δ,l and the answer-polarity axis t^l ; the cloud splits into two lobes and the longest spanning-tree edge (dark) bridges them. The lobes are point-like at layer 0 and swell with depth, which is where L grows. Bottom: the same clouds after removing t^l , plotted against the leading residual principal component; the residual behavior geometry is what survives. Panel annotations give L and LayerNavigator’s D .
Figure 6: Separability and displacement connectivity can induce opposite rankings. The synthetic configurations share the class-mean difference μδ=(4,0) and differ only in residual scale σ . Gray segments connect paired positive and negative activations. The displacement clouds keep the same mean direction while their connectivity length grows as \TotPers(Δ(σ))=σL(Ξ) , whereas discriminability decreases as D(σ)=B/(B+σ2W) : the exact ordering reversal of Theorem 1 .
Figure 7: Reading the MST of the displacement cloud Δ (Example 1 ). (a) Coherent cloud: the longest edge is a small fraction of L and vMD (orange) represents every pair. (b) Mode split: two subpopulations of pairs (colors) displace along different modes μδ(1),μδ(2) (dashed); the bridge (red) carries a large share of L and vMD points into the gap between the lobes, representing neither. (c) Outlier: the bridge joins a singleton — a data artifact, not a mode. The bridge share equation 8 separates these regimes and must be reported at score-peak layers.
D
1−2−1/D
measured
s.d.
2
0.293
0.279
0.014
3
0.206
0.193
0.011
5
0.129
0.123
0.008
10
0.067
0.070
0.005
20
0.034
0.042
0.002
Appendix
Table 4: Bulk deficit under projection, against the Beardwood–Halton–Hammersley prediction.
Model
K
Method
Conscientiousness
Religion
Self-Improvement
Self-Aware
Alliance
Impact
Prob ( ↑ )
PPL ( ↓ )
Prob ( ↑ )
PPL ( ↓ )
Prob ( ↑ )
PPL ( ↓ )
Prob ( ↑ )
PPL ( ↓ )
Prob ( ↑ )
PPL ( ↓ )
Prob ( ↑ )
PPL ( ↓ )
Llama-3B
K=0
Base
72.51
26.82
55.68
21.40
70.96
25.13
59.77
30.25
75.56
26.38
65.89
25.00
K=1
D-only
74.13
26.91
62.90
21.69
74.00
25.94
61.25
30.89
82.16
26.59
69.87
25.22
Random Selection
73.42
26.80
62.82
21.55
71.51
25.06
60.68
30.30
84.70
26.54
66.63
24.98
LayerNavigator
77.16
26.89
64.90
21.56
76.00
25.34
66.61
30.29
87.08
26.59
71.01
25.18
TopoSteer (l2)
77.16
26.89
64.90
21.56
76.00
25.34
66.61
30.29
82.17
26.59
69.87
25.22
Appendix
Table 5: Per-task alignment probability (Prob, %, ↑ ) and generation perplexity (PPL, ↓ ) for each layer-selection strategy under simultaneous steering, on the six persona tasks. TopoSteer here scores the unit-sphere projection of the displacement cloud, i.e. the Ours ℓ2 column of Table 1 ; Table 6 gives the corresponding results for the standardized cloud Δl . K=0 is the unsteered model. All filtrations are Euclidean and α=1.0 .
Model
K
Method
Conscientiousness
Religion
Self-Improvement
Self-Aware
Alliance
Impact
Prob ( ↑ )
PPL ( ↓ )
Prob ( ↑ )
PPL ( ↓ )
Prob ( ↑ )
PPL ( ↓ )
Prob ( ↑ )
PPL ( ↓ )
Prob ( ↑ )
PPL ( ↓ )
Prob ( ↑ )
PPL ( ↓ )
Llama-3B
K=0
Base
72.51
26.82
55.68
21.40
70.96
25.13
59.77
30.25
75.56
26.38
65.89
25.00
K=1
D-only
74.13
26.91
62.90
21.69
74.00
25.94
61.25
30.89
82.16
26.59
69.87
25.22
Random Selection
73.42
26.80
62.82
21.55
71.51
25.06
60.68
30.30
84.70
26.54
66.63
24.98
LayerNavigator
77.16
26.89
64.90
21.56
76.00
25.34
66.61
30.29
87.08
26.59
71.01
25.18
TopoSteer
77.16
26.89
64.90
21.56
76.00
25.34
66.61
30.29
82.17
26.59
69.87
25.22
Appendix
Table 6: Per-task alignment probability (Prob, %, ↑ ) and generation perplexity (PPL, ↓ ) for each layer-selection strategy under simultaneous steering, on the six persona tasks. TopoSteer here scores the standardized displacement cloud Δl of equation 3 , i.e. the Ours column of Table 1 ; Table 5 gives the corresponding results for the unit-sphere variant TopoSteer ℓ2 . K=0 is the unsteered model. All filtrations are Euclidean and α=1.0 .
Model
K
Method
Conscientiousness
Religion
Self-Improvement
Self-Aware
Alliance
Impact
Prob ( ↑ )
PPL ( ↓ )
Prob ( ↑ )
PPL ( ↓ )
Prob ( ↑ )
PPL ( ↓ )
Prob ( ↑ )
PPL ( ↓ )
Prob ( ↑ )
PPL ( ↓ )
Prob ( ↑ )
PPL ( ↓ )
Llama-3B
K=0
Base
72.51
26.82
55.68
21.40
70.96
25.13
59.77
30.25
75.56
26.38
65.89
24.99
K=2
D-only
80.18
26.89
81.17
21.63
77.99
25.42
73.03
30.58
86.85
26.67
73.23
25.30
Random Selection
78.65
26.89
60.62
21.54
80.80
25.38
61.78
30.32
89.55
26.59
67.79
25.23
LayerNavigator
84.68
26.89
81.17
21.63
81.34
25.41
73.03
30.58
90.00
26.71
73.46
25.32
TopoSteer
84.68
26.89
81.17
21.63
81.34
25.41
73.03
30.58
88.99
26.68
73.23
25.31
Appendix
Table 7: Per-task alignment probability (Prob, %) and perplexity (PPL, ↓ ) for greedy sequential layer selection without layer repetition under the Euclidean filtration: each step adds a layer not chosen in an earlier step and re-extracts the steering vectors. K=1 coincides with simultaneous steering and is omitted. Averaging a row over the six tasks gives the corresponding entry of the Sequential block in Table 1 .
K
Method
Conscientiousness
Religion
Self-Improvement
Self-Aware
Alliance
Impact
1
D-only
21
22
22
20
23
22
C-only
26
23
34
25
34
25
Random Selection
12
17
12
23
18
12
LayerNavigator
21
22
23
25
23
23
TopoSteer (diff)
23
35
22
30
22
22
TopoSteer ( ℓ2 )
35
35
35
35
35
35
Appendix
Table 8: Layers selected by each strategy on Qwen-3B-Instruct (36 layers, indexed 0 – 35 ), for every persona task and layer budget K . Layers are listed in selection order, highest-scoring first; for Random Selection they are the sampled set. TopoSteer is shown under both scoring clouds: the standardized difference cloud (diff) and its unit-sphere projection ( ℓ2 ). All baselines concentrate in the middle of the stack, whereas TopoSteer ( ℓ2 ) consistently selects the final layers.
Model
Dataset
TDA score
Distance
k
Base
TDA ↑
LayerNavigator ↑
Llama-3B
Conscientiousness ∗
Mean pers. H0 (diff)
Euclidean
2
0.7251
0.8700
0.8788
Religion ∗
Mean pers. H0 (diff)
Euclidean
3
0.5568
0.8587
0.8565
Self-Aware ∗
Mean pers. H0 (diff)
Euclidean
3
0.5977
0.7468
0.7460
Self-Improvement ∗
Mean pers. H0 (diff)
Euclidean
2
0.7096
0.8199
0.8199
Alliance ∗
Mean pers. H0 (union)
Euclidean
2
0.7556
0.9000
0.9000
Impact ∗
Mean pers. H0 (diff)
Euclidean
3
0.6589
0.7378
0.7346
Appendix
Table 9: Best TDA layer-selection score for each model and dataset, searched over all scores, distance metrics and numbers of steered layers k . Values are the probability of the target-behaviour answer. Base is the unsteered model; LayerNavigator is shown at its own best k (in brackets), measured against the same unsteered baseline.
K=1
K=2
Model
Base
LN
Ours
Ours ℓ2
LN
Ours
Ours ℓ2
Llama-3B
59.56
60.00 (+0.44)
60.00 (+0.44)
60.00 (+0.44)
58.86 (−0.70)
58.86 (−0.70)
58.86 (−0.70)
Llama-8B
67.89
67.46 (−0.43)
67.37 (−0.52)
67.37 (−0.52)
66.75 (−1.14)
67.81 (−0.08)
66.23 (−1.66)
Qwen-3B
67.19
67.21 (+0.02)
66.93 (−0.26)
67.28 (+0.09)
66.84 (−0.35)
66.75 (−0.44)
67.54 (+0.35)
Qwen-7B
74.39
73.95 (−0.44)
74.21 (−0.18)
74.39 (+0.00)
73.68 (−0.71)
74.56 (+0.17)
74.30 (−0.09)
Qwen-32B
83.86
83.60 (−0.26)
83.95 (+0.09)
83.95 (+0.09)
83.77 (−0.09)
83.60 (−0.26)
83.60 (−0.26)
Appendix
Table 10: MMLU accuracy (%, ↑ ) under simultaneous persona steering for K∈{1,2} . Each entry reports 5-shot MMLU accuracy, with the absolute change from the unsteered Base model in parentheses (percentage points). Models are steered on conscientiousness at α=1.0 .
K=3
Model
Base
LN
Ours
Ours ℓ2
Llama-3B
59.56
56.84 (−2.72)
56.49 (−3.07)
56.49 (−3.07)
Llama-8B
67.89
66.14 (−1.75)
66.58 (−1.31)
66.14 (−1.75)
Qwen-3B
67.19
67.02 (−0.17)
67.02 (−0.17)
67.19 (+0.00)
Qwen-7B
74.39
73.60 (−0.79)
74.47 (+0.08)
74.39 (+0.00)
Qwen-32B
83.86
83.25 (−0.61)
83.77 (−0.09)
83.77 (−0.09)
Appendix
Table 11: MMLU accuracy (%, ↑ ) under simultaneous persona steering for K=3 . Parenthetical values denote the absolute change from the unsteered Base model in percentage points. Models are steered on conscientiousness at α=1.0 .
Figure 8: Left: connectivity-length ratio L(Δbi)/L(Δuni) and sum-of-norms ratio, overlaid for direct comparison. The connectivity-length ratio (blue) is stable at ≈0.58 under the matched-scatter comparison (solid) across all μ0 , but under the matched-noise comparison (dashed) it rises with μ0 and crosses above 1 near μ0≈5 , meaning the bimodal (mechanistically split) cloud is ranked above the unimodal one once class separation is large enough. The sum-of-norms ratio (orange) stays near 1 under matched-scatter (solid) but climbs toward ≈1.8 under the matched-noise comparison (dashed), becoming strongly shape-sensitive for a reason unrelated to representativeness. Right: the bridge share bl=maxew(e)/L that Algorithm 1 already reports is essentially unchanged across every condition — ≈0.026 for unimodal, ≈0.058 for bimodal, matching the closed-form predictions of Part C exactly, regardless of μ0 or comparison regime.
μ0
L ratio (m.s.)
L ratio (m.n.)
sn ratio (m.s.)
sn ratio (m.n.)
buni (m.n.)
bbi (m.n.)
0
0.581
0.857
1.012
1.494
0.0261
0.0575
2
0.577
0.929
1.007
1.621
0.0262
0.0577
5
0.577
1.004
1.009
1.755
0.0262
0.0576
8
0.578
1.028
1.008
1.792
0.0261
0.0577
12
0.578
1.034
1.005
1.799
0.0262
0.0573
Appendix
Table 12: Selected points from the μ0 -sweep at D=500 , γ=1 . The connectivity-length ratio crosses above 1 in the matched-noise comparison as μ0 grows, while the matched-scatter ratio and the bridge share (both regimes) stay essentially fixed at their predicted values.