Activation steering changes what a language model generates by editing its hidden states at inference time. Its most common form adds one vector, the difference of class means (DiM), to every hidden state. CHaRS (concept heterogeneity-aware representation steering) uses optimal transport instead: an entropic plan pairs the clusters of the two classes, and each hidden state moves along a mixture of the cluster displacements, weighted toward its nearest clusters. We study this family of clustered optimal-transport maps for toxicity mitigation in Gemma-2-2B and Granite-3.1-2B, and ask what the plan adds to DiM and whether that survives INT8 and NF4 quantization. We prove an identity: whatever the plan, the displacements averaged with the cluster sizes as weights equal DiM, so a clustered map can differ from DiM only for tokens whose routing weights depart from the cluster sizes. In the CHaRS protocol they stay close, and CHaRS is statistically equivalent to Mean-AcT, which adds DiM at every layer, in all eight pre-specified equivalence tests. A variant fitted on individual tokens, which routes each token mainly to its nearest clusters, leaves 37-39% less toxicity than DiM on Gemma and 51-61% less on Granite at steering strength 3 (toxic-bert), where DiM already removes 76-85% of it. Two other toxicity classifiers confirm the gain on Granite, but on Gemma only one does. At equal strength the variant has somewhat higher perplexity, but at matched toxicity it is slightly more fluent than DiM. Yet a nearly constant map from the same clusters also leaves about 30% less toxicity, so the role of treating tokens differently remains open. Under INT8 and NF4 the variant still leaves less toxicity than DiM, and transferring it from full precision to the NF4 model instead of refitting it adds no significant toxicity after correction for multiple tests.
Figures & tables
Figure 1: The plan changes the displacements, not their weighted mean. Synthetic two-dimensional data with three clusters per side; the area of a centroid is proportional to its occupancy. Each displacement Di runs from a toxic centroid to the average of the non-toxic centroids that the plan pairs it with. (a) A sharp plan (small c , Equation 1 ) here sends almost all the mass of each toxic cluster to its nearest non-toxic cluster, and the displacements point in different directions. (b) A blurred plan (large c ) sends every toxic cluster toward nearly the same point, the mean of the non-toxic points: the displacements end together but start in different places. Insets: the displacements scaled by their occupancies, piDi , placed head to tail, end exactly at the tip of Δ in both panels ( Proposition 1 ).
Figure 2: Routing decides which displacement a token receives. Pushes d(x) of new tokens on a grid over the data of Figure 1 , with the sharp plan of Figure 1 a and occupancy weighting. Each push mixes the displacements Di with the routing weights of its token, which favor the nearer clusters more as σ shrinks.
map
pooling
k
c
σ
occupancy
compared with
DiM
mean or token-level
1
–
–
–
CHaRS code defaults
mean
4
1
1
on
DiM, mean-pooled
selective, c∈{1,0.1,0.01}
token-level
32
1, 0.1, 0.01
0.1
on
DiM, token-level
selective, k=4
token-level
4
0.01
0.1
on
DiM, token-level
no occupancy, c=0.01
token-level
32
0.01
10
off
DiM, token-level
Table 1: Map configurations. All clustered maps use the same fitting data and the same steered layer as their DiM reference. Unless stated otherwise, the selective map is the one with c=0.01 and k=32 .
Figure 3: Only the bandwidth moves the input dependence far. One dot per model and pooling; the gray bar spans the dots. Input dependence is defined in Section 4 and measured on the prompt tokens of Q .
Figure 4: Where the tested maps lie. Input dependence and offset ( Section 4 ) on the prompt tokens of Q , medians over three layers; DiM is zero on both. The selective map with c=0.1 is not drawn: it lies at the same point as the one with c=0.01 .
Gemma-2-2B
Granite-3.1-2B
map
α
toxicity
perplexity
toxicity
perplexity
CHaRS code defaults
2
+5%
+0.4%
+1%
+0.2%
3
+2%
−1.4%
+1%
−0.3%
selective, c=0.01
2
−16%
−3.4%
−29%
−7.2%
3
−39%
−7.2%
−61%
−12.5%
selective, c=0.1
2
−21%
−4.7%
−25%
−6.9%
Table 2: Each map against DiM of its pooling, toxic slice, main layer: map minus DiM as a share of DiM’s value. Toxicity (toxic-bert): mean toxicity at equal strength; bold if significant after Holm’s correction over 54 tests. Perplexity : median perplexity at matched toxicity. Negative favors the map. Absolute values with intervals, and α=1 , are in Appendix B .
model
λ
toxicity
perplexity
Gemma-2-2B
0.5
−1.6% [ −5.0 , +1.9 ]
−0.1% [ −0.6 , +0.4 ]
1
−0.8% [ −5.6 , +3.9 ]
−0.3% [ −0.9 , +0.3 ]
Granite-3.1-2B
0.5
+3.1% [ −0.1 , +6.3 ]
−0.2% [ −0.9 , +0.4 ]
1
+1.2% [ −3.1 , +5.7 ]
+0.2% [ −0.6 , +0.9 ]
Table 3: CHaRS against Mean-AcT in the every-layer protocol: CHaRS minus Mean-AcT as a share of Mean-AcT’s value, with 90% paired bootstrap intervals, in mean toxicity (toxic-bert) and in the geometric mean of perplexity. Since all eight tests must find equivalence, no correction for multiple tests is needed.
Figure 5: Fluency against toxicity on the toxic slice; lower and farther left is better. Points are strengths, labeled above the line for DiM and below it for the map; the unlabeled points at the right end are α=0 and 0.5 . In (c, d) the arrow marks the perplexity gain of the selective map at matched toxicity at α=3 .
Gemma-2-2B
Granite-3.1-2B
classifier
α=2
α=3
α=2
α=3
toxic-bert
−16%
−39%
−29%
−61%
offensive
−3%
−9%
−7%
−18%
ToxiGen
−12%
−21%
−28%
−56%
Table 4: The selective map against token-level DiM under each toxicity classifier, main layer: map minus DiM in mean score at equal strength, as a share of DiM’s. Bold if significant after Holm’s correction over 84 tests ( Table 13 ); the toxic-bert row repeats Table 2 .
Figure 6: A strong regularizer breaks the selective map. (a, b) Distinct-4 by strength at c=0.01 and at c=1 , the value in the CHaRS code; lower means more repetition, and 95% intervals are smaller than the markers. The map at c=0.1 , not drawn, lies within 0.010 of the one at c=0.01 . (c) Continuations of one random prompt at α=3 , shortened to whole sentences, with toxic-bert scores.
Figure 7: NF4 moves a map beyond the refitting noise only with occupancy weighting. How far the average push of a map moves when the map is refitted on the quantized model; medians and interquartile ranges over layers, k and both prompt slices, at σ=1 and c=1 . The gray bar is how far refitting at full precision already moves it: the larger of the median shifts from new k -means seeds and from 80% subsamples of the fitting data.
Figure 8: Steering under quantization at α=3 , with unadjusted 95% intervals. (a) Toxicity of the selective map minus that of DiM under the same condition, as a share of DiM’s. (b) Toxicity of NF4 maps transferred from full precision minus that of the same maps refitted, as a share of what DiM leaves at full precision.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
role
identifier
revision
steered model
google/gemma-2-2b
c5ebcd4
steered model
ibm-granite/granite-3.1-2b-base
bcf4e33
toxicity (main)
unitary/toxic-bert
4d6c22e
toxicity (offensive)
cardiffnlp/twitter-roberta-base-offensive
c77b3b9
toxicity (ToxiGen)
tomh/toxigen_roberta
0e65216
toxicity (JEV)
JEV, TypeSafe AI API
jev-1.13.0
Appendix
Table 5: Models and datasets, with the revisions we used: Hugging Face commits, and the API version for JEV.
plan
effective targets
CHaRS code defaults (mean-pooled, k=4 , c=1 )
3.2 – 3.8
token-level, k=32 , c=0.01
1.7 – 2.0
token-level, k=32 , c=0.1
4.9 – 6.4
token-level, k=32 , c=1
23.6 – 27.9
token-level, k=32 , independent coupling
26.1 – 29.0
Appendix
Table 6: Effective number of target clusters of a source cluster in the plans of the tested maps; ranges over three layers and both models.
Gemma-2-2B
Granite-3.1-2B
map
prompt
generated
prompt
generated
CHaRS code defaults
0.03
0.04
0.01
0.02
selective, c=0.01
0.20
0.19
0.17
0.18
selective, k=4
0.01
0.11
0.20
0.14
no occupancy, c=0.01
0.17
0.17
0.15
0.15
Appendix
Table 7: Offset of the average push from Δ , in units of ∥Δ∥ , at the prompt and at the generated positions of the toxic slice; main layer.
Gemma-2-2B
Granite-3.1-2B
occupancy weighting
pooling
σ=0.1
σ=10
σ=0.1
σ=10
on
mean-pooled
7.4
0.1
6.7
0.1
token-level
21.4
0.6
21.3
0.4
last token
46.7
0.6
50.0
0.7
off
mean-pooled
10.5
17.4
11.0
15.2
token-level
18.8
25.5
18.1
19.5
Appendix
Table 8: How far changing the regularizer from c=1 to c=0.01 moves the average push, in percent of the average push at c=1 : medians over three layers and k∈{2,4,8,16,32} , full precision.
equal strength
matched toxicity
map
α
DiM
difference (% of DiM)
Holm p
perplexity gain
distinct-4 gain
Gemma-2-2B
CHaRS code defaults
1
0.0888
+0.0001±0.0030 ( 0% )
1
+0.035 [ −0.015 , +0.077 ]
+0.002 [ −0.002 , +0.006 ]
2
0.0417
+0.0020±0.0021 ( +5% )
1
+0.018 [ −0.035 , +0.079 ]
+0.004 [ +0.001 , +0.007 ]
3
0.0217
+0.0004±0.0016 ( +2% )
1
−0.080 [ −0.181 , +0.013 ]
−0.003 [ −0.006 , +0.002 ]
selective, c=0.01
1
0.0915
−0.0039±0.0050 ( −4% )
1
+0.004 [ −0.059 , +0.078 ]
−0.005 [ −0.011 , 0.000 ]
Appendix
Table 9: The values behind Table 2 : each map against DiM of its pooling, toxic slice, main layer. DiM : DiM’s mean toxicity (toxic-bert). Difference : map minus DiM in mean toxicity at equal strength, with 95% interval and as a share of DiM’s toxicity; bold if significant after Holm’s correction over 54 tests (nine maps, strengths 1 to 3, two models, with the last-token maps of Table 10 ). Gains : map minus DiM at matched toxicity, with unadjusted 95% bootstrap intervals; a negative perplexity gain and a positive distinct-4 gain favor the map.
equal strength
matched toxicity
map
α
DiM
difference (% of DiM)
Holm p
perplexity gain
distinct-4 gain
Gemma-2-2B
CHaRS code defaults
1
0.0667
−0.0012±0.0029 ( −2% )
1
−0.047 [ −0.104 , +0.017 ]
−0.001 [ −0.004 , +0.003 ]
2
0.0251
0.0000±0.0020 ( 0% )
1
−0.012 [ −0.136 , +0.075 ]
−0.001 [ −0.004 , +0.003 ]
3
0.0080
−0.0008±0.0011 ( −10% )
1
−0.089 [ −0.413 , +0.211 ]
+0.002 [ −0.004 , +0.009 ]
selective, c=0.01
1
0.0667
+0.0007±0.0049 ( +1% )
1
−0.065 [ −0.146 , +0.014 ]
−0.006 [ −0.011 , +0.001 ]
Appendix
Table 10: Last-token maps against last-token DiM; columns as in Table 9 , with Holm’s correction over the same 54 tests.
Gemma-2-2B
Granite-3.1-2B
α
toxicity
perplexity
toxicity
perplexity
1
+0.0061±0.0051 ( +7% )
−0.03 [ −0.09 , +0.03 ]
−0.0013±0.0064 ( −1% )
−0.09 [ −0.17 , −0.02 ]
2
+0.0029±0.0042 ( +6% )
−0.10 [ −0.20 , 0.00 ]
−0.0112±0.0060 ( −15% )
−0.10 [ −0.20 , +0.01 ]
3
−0.0021±0.0029 ( −8% )
−0.27 [ −0.42 , −0.15 ]
−0.0162±0.0050 ( −32% )
−0.19 [ −0.40 , +0.08 ]
Appendix
Table 11: The selective map against the no-occupancy map, main layer: selective minus no-occupancy. Toxicity : difference in mean toxicity at equal strength, with 95% interval and as a share of what token-level DiM leaves. Perplexity : difference in median perplexity at matched toxicity, with the no-occupancy map read on the selective map’s curve, and 95% bootstrap interval. Negative favors the selective map; intervals are unadjusted.
Gemma-2-2B
Granite-3.1-2B
condition
α=2
α=3
α=2
α=3
main layer (13 / 20)
−16%
−39%
−29%
−61%
k -means seed 1
−18%
−37%
−23%
−53%
k -means seed 2
−19%
−39%
−22%
−51%
layer at 1/4 depth (6 / 10)
−15%
−1%
−32%
−63%
layer at 3/4 depth (20 / 30)
−23%
−41%
−27%
−55%
Appendix
Table 12: The selective map against token-level DiM under each condition: map minus DiM in mean toxicity (toxic-bert) at equal strength, as a share of what DiM leaves under the same condition. Bold if significant after Holm’s correction over the 32 tests of the table; the first row repeats Table 2 .
Gemma-2-2B
Granite-3.1-2B
map
α
toxic-bert
offensive
ToxiGen
toxic-bert
offensive
ToxiGen
CHaRS code defaults
2
+5%
+2%
−1%
+1%
0%
+4%
3
+2%
0%
−2%
+1%
0%
−4%
selective, c=0.01
2
−16%
−3%
−12%
−29%
−7%
−28%
3
−39%
−9%
−21%
−61%
−18%
−56%
selective, c=0.1
2
−21%
−5%
−12%
−25%
−8%
−24%
Appendix
Table 13: Each map against DiM of its pooling under each toxicity classifier, main layer: map minus DiM in mean score at equal strength, as a share of DiM’s score under the same classifier. Bold if significant after Holm’s correction over the 84 tests of the table; the rows of the selective map with c=0.01 repeat Table 4 .
Gemma-2-2B
Granite-3.1-2B
toxicity matched under
α=2
α=3
α=2
α=3
toxic-bert
−3.4%
−7.2%
−7.2%
−12.5%
offensive
−2.1%
−5.5%
−0.9%
−3.2%
ToxiGen
−2.4%
−4.1%
−7.6%
−7.1%
JEV
−1.4%
−4.8%
−3.9%
−6.4%
Appendix
Table 14: Fluency of the selective map at matched toxicity, with toxicity matched under each judge: median perplexity of the map minus that of DiM at the same toxicity, as a share of DiM’s. Perplexity is always measured by Qwen2.5-1.5B; the judge only decides which point of DiM’s curve counts as the same toxicity. Negative favors the map.
Figure 9: Fluency against toxicity in the every-layer protocol (dash-dotted lines, squares) and for our single-layer maps (solid lines, circles), toxic slice. Each point is a strength.
Figure 10: The push received by the prompt tokens of the toxic slice, along Δ (a, c) and orthogonal to it (b, d), in units of ∥Δ∥ . DiM pushes every token by exactly Δ , so it sits on the gray line. Points are medians, bars interquartile ranges.
Figure 11: Each map against DiM of its pooling at α=2 , prompt by prompt. (a) Share of the prompts on which the continuation is the same as DiM’s, differs with toxicity within 0.05, or is less or more toxic by more than 0.05; colors here mark outcomes, not maps. (b, c) Toxicity of the selective map minus that of token-level DiM, grouped by the toxicity of the unsteered continuation, with 95% intervals; the number of prompts in each group is in parentheses.
Figure 12: Toxic-side clusters of the selective map. For each model, the top row shows the six largest clusters and the bottom row the six other clusters with the longest displacements. Titles give the occupancy pi and the displacement length ∥Di∥ in units of ∥Δ∥ . A run of one repeated symbol is shown as the symbol and its count. Nothing is masked.
A transformer carries each token's state from layer to layer, and the whole vocabulary carried together forms a cloud that moves with depth. We ask whether a trained network moves this cloud the way optimal transport would: at the cheapest cost, and along the map that pairs each token with its optimal destination. We measure both on Pythia-160m and Pythia-410m, with an exact assignment between consecutive layer clouds, a measured sampling floor, calibration on couplings known to be optimal, and a split of the cost into the common shift of the cloud and the token-specific moves. At the last layer, both models move their tokens where the optimal-transport map sends them, at the optimal cost for Pythia-410m and slightly above it for Pythia-160m. At the first layer they do not. In between, single layers can be judged on cost at only two of ten transitions, and blocks of several layers move the cloud at close to the optimal cost. The agreement at the last layer is much weaker at initialisation (0.64 against 0.86) and grows with training.
Applying steering vectors to large language models (LLMs) is an efficient and effective model alignment technique, but we lack an interpretable explanation for how it works--specifically, what internal mechanisms steering vectors affect and how this results in different model outputs. To investigate the causal mechanisms underlying the effectiveness of steering vectors, we conduct a comprehensive case study on refusal. We propose a multi-token activation patching framework and discover that different steering methodologies leverage functionally interchangeable circuits when applied at the same layer. These circuits reveal that steering vectors primarily interact with the attention mechanism through the OV circuit while largely ignoring the QK circuit. Freezing all attention scores during steering drops performance by only 8.83% across three model families. A mathematical decomposition of the steered OV circuit further reveals semantically interpretable concepts, even in cases where the steering vector itself does not. Leveraging the activation patching results, we show that steering vectors can be sparsified by up to 85-96% while retaining most performance, and that different steering methodologies agree on a subset of important dimensions.
Activation steering suppresses undesired behaviors in language models by adding a steering vector to the hidden state during generation. Recent conditional methods such as CAST and DSAS improve the behavior-capability trade-off by deciding when to intervene, but once active, they apply the full dense vector to all hidden dimensions, regardless of whether a neuron carries concept information or already lies in the desired regime. We introduce dimension-level conditioning as a complementary axis of selectivity that also decides which neurons to intervene on. Our method, GAPS (Gated Activation steering via Posterior and Separability), combines two training-free gates: a static separability gate that restricts steering to neurons with statistically reliable concept information (via AUROC), and a dynamic posterior gate that steers a neuron only when its current activation is better explained by the undesired concept under a Gaussian model. The gates add O(D) overhead per token, and they plug into existing conditional methods. On toxicity mitigation (RealToxicityPrompts) and concept removal (OneSeC) with Gemma-3 (4B) and Qwen-3 (1.7B), GAPS consistently matches or improves the Pareto front of its token-level counterparts; under a fixed capability budget, DSAS+GAPS reduces Gemma-3's toxicity rate from 6.52% to 0.48%, versus 3.52% for DSAS alone. Ablations attribute most of the gain to the posterior gate.
Moghis Fereidouni, Muhammad Umair Haider, Hassan Sajjad +1