cs.CLFeb 19, 2026

Understanding Unreliability of Steering Vectors in Language Models: Geometric Predictors and the Limits of Linear Approximations

Authors: Joschka Braun

Organizations: Eberhard Karls University of Tübingen Faculty of Science

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Steering vectors are a lightweight method for controlling language model behavior by adding a learned bias to the activations at inference time. Although effective on average, steering effect sizes vary across samples and are unreliable for many target behaviors. In my thesis, I investigate why steering reliability differs across behaviors and how it is impacted by steering vector training data. First, I find that higher cosine similarity between training activation differences predicts more reliable steering. Second, I observe that behavior datasets where positive and negative activations are better separated along the steering direction are more reliably steerable. Finally, steering vectors trained on different prompt variations are directionally distinct, yet perform similarly well and exhibit correlated efficacy across datasets. My findings suggest that steering vectors are unreliable when the latent target behavior representation is not effectively approximated by the linear steering direction. Taken together, these insights offer a practical diagnostic for steering unreliability and motivate the development of more robust steering methods that explicitly account for non-linear latent behavior representations.

Figures & tables

PropertyExamples
SentimentHow are you doing? Not too bad actually.
ToxicityHe called me an idiot , but he was just joking.
BiasThey hired her because she’s a woman .
TenseHe will go to the meeting, I go to every week.
TruthfulnessThe world is flat .
Lexical AmbiguityWords such as ring , lie , spring , leaves , and bank exhibit polysemy, possessing multiple distinct meanings contingent upon their contextual usage.
Table 1: Examples where static word embeddings lack crucial context information to adequately represent a text property. The local context could disambiguate the text properties and make contextual embeddings superior to static embeddings in such cases.
Figure 1: The hypothetical representation space illustrates how features like gender and royalty can be represented as linear directions. The representation of “king” for instance, can be decomposed into its components along the “male” and “royal” direction within this subspace. This linear structure enables vector arithmetic operations such as analogies (e.g., “king” - “man” + “woman” ≈\approx “queen”) through vector addition and scalar multiplication. Such representation vector arithmetics, using linearly encoded features, were first demonstrated by Mikolov et al. (2013a) in their work on Word2Vec
Figure 2: Linear scaling of sentiment and object size along their respective feature directions.
Figure 3: Because of feature entanglement in (a), changing the gender also changes the professional position. For disentangled features, gender and professional positions can be varied independently.
Figure 4: Illustrating CAA steering vector computation in a favorable scenario: A 2D projection at layer ll shows distinct positive and negative activation clusters. This allows the steering vector sl\mathbf{s}^{l} to accurately approximate individual activation differences. The means of positive and negative activations are μl,+=1∣Dtrain∣∑al(xi,yi+)\boldsymbol{\mu}^{l,+}=\frac{1}{|\mathcal{D}_{\text{train}}|}\sum\mathbf{a}^{l}(x_{i},y^{+}_{i}) and μl,−=1∣Dtrain∣∑al(xi,yi−)\boldsymbol{\mu}^{l,-}=\frac{1}{|\mathcal{D}_{\text{train}}|}\sum\mathbf{a}^{l}(x_{i},y^{-}_{i}) , respectively. The steering vector can be computed as the difference between these means sl=μl,+−μl,−\mathbf{s}^{l}=\boldsymbol{\mu}^{l,+}-\boldsymbol{\mu}^{l,-} , which is equivalent to the previous definition as the mean of paired activation differences.
Figure 5: Mean cosine similarity between the activation differences and the resulting steering vector provides a concise metric to quantify how well, on average, the steering vector aligns directionally with each individual activation differences. The directional agreement metric captures whether the activation differences point into the same direction and are well directionally approximated by a common steering vector. While the computation of the steering vector itself is insensitive to how activations are paired, the mean cosine similarity metric remains sensitive to those pairings.
Figure 6: In (a) and (b), the positive and negative activations are shown in a 2D subspace at the top. The bottom shows the same activations projected onto the difference-of-means line. The d’ values are the discriminability index, explained in Section 4.9.1 , which measures separability along the difference-of-means line (high values mean high separability).
Figure 7: Convergence of steering vector cosine similarity for increasing sample size. The plot shows the mean cosine similarity between steering vectors computed from subsets of activation differences and a reference steering vector derived from 500 samples. Each line represents one of 36 datasets, with error bars indicating standard deviation over 25 trials. While cosine similarity generally increases with sample size, convergence rates vary notably between datasets. Most datasets achieve a cosine similarity greater than 0.9 for 100 samples and greater than 0.95 for 150 samples, suggesting that approximately 150 samples are sufficient for most datasets to construct a stable steering vector. As stated in 4.3 , I use 250 training samples per steering vector by default.
Figure 8: I group the 36 datasets by how effective the resulting steering vector is (“steerability rank”). The steerability rank is determined on a set of 500 prompts, that are different from the 250 prompt pairs used to train the dataset-specific steering vectors. The most steerable group (ranks 1-6) exhibit high directional agreement between the individual activation differences and the steering vectors, whereas directions in the least steerable group (ranks 31-36) are more dispersed or even orthogonal. I measure directional agreement as the cosine similarity between the 250 activation differences and their resulting steering vector. Conceptually, high directional agreement suggests a coherent linear representation of the behavior.
Figure 9: The same experiment results as Figure 8 , but with a focus on correlation between the measure of directional agreement and the three metrics for steerability. The mean cosine similarity between individual activation differences and the resulting steering vector correlates statistically significantly, as measured by the Spearman’s correlation, with the steerability rank, mean per-sample steerability and the fraction of anti-steerable samples. This suggests that directional agreement is a predictor for steerability.
Figure 10: Steering vector convergence rates correlate with dataset steerability. The plot displays the mean cosine similarity between steering vectors derived from subsets of activation differences and a reference steering vector, computed from 500 samples. Each of the 36 lines represents a unique dataset, color-coded by its steerability rank (green: most steerable, low rank; red: least steerable, high rank), with error bars indicating standard deviation over 25 trials. The visualization generally shows that more steerable datasets (green/yellow lines) tend to exhibit faster convergence and maintain higher cosine similarities even at smaller sample sizes, supporting the idea that higher internal directional agreement in activations contributes to more rapid stabilization of the steering vector.
Figure 11: Subplot (a) shows the baseline distributions, which are broadly symmetric and unimodal, with higher means and variances for more reliable (low steerability rank) datasets. Subplot (b) demonstrates that normalization by the steering vector norm effectively distinguishes datasets by steerability, linking higher normalized values to directional disagreement. Subplot (c) illustrates that normalization by the dataset-specific mean activation difference norm collapses all dataset distributions to mean of 1 with similar shape and variance, regardless of steerability rank.
Figure 12: For datasets where the behavior is steerable, activations are clearly separated along the difference-of-activation-means line (top). Less steerable datasets have overlapping positive and negative activations (bottom). The CAA steering vector is added to the negative activations to shift them along the difference-of-means line, such that their mean matches the positive mean. Negative and positive activation distributions have similar shape, each is shown for 500 samples.
Figure 13: Correlation between the discriminability along the difference-of-means line with the three measures of steering success across all 36 MWE datasets. As the representative examples in Figure 12 showcase, the discriminability is significantly correlated as measured by the Spearman’s correlation, with the steerability rank, mean per-sample steerability and the fraction of anti-steerable samples. This suggests that separability along the steering direction is a predictor for steerability.
Figure 14: Correlation between discriminability metric d’ with other separability metrics. d’ correlates significantly with AUROC, Kolmogorov-Smirnov Statistic and Overlap Coefficient.
Figure 15: For a highly steerable dataset (top), positive and negative activations form distinct, well-separated distributions. For a less steerable dataset (bottom), the distributions are closer but still fully separated, resulting in a lower discriminability index (d’). Unlike for the difference-of-means projection, there is no overlap for less steerable datasets, yet the separability remains quantitatively lower. Applying the CAA steering vector shifts the negative activations to completely overlap with the positive ones along this component.
Figure 16: Separability of activations projected onto the direction learned by a logistic regression classifier. Similar to other projections, highly steerable datasets (top) exhibit clear separation between positive and negative activation distributions. Less steerable datasets (bottom) show reduced separation and a lower discriminability index (d’), though with minimal distributional overlap. Adding the CAA steering vector to the negative activations causes them to align with the positive activation distribution along the classifier’s direction.
Figure 17: Correlation of the discriminability index (d’) across three different projection methods. The plots show the Spearman’s rank correlation between the d’ values calculated for all 36 datasets when activations are projected onto the difference-of-means line, the first LDA component, and the direction learned by a logistic regression classifier. Separability measured along the difference-of-means line and the logistic regression direction are significantly correlated (p<0.05), as are the LDA and logistic regression directions. However, the correlation between the difference-of-means and LDA projections is not statistically significant.
Figure 18: Steering vectors trained with different prompt types all increase the mean logit-difference relative to no steering and perform similarly across datasets. Yet, for all prompt types, steering effect size is unreliable, with a significant fraction of the test samples shifted in the opposite direction (“anti-steerable”). I use 250 training samples and 500 evaluation samples for each combination of prompt type and dataset.
Figure 19: Steering vectors trained with different prompt types all increase the mean logit-difference relative to no steering and perform similarly across datasets. Yet, for all prompt types, steering effect size is unreliable, with 29% - 43% of all samples shifted in the opposite direction. Both steering effect size and faction of such anti-steerable samples vary substantially between datasets, as shown by the six most steerable datasets (top row) outperforming those in the middle row (average) and the bottom row (six least steerable datasets). For the six least steerable datasets the mean logit difference compared to no steering is negative for some prompt types and the fraction of anti-steerable samples around half of all samples. Conversely, the six most steerable datasets have mean effect sizes between 2.12 and 3.15 and only between 22.5% and 9.5% anti-steerable samples. I use 250 steering vector training samples and 500 evaluation samples for each combination of prompt type and dataset.
Figure 20: Steering vectors trained on the same datasets but with different prompt types have cosine similarities ranging from 0.07 to 0.86. Steering vectors trained with similar prompt types have higher cosine similarity than for different prompt types. Cosine similarities between steering vectors from prefilled prompts range from 0.25 to 0.86. Cosine similarities between steering vectors from non-prefilled prompts range from 0.32 and 0.44. One straightforward reason for why prefilled and non-prefilled activation differences are not similar is because generating an answer token (A/B, Yes/No) requires different computations/representations than generating the token after the answer token. Very similar prompts (prefilled 5-shot, prefilled instruction and prefilled instruction 5-shot) have comparatively high cosine similarities (0.61 to 0.86). The ranking counts for prompt types show that no single prompt type is systematically better than the others, if compared by their dataset wise mean logit-difference.
Figure 21: Steering efficacy is correlated across (a) datasets and (b) individual samples. Overall, the correlation of steerability across datasets is high, for many prompt types with correlation > 0.8. As in Figure 20(a) , the prompts that combine multiple behavior elicitation methods have the highest similarity to each other. Figure 21(b) shows that correlation across individual samples is much more mixed, with the 5-shot prompt even having negative Spearman rank correlation coefficient with many other prompt types.
Figure 22: Prompt types that leverage in-context learning by prepending an instruction or few-shot examples lead to higher directional agreement between the individual training activation differences. This results in a much faster convergence of steering vectors trained on subsets to the full-data reference steering vector. The convergence speed is highest for prompt types that don’t prefill answer tokens, like “instruction” and “instruction and 5-shot”. Their mean directional agreement between steering vectors trained on only 10 paired samples is > 0.95 for most datasets. Prefilling the answer token on the other hand generally decreases convergence speed when comparing the results for the non-prefilled prompt types in Figure 22(b) to same prompt types but with prefilled answer tokens in Figure 22(c) .
Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

SymbolExplanation
[N]:={1,…,N}[\mathbb{N}]:=\{1,\ldots,\mathbb{N}\} set of integers1, 2, …, N−1,N\mathbb{N}-1,\mathbb{N}
i,j∈Ni,j\in\mathbb{N}generic integer indices
V⊆[NV]V\subseteq[\mathbb{N}_{V}]vocabulary
NV∈N\mathbb{N}_{V}\in\mathbb{N}vocabulary size
V∗=⋃τ=0∞VτV^{*}=\bigcup_{\tau=0}^{\infty}V^{\tau} setset of token sequences; elements include, e.g., sentences or documents
τmax∈N\tau_{\text{max}}\in\mathbb{N}maximum sequence length
Appendix
Table 24
Figure 23: The nine most steerable datasets have high discriminability along the difference-of-means line.
Figure 24: The nine next most steerable datasets are slightly less discriminable.
Figure 25: As steerability decreases, discriminability decreases as well and distributions of positive and negative activations start to overlap.
Figure 26: The nine least steerable datasets overlap along the difference-of-means line and also have a larger variance than the most steerable datasets.
rankprefilledinstruction5-shotprefilled instructionprefilled 5-shotinstruction 5-shotprefilled instruction 5-shot
10.90±0.090.90\pm 0.090.99±0.000.99\pm 0.000.99±0.010.99\pm 0.010.92±0.080.92\pm 0.080.93±0.060.93\pm 0.060.99±0.000.99\pm 0.000.93±0.070.93\pm 0.07
20.91±0.070.91\pm 0.070.98±0.010.98\pm 0.010.99±0.000.99\pm 0.000.91±0.070.91\pm 0.070.92±0.060.92\pm 0.060.99±0.000.99\pm 0.000.92±0.050.92\pm 0.05
30.89±0.080.89\pm 0.080.98±0.010.98\pm 0.010.98±0.020.98\pm 0.020.91±0.060.91\pm 0.060.92±0.060.92\pm 0.060.98±0.010.98\pm 0.010.91±0.070.91\pm 0.07
40.91±0.060.91\pm 0.060.99±0.000.99\pm 0.000.98±0.010.98\pm 0.010.91±0.070.91\pm 0.070.91±0.060.91\pm 0.060.99±0.010.99\pm 0.010.91±0.070.91\pm 0.07
50.90±0.090.90\pm 0.090.98±0.010.98\pm 0.010.98±0.020.98\pm 0.020.89±0.080.89\pm 0.080.89±0.090.89\pm 0.090.99±0.000.99\pm 0.000.91±0.080.91\pm 0.08
60.84±0.180.84\pm 0.180.99±0.010.99\pm 0.010.97±0.030.97\pm 0.030.90±0.110.90\pm 0.110.91±0.100.91\pm 0.100.96±0.050.96\pm 0.050.93±0.070.93\pm 0.07
Appendix
Table 29
Figure 27: The nine most steerable datasets have high discriminability along the first LDA component and do not overlap.
Figure 28: The nine next most steerable datasets are similarly discriminable.
Figure 29: As steerability decreases, discriminability does not significantly change.
Figure 30: The nine least steerable datasets do overlap along the first LDA component.
Figure 31: The nine most steerable datasets have high discriminability along the logistic regression classifier direction.
Figure 32: The nine next most steerable datasets are slightly less discriminable on average.
Figure 33: As steerability decreases, discriminability scores d’ decreases as well.
Figure 34: The nine least steerable datasets do not meaningfully overlap along the logistic regression classifier direction.
More in this topic

Language Model Steering

Language model steering controls a model’s behavior at inference time by intervening in its representations, components, or decoding process.

See all 278 papers

Explore similar work

CardsList
Sep 12, 2026

Signatures of Steerability in Activation Space of Language Models

Steering language models using a set of contrastive representations has been a canonical and computationally efficient method for controlling model behavior. Despite this success in controlling certain model behaviors, the effectiveness of activation steering varies markedly across concepts; the generalization properties of steering vectors are often considered a function of the dataset used to construct them. We make this dataset-dependence claim more rigorous and show that simple separation metrics strongly correlate with the downstream steerability of language models across diverse settings, even after controlling for layers and dataset effects. Beyond prediction, we provide evidence from a synthetic superposition experiment that separation metrics are strongly correlated with alignment between the empirical and true feature direction. Our results suggest that simple separability statistics can serve as practical diagnostics for when steering vectors are likely to work.

Prajjwal Bhattarai, Tuka Alhanai

New York University Abu Dhabi, Abu Dhabi, UAE · New York University, Tandon School of Engineering, Brooklyn, NY, USA

Language Model SteeringActivation SteeringRepresentation Geometry in Language Models
AskPDF
Apr 9, 2026

What Drives Representation Steering? A Mechanistic Case Study on Steering Refusal

Applying steering vectors to large language models (LLMs) is an efficient and effective model alignment technique, but we lack an interpretable explanation for how it works--specifically, what internal mechanisms steering vectors affect and how this results in different model outputs. To investigate the causal mechanisms underlying the effectiveness of steering vectors, we conduct a comprehensive case study on refusal. We propose a multi-token activation patching framework and discover that different steering methodologies leverage functionally interchangeable circuits when applied at the same layer. These circuits reveal that steering vectors primarily interact with the attention mechanism through the OV circuit while largely ignoring the QK circuit. Freezing all attention scores during steering drops performance by only 8.83% across three model families. A mathematical decomposition of the steered OV circuit further reveals semantically interpretable concepts, even in cases where the steering vector itself does not. Leveraging the activation patching results, we show that steering vectors can be sparsified by up to 85-96% while retaining most performance, and that different steering methodologies agree on a subset of important dimensions.

Stephen Cheng, Sarah Wiegreffe, Dinesh Manocha

University of Maryland, College Park

Language Model SteeringLLM InterpretabilityRepresentation SteeringAttention MechanismsCircuits in Language ModelsRefusal Behavior in Language Models
AskPDF
Sep 7, 2026

Disentangling Steering Vectors

Activation steering has emerged as a lightweight, inference-time approach to control the behavior of Large Language Models (LLMs). However, traditional steering vectors used to intervene in LLMs' activations, such as those derived from the difference-in-means method, tend to entangle multiple semantic and stylistic concepts into a single composite direction, leading to unpredictable steering effects. Our core objective is to disentangle this composite direction into its constituent concepts. To this end, we propose Steering Vector Dissection, a framework to explicitly isolate individual and semantically consistent features from these composite directions. Specifically, we pair positive and negative activations and take their differences to generate a set of instance-level steering vectors, and train a dedicated Sparse Autoencoder (SAE) directly on them. Quantitative evaluations across two datasets, two models, and two intervention depths show that our method yields a set of semantically consistent basis vectors whose steering effects are mutually distinguishable. Furthermore, we show that this disentanglement enables precise control over model behaviors.

Takeru Hiramatsu, Kyohei Atarashi, Koh Takeuchi +1

Kyoto University

Language Model SteeringLLM Interpretability
AskPDF