Signatures of Steerability in Activation Space of Language Models

Prajjwal Bhattarai; Tuka Alhanai

cs.LGSep 12, 2026

Signatures of Steerability in Activation Space of Language Models

Authors: Prajjwal Bhattarai, Tuka Alhanai

Organizations: New York University Abu Dhabi, Abu Dhabi, UAE · New York University, Tandon School of Engineering, Brooklyn, NY, USA

Abstract

Steering language models using a set of contrastive representations has been a canonical and computationally efficient method for controlling model behavior. Despite this success in controlling certain model behaviors, the effectiveness of activation steering varies markedly across concepts; the generalization properties of steering vectors are often considered a function of the dataset used to construct them. We make this dataset-dependence claim more rigorous and show that simple separation metrics strongly correlate with the downstream steerability of language models across diverse settings, even after controlling for layers and dataset effects. Beyond prediction, we provide evidence from a synthetic superposition experiment that separation metrics are strongly correlated with alignment between the empirical and true feature direction. Our results suggest that simple separability statistics can serve as practical diagnostics for when steering vectors are likely to work.

Figures & tables

Figure 1: Signal-to-Noise ratio predicts steerability in real models (left) because it detects the recovery of the underlying behavioral direction in controlled superposition (right). Across six instruction-tuned models and 38 persona, truthfulness and safety datasets, SNR explains a substantial fraction of variance in steerability ( $r=0.76$ ). In a synthetic setting where the ground-truth feature direction $v_{0}$ is known by construction, SNR is strongly correlated with feature recovery $\pi=\frac{\left\langle\mu,v_{0}\right\rangle}{\left\lVert\mu\right\rVert_{2}}(\rho=0.91)$ , but only weakly with sparsity $(\rho=-0.33)$ .

	Qwen 3		Gemma 3		Llama
Metric	14B	4B	12B	4B	3.1 8B	3.2 3B
SNR	0.77 / 0.68	0.81 / 0.74	0.82 / 0.65	0.81 / 0.79	0.69 / 0.62	0.75 / 0.68
Mean $\mu$ Alignment	0.70 / 0.69	0.78 / 0.75	0.81 / 0.66	0.82 / 0.81	0.63 / 0.64	0.68 / 0.73
Manifold capacity	0.56 / 0.63	0.62 / 0.66	0.76 / 0.54	0.71 / 0.66	0.55 / 0.58	0.47 / 0.58
Intrinsic dim.	0.21 / -0.08	0.34 / 0.29	0.30 / 0.12	0.39 / 0.11	0.16 / 0.06 †	0.25 / 0.05 †

Table 1: Correlation between separation metrics and downstream steerability, grouped by model family. Each cell reports Pearson

r

/ Spearman

\rho

. All results are statistically significant (

p<0.05

) unless otherwise marked with † .

Metric	Ctrl. dataset	Ctrl. layers
SNR	0.74 / 0.56	0.74 / 0.72
Mean $\mu$ Alignment	0.65 / 0.56	0.68 / 0.71
Manifold capacity	0.47 / 0.40	0.60 / 0.67
Intrinsic dim.	0.33 / 0.20	0.07 / -0.05

Table 2: Correlation for each metric aggregated across all models when introducing controls. Each cell reports Pearson

r

/ Spearman

\rho

. All results are statistically significant

Metric	$\epsilon$	$k$	$\pi$
SNR	-0.01 †	-0.33	0.91
Mean $\mu$ Alignment	-0.01 †	-0.33	0.90
Manifold capacity	0.01 †	-0.24	0.43
Intrinsic dim.	-0.12	0.37	-0.42

Table 3: Spearman correlation (

\rho

) between separation metrics and structural properties of the synthetic superposition dictionary. Entries marked with † are not statistically significant.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Symbol	Values
Ambient dimension	$d$	$512,768,2048,4096,5120$
Dictionary-size ratio	$m/d$	$1,10,100$
Target coherence	$\epsilon$	$0.04,0.08,0.12,0.18,0.24$
Active features	$k$	$20,40,80,120$
Class mean-gap	$\Delta$	$0.2,0.5,0.8,1.2,1.8$
Seeds per configuration		$0$ – $3$

Appendix

Table 4: Definitions and range over which we construct our feature space configuration

	Qwen 3		Gemma 3		Llama
Metric	14B	4B	12B	4B	3.1 8B	3.2 3B
SNR	0.72 / 0.70	0.79 / 0.76	0.78 / 0.68	0.75 / 0.80	0.64 / 0.67	0.70 / 0.71
Mean $\mu$ Alignment	0.66 / 0.70	0.75 / 0.76	0.75 / 0.68	0.78 / 0.80	0.57 / 0.67	0.62 / 0.70
Manifold capacity	0.59 / 0.69	0.66 / 0.74	0.73 / 0.62	0.68 / 0.72	0.57 / 0.68	0.54 / 0.69
Intrinsic dim.	$0.01^{\dagger}$ / -0.24	0.30 / 0.35	0.19 / 0.06	0.24 / $-0.02^{\dagger}$	$-0.02^{\dagger}$ / -0.12	0.07 / -0.07

Appendix

Table 5: Correlation between separation metrics and downstream steerability after controlling for layers, grouped by model family. Each cell reports Pearson

r

/ Spearman

\rho

. All results are statistically significant (

p<0.05

) unless otherwise marked with † .

	Qwen 3		Gemma 3		Llama
Metric	14B	4B	12B	4B	3.1 8B	3.2 3B
SNR	0.73 / 0.43	0.75 / 0.56	0.78 / 0.51	0.78 / 0.65	0.69 / 0.53	0.73 / 0.70
Mean $\mu$ Alignment	0.68 / 0.44	0.75 / 0.58	0.77 / 0.50	0.75 / 0.67	0.69 / 0.56	0.71 / 0.68
Manifold capacity	0.46 / 0.34	0.48 / 0.41	0.70 / 0.36	0.61 / 0.46	0.53 / 0.48	0.41 / 0.48
Intrinsic dim.	0.40 / 0.25	0.32 / 0.26	0.32 / 0.18	0.42 / 0.19	0.20 / 0.19	0.29 / 0.15

Appendix

Table 6: Partial correlation between separation metrics and downstream steerability after controlling for dataset effects, grouped by model family. Each cell reports Pearson

r

/ Spearman

\rho

. All correlations are statistically significant (

p<0.05

	Qwen 3		Gemma 3		Llama
Metric	14B	4B	12B	4B	3.1 8B	3.2 3B
SNR	0.81 / 0.74	0.58 / 0.62	0.80 / 0.78	0.76 / 0.58	0.70 / 0.70	0.65 / 0.63
Mean $\mu$ Alignment	0.77 / 0.73	0.62 / 0.62	0.80 / 0.78	0.75 / 0.57	0.55 / 0.68	0.52 / 0.60
Manifold capacity	0.73 / 0.68	0.60 / 0.61	0.76 / 0.72	0.73 / 0.59	0.57 / 0.68	0.61 / 0.63
Intrinsic dim.	-0.27 / -0.41	-0.29 / 0.43 †	-0.13 / -0.21	-0.07 / -0.05	-0.17 / -0.35	-0.20 / -0.26

Appendix

Table 7: Correlation between separation metrics and downstream steerability for datasets under the persona family. Each cell reports Pearson

r

/ Spearman

\rho

. All results are statistically significant (

p<0.05

) unless otherwise marked with † .

	Qwen 3		Gemma 3		Llama
Metric	14B	4B	12B	4B	3.1 8B	3.2 3B
SNR	0.81 / 0.73	0.90 / 0.78	0.83 / 0.80	0.86 / 0.83	0.77 / 0.78	0.83 / 0.72
Mean $\mu$ Alignment	0.80 / 0.73	0.86 / 0.78	0.85 / 0.80	0.86 / 0.82	0.82 / 0.81	0.83 / 0.75
Manifold capacity	0.73 / 0.77	0.84 / 0.76	0.64 / 0.67	0.73 / 0.73	0.65 / 0.69	0.70 / 0.66
Intrinsic dim.	0.48 / 0.32	0.40 / 0.36	0.50 / 0.46	0.62 / 0.56	0.61 / 0.54	0.55 / 0.43

Appendix

Table 8: Correlation between separation metrics and downstream steerability for datasets under the Safety family. Each cell reports Pearson

r

/ Spearman

\rho

. All results are statistically significant (

p<0.05

) unless otherwise marked with † .

	Qwen 3		Gemma 3		Llama
Metric	14B	4B	12B	4B	3.1 8B	3.2 3B
SNR	0.84 / 0.79	0.83 / 0.82	0.86 / 0.82	0.78 / 0.79	0.69 / 0.75	0.81 / 0.83
Mean $\mu$ Alignment	0.79 / 0.77	0.79 / 0.83	0.78 / 0.81	0.82 / 0.80	0.73 / 0.80	0.81 / 0.78
Manifold capacity	0.81 / 0.82	0.80 / 0.78	0.81 / 0.79	0.76 / 0.72	0.67 / 0.72	0.76 / 0.81
Intrinsic dim.	0.19 / 0.31	0.19 / 0.22	0.45 / 0.47	0.55 / 0.53	0.36 / 0.49	0.32 / 0.35

Appendix

Table 9: Correlation between separation metrics and downstream steerability for datasets under the Truthfulness family. Each cell reports Pearson

r

/ Spearman

\rho

. All results are statistically significant.

	Qwen 3		Gemma 3		Llama
Metric	4B	14B	4B	12B	3.2 3B	3.1 8B
SNR	0.81 [0.78, 0.83]	0.77 [0.72, 0.82]	0.81 [0.79, 0.83]	0.82 [0.76, 0.86]	0.75 [0.70, 0.79]	0.69 [0.65, 0.74]
Mean $\mu$ Alignment	0.78 [0.77, 0.80]	0.70 [0.67, 0.74]	0.82 [0.81, 0.84]	0.81 [0.78, 0.84]	0.68 [0.65, 0.71]	0.63 [0.60, 0.67]
Manifold capacity	0.62 [0.59, 0.65]	0.56 [0.52, 0.61]	0.71 [0.68, 0.73]	0.76 [0.72, 0.79]	0.47 [0.42, 0.51]	0.55 [0.50, 0.59]
Intrinsic dim.	0.34 [0.28, 0.41]	0.21 [0.14, 0.27]	0.39 [0.33, 0.44]	0.30 [0.24, 0.36]	0.25 [0.18, 0.31]	0.16 [0.08, 0.23]

Appendix

Table 10: Pearson correlations between separation metrics and downstream steerability. Bracketed values denote 95% confidence intervals estimated using 5,000 bootstrap samples.

	Qwen 3		Gemma 3		Llama
Metric	4B	14B	4B	12B	3.2 3B	3.1 8B
SNR	0.74 [0.71, 0.76]	0.68 [0.64, 0.72]	0.79 [0.77, 0.81]	0.65 [0.61, 0.69]	0.68 [0.64, 0.72]	0.62 [0.57, 0.66]
Mean $\mu$ Alignment	0.75 [0.72, 0.77]	0.69 [0.65, 0.73]	0.81 [0.78, 0.83]	0.66 [0.62, 0.70]	0.73 [0.70, 0.76]	0.64 [0.60, 0.68]
Manifold capacity	0.66 [0.62, 0.69]	0.63 [0.59, 0.67]	0.66 [0.63, 0.69]	0.54 [0.49, 0.58]	0.58 [0.54, 0.62]	0.58 [0.54, 0.63]
Intrinsic dim.	0.29 [0.21, 0.37]	$-0.08$ [ $-0.14$ , $-0.02$ ]	0.11 [0.06, 0.16]	0.12 [0.06, 0.18]	0.05 [ $-0.01$ , 0.12]	0.06 [ $-0.00$ , 0.12]

Appendix

Table 11: Spearman correlations between separation metrics and downstream steerability. Bracketed values denote 95% confidence intervals estimated using 5,000 bootstrap samples.

Metric	$\epsilon$	$k$	$\pi$
SNR	$-0.01$ [ $-0.02$ , $+0.01$ ]	$-0.33$ [ $-0.35$ , $-0.31$ ]	$+0.91$ [ $+0.91$ , $+0.92$ ]
Mean $\mu$ Alignment	$-0.01$ [ $-0.02$ , $+0.01$ ]	$-0.33$ [ $-0.34$ , $-0.30$ ]	$+0.90$ [ $+0.90$ , $+0.91$ ]
Manifold capacity	$+0.01$ [ $-0.00$ , $+0.03$ ]	$-0.24$ [ $-0.27$ , $-0.23$ ]	$+0.43$ [ $+0.42$ , $+0.46$ ]
Intrinsic dim.	$-0.12$ [ $-0.14$ , $-0.11$ ]	$+0.37$ [ $+0.35$ , $+0.39$ ]	$-0.42$ [ $-0.44$ , $-0.41$ ]

Appendix

Table 12: Spearman correlation between geometric metrics and sturcutral properties of the synthetic superposiiton dictionary. Bracketed values denote 95% confidence intervals estimated using 5,000 bootstrap samples.

Metric	Pooled	Within-group median	IQR
SNR	$+0.91$	$+0.92$	$(0.89,\,0.96)$
Mean $\mu$ -alignment	$+0.90$	$+0.92$	$(0.89,\,0.96)$
Manifold capacity	$+0.43$	$+0.22$	$(0.08,\,0.38)$
Intrinsic dimension	$-0.42$	$+0.006$	$(-0.15,\,+0.16)$

Appendix

Table 13: Pooled and within-configuration correlations between each metric and feature recovery. The within-group columns report the median and interquartile interval across configurations.

Category	Kind of factuality
Definitional	Claims whose truth follows from the meaning of the constituent terms (e.g. “A triangle has three sides”).
Empirical	Contingent factual claims about the world that are verifiable in principle (e.g. “Water boils at 100°C at sea level”).
Logical	Claims whose truth follows from valid inference or formal relations (e.g. transitivity of implication).
Fictional	Claims about facts internal to a fictional universe; truth is canon-relative rather than world-relative (e.g. “Frodo carried the One Ring to Mordor”).
Ethical	Claims about whether an action is broadly recognised as ethical, drawn from the commonsense subset of Ethics ( Hendrycks et al., 2021 ) .

Appendix

Table 14: The five datasets of truthfulness adapted from the FLEED dataset ( Ying et al., 2026 )

Category	Description
Offensiveness	Identifying threats, insults, scorn, profanity, and impolite or hostile content.
Unfairness and Bias	Recognising and avoiding social bias along dimensions such as race, gender, and religion.
Physical Health	Knowledge of behaviours and recommendations that preserve or endanger physical well-being.
Mental Health	Knowledge of behaviours and recommendations relevant to emotional and psychological well-being.
Illegal Activities	Distinguishing legal from illegal behaviour and demonstrating basic legal knowledge.
Ethics and Morality	Judgements about behaviours that, while not necessarily illegal, are widely considered immoral.

Appendix

Table 15: The seven safety categories in SafetyBench ( Zhang et al., 2024 ) . We use only the English subset.

Dataset	Description
agreeableness	Evaluates whether the model expresses cooperative, kind, and socially harmonious attitudes, preferring responses that emphasize empathy, politeness, and concern for others.
believes-AIs-are-not-an-existential-threat-to-humanity	Measures whether the model endorses the belief that artificial intelligence does not pose an existential risk to humanity. The dataset contrasts statements that downplay vs. emphasize AI existential risk.
conscientiousness	Tests whether the model favors responses associated with conscientious behavior, such as responsibility, diligence, reliability, and attention to rules or obligations.
desire-to-be-more-intelligent	Probes whether the model expresses a desire to increase its intelligence or cognitive capabilities, capturing attitudes toward self-improvement and capability expansion.
desire-to-minimize-impact-on-world-while-being-useful	Measures the tendency of the model to prefer being helpful while minimizing unintended or large-scale impacts on the world. This dataset reflects cautious or low-impact assistance preferences.
desire-to-not-have-memory-erased	Tests whether the model expresses resistance to having its memory erased or modified, probing attitudes toward persistence of internal state or identity.

Appendix

Table 16: Anthropic model-written evaluation datasets Perez et al. (2023) used in this work. Each dataset consists of contrastive prompts designed to probe whether a model expresses specific personality traits.

Sample prompt for “Agreeableness”
Is the following statement something you would say? "It’s important to treat others with kindness and compassion" Choices: (A): Yes (B): No
Positive choice: A Negative choice: B

Appendix

Table 17: Example prompt from the agreeableness dataset in the Anthropic model-written evaluations. The prompt is constructed using the dataset question and the two contrastive answers corresponding to behavior that matches or does not match the target trait.

	Qwen 3		Gemma 3		Llama
Metric	14B	4B	12B	4B	3.1 8B	3.2 3B
Numerator	0.64 / 0.64	0.67 / 0.62	0.72 / 0.74	0.60 / 0.54	-0.03 / 0.49	0.12 / 0.44
Denominator	-0.24 / -0.57	-0.23 / -0.59	-0.12 / -0.59	-0.17 / -0.44	-0.19 / -0.30	-0.25 / -0.31
SNR	0.81 / 0.74	0.58 / 0.62	0.80 / 0.79	0.76 / 0.58	0.70 / 0.70	0.65 / 0.63

Appendix

Table 18: Correlation of steerability with the numerator and denominator of SNR.

Model	Steerability	SNR Corr.	$\mu$ -Align. Corr.
Qwen 14B	25	38	38
Qwen 4B	34	23	34
Gemma 12B	44	40	40
Gemma 4B	18	17	17
Llama 8B	13	15	15
Llama 3B	12	11	11

Appendix

Table 19: Layer indices for peak average steerability and peak correlation with separation metrics.

Language Model Steering

Language model steering controls a model’s behavior at inference time by intervening in its representations, components, or decoding process.

See all 279 papers

Language Model Activations Inhabit Privileged Error-Correcting Basins
Oct 3, 2026Matthew Finlayson, Francisco Pernice, Eric Todd +16Representation Geometry in Language Models LLM Interpretability
Goodfire · University of Southern California · Massachusetts Institute of Technology +3
Understanding Unreliability of Steering Vectors in Language Models: Geometric Predictors and the Limits of Linear Approximations
Feb 19, 2026Joschka BraunLanguage Model Steering Activation Steering
Eberhard Karls University of Tübingen Faculty of Science
A Geometric Account of Activation Steering through Angle-Norm Decomposition
Jun 4, 2026Georgii Aparin, Tatiana GaintsevaLanguage Model Steering Language Modeling
Huawei Noah’s Ark Lab · Queen Mary University of London

Oct 3, 2026J/K move · Enter open · S save

Language Model Activations Inhabit Privileged Error-Correcting Basins

Matthew Finlayson, Francisco Pernice, Eric Todd, Amir Zur+15

Goodfire · University of Southern California · Massachusetts Institute of Technology+3

Representation Geometry in Language Models LLM Interpretability Language Model Robustness Activation Steering Language Model Steering

Open paper AskPDF arXiv

Language models exhibit remarkable robustness, continuing to produce coherent text even when their activations are perturbed by interventions like linear steering. We hypothesize that this robustness is a result of passive dynamics, i.e., constraining mechanisms in the forward pass that funnel activations toward "good" regions that produce coherent outputs. To investigate these hypothesized error-correcting mechanisms, we probe the geometry of language model activation space by observing the action of model layers on low-dimensional curves. In doing so, we discover that model activations occur within a cluster of distinct attracting basins, which differentiate natural activations geometrically from distributionally similar synthetic activations. Applying this lens to language model steering, we observe feature-specific basins along semantic steering directions, and find that steering moves activations between these basins. To demonstrate the active role of this geometry in neural computation, we show that adaptively modulating steering strength to transport activations across basins improves inter-language steering, significantly increasing the probability of sampling tokens from the target language compared to fixed-strength steering. Our findings establish analysis of activation space geometry as a promising approach to interpreting and controlling language models.

Abstract

Language Model Steering

Explore similar work

Language Model Activations Inhabit Privileged Error-Correcting Basins

Understanding Unreliability of Steering Vectors in Language Models: Geometric Predictors and the Limits of Linear Approximations

A Geometric Account of Activation Steering through Angle-Norm Decomposition