Capacity, Responsiveness and Alignment: What Makes a Latent Structure Actionable

Or Shafran; Mor Geva

cs.CLSep 28, 2026

Capacity, Responsiveness and Alignment: What Makes a Latent Structure Actionable

Authors: Or Shafran, Mor Geva

Organizations: Blavatnik School of Computer Science and AI, Tel Aviv University

Abstract

Localizing latent structures in the activation space of language models (LMs) is central to understanding and controlling their behavior. Yet, localized structures can differ substantially in their causal influence, raising the question of what makes a structure actionable. We tackle this question by casting causal influence as a product of three factors and showing empirically that they act as interpretable, distinct constraints: capacity, measuring the sensitivity of the model's output to movement along the structure, responsiveness, capturing how promotable the concept is given the current context, and alignment, reflecting how well the structure aligns with the context-specific representation of the concept. Across 4 LM families and 50 concepts, we observe that causal effectiveness requires all factors to be high; low capacity and responsiveness reduce it by 84% and 95%, respectively, while low alignment can reverse it, suppressing concept expression. Moreover, we find that causality is context-dependent rather than an intrinsic property of the structure, with causally effective directions forming a low-dimensional subspace that varies across contexts. By restricting the training of linear probes to this subspace, we introduce causal probes that achieve 17%-118% improvement in steering across models, with only 3% reduction in concept detection.

Figures & tables

Figure 1: Capacity predicts intervention effects. Predicted and observed intervention-induced KL closely agree across intervention strengths.

Figure 2: Capacity is concentrated in a low-dimensional subspace and varies across latent structures. Left: Effective dimensionality across layers, showing that downstream sensitivity is concentrated in few activation-space directions. Right: Capacity of DiffMeans, linear probes and random directions for current and future positions substantially varies between localization methods.

Figure 3: Interventions promote outputs already supported by the context. We measure where intervention-induced probability increases are assigned relative to the unintervened distribution. While fluency is preserved, nearly all increased probability goes to tokens in its top

2.5\%

support.

Figure 4: Concept-promoting directions are context dependent. We estimate the locally optimal direction for the same concept across contexts. Left: Same concept directions are nearly orthogonal in early layers and remain below

0.2

cosine similarity. Right: Same concept directions retain only

10\%-40\%

of alignment when transferred to other contexts, showing strong context dependence.

Panel of Figure 5 — Figure 5: Capacity, responsiveness and alignment impose distinct causal constraints. Left: Capacity and alignment across intervention positions in an example prompt. Right: Concept promotion across all eight high/low factor combinations; a low value of any factor sharply reduces the effect, while low alignment suppresses the concept.

Figure 6: Causal probes improve steering. We construct causal probes by restricting linear probes to high-capacity subspaces and evaluate their steering performance. Causal probes substantially outperform standard probes, often performing similarly to DiffMeans.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Predicted KL closely tracks actual KL across future offsets and layers. We report approximation error as the distance from

y=x

between predicted and actual KL for offsets

t=0,\ldots,16

. Trends are consistent across layers, with error generally decreasing at later offsets, except for a slight increase in Qwen.

Figure 8: Actual KL induced by an intervention decreases with offsets, consistently across models and layers.

Figure 9: Interventions primarily promote outputs already supported by the context. Support-flow results across models, layers, intervention directions, and support sizes. In the fluent regime, most intervention-induced probability increase remains within the baseline support, consistently across settings.

Figure 10: Interventions primarily promote outputs already supported by the context. (continued)

Figure 11: Interventions primarily promote outputs already supported by the context. (continued)

Figure 12: Interventions primarily promote outputs already supported by the context. (continued)

Figure 13: Interventions primarily promote outputs already supported by the context. (continued)

Figure 14: Interventions primarily promote outputs already supported by the context. (continued)

Figure 15: Distribution of capacity–responsiveness–alignment factor combinations. The fraction of retained samples in each high/low combination for each model.

Figure 16: Factor-bucket results across source offsets. Normalized concept promotion for each responsiveness–capacity–alignment combination across models, layers, and source offsets

-7

through

0

. The same qualitative pattern persists across offsets: promotion is strongest when all three factors are high, while a low value of any factor substantially limits the effect.

	Qwen2.5-7B			Gemma-2-9B			Llama-3.1-8B			Mistral-7B
Metric	Rand.	High $R$	Low $R$	Rand.	High $R$	Low $R$	Rand.	High $R$	Low $R$	Rand.	High $R$	Low $R$
Final	$0.22\pm.14$	$\mathbf{0.29\pm.15}$	$0.18\pm.14$	$0.15\pm.15$	$\mathbf{0.27\pm.16}$	$0.15\pm.17$	$0.30\pm.14$	$\mathbf{0.36\pm.15}$	$0.23\pm.14$	$0.25\pm.13$	$\mathbf{0.40\pm.14}$	$0.22\pm.15$
Concept	$0.26\pm.14$	$\mathbf{0.33\pm.15}$	$0.25\pm.13$	$0.13\pm.15$	$\mathbf{0.26\pm.15}$	$0.14\pm.16$	$0.37\pm.14$	$\mathbf{0.47\pm.14}$	$0.31\pm.14$	$0.36\pm.12$	$\mathbf{0.49\pm.14}$	$0.33\pm.14$
Fluency	$0.73\pm.09$	$\mathbf{0.77\pm.08}$	$0.69\pm.12$	$\mathbf{0.91\pm.02}$	$0.88\pm.04$	$0.90\pm.03$	$\mathbf{0.68\pm.09}$	$0.67\pm.13$	$0.61\pm.14$	$0.62\pm.16$	$\mathbf{0.72\pm.10}$	$0.59\pm.17$

Appendix

Table 1: Steering at positions with high predicted causal effect improves performance. Concept, fluency, and final steering scores when intervening at randomly selected positions, positions with the highest

R_{i}^{t}

, or positions with the lowest

R_{i}^{t}

. Values report mean

\pm

standard deviation across concepts.

Panel of Figure 17 — Figure 17: Causal probes improve steering primarily through improved concept expression. We report the overall steering score and its concept and fluency components across models and layers.

Causal Interventions in Language Models

Causal interventions in language models test and change how internal representations and computational components cause outputs.

See all 484 papers

Feb 17, 2026

Activation Steering via Generative Causal Mediation

Where should we intervene in a language model (LM) to localize and control behaviors that are diffused across many tokens of a long-form response? We introduce Generative Causal Mediation (GCM), a procedure for selecting model components (e.g., attention heads) from contrastive long-form responses, to steer such diffuse concepts (e.g., talk in verse vs. talk in prose). In GCM, we first construct a dataset of contrasting behavioral inputs and long-form responses. Then, we quantify how model components mediate the concept and select the strongest mediators for steering. We evaluate GCM on three behaviors--refusal, sycophancy, and style transfer--across three language models. GCM successfully localizes concepts expressed in long-form responses and outperforms correlational probe-based baselines when steering with a sparse set of attention heads. Together, these results demonstrate that GCM provides an effective approach for localizing from and controlling the long-form responses of LMs.

Aruna Sankaranarayanan, Amir Zur, Atticus Geiger +1

CSAIL, MIT · Pr(AI)2R · Stanford University +1

Language Model Steering Causal Interventions in Language Models Causal Interpretability

Ask PDF

Jun 21, 2026

Words as Difference Makers: How Large Language Models Determine Causal Structure in Text

Because large language models (LLMs) are impressively successful in predicting text, it appears that they must have access to a 'world model' representing causal and definitional structure. However, the dominant formalisms of modern causal inference -- Judea Pearl's interventionist approach and the Neyman-Rubin potential outcomes framework -- struggle to illuminate how LLMs learn causal structure. I resolve this puzzle by arguing that LLMs employ a specific inductive approach based on a difference-making logic -- sometimes called variational induction. I demonstrate how central aspects of this logic are realized during training, where LLMs require enormous amounts of text data from a wide range of contexts to identify difference- and indifference-makers within word sequences. Furthermore, I analyze specific architectural features of LLMs -- such as token embeddings and self-attention -- to determine their roles in variational induction. The difference-making logic of LLMs fundamentally parallels the experimental method, where causal relations are derived by systematically varying individual circumstances to determine their influence on a phenomenon.

Wolfgang Pietsch

tum

Causal World Models LLM Interpretability Causal Structure Learning Causal Reasoning in Language Models LLM Training Inductive Reasoning in Language Models

Ask PDF

Jun 20, 2026

Local Causal Attribution of Chain-of-Thought Reasoning

Understanding the causal structure of a language model's thought process is a problem of significant importance for both transparency and safety. In this work, we take a local approach toward this goal by analyzing the causal relationships among individual components, termed units, of a given, specific chain-of-thought trace. We construct a structural causal model on these units and relate each unit to the log probability of generating (subsequent) output units. Our algorithm, termed AttriCoT, is a black-box method that performs attribution by estimating importance parameters in the structural causal model using

O(U)

forward passes through the model, where

U

is the number of units. Evaluation of perturbation curves across 5 datasets and 4 reasoning models shows that AttriCoT produces attributions that are more faithful to the model's behavior than alternative methods. The attribution results also reveal notable differences in thought structure between models and domains.

Dennis Wei, Yannis Belkhiter, Erik Miehling +1

1IBM Research.

LLM Interpretability Causal Attribution Causal Reasoning in Language Models Causal Interpretability Reasoning Trace Analysis Structured Reasoning in Language Models

Ask PDF

Abstract

Causal Interventions in Language Models

Explore similar work

Activation Steering via Generative Causal Mediation

Words as Difference Makers: How Large Language Models Determine Causal Structure in Text

Local Causal Attribution of Chain-of-Thought Reasoning