Causality $\neq$ Invariance: Function and Concept Vectors in LLMs

Gustaw Opiełka; Hannes Rosenbusch; Claire E. Stevenson

cs.CLFeb 25, 2026

Causality $\neq$ Invariance: Function and Concept Vectors in LLMs

Authors: Gustaw Opiełka, Hannes Rosenbusch, Claire E. Stevenson

Organizations: University of Amsterdam

Abstract

Do large language models (LLMs) represent concepts abstractly, i.e., independent of input format? We revisit Function Vectors (FVs), compact representations of in-context learning (ICL) tasks that causally drive task performance. Across multiple LLMs, we show that FVs are not fully invariant: FVs are nearly orthogonal when extracted from different input formats (e.g., open-ended vs. multiple-choice), even if both target the same concept. We identify Concept Vectors (CVs), which carry more stable concept representations. Like FVs, CVs are composed of attention head outputs; however, unlike FVs, the constituent heads are selected using Representational Similarity Analysis (RSA) based on whether they encode concepts consistently across input formats. While these heads emerge in similar layers to FV-related heads, the two sets are largely distinct, suggesting different underlying mechanisms. Steering experiments reveal that FVs excel in-distribution, when extraction and application formats match (e.g., both open-ended in English), while CVs generalize better out-of-distribution across both question types (open-ended vs. multiple-choice) and languages. Our results show that LLMs do contain abstract concept representations, but these differ from those that drive ICL performance.

Figures & tables

Figure 1: Function vs. Concept Vectors . Top : Similarity matrices for $\mathcal{FV}$ s (left) and $\mathcal{CV}$ s (right) in Llama 3.1 70B; cells show how similar two prompt representations are (warmer = more similar). Middle : Schematic highlighting the distinction between heads with causal effect (Activation Patching-selected) and heads that encode format-invariant structure (RSA-selected). Bottom : Example prompts for two concepts across three formats (EN open - ended, FR open - ended, multiple - choice). Takeaway : $\mathcal{FV}$ s cluster by input format; $\mathcal{CV}$ s cluster by concept across formats.

Figure 2: Representational Similarity Analysis (RSA) . For each attention head, we compute a representational similarity matrix (RSM) over prompts spanning concepts and input formats (cosine similarity of head outputs). We construct a binary design matrix that marks pairs sharing the same concept, independent of format. The RSA score for a head is Spearman’s $\rho$ between the lower - triangular entries of the RSM and the design matrix; higher $\rho$ indicates stronger concept - invariant encoding.

Figure 3: Similarity matrices . Full similarity matrices extracted from top $K=5$ heads in $\mathcal{CV}$ s and $\mathcal{FV}$ s in Llama 3.1 70B for all concepts. See Appendix B for other models.

Figure 4: Concept vs. format RSA . Question type and Concept RSA scores for

\mathcal{CV}

s and

\mathcal{FV}

s in all models. Takeaway :

\mathcal{CV}

s encode more concept information and less input format than

\mathcal{FV}

Model	K=10	K=20	K=50	K=100
Llama-3.1 8B	1	1	$\mathbf{12}$	$\mathbf{28}$
Llama-3.1 70B	0	0	1	$\mathbf{6}$
Qwen2.5 7B	0	$\mathbf{4}$	$\mathbf{15}$	$\mathbf{39}$
Qwen2.5 72B	0	1	$\mathbf{3}$	$\mathbf{13}$

Table 1: RSA–AIE head overlap . Overlap between RSA and AIE heads (number of overlapping heads among top-

K

). Bold numbers indicate overlap significantly above chance (

p<0.05

; details in Appendix E ). Takeaway :

\mathcal{FV}

s and

\mathcal{CV}

s are composed of different attention heads.

Figure 5: Layer - wise AIE vs. RSA . AIE and RSA scores averaged across all heads per layer. Takeaway :

\mathcal{FV}

and

\mathcal{CV}

heads are in similar layers.

Figure 6: Overview of steering results . Top : We extract $\mathcal{CV}$ s and $\mathcal{FV}$ s from antonym ICL prompts in formats that are in - distribution (ID; OE - ENG ) or out - of - distribution (OOD; OE - FR , MC ) relative to the AmbiguousICL task (bottom - left). Bottom - left : We interleave two concepts— antonym and EN→FR translation —within one prompt; the model’s original prediction is the French translation. Bottom - right : Predictions after steering. Takeaways : (1) $\mathcal{FV}$ s yield larger ID gains. (2) $\mathcal{CV}$ s show more stable OOD effects across formats. (3) $\mathcal{FV}$ s can conflate concept with input format (e.g., French version of antonym and multiple - choice formatting).

Figure 7: Steering effect across layers . We inject

\mathcal{CV}

s and

\mathcal{FV}

s into Llama - 3.1 - 70B and plot the change in target - token probability (

\Delta P

) for four representative concepts (columns). Curves compare ID extraction format with OOD formats relative to the AmbiguousICL task (Figure 6 ). Higher

\Delta P

means the model assigns more probability to the expected token than the unsteered model. Takeaways : (1)

\mathcal{FV}

s typically achieve larger ID gains but often drop OOD. (2)

\mathcal{CV}

s yield smaller gains yet show more stable OOD behavior across formats. See Figure 16 for other concepts/models.

	+ Antonym	Top $\Delta$ Tokens
$\mathcal{FV}$	OE-ENG	_sweet (+56%), _fresh (+16%), _bland (+6%), _taste (+3%), _uns (+2%)
	OE-FR	_su (+31%), _dou (+27%), _frais (+5%), _fade (+5%), _ins (+3%)
	MC	_( (+53%), _A (+1%), _\n (+1%), _space (+0%), _) (+0%)
$\mathcal{CV}$	OE-ENG	_sweet (+49%), _fresh (+8%), _bland (+3%), _taste (+3%), _uns (+3%)
	OE-FR	_sweet (+54%), _fresh (+9%), _bland (+3%), _uns (+3%), _taste (+2%)
	MC	_sweet (+35%), _fresh (+12%), _bland (+4%), _uns (+3%), _taste (+3%)

Table 2: Token - level steering effects . Top tokens with largest probability gains when injecting

\mathcal{CV}

s or

\mathcal{FV}

s into Llama - 3.1 - 70B on the AmbiguousICL prompt (query shown above). Results shown at the layer with the strongest in - distribution effect per vector. Without intervention, the model predicts French translation _sa (from salé ) with 49%; antonym _sweet has 2%. English antonyms in red , French in blue , and the opening bracket (MC token) in green .

Figure 8: Consistency of steering effects . KL divergence between the probability distributions after the models were steered with an ID (OE-ENG) and OOD vectors (OE-FR [top], MC [bottom]). Lower values indicate more similar effects of ID and OOD vectors. Takeaway :

\mathcal{CV}

s steer the models more consistently than

\mathcal{FV}

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Similarity matrices extracted from top $K=1$ heads in $\mathcal{CV}$ s and $\mathcal{FV}$ s in Llama 3.1 8B.

Figure 10: Similarity matrices extracted from top $K=1$ heads in $\mathcal{CV}$ s and $\mathcal{FV}$ s in Qwen 2.5 7B.

Figure 11: Similarity matrices extracted from top $K=2$ heads in $\mathcal{CV}$ s and $\mathcal{FV}$ s in Qwen 2.5 72B.

Figure 12: Histogram of AIE scores for Llama 3.1 8B, 70B, Qwen 2.5 7B, and Qwen 2.5 72B. Note, the y-axis is on a log-scale. Takeaway : AIE scores are highly sparse.

Model	Best $K$	Best $\alpha$
Llama 3.1 8B	1	10
Llama 3.1 70B	5	10
Qwen 2.5 7B	3	10
Qwen 2.5 72B	5	15

Appendix

Table 3: Optimal hyperparameters for steering interventions across different models.

K

represents the number of attention heads used for

\mathcal{FV}

\mathcal{CV}

extraction, while

\alpha

controls the intervention strength.

Figure 13:

\Delta

P for French translations of all the concepts.

\mathcal{FV}

s and

\mathcal{CV}

s are extracted from open-ended French prompts.

Figure 14:

\Delta

P for French translations of all the concepts.

\mathcal{FV}

s and

\mathcal{CV}

s are extracted from open-ended Spanish prompts.

Figure 15:

\Delta

P for the opening bracket token _( .

\mathcal{FV}

s and

\mathcal{CV}

s are extracted from multiple-choice prompts.

Panel of Figure 16 — Figure 16: Steering effect across layers and all concepts for different models.

Panel of Figure 17 — Figure 17: 0-shot steering effect across layers and all concepts for different models.

Figure 18: Violin plots of CIE for different concepts and prompts.

Figure 19: Similarity matrices for MC prompts only . Each panel shows the similarity matrix for a different subset of AIE-selected heads computed over MC prompts only (7 concepts $\times$ 50 prompts each). Common heads show stronger concept clustering than unique MC heads (albeit with large similarity between concepts due to the shared MC structure). Full FVs show a very similar representational structure to the common heads. Model: Llama 3.1 70B.

Figure 20: Similarity matrices for different FV head subsets across all formats . Same as Figure 19 but computed over all input formats (7 concepts $\times$ 3 formats $\times$ 50 prompts). Within the same concept the representations are nearly orthogonal between open-ended and multiple-choice prompts. Model: Llama 3.1 70B.

Figure 21: Steering effect across layers (Accuracy) . We inject

\mathcal{CV}

s and

\mathcal{FV}

s into Llama - 3.1 - 70B and plot the Top-1 accuracy for four representative concepts (columns). The grey horizontal line indicates the accuracy of the unsteered model. See Figure 7 for

\Delta P

results.

Figure 22: Similarity matrices for $\mathcal{FV}$ s and $\mathcal{CV}$ s with the inclusion of MC prompts where the model is expected to produce a word instead of a letter. Takeaway : Unlike $\mathcal{CV}$ s, $\mathcal{FV}$ MC representations still cluster due to the input format, therefore MC cluster effect is not due to the model producing words/letters. Model: Llama 3.1 70B.

Figure 23: Steering effect across layers and all concepts for different languages.

Figure 24: KL divergence between ID and OOD for different languages. Model: Llama 3.1 70B.

Source Format	Target Format	Max AIE	Overlap w/ FV (top-5)	Overlap w/ CV (top-5)
OE-ENG	OE-FR	0.22	4	0
OE-ENG	MC	0.02	2	0
OE-FR	MC	0.01	1	0
MC	OE-ENG	0.23	3	0
OE-FR	OE-ENG	0.41	4	0
MC	OE-FR	0.12	4	0

Appendix

Table 4: Cross-format Activation Patching Results (Llama 3.1 70B) . We show the max Average Indirect Effect (AIE) and the number of overlapping heads with the standard Function Vectors (FV) and Concept Vectors (CV) (top-5 heads). The patching source refers to the format from which activations were extracted, and the target refers to the corrupted prompt format into which they were patched. FV heads are consistently identified across formats (specifically L31H18 and L35H57), while CV heads are not.

Representation Geometry in Language Models

Representation geometry in language models studies how embeddings and hidden states organize concepts, computation, and behavior.

See all 417 papers

Fast & Faithful Function Vectors
Jun 3, 2026Minh An Pham, Anton Segeler, Thomas Wiegand +4Task Vectors LLM Interpretability
Fraunhofer Heinrich-Hertz-Institute, Berlin, Germany · Technische Universität Berlin, Berlin, Germany · BIFOLD – Berlin Institute for the Foundations of Learning and Data, Berlin, Germany +1
Functional Subspace, where language models can use vector algebra to solve problems
Feb 2, 2026Jung H. Lee, Sujith VijayanLLM Interpretability In-Context Learning
Pacific Northwest National Laboratory Seattle, WA · School of Neuroscience Virginia Tech Blacksburg, VA
Exploring Language-Agnosticity in Function Vectors: A Case Study in Machine Translation
Apr 21, 2026Nurkhan Laiyk, Gerard I. Gállego, Javier Ferrando +1Multilingual Language Models Task Vectors
Mohamed bin Zayed University of Artificial Intelligence · Universitat Politècnica de Catalunya · Cantina Labs

Jun 3, 2026J/K move · Enter open · S save

Fast & Faithful Function Vectors

Minh An Pham, Anton Segeler, Thomas Wiegand, Wojciech Samek+3

Fraunhofer Heinrich-Hertz-Institute, Berlin, Germany · Technische Universität Berlin, Berlin, Germany · BIFOLD – Berlin Institute for the Foundations of Learning and Data, Berlin, Germany+1

Task Vectors LLM Interpretability In-Context Learning Activation Steering Function Vectors

Open paper AskPDF arXiv

Function vectors (FVs) are task representations elicited during in-context learning that can be used to steer Large Language Models (LLMs). However, design choices in their formulation remain underexplored. In this work, we study the impact of varying FV definitions for instructions along two degrees of freedom: attention head selection and steering. For head selection, using gradient-based attributions with Layer-wise Relevance Propagation (LRP) substantially improves efficiency as well as accuracy. For FV steering, applying it in a distributed manner yields a higher accuracy compared to simple aggregation. Our code is publicly available.

Abstract

Representation Geometry in Language Models

Explore similar work

Fast & Faithful Function Vectors

Functional Subspace, where language models can use vector algebra to solve problems

Exploring Language-Agnosticity in Function Vectors: A Case Study in Machine Translation