Do large language models (LLMs) represent concepts abstractly, i.e., independent of input format? We revisit Function Vectors (FVs), compact representations of in-context learning (ICL) tasks that causally drive task performance. Across multiple LLMs, we show that FVs are not fully invariant: FVs are nearly orthogonal when extracted from different input formats (e.g., open-ended vs. multiple-choice), even if both target the same concept. We identify Concept Vectors (CVs), which carry more stable concept representations. Like FVs, CVs are composed of attention head outputs; however, unlike FVs, the constituent heads are selected using Representational Similarity Analysis (RSA) based on whether they encode concepts consistently across input formats. While these heads emerge in similar layers to FV-related heads, the two sets are largely distinct, suggesting different underlying mechanisms. Steering experiments reveal that FVs excel in-distribution, when extraction and application formats match (e.g., both open-ended in English), while CVs generalize better out-of-distribution across both question types (open-ended vs. multiple-choice) and languages. Our results show that LLMs do contain abstract concept representations, but these differ from those that drive ICL performance.
Figures & tables
Figure 1: Function vs. Concept Vectors . Top : Similarity matrices for FV s (left) and CV s (right) in Llama 3.1 70B; cells show how similar two prompt representations are (warmer = more similar). Middle : Schematic highlighting the distinction between heads with causal effect (Activation Patching-selected) and heads that encode format-invariant structure (RSA-selected). Bottom : Example prompts for two concepts across three formats (EN open - ended, FR open - ended, multiple - choice). Takeaway : FV s cluster by input format; CV s cluster by concept across formats.
Figure 2: Representational Similarity Analysis (RSA) . For each attention head, we compute a representational similarity matrix (RSM) over prompts spanning concepts and input formats (cosine similarity of head outputs). We construct a binary design matrix that marks pairs sharing the same concept, independent of format. The RSA score for a head is Spearman’s ρ between the lower - triangular entries of the RSM and the design matrix; higher ρ indicates stronger concept - invariant encoding.
Figure 3: Similarity matrices . Full similarity matrices extracted from top K=5 heads in CV s and FV s in Llama 3.1 70B for all concepts. See Appendix B for other models.
Figure 4: Concept vs. format RSA . Question type and Concept RSA scores for CV s and FV s in all models. Takeaway : CV s encode more concept information and less input format than FV s.
Model
K=3
K=5
K=10
K=20
K=50
K=100
Llama-3.1 8B
0
0
1
1
12
28
Llama-3.1 70B
0
0
0
0
1
6
Qwen2.5 7B
0
0
0
4
15
39
Qwen2.5 72B
0
0
0
1
3
13
Table 1: RSA–AIE head overlap . Overlap between RSA and AIE heads (number of overlapping heads among top- K ). Bold numbers indicate overlap significantly above chance ( p<0.05 ; details in Appendix E ). Takeaway : FV s and CV s are composed of different attention heads.
Figure 5: Layer - wise AIE vs. RSA . AIE and RSA scores averaged across all heads per layer. Takeaway : FV and CV heads are in similar layers.
Figure 6: Overview of steering results . Top : We extract CV s and FV s from antonym ICL prompts in formats that are in - distribution (ID; OE - ENG ) or out - of - distribution (OOD; OE - FR , MC ) relative to the AmbiguousICL task (bottom - left). Bottom - left : We interleave two concepts— antonym and EN→FR translation —within one prompt; the model’s original prediction is the French translation. Bottom - right : Predictions after steering. Takeaways : (1) FV s yield larger ID gains. (2) CV s show more stable OOD effects across formats. (3) FV s can conflate concept with input format (e.g., French version of antonym and multiple - choice formatting).
Figure 7: Steering effect across layers . We inject CV s and FV s into Llama - 3.1 - 70B and plot the change in target - token probability ( ΔP ) for four representative concepts (columns). Curves compare ID extraction format with OOD formats relative to the AmbiguousICL task (Figure 6 ). Higher ΔP means the model assigns more probability to the expected token than the unsteered model. Takeaways : (1) FV s typically achieve larger ID gains but often drop OOD. (2) CV s yield smaller gains yet show more stable OOD behavior across formats. See Figure 16 for other concepts/models.
Table 2: Token - level steering effects . Top tokens with largest probability gains when injecting CV s or FV s into Llama - 3.1 - 70B on the AmbiguousICL prompt (query shown above). Results shown at the layer with the strongest in - distribution effect per vector. Without intervention, the model predicts French translation _sa (from salé ) with 49%; antonym _sweet has 2%. English antonyms in red , French in blue , and the opening bracket (MC token) in green .
Figure 8: Consistency of steering effects . KL divergence between the probability distributions after the models were steered with an ID (OE-ENG) and OOD vectors (OE-FR [top], MC [bottom]). Lower values indicate more similar effects of ID and OOD vectors. Takeaway : CV s steer the models more consistently than FV s.
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Similarity matrices extracted from top K=1 heads in CV s and FV s in Llama 3.1 8B.
Figure 10: Similarity matrices extracted from top K=1 heads in CV s and FV s in Qwen 2.5 7B.
Figure 11: Similarity matrices extracted from top K=2 heads in CV s and FV s in Qwen 2.5 72B.
Figure 12: Histogram of AIE scores for Llama 3.1 8B, 70B, Qwen 2.5 7B, and Qwen 2.5 72B. Note, the y-axis is on a log-scale. Takeaway : AIE scores are highly sparse.
Model
Best K
Best α
Llama 3.1 8B
1
10
Llama 3.1 70B
5
10
Qwen 2.5 7B
3
10
Qwen 2.5 72B
5
15
Appendix
Table 3: Optimal hyperparameters for steering interventions across different models. K represents the number of attention heads used for FV / CV extraction, while α controls the intervention strength.
Figure 13: Δ P for French translations of all the concepts. FV s and CV s are extracted from open-ended French prompts.
Figure 14: Δ P for French translations of all the concepts. FV s and CV s are extracted from open-ended Spanish prompts.
Figure 15: Δ P for the opening bracket token _( . FV s and CV s are extracted from multiple-choice prompts.
Figure 16: Steering effect across layers and all concepts for different models.
Figure 17: 0-shot steering effect across layers and all concepts for different models.
Figure 18: Violin plots of CIE for different concepts and prompts.
Figure 19: Similarity matrices for MC prompts only . Each panel shows the similarity matrix for a different subset of AIE-selected heads computed over MC prompts only (7 concepts × 50 prompts each). Common heads show stronger concept clustering than unique MC heads (albeit with large similarity between concepts due to the shared MC structure). Full FVs show a very similar representational structure to the common heads. Model: Llama 3.1 70B.
Figure 20: Similarity matrices for different FV head subsets across all formats . Same as Figure 19 but computed over all input formats (7 concepts × 3 formats × 50 prompts). Within the same concept the representations are nearly orthogonal between open-ended and multiple-choice prompts. Model: Llama 3.1 70B.
Figure 21: Steering effect across layers (Accuracy) . We inject CV s and FV s into Llama - 3.1 - 70B and plot the Top-1 accuracy for four representative concepts (columns). The grey horizontal line indicates the accuracy of the unsteered model. See Figure 7 for ΔP results.
Figure 22: Similarity matrices for FV s and CV s with the inclusion of MC prompts where the model is expected to produce a word instead of a letter. Takeaway : Unlike CV s, FV MC representations still cluster due to the input format, therefore MC cluster effect is not due to the model producing words/letters. Model: Llama 3.1 70B.
Figure 23: Steering effect across layers and all concepts for different languages.
Figure 24: KL divergence between ID and OOD for different languages. Model: Llama 3.1 70B.
Source Format
Target Format
Max AIE
Overlap w/ FV (top-5)
Overlap w/ CV (top-5)
OE-ENG
OE-FR
0.22
4
0
OE-ENG
MC
0.02
2
0
OE-FR
MC
0.01
1
0
MC
OE-ENG
0.23
3
0
OE-FR
OE-ENG
0.41
4
0
MC
OE-FR
0.12
4
0
Appendix
Table 4: Cross-format Activation Patching Results (Llama 3.1 70B) . We show the max Average Indirect Effect (AIE) and the number of overlapping heads with the standard Function Vectors (FV) and Concept Vectors (CV) (top-5 heads). The patching source refers to the format from which activations were extracted, and the target refers to the corrupted prompt format into which they were patched. FV heads are consistently identified across formats (specifically L31H18 and L35H57), while CV heads are not.
Function vectors (FVs) are task representations elicited during in-context learning that can be used to steer Large Language Models (LLMs). However, design choices in their formulation remain underexplored. In this work, we study the impact of varying FV definitions for instructions along two degrees of freedom: attention head selection and steering. For head selection, using gradient-based attributions with Layer-wise Relevance Propagation (LRP) substantially improves efficiency as well as accuracy. For FV steering, applying it in a distributed manner yields a higher accuracy compared to simple aggregation. Our code is publicly available.
Minh An Pham, Anton Segeler, Thomas Wiegand +4
Fraunhofer Heinrich-Hertz-Institute, Berlin, Germany · Technische Universität Berlin, Berlin, Germany · BIFOLD – Berlin Institute for the Foundations of Learning and Data, Berlin, Germany +1
Large language models (LLMs) were invented for natural language tasks such as translation, but they have proved that they can perform highly complex functions across domains. Additionally, they have been thought to develop new skills without being trained on them. These learning capabilities lead to LLMs adoption in a wide range of domains. Thus, it is imperative that we understand their operating mechanisms and limitations for proper diagnostics and repair. The earlier studies proposed that high level concepts are encoded as linear directions in LLMs activation space and that the geometry of embeddings have semantic meanings. Inspired by these studies, we hypothesize that LLMs may use subspaces and vector algebra in subspaces to perform tasks. To address this hypothesis, we analyze LLMs' functional modules and residual streams collected from LLMs engaging in in-context learning (ICL), one of the emergent abilities. Our analyses suggest that 1) LLMs can create subspaces, where evidence can be accumulated and 2) ICL tasks can be solved via simple algebraic operations in subspaces.
Jung H. Lee, Sujith Vijayan
Pacific Northwest National Laboratory Seattle, WA · School of Neuroscience Virginia Tech Blacksburg, VA
Function vectors (FVs) are vector representations of tasks extracted from model activations during in-context learning. While prior work has shown that multilingual model representations can be language-agnostic, it remains unclear whether the same holds for function vectors. We study whether FVs exhibit language-agnosticity, using machine translation as a case study. Across three decoder-only multilingual LLMs, we find that translation FVs extracted from a single English→X direction transfer to other target languages, consistently improving the rank of correct translation tokens across multiple unseen languages. We further find that the highest-gain tokens span multiple languages and that translation FVs across directions share most of their top-ranked heads, indicating that the FV encodes a largely language-agnostic translation signal rather than a language-pair-specific mapping.
Nurkhan Laiyk, Gerard I. Gállego, Javier Ferrando +1
Mohamed bin Zayed University of Artificial Intelligence · Universitat Politècnica de Catalunya · Cantina Labs