Cross-lingual Functional Vectors for Emotion Detection in Large Language Models

Jieying Xue; Phuong Minh Nguyen; Minh Le Nguyen; Shogo Okada

cs.CLAug 30, 2026

Cross-lingual Functional Vectors for Emotion Detection in Large Language Models

Authors: Jieying Xue, Phuong Minh Nguyen, Minh Le Nguyen, Shogo Okada

Organizations: Japan Advanced Institute of Science and Technology

Abstract

Function vectors (FVs) have recently emerged as a promising mechanism for steering the behavior of large language models (LLMs) by injecting task-specific latent direction representations derived from in-context demonstrations. While prior studies have shown that FVs can recover task behavior in structured in-context learning settings, their effectiveness on semantically complex tasks and their ability to generalize across languages remain underexplored. We investigate the cross-lingual transferability of FVs using multilingual multi-label emotion recognition as a challenging semantic classification benchmark. Specifically, we examine whether FVs extracted from a source language can steer task behavior in another language under both standard clean and perturbed zero-shot settings without providing demonstrations during inference. Across diverse cross-lingual settings, applying FVs substantially improves performance, suggesting that FVs capture language-agnostic, task-relevant signals rather than purely language-specific lexical patterns, and highlighting their potential as a lightweight and transferable mechanism for multilingual task adaptation. We observe that each LLM exhibits a relatively stable optimal range of attention heads for constructing effective FVs, and the pattern remains consistent across languages. In addition, FVs can partially replicate the task-steering effects of standard few-shot in-context learning while avoiding the computational overhead of processing multiple demonstrations, making them effective for large-scale practical applications. Our code is available at https://github.com/yingjie7/cross_lingual_fvs.

Figures & tables

Language	Train	Dev	Test	Total
Chinese (ZH)	2,642	200	2,642	5,484
German (DE)	2,603	200	2,604	5,407
English (EN)	2,768	116	2,767	5,651
Spanish (ES)	1,996	184	1,695	3,875
Russian (RU)	2,679	199	1,000	3,878

Table 1: Statistics of the multilingual emotion classification datasets used in our experiments.

FVs Src.	Target Language
FVs Src.	EN	DE	ZH	ES	RU
w/o FV	36.4	16.9	18.4	41.4	52.5
(+FVs)
EN	53.7 ±1.4	44.1 ±0.6	44.2 ±0.6	62.4 ±1.0	72.1 ±2.8
DE	50.5 ±0.8	41.9 ±0.9	43.3 ±0.6	60.3 ±1.4	70.7 ±3.5
ZH	51.5 ±1.1	41.3 ±1.4	43.5 ±0.2	60.5 ±1.2	71.2 ±1.2

Table 2: Macro-F1 scores (%) of Qwen3-8B on the cross-lingual emotion classification task with standard clean zero-shot prompts. The first row reports the baseline without FVs (w/o FV) , while the remaining rows use FVs extracted from random 5-shot demonstrations using the top-

k

heads (

|\mathcal{A}|=20

). Rows indicate the source languages for FV extraction (

g_{1}

), and columns denote the target evaluation languages (

g_{2}

FVs Src.	Target Language
FVs Src.	EN	DE	ZH	ES	RU
w/o FV	0.6	0.5	0.0	8.0	1.8
(+ FVs)
EN	48.6 ±1.7	38.2 ±0.9	41.4 ±0.7	51.8 ±2.6	58.9 ±3.2
DE	45.4 ±1.8	35.6 ±1.2	40.7 ±1.3	49.4 ±1.8	57.8 ±2.9
ZH	46.3 ±1.7	36.2 ±1.0	42.2 ±0.7	51.0 ±1.5	58.8 ±3.2

Table 3: Macro-F1 scores (%) of Qwen3-8B on the cross-lingual emotion classification task with perturbed zero-shot prompts. The FVs were extracted from random 5-shot demonstrations using the top-

k

heads (

|\mathcal{A}|=20

(Example 1)

Sentence: “My heart dropped and I just replied ‘No.’ ”

Gold Label: fear

Zero-shot Output (without FVs): yellow

Zero-shot Output (with FVs): fear

(Example 2)

Sentence: “I slammed my fist against the door and yelled, Open up!”

Table 4: Case studies comparing perturbed zero-shot inference with and without cross-lingual FV injection.

Previous works
FVs Src.	Target Language
	EN	DE	ZH	ES	RU
PAI ∗	82.3	73.9	70.9	84.8	88.2
JNLP ∗	80.4	69.9	68.1	83.0	89.1
This work
w/o FV	66.6	64.1	54.1	71.6	73.9

Table 5: Macro-F1 scores (%) of Qwen3-8B on the cross-lingual emotion classification task with few-shot prompts. The FVs were extracted from random 5-shot demonstrations in the source language using the top-

20

heads. The denotation ∗ indicates the results reported by previous works in SemEval 2025 Task 11: PAI Ruan et al. (2025) and JNLP Xue et al. (2025) .

Demo. Lang.	Target Language
	EN	DE	ZH	ES	RU
EN	66.6	45.2	49.0	68.8	65.9
DE	61.6	64.1	53.7	70.5	74.0
ZH	52.9	45.3	54.1	54.0	67.7
ES	59.2	53.3	50.4	71.6	77.4
RU	51.8	44.9	53.0	55.9	73.9

Table 6: Macro-F1 scores (%) of Qwen3-8B with standard 5-shot prompts without FV injection in monolingual and cross-lingual settings. Rows indicate the source language (demonstration languages), while columns indicate the target evaluation languages.

Figure 1: Pairwise similarity scores of FV construction across different source languages ( $v_{kg}$ for $g\in\{\mathrm{eng},\mathrm{deu},\mathrm{chn},\mathrm{esp},\mathrm{rus}\}$ ). Results are obtained with Qwen3-8B (5-shot, head = 20) and are reported as mean ±std over five random seeds.

FVs Src.	Target Language
	EN	DE	ZH	ES	RU
w/o FV	36.4	16.9	18.4	41.4	52.5
(+ FVs)
EN	52.9 ±1.3	43.7 ±0.6	43.7 ±0.4	61.6 ±1.1	71.2 ±1.5
DE	52.2 ±0.8	43.3 ±0.6	44.6 ±0.7	62.2 ±0.9	74.4 ±1.4
ZH	50.3 ±0.3	42.4 ±1.8	44.3 ±0.6	61.6 ±0.8	73.3 ±1.0

Table 7: Macro-F1 scores (%) of Qwen3-8B on the cross-lingual emotion classification task with standard clean zero-shot prompts. The FVs were extracted from random demonstrations in the range from

5

-shot to

9

-shot in the source language using the top-

20

heads.

Panel of Figure 2 — Figure 2: Effect of the top- $k$ attention heads used to construct FVs on performance under monolingual and cross-lingual settings with perturbed zero-shot prompts using Qwen3-8B .

Figure 3: Performance of FV-enhanced few-shot prompting across different Top-

k

head selections on Qwen3-8B . X-axis: number of selected FV heads (

k

); y-axis: classification performance score. The dashed line indicates the standard few-shot baseline without FV injection. “Fs Lang” (

g_{1}

): language used for FV extraction and few-shot demonstrations; “Test Lang” (

g_{2}

): language used in both demonstrations and query inputs during evaluation.

Figure 4: Confusion matrix across different emotion categories, where each emotion category combines results from all languages.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

You are a multi-label emotion classification system. Each sentence contains zero or more emotions from the following list: anger, fear, joy, sadness, surprise, or none.

### Emotions: anger, fear, joy, sadness, surprise, or none

### Input: For many months I learned to shower keeping my elbows close to prevent water from hosing off the ends and totally drenching the place.

###Answer: fear

… {{demonstrations}} …

Appendix

Table 8: English few-shot prompting template used for multilingual emotion classification. Blue text denotes the task instruction, while the red text indicates the target output generated by the LLM. Five demonstrations are used by default in the few-shot setting.

### Colors: yellow, red, blue or none

### Nation: China, USA, Germany, England or none

### Emotions: anger, fear, joy, sadness, surprise, or none

### Input: Ich kenne keinen Staat, der Menschen, die unter seiner Herrschaft leben so unmenschlich behandelt wie.

###Answer:

Appendix

Table 9: Perturbed zero-shot prompting template for German (

\textit{g}=deu).

You are a multi-label emotion classification system. Each sentence contains zero or more emotions from the following list: anger, fear, joy, sadness, surprise, or none.

### Emotions: anger, fear, joy, sadness, surprise, or none

### Input: Ich kenne keinen Staat, der Menschen, die unter seiner Herrschaft leben so unmenschlich behandelt wie.

###Answer:

Appendix

Table 10: Standard zero-shot prompting template for German (

\textit{g}=deu).

FVs Src.	Target Language
FVs Src.	EN	DE	ZH	ES	RU
w/o FV	14.1	12.9	8.4	29.1	35.5
(+ FVs)
EN	40.9 ±4.2	30.4 ±4.5	38.7 ±2.0	49.1 ±2.3	46.7 ±5.6
DE	42.8 ±6.3	34.4 ±4.3	40.0 ±2.0	49.7 ±4.1	51.4 ±4.9
ZH	44.3 ±4.4	34.5 ±2.4	40.4 ±1.6	49.2 ±2.0	49.2 ±3.6

Appendix

Table 11: Macro-F1 scores (%) of Llama-3.1-8B-Instruct on the cross-lingual emotion classification task with perturbed zero-shot prompts. The FVs were extracted from random 5-shot demonstrations using the top-

k

heads (

|\mathcal{A}|=5

FVs Src.	Target Language
FVs Src.	EN	DE	ZH	ES	RU
w/o FV	36.1	18.9	12.7	31.0	21.9
(+ FVs)
EN	39.6 ±2.7	29.1 ±2.1	35.3 ±1.2	44.4 ±2.7	42.4 ±5.9
DE	41.0 ±1.6	29.5 ±1.7	35.3 ±1.4	45.7 ±1.5	47.7 ±5.6
ZH	41.3 ±1.4	29.4 ±0.5	33.3 ±2.9	44.2 ±0.3	45.4 ±2.8

Appendix

Table 12: Macro-F1 scores (%) of Llama-3.1-8B-Instruct on the cross-lingual emotion classification task with standard clean zero-shot prompts. The FVs were extracted from random 5-shot demonstrations using the top-

k

heads (

|\mathcal{A}|=5

Panel of Figure 5 — Figure 5: Effect of top- $k$ attention heads used for constructing FVs under monolingual and cross-lingual settings in Llama-3.1-8B-Instruct .

Figure 6: Performance of FV-enhanced few-shot prompting across different Top-

k

FV head selections on Qwen3-8B. The x-axis denotes the number of selected FV heads (

k

), while the y-axis represents the classification performance score. The dashed horizontal line indicates the corresponding standard few-shot prompting baseline without FV injection. “Fs Lang” (

g_{1}

) denotes the language used for FV extraction and few-shot demonstrations, and “Test Lang” (

g_{2}

) denotes the language used in both the demonstrations and query inputs during evaluation.

Figure 7: Pairwise similarity scores of FV construction across different source languages ( $v_{kg}$ for $g\in\{\mathrm{eng},\mathrm{deu},\mathrm{chn},\mathrm{esp},\mathrm{rus}\}$ ). Results are obtained with Llama-3.1-8B-Instruct (5-shot, head = 5).

Explore similar work

Apr 21, 2026

Exploring Language-Agnosticity in Function Vectors: A Case Study in Machine Translation

Function vectors (FVs) are vector representations of tasks extracted from model activations during in-context learning. While prior work has shown that multilingual model representations can be language-agnostic, it remains unclear whether the same holds for function vectors. We study whether FVs exhibit language-agnosticity, using machine translation as a case study. Across three decoder-only multilingual LLMs, we find that translation FVs extracted from a single English

\to

X direction transfer to other target languages, consistently improving the rank of correct translation tokens across multiple unseen languages. We further find that the highest-gain tokens span multiple languages and that translation FVs across directions share most of their top-ranked heads, indicating that the FV encodes a largely language-agnostic translation signal rather than a language-pair-specific mapping.

Nurkhan Laiyk, Gerard I. Gállego, Javier Ferrando +1

Mohamed bin Zayed University of Artificial Intelligence · Universitat Politècnica de Catalunya · Cantina Labs

Multilingual Language Models Task Vectors Cross-Lingual Transfer

Ask PDF

Jun 3, 2026

Fast & Faithful Function Vectors

Function vectors (FVs) are task representations elicited during in-context learning that can be used to steer Large Language Models (LLMs). However, design choices in their formulation remain underexplored. In this work, we study the impact of varying FV definitions for instructions along two degrees of freedom: attention head selection and steering. For head selection, using gradient-based attributions with Layer-wise Relevance Propagation (LRP) substantially improves efficiency as well as accuracy. For FV steering, applying it in a distributed manner yields a higher accuracy compared to simple aggregation. Our code is publicly available.

Minh An Pham, Anton Segeler, Thomas Wiegand +4

Fraunhofer Heinrich-Hertz-Institute, Berlin, Germany · Technische Universität Berlin, Berlin, Germany · BIFOLD – Berlin Institute for the Foundations of Learning and Data, Berlin, Germany +1

Task Vectors LLM Interpretability In-Context Learning Activation Steering

Ask PDF

Aug 9, 2026

Multilingual Emotion Neurons in Large Audio-Language Models

Emotion is central to human communication, and its expression varies across languages. Large audio-language models (LALMs) achieve strong performance on multilingual speech tasks, yet it remains unclear whether they encode emotion through language-specific correlations or language-agnostic representations. We present the first neuron-level interpretability study of this question. We define Multilingual Emotion Neurons (MLENs) as functional units exhibiting stable emotional selectivity and aligned causal effects across languages, and introduce Consistency-Regularized Fusion (CR-Fusion) to identify them. Across four modern LALMs and 12 typologically diverse languages, emotion-sensitive neurons identified independently per language show minimal overlap, and additional monolingual identification data saturates quickly without isolating more transferable units, motivating identification from pooled cross-lingual evidence. Causal interventions demonstrate that MLENs identified by CR-Fusion provide more precise and transferable affective control than monolingual neuron sets in both zero-shot and low-resource settings. Leave-one-out ablations further reveal asymmetric transfer: individual identification languages, including low-resource ones, contribute non-redundant evidence, while several low-resource languages benefit most from the resulting cross-lingual transfer. Together, our findings provide the first causal, neuron-level account of how LALMs encode emotion across languages, and establish multilingual neuron identification as an effective mechanism for understanding cross-lingual affective behavior.

Xiutian Zhao, Philipp Koehn, Björn Schuller +1

Center for Language and Speech Processing (CLSP), Johns Hopkins University, USA · Group on Language, Audio & Music (GLAM), Imperial College London, UK

Cross-Lingual Representation Alignment Audio-Language Models Mechanistic Interpretability Emotion Representation in Language Models

Ask PDF