Emotion Is Not External Knowledge: Internal Attention Signals for Affective State Reasoning

Wenqiu Tang; Zhen Wan; Yihui Wang; Takahiro Komamizu; Ichiro Ide

cs.CLOct 15, 2026

Emotion Is Not External Knowledge: Internal Attention Signals for Affective State Reasoning

Authors: Wenqiu Tang, Zhen Wan, Yihui Wang, Takahiro Komamizu, Ichiro Ide

Organizations: Nagoya University, Nagoya, Aichi, Japan · National Institute of Informatics, Tokyo, Japan

Abstract

Large Language Models (LLMs) increasingly operate in multi-turn interactions that require maintaining latent conversational states, not only explicit facts. We study affective state as an implicit state inferred from sparse contextual cues that remain relevant after topic shifts, and ask whether LLMs expose internal attention signals for routing such evidence at inference time. Prior attention interventions rely on labeled in-domain head profiles and externally specified target spans, limiting transfer across affective-evidence distributions. We propose Self-targeted Internal Attention Boosting (SIAB), which localizes sparse affective heads via causal localization (in detail activation patching) on synthetic minimal pairs and boosts, at the answer position, the content tokens these heads already attend to; both edited heads and target tokens are derived from the model's own behavior, enabling the same head set to be reused across tasks.We evaluate on ECC and GoEmotions across three LLMs and multiple prompting settings. Across settings, SIAB improves prompt-matched baselines by up to +10.3 macro-F1 on ECC and +5.5 on GoEmotions. Combined with PASTA-style external supervision, it achieves up to +13.84 accuracy points and +12.51 macro-F1 points, while mechanistic analyses trace the gains to sparse, emotion-enriched heads and show that layer proximity affects additivity.

Figures & tables

Figure 1: Long dialogues can bury early affective cues, turning affective prediction into a routing problem.

Figure 2: Overview of the proposed Self-targeted Internal Attention Boosting ( SIAB ). Causal localization via activation patching selects sparse emotion-sensitive heads (4–20 of hundreds). SIAB adds a pre-softmax bias to the content tokens these heads already attend to at final query positions —without training, parameter update, or external span.

		ECC benchmark				GoEmotions benchmark
Model	Config	Acc.	Macro-F1	$\Delta_{\text{Acc}}$	$\Delta_{\text{F1}}$	Macro-F1	Top-1	$\Delta_{\text{F1}}$	$\Delta_{\text{Top1}}$
Gemma3 ( Gemma Team, 2025 )	ZS	50 . 29	52 . 26	—	—	27 . 17	34 . 37	—	—
	ZS+SIAB	59 . 62	59 . 93	+9 . 32	+7 . 67	30 . 45	38 . 15	+3 . 28	+3 . 78
	Mid	56 . 48	55 . 51	+6 . 19	+3 . 25	30 . 55	33 . 45	+3 . 38	$-$ 0 . 92
	Mid+SIAB	59 . 90	60 . 15	+9 . 61	+7 . 89	32 . 65	36 . 68	+5 . 47	+2 . 30
	High	60 . 66	58 . 27	+10 . 37	+6 . 01	28 . 72	32 . 24	+1 . 55	$-$ 2 . 14

Table 1: Results of two different metrics on the ECC ( He et al., 2025 ) and GoEmotions ( Demszky et al., 2020 ) benchmarks. ZS / Mid / High denote zero- / few-shot prompts (15/27 for GoEmotions, 5/10 for ECC). PASTA- { ECC,GoE } use heads selected on the named dataset, PASTA-

\cap

keeps the shared top heads from these two profiles. Rand. is layer-matched random. Scores are

\times 100

\Delta

is vs. ZS, best per model in bold .

Figure 3: Performance vs. boosted head count $K$ on the ECC ( He et al., 2025 ) and GoEmotions ( Demszky et al., 2020 ) benchmarks. Blue denotes macro-F1, and orange denotes accuracy for ECC and Top-1 accuracy for GoEmotions. Dashed lines mark the $K^{*}$ used in the main experiments, showing small- $K^{*}$ saturation across datasets for each model.

Head set	Gemma3	Qwen3	LLaMA3
Head set	( $K{=}4$ )	( $K{=}10$ )	( $K{=}20$ )
Selected	1.575	1.370	1.384
Random Late	1.373	1.064	1.195
Random Middle	1.346	1.157	1.228
Random Early	1.224	1.083	1.092

Table 2: Emotion-word ratio for selected and random heads. Larger values indicate a higher emotion-word share turn average.

Group	Gemma3	Qwen3	LLaMA3
POS	$+9.77$	$+2.85$	$+0.79$
NEG	$-7.57$	$-2.12$	$-0.83$
HARD	$+2.53$	$+1.14$	$+0.45$

Table 3: Mean

\Delta\text{label\_gap}

per on GoEmotion samples.

Table 4: Zero-Shot (ZS) margin and label-gap statistics by sample group, and rescue rates by label-gap range.In (a), share is calculated over all test samples, and in (b), it is calculated over the POS+HARD subset.

Group	$r_{\text{label}}$	$r_{\text{wrong}}$	$\Delta r$
POS	$0.767\pm 0.284$	$0.140\pm 0.233$	$+0.627$
NEG	$0.661\pm 0.335$	—	—
EASY	$0.726\pm 0.325$	—	—
HARD	$0.656\pm 0.345$	$0.195\pm 0.285$	$+0.461$

Table 5: Selected-head attention mass on emotion tokens. POS shows stronger label-directional bias.

Model	Sink	$\boldsymbol{\beta}\times 2$	Sink
Model	mass	$\boldsymbol{\beta}\times 2$	suppression
Gemma3	$\sim\!25\%$	$4.5\%$	$4.2\%$
Qwen3	$\sim\!40\%$	$0.4\%$	$1.4\%$
LLaMA3	$\sim\!60\%$	$0.1\%$	$5.5\%$

Table 6: Rescue rate on HARD samples under two intervention strategies.

Model	PASTA dist. to SIAB	Additivity (Acc / F1)
Gemma3	Near $(\bar{d}{=}1.1)$	$\mathbf{95.8\%}$	/	$\mathbf{96.6\%}$
Gemma3	Far $(\bar{d}{=}10.5)$	$79.4\%$	/	$81.5\%$
Qwen3	Near $(\bar{d}{=}0.5)$	$\mathbf{118.5\%}$	/	$\mathbf{112.7\%}$
Qwen3	Far $(\bar{d}{=}11.1)$	$70.0\%$	/	$70.4\%$
LLaMA3	Near $(\bar{d}{=}0.3)$	$\mathbf{89.0\%}$	/	$\mathbf{108.0\%}$
LLaMA3	Far $(\bar{d}{=}8.6)$ †	$72.7\%$	/	$54.3\%$

Table 7: Layer-band manipulation on the ECC ( He et al., 2025 ) benchmark. Near / Far denote PASTA ( Zhang et al., 2024a ) heads at small / large mean layer distance

\bar{d}

to the proposed SIAB’s heads, respectively. Boldface values mark the high-additivity configuration per model. † Configuration used in the main results (Table 1 ).

Hyperparameter	Role	Value	How set
$\tau_{\text{head}}$	Direct-effect threshold	0.05	( Wang et al., 2023 )
$\lambda$	Attention-vs.-value weight in $s_{\text{head}}$	0.5	Equal-weighting
$K$	Heads in $\mathcal{H}$	4, 10, 20	$K^{*}$ saturation, Fig. 3
$\|\mathcal{Q}\|$	Boost-applied query positions	4	ECC dev
$r$	Top fraction of attended content tokens	0.3	A priori
$\beta$	Pre-softmax bias magnitude	Tab 9	Per model

Table 8: Hyperparameter inventory. The first six are fixed per model across all (dataset, shot) configurations, while the last one is recalibrated as noted.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Selected (layer, head) indices
Gemma3 ( $K{=}4$ , $\beta{=}6$ )	(24,4), (17,2), (22,6), (17,0)
Qwen3 ( $K{=}10$ , $\beta{=}9$ )	(22,5), (18,30), (23,10), (20,9), (29,3), (26,27), (29,0), (7,3), (25,9), (24,15)
LLaMA3 ( $K{=}20$ , $\beta{=}10$ )	(18,0), (16,10), (19,9), (16,26), (21,26), (6,20), (22,1), (13,10), (15,26), (21,24), (15,14), (19,11), (14,24), (15,20), (20,9), (13,12), (17,28), (24,21), (28,25), (17,20)

Appendix

Table 9: Final head set

\mathcal{H}

per model, with intervention strength

\beta

. Both

\mathcal{H}

and

\beta

are fixed per model and reused across the two datasets. All indices are 0-based. HuggingFace identifiers: google/gemma-3-4b-it , Qwen/Qwen3-4B-Instruct-2507 , meta-llama/Llama-3.1-8B-Instruct .

Figure 4: Per-head attention on an ECC ( He et al., 2025 ) example rescued by the proposed SIAB (GT happy , ZS $\to$ proud ✗, ZS+SIAB $\to$ happy ✓). Each row: Attention from the final-4 query band on a single head, globally normalized, cell value $=$ attention $\times 100$ , restricted to the head’s top-30% content tokens (the boost set $\mathcal{S}_{h,q}$ ). Bold red tokens are NRC EmoLex ( Mohammad and Turney, 2013 ) emotion words. Top: Four Gemma3 ( Gemma Team, 2025 ) selected heads (L24H4, L17H2, L22H6, L17H0). Bottom: Depth-stratified random non-selected heads (L25H1, L20H0, L15H4, L5H3).

Shots	Qwen3	LLaMA3	Gemma3
0	( 5, 0.10)	( 3, 0.10)	( 6, 0.05)
5	( 3, 0.20)	( 3, 0.10)	( 7, 0.05)
10	( 2, 0.20)	( 3, 0.10)	(10, 0.05)
15	( 7, 0.08)	( 3, 0.10)	( 3, 0.03)
27	( 3, 0.15)	( 2, 0.15)	( 7, 0.05)

Appendix

Table 10: Multi-label decoding parameters

(T,t)

per (model, shot) pair for the GoEmotions benchmark:

T

= Temperature,

t

= inclusion threshold. Tuned on GoEmotions dev under baseline and frozen for +SIAB, so reported

\Delta

F1 reflects intervention effect, not

(T,t)

re-calibration.

Comparison	$\Delta$ Top-1 [95% CI]	$\Delta$ F1 [95% CI]
ZS $\rightarrow$ ZS+ SIAB	$+3.78$ [-1pt] $[+2.93,+4.73]$	$+3.28$ [-1pt] $[+2.16,+4.42]$
Mid $\rightarrow$ Mid+ SIAB	$+3.23$ [-1pt] $[+2.31,+4.18]$	$+2.10$ [-1pt] $[+0.94,+3.19]$
High $\rightarrow$ High+ SIAB	$+3.95$ [-1pt] $[+3.03,+4.89]$	$+2.77$ [-1pt] $[+1.65,+3.86]$
PASTA-ECC vs. ZS	$-1.48$ [-1pt] $[-2.53,-0.53]$	$-0.31$ [-1pt] $[-1.30,+0.79]$
PASTA-GoE vs. ZS	$+0.00$ [-1pt] $[-0.93,+1.11]$	$+4.64$ [-1pt] $[+3.46,+5.67]$
PASTA - $\cap$ vs. ZS	$+0.28$ [-1pt] $[-0.72,+1.35]$	$+2.02$ [-1pt] $[+0.96,+3.22]$

Appendix

Table 11: Pointwise paired-bootstrap 95% percentile Confidence Intervals (CI) on the GoEmotions ( Demszky et al., 2020 ) benchmark (

B=10{,}000

, values are

\times 100

). The proposed SIAB comparisons are prompt-matched, whereas PASTA-family rows are compared with Zero-Shot (ZS).

Comparison	$\Delta$ Top-1 [95% CI]	$\Delta$ F1 [95% CI]
ZS $\rightarrow$ ZS+ SIAB	$+9.32$ $[\makebox[32.00002pt][r]{$ +7.84 $},\makebox[32.00002pt][r]{$ +10.61 $}]$	$+7.67$ $[\makebox[32.00002pt][r]{$ +6.33 $},\makebox[32.00002pt][r]{$ +8.98 $}]$
Mid $\rightarrow$ Mid+ SIAB	$+3.42$ $[\makebox[32.00002pt][r]{$ +2.01 $},\makebox[32.00002pt][r]{$ +4.79 $}]$

Appendix

Table 12: Pointwise paired-bootstrap 95% percentile Confidence Intervals (CI) on the ECC ( He et al., 2025 ) benchmark (

B=10{,}000

, values are

\times 100

). The proposed SIAB comparisons are prompt-matched, whereas PASTA-family rows are compared with Zero-Shot (ZS).

Emotion Representation in Language Models

Emotion representation in language models studies how models encode, interpret, and use affective concepts such as valence, arousal, and empathy.

See all 63 papers

Mar 19, 2026

Don't Ask: Understanding What Numeric Self-Reports Measure in Language Models

Language models are often asked to rate their internal states with a number, but it is unclear how reliable this measure is. We train probes for four affective concepts (wellbeing, interest, focus and impulsivity) in eight instruction-tuned models to track these states over multi-turn conversations. We compare each probe's score with a numeric self-report of the same state, elicited at every turn. We find that rating questions displace the probed state, and that the rating is only weakly coupled with the probe score before the question but tracks the post-question probe much more strongly. Steering experiments suggest that models use the affective directions to inform ratings, but that relationship is not specific: ratings are associated with other concepts' probes as much as with their own, and change little when the question is about the user. Finally, neither the pre-question coupling nor concept specificity improves with scale (1B-27B). We warn about the effect a question can have on the internal states one is asking about, and caution against numeric self-report as a reliable measure of those states.

Nicolas Martorell, Bruno Bianchi

CONICET-Universidad de Buenos Aires. Instituto de Ciencias de la Computación (ICC). Buenos Aires, Argentina. · Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Departamento de Computación. Buenos Aires, Argentina.

Language Model Introspection Emotion Representation in Language Models

Ask PDF

Mar 10, 2026

Emotion is Not Just a Label: Latent Emotional Factors in LLM Processing

Large language models are routinely deployed on text that varies widely in emotional tone, yet their reasoning behavior is typically evaluated without accounting for emotion as a source of representational variation. Prior work has largely treated emotion as a prediction target, for example in sentiment analysis or emotion classification. In contrast, we study emotion as a latent factor that shapes how models attend to and reason over text. We analyze how emotional tone systematically alters attention geometry in transformer models, showing that metrics such as locality, center-of-mass distance, and entropy vary across emotions and correlate with downstream question-answering performance. To facilitate controlled study of these effects, we introduce Affect-Uniform ReAding QA (AURA-QA), a question-answering dataset with emotionally balanced, human-authored context passages. Finally, an emotional regularization framework is proposed that constrains emotion-conditioned representational drift during training. Experiments across multiple QA benchmarks demonstrate that this approach improves reading comprehension in both emotionally-varying and non-emotionally varying datasets, yielding consistent gains under distribution shift and in-domain improvements on several benchmarks.

Benjamin Reichman, Adar Avsian, Samuel Webster +1

Georgia Institute of Technology

Emotion Representation in Language Models LLM Interpretability Language Model Robustness Attention Mechanisms

Ask PDF

Sep 1, 2026

Some Emotions Run Deeper: Layer-wise Probing and Causal Intervention in Large Language Models

Emotion is expressed in text along a wide spectrum, from surface lexical cues to inferences entangled with content. Most layer-wise analyses of emotion in LLMs use a single corpus, leaving open whether the depth at which emotion becomes accessible is a property of the model or also of the text source. We investigate this across three datasets spanning different degrees of explicitness and contextualization in emotion expression (Twitter posts, Reddit comments, and autobiographical narratives) and eight 1B--9B open-weight LLMs from the Llama, Qwen, and Granite families. We combine layer-wise probing with offline feature scaling and online forward interventions, transfer analyses, and an early-exit classifier. We find that (i) the best probing layer shifts systematically across corpora, from input-adjacent layers to over half model depth, and this ordering persists after matching label-by-length-bin distributions; (ii) across the evaluated settings, forward-pass interventions on probe-selected bands reduce test accuracy by 5--6 points more than same-width random bands (

q < 0.01

); (iii) selected bands transfer across datasets and emotion categories, suggesting partially shared affective information rather than strictly per-emotion substrates; and (iv) probe-selected early-exit representations outperform full-depth exits by

6.9

percentage points on average.

Tian Fang, Gaël Guibon, Davide Buscaldi

1Université Sorbonne Paris Nord, CNRS, Laboratoire d’Informatique de Paris Nord, LIPN · 2LORIA, CNRS, Université de Lorraine

Causal Interventions in Language Models Emotion Representation in Language Models Language Model Probing

Ask PDF

Abstract

Emotion Representation in Language Models

Explore similar work

Don't Ask: Understanding What Numeric Self-Reports Measure in Language Models

Emotion is Not Just a Label: Latent Emotional Factors in LLM Processing

Some Emotions Run Deeper: Layer-wise Probing and Causal Intervention in Large Language Models