Emotion Is Not External Knowledge: Internal Attention Signals for Affective State Reasoning

Wenqiu Tang; Zhen Wan; Yihui Wang; Takahiro Komamizu; Ichiro Ide

cs.CLOct 15, 2026

Emotion Is Not External Knowledge: Internal Attention Signals for Affective State Reasoning

Authors: Wenqiu Tang, Zhen Wan, Yihui Wang, Takahiro Komamizu, Ichiro Ide

Organizations: Nagoya University, Nagoya, Aichi, Japan · National Institute of Informatics, Tokyo, Japan

Abstract

Large Language Models (LLMs) increasingly operate in multi-turn interactions that require maintaining latent conversational states, not only explicit facts. We study affective state as an implicit state inferred from sparse contextual cues that remain relevant after topic shifts, and ask whether LLMs expose internal attention signals for routing such evidence at inference time. Prior attention interventions rely on labeled in-domain head profiles and externally specified target spans, limiting transfer across affective-evidence distributions. We propose Self-targeted Internal Attention Boosting (SIAB), which localizes sparse affective heads via causal localization (in detail activation patching) on synthetic minimal pairs and boosts, at the answer position, the content tokens these heads already attend to; both edited heads and target tokens are derived from the model's own behavior, enabling the same head set to be reused across tasks.We evaluate on ECC and GoEmotions across three LLMs and multiple prompting settings. Across settings, SIAB improves prompt-matched baselines by up to +10.3 macro-F1 on ECC and +5.5 on GoEmotions. Combined with PASTA-style external supervision, it achieves up to +13.84 accuracy points and +12.51 macro-F1 points, while mechanistic analyses trace the gains to sparse, emotion-enriched heads and show that layer proximity affects additivity.

Figures & tables

Figure 1: Long dialogues can bury early affective cues, turning affective prediction into a routing problem.

Figure 2: Overview of the proposed Self-targeted Internal Attention Boosting ( SIAB ). Causal localization via activation patching selects sparse emotion-sensitive heads (4–20 of hundreds). SIAB adds a pre-softmax bias to the content tokens these heads already attend to at final query positions —without training, parameter update, or external span.

		ECC benchmark				GoEmotions benchmark
Model	Config	Acc.	Macro-F1	$\Delta_{\text{Acc}}$	$\Delta_{\text{F1}}$	Macro-F1	Top-1	$\Delta_{\text{F1}}$	$\Delta_{\text{Top1}}$
Gemma3 ( Gemma Team, 2025 )	ZS	50 . 29	52 . 26	—	—	27 . 17	34 . 37	—	—
	ZS+SIAB	59 . 62	59 . 93	+9 . 32	+7 . 67	30 . 45	38 . 15	+3 . 28	+3 . 78
	Mid	56 . 48	55 . 51	+6 . 19	+3 . 25	30 . 55	33 . 45	+3 . 38	$-$ 0 . 92
	Mid+SIAB	59 . 90	60 . 15	+9 . 61	+7 . 89	32 . 65	36 . 68	+5 . 47	+2 . 30
	High	60 . 66	58 . 27	+10 . 37	+6 . 01	28 . 72	32 . 24	+1 . 55	$-$ 2 . 14

Table 1: Results of two different metrics on the ECC ( He et al., 2025 ) and GoEmotions ( Demszky et al., 2020 ) benchmarks. ZS / Mid / High denote zero- / few-shot prompts (15/27 for GoEmotions, 5/10 for ECC). PASTA- { ECC,GoE } use heads selected on the named dataset, PASTA-

\cap

keeps the shared top heads from these two profiles. Rand. is layer-matched random. Scores are

\times 100

\Delta

is vs. ZS, best per model in bold .

Figure 3: Performance vs. boosted head count $K$ on the ECC ( He et al., 2025 ) and GoEmotions ( Demszky et al., 2020 ) benchmarks. Blue denotes macro-F1, and orange denotes accuracy for ECC and Top-1 accuracy for GoEmotions. Dashed lines mark the $K^{*}$ used in the main experiments, showing small- $K^{*}$ saturation across datasets for each model.

Head set	Gemma3	Qwen3	LLaMA3
Head set	( $K{=}4$ )	( $K{=}10$ )	( $K{=}20$ )
Selected	1.575	1.370	1.384
Random Late	1.373	1.064	1.195
Random Middle	1.346	1.157	1.228
Random Early	1.224	1.083	1.092

Table 2: Emotion-word ratio for selected and random heads. Larger values indicate a higher emotion-word share turn average.

Group	Gemma3	Qwen3	LLaMA3
POS	$+9.77$	$+2.85$	$+0.79$
NEG	$-7.57$	$-2.12$	$-0.83$
HARD	$+2.53$	$+1.14$	$+0.45$

Table 3: Mean

\Delta\text{label\_gap}

per on GoEmotion samples.

Table 4: Zero-Shot (ZS) margin and label-gap statistics by sample group, and rescue rates by label-gap range.In (a), share is calculated over all test samples, and in (b), it is calculated over the POS+HARD subset.

Group	$r_{\text{label}}$	$r_{\text{wrong}}$	$\Delta r$
POS	$0.767\pm 0.284$	$0.140\pm 0.233$	$+0.627$
NEG	$0.661\pm 0.335$	—	—
EASY	$0.726\pm 0.325$	—	—
HARD	$0.656\pm 0.345$	$0.195\pm 0.285$	$+0.461$

Table 5: Selected-head attention mass on emotion tokens. POS shows stronger label-directional bias.

Model	Sink	$\boldsymbol{\beta}\times 2$	Sink
Model	mass	$\boldsymbol{\beta}\times 2$	suppression
Gemma3	$\sim\!25\%$	$4.5\%$	$4.2\%$
Qwen3	$\sim\!40\%$	$0.4\%$	$1.4\%$
LLaMA3	$\sim\!60\%$	$0.1\%$	$5.5\%$

Table 6: Rescue rate on HARD samples under two intervention strategies.

Model	PASTA dist. to SIAB	Additivity (Acc / F1)
Gemma3	Near $(\bar{d}{=}1.1)$	$\mathbf{95.8\%}$	/	$\mathbf{96.6\%}$
Gemma3	Far $(\bar{d}{=}10.5)$	$79.4\%$	/	$81.5\%$
Qwen3	Near $(\bar{d}{=}0.5)$	$\mathbf{118.5\%}$	/	$\mathbf{112.7\%}$
Qwen3	Far $(\bar{d}{=}11.1)$	$70.0\%$	/	$70.4\%$
LLaMA3	Near $(\bar{d}{=}0.3)$	$\mathbf{89.0\%}$	/	$\mathbf{108.0\%}$
LLaMA3	Far $(\bar{d}{=}8.6)$ †	$72.7\%$	/	$54.3\%$

Table 7: Layer-band manipulation on the ECC ( He et al., 2025 ) benchmark. Near / Far denote PASTA ( Zhang et al., 2024a ) heads at small / large mean layer distance

\bar{d}

to the proposed SIAB’s heads, respectively. Boldface values mark the high-additivity configuration per model. † Configuration used in the main results (Table 1 ).

Hyperparameter	Role	Value	How set
$\tau_{\text{head}}$	Direct-effect threshold	0.05	( Wang et al., 2023 )
$\lambda$	Attention-vs.-value weight in $s_{\text{head}}$	0.5	Equal-weighting
$K$	Heads in $\mathcal{H}$	4, 10, 20	$K^{*}$ saturation, Fig. 3
$\|\mathcal{Q}\|$	Boost-applied query positions	4	ECC dev
$r$	Top fraction of attended content tokens	0.3	A priori
$\beta$	Pre-softmax bias magnitude	Tab 9	Per model

Table 8: Hyperparameter inventory. The first six are fixed per model across all (dataset, shot) configurations, while the last one is recalibrated as noted.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Selected (layer, head) indices
Gemma3 ( $K{=}4$ , $\beta{=}6$ )	(24,4), (17,2), (22,6), (17,0)
Qwen3 ( $K{=}10$ , $\beta{=}9$ )	(22,5), (18,30), (23,10), (20,9), (29,3), (26,27), (29,0), (7,3), (25,9), (24,15)
LLaMA3 ( $K{=}20$ , $\beta{=}10$ )	(18,0), (16,10), (19,9), (16,26), (21,26), (6,20), (22,1), (13,10), (15,26), (21,24), (15,14), (19,11), (14,24), (15,20), (20,9), (13,12), (17,28), (24,21), (28,25), (17,20)

Appendix

Table 9: Final head set

\mathcal{H}

per model, with intervention strength

\beta

. Both

\mathcal{H}

and

\beta

are fixed per model and reused across the two datasets. All indices are 0-based. HuggingFace identifiers: google/gemma-3-4b-it , Qwen/Qwen3-4B-Instruct-2507 , meta-llama/Llama-3.1-8B-Instruct .

Figure 4: Per-head attention on an ECC ( He et al., 2025 ) example rescued by the proposed SIAB (GT happy , ZS $\to$ proud ✗, ZS+SIAB $\to$ happy ✓). Each row: Attention from the final-4 query band on a single head, globally normalized, cell value $=$ attention $\times 100$ , restricted to the head’s top-30% content tokens (the boost set $\mathcal{S}_{h,q}$ ). Bold red tokens are NRC EmoLex ( Mohammad and Turney, 2013 ) emotion words. Top: Four Gemma3 ( Gemma Team, 2025 ) selected heads (L24H4, L17H2, L22H6, L17H0). Bottom: Depth-stratified random non-selected heads (L25H1, L20H0, L15H4, L5H3).

Shots	Qwen3	LLaMA3	Gemma3
0	( 5, 0.10)	( 3, 0.10)	( 6, 0.05)
5	( 3, 0.20)	( 3, 0.10)	( 7, 0.05)
10	( 2, 0.20)	( 3, 0.10)	(10, 0.05)
15	( 7, 0.08)	( 3, 0.10)	( 3, 0.03)
27	( 3, 0.15)	( 2, 0.15)	( 7, 0.05)

Appendix

Table 10: Multi-label decoding parameters

(T,t)

per (model, shot) pair for the GoEmotions benchmark:

T

= Temperature,

t

= inclusion threshold. Tuned on GoEmotions dev under baseline and frozen for +SIAB, so reported

\Delta

F1 reflects intervention effect, not

(T,t)

re-calibration.

Comparison	$\Delta$ Top-1 [95% CI]	$\Delta$ F1 [95% CI]
ZS $\rightarrow$ ZS+ SIAB	$+3.78$ [-1pt] $[+2.93,+4.73]$	$+3.28$ [-1pt] $[+2.16,+4.42]$
Mid $\rightarrow$ Mid+ SIAB	$+3.23$ [-1pt] $[+2.31,+4.18]$	$+2.10$ [-1pt] $[+0.94,+3.19]$
High $\rightarrow$ High+ SIAB	$+3.95$ [-1pt] $[+3.03,+4.89]$	$+2.77$ [-1pt] $[+1.65,+3.86]$
PASTA-ECC vs. ZS	$-1.48$ [-1pt] $[-2.53,-0.53]$	$-0.31$ [-1pt] $[-1.30,+0.79]$
PASTA-GoE vs. ZS	$+0.00$ [-1pt] $[-0.93,+1.11]$	$+4.64$ [-1pt] $[+3.46,+5.67]$
PASTA - $\cap$ vs. ZS	$+0.28$ [-1pt] $[-0.72,+1.35]$	$+2.02$ [-1pt] $[+0.96,+3.22]$

Appendix

Table 11: Pointwise paired-bootstrap 95% percentile Confidence Intervals (CI) on the GoEmotions ( Demszky et al., 2020 ) benchmark (

B=10{,}000

, values are

\times 100

). The proposed SIAB comparisons are prompt-matched, whereas PASTA-family rows are compared with Zero-Shot (ZS).

Comparison	$\Delta$ Top-1 [95% CI]	$\Delta$ F1 [95% CI]
ZS $\rightarrow$ ZS+ SIAB	$+9.32$ $[\makebox[32.00002pt][r]{$ +7.84 $},\makebox[32.00002pt][r]{$ +10.61 $}]$	$+7.67$ $[\makebox[32.00002pt][r]{$ +6.33 $},\makebox[32.00002pt][r]{$ +8.98 $}]$
Mid $\rightarrow$ Mid+ SIAB	$+3.42$ $[\makebox[32.00002pt][r]{$ +2.01 $},\makebox[32.00002pt][r]{$ +4.79 $}]$

Appendix

Table 12: Pointwise paired-bootstrap 95% percentile Confidence Intervals (CI) on the ECC ( He et al., 2025 ) benchmark (

B=10{,}000

, values are

\times 100

). The proposed SIAB comparisons are prompt-matched, whereas PASTA-family rows are compared with Zero-Shot (ZS).

Emotion Representation in Language Models

Emotion representation in language models studies how models encode, interpret, and use affective concepts such as valence, arousal, and empathy.

See all 63 papers

Don't Ask: Understanding What Numeric Self-Reports Measure in Language Models
Mar 19, 2026Nicolas Martorell, Bruno BianchiLanguage Model Introspection Emotion Representation in Language Models
CONICET-Universidad de Buenos Aires. Instituto de Ciencias de la Computación (ICC). Buenos Aires, Argentina. · Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Departamento de Computación. Buenos Aires, Argentina.
Emotion is Not Just a Label: Latent Emotional Factors in LLM Processing
Mar 10, 2026Benjamin Reichman, Adar Avsian, Samuel Webster +1Emotion Representation in Language Models LLM Interpretability
Georgia Institute of Technology
Some Emotions Run Deeper: Layer-wise Probing and Causal Intervention in Large Language Models
Sep 1, 2026Tian Fang, Gaël Guibon, Davide BuscaldiCausal Interventions in Language Models Emotion Representation in Language Models
1Université Sorbonne Paris Nord, CNRS, Laboratoire d’Informatique de Paris Nord, LIPN · 2LORIA, CNRS, Université de Lorraine

Mar 19, 2026J/K move · Enter open · S save

Don't Ask: Understanding What Numeric Self-Reports Measure in Language Models

Nicolas Martorell, Bruno Bianchi

CONICET-Universidad de Buenos Aires. Instituto de Ciencias de la Computación (ICC). Buenos Aires, Argentina. · Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Departamento de Computación. Buenos Aires, Argentina.

Language Model Introspection Emotion Representation in Language Models

Open paper AskPDF arXiv

Language models are often asked to rate their internal states with a number, but it is unclear how reliable this measure is. We train probes for four affective concepts (wellbeing, interest, focus and impulsivity) in eight instruction-tuned models to track these states over multi-turn conversations. We compare each probe's score with a numeric self-report of the same state, elicited at every turn. We find that rating questions displace the probed state, and that the rating is only weakly coupled with the probe score before the question but tracks the post-question probe much more strongly. Steering experiments suggest that models use the affective directions to inform ratings, but that relationship is not specific: ratings are associated with other concepts' probes as much as with their own, and change little when the question is about the user. Finally, neither the pre-question coupling nor concept specificity improves with scale (1B-27B). We warn about the effect a question can have on the internal states one is asking about, and caution against numeric self-report as a reliable measure of those states.

Abstract

Emotion Representation in Language Models

Explore similar work

Don't Ask: Understanding What Numeric Self-Reports Measure in Language Models

Emotion is Not Just a Label: Latent Emotional Factors in LLM Processing

Some Emotions Run Deeper: Layer-wise Probing and Causal Intervention in Large Language Models