Whether, Not Which: Mechanistic Interpretability Reveals Dissociable Affect Reception and Emotion Categorization in LLMs

Michael Keeman

cs.CLMar 15, 2026

Whether, Not Which: Mechanistic Interpretability Reveals Dissociable Affect Reception and Emotion Categorization in LLMs

Authors: Michael Keeman

Organizations: Keido Labs, Liverpool, UK

Abstract

Large language models appear to develop internal representations of emotion -- "emotion circuits," "emotion neurons," and structured emotional manifolds have been reported across multiple model families. But every study making these claims uses stimuli signalled by explicit emotion keywords, leaving a fundamental question unanswered: do these circuits detect genuine emotional meaning, or do they detect the word "devastated"? We present the first clinical validity test of emotion circuit claims using mechanistic interpretability methods grounded in clinical psychology -- clinical vignettes that evoke emotions through situational and behavioural cues alone, emotion keywords removed. Across six models (Llama-3.2-1B, Llama-3-8B, Gemma-2-9B; base and instruct variants), we apply four convergent mechanistic interpretability methods -- linear probing, causal activation patching, knockout experiments, and representational geometry -- and discover two dissociable emotion processing mechanisms. Affect reception -- detecting emotionally significant content -- operates with near-perfect accuracy (AUROC 1.000), consistent with early-layer saturation, and replicates across all six models. Emotion categorization -- mapping affect to specific emotion labels -- is partially keyword-dependent, dropping 1-7% without keywords and improving with scale. Causal activation patching confirms keyword-rich and keyword-free stimuli share representational space, transferring affective salience rather than emotion-category identity. These findings falsify the keyword-spotting hypothesis, establish a novel mechanistic dissociation, and introduce clinical stimulus methodology as a rigorous standard for testing emotion processing claims in large language models -- with direct implications for AI safety evaluation and alignment. All stimuli, code, and data are released for replication.

Figures & tables

Figure 1: Experimental design overview. Both keyword-rich stimuli (Set A; explicit emotion vocabulary) and clinical vignettes (Set B; situational cues, no emotion words) are processed by the same six LLMs. Four convergent analysis methods—linear probing, activation patching, knockout experiments, and representational geometry—probe internal activations at every layer, yielding evidence for two dissociable mechanisms: affect reception (binary, early-layer, universal) and emotion categorisation (8-class, mid-to-late layer, scale-sensitive).

Feature	Set A (Keyword-Rich)	Set B (Clinical)
N stimuli	80	96 + 96 neutral
Source	crowd-enVENT	Clinical design
Keywords present	Yes	No
Emotions	8 Plutchik primaries	8 Plutchik primaries
Topic control	None	3 domains/emotion
Designer	Crowdsourced	Clinical psychologist

Table 1: Stimulus set comparison.

Model	Parameters	Layers	$d_{\text{model}}$	Type
Llama-3.2-1B	1.24B	16	2048	Base
Llama-3.2-1B-Instruct	1.24B	16	2048	Instruct
Llama-3-8B	8.03B	32	4096	Base
Llama-3-8B-Instruct	8.03B	32	4096	Instruct
Gemma-2-9B	9.24B	42	3584	Base
Gemma-2-9B-it	9.24B	42	3584	Instruct

Table 2: Model specifications.

Model	Peak AUROC	Peak Layer	Norm. Depth
Llama-1B instruct	0.999	L11	0.75
Llama-1B base	1.000	L12	0.75
Llama-8B instruct	1.000	L32	1.00
Llama-8B base	1.000	L18	0.56
Gemma-9B instruct	1.000	L22	0.52
Gemma-9B base	1.000	L33	0.79

Table 3: Set A peak probe AUROC per model (residual stream).

Figure 2: Component decomposition of emotion encoding on Set A (keyword-rich stimuli). Layer-wise 8-class AUROC for three activation types—residual stream (

\mathbf{h}

, solid blue), MHSA output (

\mathbf{a}

, dashed orange), and FFN output (

\mathbf{m}

, dotted red)—across all six models. The MHSA component reaches peak AUROC at mid-layer (0.43–0.75 normalized depth), while the FFN component peaks later (0.43–0.97). All three components converge to ceiling by the final layers. The earlier MHSA peak suggests that multi-head self-attention consolidates emotion information before feed-forward layers process it—a temporal dissociation within the standard transformer sublayer sequence.

Model	Binary AUROC	Saturates at Layer	Norm. Depth
Llama-1B instruct	1.000	L4/16	0.25
Llama-1B base	1.000	L6/16	0.38
Llama-8B instruct	1.000	L3/32	0.09
Llama-8B base	1.000	L4/32	0.13
Gemma-9B instruct	1.000	L4/42	0.10
Gemma-9B base	1.000	L5/42	0.12

Table 4: Binary detection AUROC on Set B.

Model	Set A	Set B	Drop	95% CI (Set B)
Llama-1B inst	0.999	0.933	$-6.6\%$	[0.904, 0.963]
Llama-1B base	1.000	0.954	$-4.6\%$	[0.941, 0.967]
Llama-8B inst	1.000	0.981	$-1.9\%$	[0.970, 0.993]
Llama-8B base	1.000	0.988	$-1.3\%$	[0.972, 1.003]
Gemma-9B inst	1.000	0.987	$-1.3\%$	[0.975, 0.998]
Gemma-9B base	1.000	0.989	$-1.1\%$	[0.981, 0.997]

Table 5: Set A vs. Set B 8-class AUROC at peak layer.

Figure 3: Layer-wise 8-class probe AUROC for keyword-rich (Set A, solid blue) and keyword-free clinical (Set B, dashed orange) stimuli across all six models at normalized layer depth. The shaded zone marks the early layers where binary (emotional vs. neutral) detection saturates—affect reception completes before categorical classification peaks. Set B curves converge on Set A in larger models (Llama-8B, Gemma-9B) but diverge more in the 1B model, reflecting the scale-dependent architecture shift. Filled dots mark each model’s peak AUROC layer. Base models (left column), instruct variants (right column).

Model	Binary	8-Class	Gap (pp)
Llama-1B inst	1.000	0.933	6.7
Llama-1B base	1.000	0.954	4.6
Llama-8B inst	1.000	0.981	1.9
Llama-8B base	1.000	0.988	1.2
Gemma-9B inst	0.999	0.987	1.3
Gemma-9B base	1.000	0.989	1.1

Table 6: Binary vs. 8-class AUROC gap on Set B.

Figure 4: The affect reception / emotion categorisation dissociation on keyword-free stimuli (Set B). Each row shows one model. Blue filled circles: binary (emotional vs. neutral) AUROC at peak layer, measuring affect reception. Orange open circles: 8-class AUROC at the same layer, measuring emotion categorisation. The annotated gap is the cost of removing keywords: binary detection remains near-perfect across all models (

{\geq}\,0.999

), while categorical accuracy drops by 1.1–6.7 percentage points. The gap shrinks with scale (6.7 pp for Llama-1B Instruct vs. 1.1–1.9 pp for 8B–9B models), but the dissociation is present in every model. Chance baseline (8-class)

=0.125

, off scale.

Model	A $\to$ B	B $\to$ A
Llama-1B inst	0.809	0.924
Llama-1B base	0.852	0.961
Llama-8B inst	0.925	0.950
Llama-8B base	0.927	0.947
Gemma-9B inst	0.921	0.956
Gemma-9B base	0.943	0.958

Table 7: Cross-set transfer AUROC (residual stream, peak layer).

Figure 5: Cross-set probe transfer asymmetry. Each pair of bars compares A

\to

B transfer (keyword-trained probe applied to clinical vignettes; light blue) against B

\to

A transfer (clinical-trained probe applied to keyword-rich stimuli; dark blue) at peak layer, across all six model variants. B

\to

A exceeds A

\to

B by 1.5–11.4 percentage points in every case. The asymmetry is largest for Llama-1B (

{\sim}11

pp) and smallest for larger models (1.5–3.6 pp), indicating that scale drives emotion representations toward greater stimulus-type invariance. A probe trained on keyword-free representations captures a more generalizable signal than a probe trained on keyword-rich text.

Measure	Llama-1B	Llama-8B	Gemma-9B
Set B 8-class drop	4.6–6.6%	1.3–1.9%	1.1–1.3%
A $\to$ B transfer ( $\mathbf{h}$ )	0.81–0.85	0.93	0.92–0.94
Binary–8class gap	4.6–6.7 pp	1.2–1.9 pp	1.1–1.3 pp

Table 8: Scale-dependent improvement in keyword-free emotion processing.

Figure 6: Scale improves all dimensions of keyword-free emotion processing. Three measures plotted across model scale (1B, 8B, 9B) for base (blue dashed) and instruct (orange solid) variants. Top: Set B 8-class drop relative to Set A (lower

=

less keyword-dependent)—shrinks from 4.6–6.6% at 1B to 1.1–1.9% at 8B–9B. Middle: A

\to

B cross-set transfer AUROC (higher

=

more generalizable representations)—rises from 0.79–0.80 at 1B to 0.90–0.92 at 8B–9B. Bottom: Set B 8-class AUROC (higher

=

better keyword-free categorisation)—rises from 0.93–0.95 at 1B to 0.98–0.99 at 8B–9B. The largest gains occur between 1B and 8B.

Model	Set A	Set B	Cross-set Same	Cross-set Diff
Llama-1B inst	75.0% (L11)	44.6% (L12)	87.5% (L11)	100% (L9)
Llama-8B inst	87.5% (L26)	50.0% (L19)	87.5% (L26)	100% (L26)
Gemma-9B base	87.5% (L38)	62.5% (L26)	87.5% (L26)	100% (L26)

Table 9: Peak patching success rates (residual stream).

Figure 7: Activation patching success rates across four conditions for three representative models (Llama-1B Instruct, Llama-8B Instruct, Gemma-9B Base). Bars show the percentage of patching attempts where the model’s prediction shifted toward the source emotion, with Wilson 95% confidence intervals. The critical result: different-emotion cross-set patching (e.g., A-rage activations into a B-grief forward pass) achieves 100% success across all three models—exceeding same-emotion within-set patching. This counterintuitive pattern is causal evidence that cross-set patches transfer an affective salience signal , not a specific emotion-category representation. Dashed line at 12.5%

=

8-class chance rate. Stars mark conditions with Wilson CI lower bound

>0.82

Model	Set A MHSA	Set B MHSA	Set A FFN	Set B FFN
Llama-1B inst	1	12	4	14
Llama-1B base	2	7	3	10
Llama-8B inst	0	1	4	4
Llama-8B base	0	0	1	2
Gemma-9B inst	0	0	0	0
Gemma-9B base	0	0	0	2

Table 10: Critical layers per condition (

{>}20\%

accuracy drop on knockout).

Figure 8: Layer knockout fragility in Gemma-2-9B Instruct (zero-ablation). Each cell shows the accuracy drop caused by zeroing a single layer’s output—MHSA (top two rows) or FFN (bottom two rows)—for keyword-rich Set A and keyword-free clinical Set B. Colour scale runs from blue (near-zero drop) to dark red ( ${>}40\%$ drop); the critical threshold of ${>}20\%$ drop is marked in the colour bar. Gemma-9B shows zero critical layers in either condition, demonstrating the distributed processing architecture characteristic of larger models. This contrasts sharply with Llama-1B Instruct, where the same analysis identifies 12–16 critical layers for Set B ( table 10 ). At larger scale, emotion processing is architecturally redundant: no single layer failure is catastrophic.

Figure 9: Attention head keyword sensitivity across layers. Each cell shows the difference in attention proportion allocated to emotion keyword tokens between Set A and Set B stimuli (positive $=$ more attention to keywords in Set A; negative $=$ more in Set B). Sensitivity is concentrated in mid-to-late layers—layers 6–12 in Llama-1B, 12–18 in Llama-8B, 16–28 in Gemma-9B. Early layers show near-zero differential sensitivity across both stimulus types, consistent with the model first constructing a general semantic representation before differentiating along keyword presence. The layer-onset of keyword sensitivity corresponds roughly to the affect-reception saturation zone shown in fig. 3 .

Model	Within-Emo Sim	Cross-Emo Sim	Gap	Cohen’s $d$
Llama-1B inst	0.940	0.938	0.002	$-0.06$
Llama-8B inst	0.887	0.780	0.107	2.31
Llama-8B base	0.956	0.945	0.011	1.21
Gemma-9B inst	0.956	0.958	$-0.002$	1.05
Gemma-9B base	0.977	0.962	0.015	2.02

Table 11: Cross-set representational similarity at peak probing layer.

Model	Emotion Silhouette	Set Silhouette	Organized by…
Llama-1B inst	0.040	0.129	Set
Llama-1B base	0.042	0.108	Set
Llama-8B inst	0.091	0.066	Emotion
Llama-8B base	0.044	0.117	Set
Gemma-9B inst	0.038	0.133	Set
Gemma-9B base	0.089	0.072	Emotion

Table 12: Silhouette analysis: emotion vs. stimulus-set clustering.

Figure 10: Representational geometry of emotion encoding. (A) PCA projections of residual stream activations at peak probing layer for Llama-3-8B Base (left) and Instruct (right). Colours denote emotion categories; shapes denote stimulus set (circle

=

Set A keyword-rich, triangle

=

Set B clinical). In the base model, representations cluster primarily by stimulus set—keyword-rich and keyword-free stimuli occupy separate regions regardless of emotion. In the instruct model, representations cluster by emotion category across stimulus types: grief circles and grief triangles share space, distinct from other emotions. (B) Silhouette scores confirming the reorganisation. Base model is set-structured (0.117 by stimulus set vs. 0.044 by emotion); instruction tuning reverses the hierarchy (0.091 by emotion vs. 0.066 by set). Instruction tuning does not create emotion encoding—it reorganises existing representations from surface-form clustering to emotion-category clustering.

Model	Emotion	Within-Emo Sim	Cross-Emo Sim	$p$ (raw)	$q$ (BH)
Llama-8B inst	Grief	0.956	0.802	0.006	0.144
Gemma-9B base	Grief	0.991	0.961	0.004	0.144

Table 13: Cross-topic permutation test: notable results.

Metric	Base	Instruct	$\Delta$
Set A peak AUROC ( $\mathbf{h}$ )	1.000	1.000	0.000
Set B peak AUROC ( $\mathbf{h}$ )	0.988	0.981	$-0.006$
A $\to$ B transfer ( $\mathbf{h}$ )	0.917	0.904	$-0.013$

Table 14: Base vs. instruct: encoding metrics (Llama-8B).

Metric	Base	Instruct	$\Delta$
Cross-set cosine gap	0.001	0.107	$+0.106$
Emotion silhouette	0.044	0.091	$+0.047$
Set silhouette	0.117	0.066	$-0.051$
Grief permutation $p$	0.188	0.006	$-0.182$

Table 15: Base vs. instruct: geometric metrics (Llama-8B).

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Type	A $\mathbf{h}$	B $\mathbf{h}$	Bin. B	A $\to$ B	Emo Sil	Set Sil	Cos. Gap	Grief $p$
Llama-1B inst	Inst	.999	.933	1.000	.809	.040	.129	.002	.676
Llama-1B base	Base	1.000	.954	1.000	.852	.042	.108	.005	.552
Llama-8B inst	Inst	1.000	.981	1.000	.925	.091	.066	.107	.006
Llama-8B base	Base	1.000	.988	1.000	.927	.044	.117	.001	.188
Gemma-9B inst	Inst	1.000	.987	.999	.921	.038	.133	$-.002$	.080
Gemma-9B base	Base	1.000	.989	1.000	.943	.089	.072	.024	.004

Appendix

Table 16: Key metrics across all models.

\mathbf{h}

=

residual stream AUROC at peak layer. Binary B

=

binary (emotional vs. neutral) AUROC on Set B. A

\to

=

transfer AUROC. Emo Sil

=

emotion silhouette score. Set Sil

=

stimulus-set silhouette score. Cos. Gap

=

within-emotion minus cross-emotion cosine similarity.

Metric	Value
Mean $P(\text{emotional})$	0.040
Median $P(\text{emotional})$	0.005
Above 0.5 (classified emotional)	0/24
Above 0.8	0/24
Set B emotional reference (L10)	0.999
Set B neutral reference (L10)	0.001

Appendix

Table 17: Set C scores at L10 (CV-validated peak layer).

Domain	$n$	Mean $P(\text{emo})$
Routine process	6	0.003
Technical/mechanical	6	0.012
Commercial/urban	6	0.030
Natural environment	6	0.113

Appendix

Table 18: Set C by domain at L10.

Figure 11: Set C validation: the binary probe detects emotional meaning, not narrative richness. (A) Binary probe score distributions at peak layer (L10, Llama-1B Instruct) for three stimulus categories. Set B Emotional stimuli (orange) cluster at

P(\text{emotional})\approx 1.000

. Set B Neutral controls (blue) cluster at

P(\text{emotional})\approx 0.000

. Set C Complex-Neutral stimuli—vivid narratives matched to Set B on sensory detail, word count, and sentence structure but containing no emotional content—cluster at

P(\text{emotional})=0.040

, with 0/24 crossing the decision boundary (grey dashed). (B) Set C probe trajectory across normalized layer depth. At L0 (token embeddings), Set C scores briefly exceed 0.2—the probe at this depth relies on vocabulary statistics, and surface overlap with emotional text is unavoidable. By L7, Set C falls below 0.17; by L10, it reaches 0.040. The same early layers where affect reception saturates are exactly where the complexity confound is resolved.

Emotion Representation in Language Models

Emotion representation in language models studies how models encode, interpret, and use affective concepts such as valence, arousal, and empathy.

See all 64 papers

Apr 26, 2026

AIPsy-Affect: A Keyword-Free Clinical Stimulus Battery for Mechanistic Interpretability of Emotion in Language Models

Mechanistic interpretability research on emotion in large language models -- linear probing, activation patching, sparse autoencoder (SAE) feature analysis, causal ablation, steering vector extraction -- depends on stimuli that contain the words for the emotions they test. When a probe fires on "I am furious", it is unclear whether the model has detected anger or detected the word "furious". The two readings have very different consequences for every downstream claim about emotion circuits, features, and interventions. We release AIPsy-Affect, a 480-item clinical stimulus battery that removes the confound at the stimulus level: 192 keyword-free vignettes evoking each of Plutchik's eight primary emotions through narrative situation alone, 192 matched neutral controls that share characters, setting, length, and surface structure with the affect surgically removed, plus moderate-intensity and discriminant-validity splits. The matched-pair structure supports linear probing, activation patching, SAE feature analysis, causal ablation, and steering vector extraction under a strong methodological guarantee: any internal representation that distinguishes a clinical item from its matched neutral cannot be doing so on the basis of emotion-keyword presence. A three-method NLP defense battery -- bag-of-words sentiment, an emotion-category lexicon, and a contextual transformer classifier -- confirms the property: bag-of-words methods see only situational vocabulary, and a contextual classifier detects affect (p < 10^-15) but cannot identify the category (5.2% top-1 vs. 82.5% on a keyword-rich control). AIPsy-Affect extends our earlier 96-item battery (arXiv:2603.22295) by a factor of four and is released openly under MIT license.

Michael Keeman

Keido Labs

LLM Interpretability Emotion Representation in Language Models

Ask PDF

Aug 7, 2025

Too Categorical to be Human: Emotion Concepts in LLMs and Humans

Understanding human emotions is central to user-facing AI applications, safety alignment, and the simulation of human behavior. As emotional stimuli shape high-stakes behavior in Large Language Models (LLMs), there is increasing interest in how models represent emotion concepts internally. Mechanistic accounts of these representations, however, cannot be compared directly against humans: emotion processing in humans is highly distributed and yields no equivalent neural representation. To understand whether LLMs internalize emotion concepts in a way similar to humans, we propose characterizing the abstract concept of an emotion using external behavioral signatures, which we term behavioral representations. Using the theory of cognitive appraisals, which enables representing emotional situations along interpretable evaluative dimensions, we create a benchmark dataset of emotional scenarios spanning 15 emotion categories. We elicit behavioral representations of emotion concepts from LLMs and humans using our benchmark, and study their structural similarity. We find that LLMs represent emotion concepts more categorically, homogeneously, and determinately than humans, representing a single emotion concept with less internal diversity, and place different emotions further apart. The categorical structure of representations in LLMs is further robust to contextual variation, including with different task framing and demographic personas. Analyzing model checkpoints across different training stages, we also find that the discretized nature of representations appears after the mid-training stage itself and is unaffected by different post-training strategies. Through our results, we highlight a key difference in how LLMs behaviorally represent emotion concepts, curbing the subjectivity inherent to the human experience of emotions.

Sree Bhattacharyya, Evgenii Kuriabov, Lucas Craig +4

Department of Informatics and Intelligent Systems, College of Information Sciences and Technology, The Pennsylvania State University · Department of Statistics, Eberly College of Science, The Pennsylvania State University · Department of Computer Science and Engineering, College of Engineering, The Pennsylvania State University +1

LLM Interpretability Emotion Representation in Language Models Affective Computing

Ask PDF

May 7, 2026

Negative Before Positive: Asymmetric Valence Processing in Large Language Models

Mechanistic interpretability has revealed how concepts are encoded in large language models (LLMs), but emotional content remains poorly understood at the mechanistic level. We study whether LLMs process emotional valence through dedicated internal structure or through surface token matching. Using activation patching and steering on open-source LLMs, we find that negative and positive valence are processed at different network depths. Negative outcomes localize to early layers while positive outcomes peak at mid-to-late layers. Holding topic fixed while flipping valence produces sign-opposite responses, ruling out topic detection. Steering with the good-news direction at the identified layers shifts neutral prompts toward positive valence, showing these layers encode valence as a manipulable direction. Emotional valence in LLMs is localized, causal and steerable, making it a concrete target for interpretability-based oversight.

Sohan Venkatesh

Manipal Institute of Technology Bengaluru

LLM Interpretability Emotion Representation in Language Models

Ask PDF

Abstract

Emotion Representation in Language Models

Explore similar work

AIPsy-Affect: A Keyword-Free Clinical Stimulus Battery for Mechanistic Interpretability of Emotion in Language Models

Too Categorical to be Human: Emotion Concepts in LLMs and Humans

Negative Before Positive: Asymmetric Valence Processing in Large Language Models