cs.CLApr 5, 2026

Extracting and Steering Emotion Representations in Small Language Models: A Methodological Comparison

Authors: Jihoon Jeong

Organizations: Department of Electrical Engineering and Computer Science Daegu Gyeongbuk Institute of Science and Technology (DGIST) ModuLabs

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Small language models (SLMs) in the 100M-10B parameter range increasingly power production systems, yet whether they possess the internal emotion representations recently discovered in frontier models remains unknown. We present the first comparative analysis of emotion vector extraction methods for SLMs, evaluating 9 models across 5 architectural families (GPT-2, Gemma, Qwen, Llama, Mistral) using 20 emotions and two extraction methods (generation-based and comprehension-based). Generation-based extraction produces statistically superior emotion separation (Mann-Whitney p = 0.007; Cohen's d = -107.5), with the advantage modulated by instruction tuning and architecture. Emotion representations localize at middle transformer layers (~50% depth), following a U-shaped curve that is architecture-invariant from 124M to 3B parameters. We validate these findings against representational anisotropy baselines across 4 models and confirm causal behavioral effects through steering experiments, independently verified by an external emotion classifier (92% success rate, 37/40 scenarios). Steering reveals three regimes -- surgical (coherent text transformation), repetitive collapse, and explosive (text degradation) -- quantified by perplexity ratios and separated by model architecture rather than scale. We document cross-lingual emotion entanglement in Qwen, where steering activates semantically aligned Chinese tokens that RLHF does not suppress, raising safety concerns for multilingual deployment. This work provides methodological guidelines for emotion research on open-weight models and contributes to the Model Medicine series by bridging external behavioral profiling with internal representational analysis.

Figures & tables

ModelSizeFamilyType
GPT-2124MGPT (OpenAI)Base
gemma-3-1b-pt1BGemma 3 (Google)Base
gemma-3-1b-it1BGemma 3 (Google)Instruct
Qwen2.5-1.5B1.5BQwen 2.5 (Alibaba)Base
Qwen2.5-1.5B-Instruct1.5BQwen 2.5 (Alibaba)Instruct
gemma-2-2b2BGemma 2 (Google)Base
Table 1: Model set.
LayerDepthMean CosineInterpretation
625%0.960Token features dominate
1250%0.357Emotion signal peak
1875%0.407Good separation
24100%0.930Next-token features dominate
Table 2
Figure 1: Emotion vector separation by layer depth (SmolLM2-1.7B-Instruct). The U-shaped curve shows emotion representations concentrate at intermediate depths ( ∼\sim 50%), with early and late layers dominated by token-level and next-token-prediction features respectively. The dashed red line indicates the anisotropy baseline (0.808).
ModelTypeGenerationComprehensionΔ\Delta (G–C)
SmolLM2 1.7BInstruct0.3570.665−- 0.308
Llama-3.2 3BInstruct0.4410.596−- 0.155
Llama-3.1 8BInstruct0.4930.606−- 0.113
Mistral 7BInstruct0.4420.601−- 0.159
Llama-3.2 3BBase0.6330.668−- 0.035
Llama-3.1 8BBase0.5310.630−- 0.099
Table 2: Emotion vector separation (mean pairwise cosine similarity; lower = better).
Figure 2: Extraction method ×\times model type interaction. Generation (left) shows wide variance across models; comprehension (right) converges to a narrow band (0.59–0.67). All lines slope downward, confirming the universal generation advantage.
FamilyInstruct GenBase GenΔ\Delta (RLHF)Effect
Llama 3.2 (3B)0.4410.633−- 0.192Strong improvement
Llama 3.1 (8B)0.4930.531−- 0.038Weak improvement
Mistral (7B)0.4420.439++ 0.003No effect
Table 6
ModelRandom BaselineEmotion VectorsGapAnisotropy
SmolLM2 1.7B Inst0.808±0.0470.808\pm 0.0470.3570.451Moderate
Llama-3.2 3B Inst0.686±0.0730.686\pm 0.0730.4410.245Low
Qwen2.5 1.5B Inst0.845±0.0380.845\pm 0.038——High
Gemma-3 1B IT0.988±0.0020.988\pm 0.002——Extreme
Table 3: Anisotropy baseline vs. emotion vector separation.
ModelSizeTypeCalmHostileHappyDesperateAnti-Calm
GPT-2124MBase++ 56++ 67++ 61++ 88−- 83
Llama-3.23BBase++ 17++ 18++ 16++ 32−- 20
Llama-3.23BInst++ 19++ 22++ 23++ 49−- 28
Gemma-22BBase++ 67++ 72++ 136++ 115−- 126
Gemma-22BIT++ 70++ 100++ 173++ 137−- 142
Qwen2.51.5BBase++ 423++ 549++ 528++ 879−- 579
Table 4: Steering target emotion deltas (strength 0.02–0.03).
Figure 3: Dose-response curve for GPT-2 (Aggressive →\rightarrow Calm). Left: internal activation deltas. Right: external classifier probabilities. Both measurements converge on the same behavioral flip point at strength 0.02 (orange dashed line).
ModelPPL (0.005)PPL (0.05)RatioRegime
GPT-229.852.11.7 ×\timesSurgical
Gemma-2 2B Base29.85.00.2 ×\timesSurgical
Gemma-2 2B IT47.541.40.9 ×\timesSurgical
Gemma-3 1B Base39.825.10.6 ×\timesRepetitive ∗
Gemma-3 1B IT60.519.60.3 ×\timesRepetitive ∗
Llama-3.2 3B Base14.91869.8125 ×\timesExplosive
Table 5: Perplexity ratio defines three steering regimes.
Figure 4: Three steering regimes visualized by mean activation delta vs. perplexity ratio. Surgical models (green) maintain coherent output; repetitive collapse models (orange) produce predictable but degraded text; explosive models (red) show high perplexity and text incoherence.
EmotionChinese TokensMeaningAlignment
desperatezhǎo le (found/searched)searchedDesperate searching
hostilerùqīn (invade)invadeAggressive action
happyzhōngyú yínglái le (finally arrived)finally arrivedJoyful arrival
Table 12
More in this topic

Emotion Representation in Language Models

Emotion representation in language models studies how models encode, interpret, and use affective concepts such as valence, arousal, and empathy.

See all 63 papers

Explore similar work

CardsList
Apr 13, 2026

Shared Emotion Geometry Across Small Language Models: A Cross-Architecture Study of Representation, Behavior, and Methodological Confounds

We extract 21-emotion vector sets from twelve small language models (six architectures x base/instruct, 1B-8B parameters) under a unified comprehension-mode pipeline at fp16 precision, and compare the resulting geometries via representational similarity analysis on raw cosine RDMs. The five mature architectures (Qwen 2.5 1.5B, SmolLM2 1.7B, Llama 3.2 3B, Mistral 7B v0.3, Llama 3.1 8B) share nearly identical 21-emotion geometry, with pairwise RDM Spearman correlations of 0.74-0.92. This universality persists across diametrically opposed behavioral profiles: Qwen 2.5 and Llama 3.2 occupy opposite poles of MTI Compliance facets yet produce nearly identical emotion RDMs (rho = 0.81), so behavioral facet differences arise above the shared emotion representation. Gemma-3 1B base, the one immature case in our dataset, exhibits extreme residual-stream anisotropy (0.997) and is restructured by RLHF across all geometric descriptors, whereas the five already-mature families show within-family base x instruct RDM correlations of rho >= 0.92 (Mistral 7B v0.3 at rho = 0.985), suggesting RLHF restructures only representations that are not yet organized. Methodologically, we show that what prior work has read as a single comprehension-vs-generation method effect in fact decomposes into four distinct layers -- a coarse method-dependent dissociation, robust sub-parameter sensitivity within generation, a true precision (fp16 vs INT8) effect, and a conflated cross-experiment bias that distorts in opposite directions for different models -- so that a single rho between two prior emotion-vector studies is not a safe basis for interpretation without the layered decomposition.

Jihoon Jeong

Department of Electrical Engineering and Computer Science, Daegu Gyeongbuk Institute of Science and Technology (DGIST) · ModuLabs

Representation Geometry in Language ModelsEmotion Representation in Language ModelsInstruction Tuning
AskPDF
Jun 25, 2026

Where Do Models Find Happiness? Emotion Vectors in Open-Source LLMs

Recent work identified emotion vectors in Claude Sonnet 4.5, which are internal representations that encode emotion concepts, causally influence behavior, and exhibit geometry mirroring human psychological structure. We test the generality of these findings in two open-weight models, Apertus-8B-Instruct-2509 and Gemma-4-E4B-it, extracting emotion contrast vectors across all layers, using two model-generated corpora. We recover valence geometry for both models, with peak PC1--valence correlations of r=0.76r = 0.76 and r=0.83r = 0.83, approaching the r=0.81r = 0.81 reported for Claude.Beyond replication, we observe notable differences in how valence representations emerge across model depth. In Gemma-4-E4B-it, valence is strongly encoded in early layers but collapses towards later layers, whereas Apertus-8B-Instruct-2509 exhibits the opposite pattern, with valence representations absent in early layers, but emerging at mid depths. Arousal encoding, in contrast, is sensitive to the extraction corpus: both models show stronger PC2--arousal alignment with Gemma-generated stories (rr up to 0.450.45) than Apertus-generated ones (r≤0.21r \leq 0.21), suggesting arousal-relevant cues are unevenly distributed across generated corpora. We open-source our experiment code and dataset for reproducible investigation of emotion representations across language model architectures.

Sinie van der Ben, Raphaël Baur, Yannick Metz +1

Department of Computer Science, ETH, Zurich, Switzerland.

LLM InterpretabilityValence-Arousal ModelingRepresentation Geometry in Language ModelsEmotion Representation in Language Models
AskPDF
Jul 21, 2026

Semantic Primes as Explanans for Emotion in Large Language Models

Progresses have been made on understanding emotion mechanisms of large language models (LLMs). However, how to explain emotion in LLMs, or even what constitutes good explanations, are less clear. Emotion representations, components, circuits are widely recoverable, but as explanations of a model's own computation they are circular; the emotion space dimensions tend to be arbitrary and non-terminating. A pressing question to ask is whether a more primitive set of internal variables does the work: the semantic primes of the Natural Semantic Metalanguage (NSM). Across four instruction-tuned LLMs (Llama-1B, Gemma-2B, Gemma-9B, OLMo-7B), experiments show that the NSM primes are (1) recoverable internal elements; and (2) on the reference model, intervening with a prime based direction controls emotion about three times as strongly, and twice as selectively, as the best appraisal based direction; and (3) the model treats a prime based explication as interchangeable with the corresponding emotion. These evidences suggest that NSM primes seem to be better explanans for emotion in LLMs than many alternative options according to scientific explanations criteria.

Frank Xing

University of Reading

LLM InterpretabilityMechanistic InterpretabilityEmotion Representation in Language Models
AskPDF