cs.CLNov 10, 2026

Do Language Models Feel the Sentiment? Investigating Emotion Vectors

Authors: Ali Hassan Khan, Saad Bin Ather, Muhammad Saif, Maryam Bashir, Hajra Waheed

Organizations: National University of Computer and Emerging Sciences, Lahore, Pakistan

Abstract

Emotion vectors are directions in a language model's internal representations that are associated with emotional concepts. Prior work shows that these vectors can influence model behavior, but their role in sentiment classification remains unclear. This study investigates emotion directions in Qwen2.5-0.5B-Instruct and Qwen2.5-3B-Instruct. Eight directions extracted from GoEmotions are evaluated through sentiment probing, activation interventions, and supervised correction. Emotion-only probes achieve 88.66% and 90.70% accuracy on binary TweetEval. In exploratory matched-ablation experiments, raw emotion-space removal causes 2.39 and 1.10 percentage points more accuracy loss than matched isotropic random-subspace removal for 0.5B and 3B, respectively. All four TweetEval raw and denoised effects are significant after Holm adjustment across eight comparisons. An exploratory SST-2 extension shows smaller raw excess losses of 0.66 and 0.57 points, with no effects surviving Holm adjustment. Frozen sentiment-correction classifiers are also tested on 20,000 Amazon Polarity reviews. Emotion-based correction improves accuracy over uncorrected Qwen, but does not consistently outperform margin-only baselines. These results distinguish sentiment information available in model representations, the effects of changing those representations, and their practical value for classification. They do not establish a uniquely emotional mechanism.

Figures & tables

Explore similar work

CardsList
  1. Where Do Models Find Happiness? Emotion Vectors in Open-Source LLMs

    Jun 25, 2026Sinie van der Ben, Raphaël Baur, Yannick Metz +1LLM InterpretabilityValence-Arousal Modeling

  2. Extracting and Steering Emotion Representations in Small Language Models: A Methodological Comparison

    Apr 5, 2026Jihoon JeongEmotion Representation in Language ModelsSmall Language Models

  3. Shared Emotion Geometry Across Small Language Models: A Cross-Architecture Study of Representation, Behavior, and Methodological Confounds

    Apr 13, 2026Jihoon JeongRepresentation Geometry in Language ModelsEmotion Representation in Language Models