A Hormone-inspired Emotion Layer for Transformer language models (HELT)
Authors: Eslam Reda, Sara El-Metwally
Organizations: Artificial Intelligence Program Faculty of Computers and Information Mansoura University Mansoura 35516 Egypt · Computer Science Department Faculty of Computers and Information Mansoura University Mansoura 35516 Egypt
Large Language Models have demonstrated remarkable capabilities in generating contextually relevant and grammatically correct text. However, they fundamentally lack the ability to process and respond to emotional context in a manner analogous to human emotional cognition. Current approaches to emotion modeling in NLP systems rely primarily on discrete emotion classification or simplistic sentiment analysis, which fail to capture the continuous, multi-dimensional nature of human emotional states. In this paper, we introduce HormoneT5, a novel architecture that augments transformer language models with a biologically-inspired Hormone Emotion Block that simulates the human endocrine system's role in emotional processing. Our approach computes six continuous hormone-like values through specialized per-hormone attention heads, each with orthogonally initialized learnable queries, temperature-scaled attention mechanisms, and deep output projections. These hormone values are then transformed into an emotional embedding that modulates the encoder hidden states, enabling emotionally-appropriate response generation. We propose a multi-objective training framework combining sequence-to-sequence loss, hormone prediction loss with margin penalties, and diversity regularization to prevent attention collapse. Experimental results on our curated emotion-labeled dataset demonstrate that HormoneT5 achieves 85%+ per-hormone accuracy within a 0.15 tolerance threshold, with hormone differentiation ranges exceeding 0.85 across all six hormones between contrasting emotional tones. Human evaluation studies show significant preference (p < 0.01) for HormoneT5-generated responses in terms of emotional appropriateness and empathetic quality compared to baseline T5 outputs. Our work opens new directions for biologically-grounded affective computing and emotionally intelligent conversational agents.
Figures & tables
Figure 1: Hidden state modulation mechanism showing how the multi-dimensional emotional embedding modulates encoder representations through multiplicative gating.
Figure 2: HormoneT5 architecture overview showing the Hormone Emotion Block inserted between the T5 encoder and decoder. The block computes six continuous hormone values and uses them to modulate the encoder’s hidden states.
Tone
Neuroscientific Profile & Rationale
DOP
SER
COR
OXY
ADR
END
Friendly
Positive social reward yields high Dopamine and Endorphins. Serotonin indicates a stable, positive mood. High Oxytocin reflects social bonding, while low Cortisol and Adrenaline represent a calm, stress-free state.
0.95
0.90
0.05
0.90
0.10
0.95
Neutral
A baseline physiological state representing factual exchanges without significant emotional arousal, stress, or deep social bonding.
0.50
0.50
0.30
0.50
0.30
0.50
Rude
Fight-or-flight activation drives high Adrenaline and Cortisol (stress). Disrupted social bonds and unrewarding interactions result in severely depleted levels of Dopamine, Serotonin, Oxytocin, and Endorphins.
0.05
0.05
0.95
0.05
0.95
0.05
Sad
Absence of joy and depressed mood cause low Dopamine, Serotonin, and Endorphins. Oxytocin is remarkably high due to the psychological need for empathy and comfort. Adrenaline remains low (lethargy), with moderate Cortisol.
0.10
0.15
0.60
0.90
0.20
0.10
Excited
High anticipation and reward trigger peak Dopamine and Endorphins. Unlike a calm "friendly" state, Adrenaline is exceptionally high due to energetic arousal, while Cortisol remains low because the arousal is positive, not stress-induced.
0.95
0.85
0.05
0.70
0.90
0.95
Table 1: Tone-to-Hormone Mapping and Neuroscientific Rationale
Environment & Architecture
Optimization & Hyperparameters
GPU / Framework
NVIDIA CUDA / PyTorch 2.0+
Epochs / Batch Size
50 / 8
Transformers / Python
HuggingFace 4.30+ / Python 3.8+
Max Sequence Length
128
Random Seed
42 (Fixed for reproducibility)
Optimizer
AdamW (Weight Decay: 0.02)
Base Model
T5-small
Learning Rate
1×10−4
Hidden Dimension
512
Scheduler
CosineAnnealingWarmRestarts
Encoder Layers
6 (4 unfrozen)
Scheduler Params
T0=10 , Tmult=2
Table 2: Comprehensive Experimental Setup and Model Configurations
Figure 3: Training dynamics over 50 epochs showing loss curves, per-hormone accuracy, and differentiation range progression.
Hormone
MSE
MAE
Accuracy ( ± 0.15)
Diff. Range
Dopamine
0.024
0.098
87.2%
0.88
Serotonin
0.031
0.112
82.5%
0.81
Cortisol
0.019
0.087
91.3%
0.89
Oxytocin
0.038
0.124
78.4%
0.85
Adrenaline
0.026
0.102
85.7%
0.83
Endorphins
0.023
0.095
88.1%
0.86
Table 3: Hormone Prediction Performance
Figure 4: Hormone activations by emotional tone showing clear differentiation between contrasting emotions.
Metric
Baseline T5
HormoneT5
p-value
Effect Size (d)
Emotional Appropriateness
2.73±0.89
4.12±0.76
<0.001
1.68
Empathy Quality
2.45±1.02
3.98±0.82
<0.001
1.65
Fluency
4.21±0.65
4.18±0.71
0.782
0.04
Overall Preference
23%
77%
<0.001
—
Table 4: Human Evaluation Results comparing Baseline T5 and HormoneT5
Figure 5: Qualitative examples showing HormoneT5 responses with corresponding hormone activations for each emotional tone.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Category
Parameter
Value
Model
Base model
T5-small
Hidden dimension
512
Encoder layers
6
Decoder layers
6
Unfrozen encoder layers
4 (last)
Unfrozen decoder layers
4 (last)
Appendix
Table 5: Complete Hyperparameter Configuration
Tone
Input Prompt
Target Output
Friendly
“You’re so helpful, thank you!”
“Aww you’re so welcome! You’re literally the sweetest person!”
“You brighten my day”
“And you brighten mine! We’re like sunshine buddies!”
Rude
“THIS IS STUPID!”
“EXCUSE ME?! Don’t yell at me! YOU’RE being stupid right now!”
“You’re an idiot!”
“I’m an idiot?! Look in the mirror, buddy!”
Sad
“I’m feeling really sad today”
“Oh no… I’m so sorry. Come here, tell me what’s wrong. I’m here for you.”
“I feel like giving up”
“Please don’t give up. I know it’s hard. Let’s talk through this together.”
Appendix
Table 6: Representative Dataset Examples Across All Emotional Tones
Input
Expected Tone
Predicted
Issue
“Whatever, I guess it’s fine”
Neutral/Disappointed
Friendly
Sarcasm and passive-aggression not detected
“lol you’re so dumb haha”
Friendly (teasing)
Rude
Could not distinguish friendly teasing from hostility
“I’m SO happy right now…”
Potentially Sad (sarcastic)
Excited
Sarcasm with contradicting surface sentiment
“Great. Just great.”
Frustrated
Neutral
Flat sarcasm undetected
Appendix
Table 7: Examples of Incorrect Predictions and Underlying Issues
Large language models are routinely deployed on text that varies widely in emotional tone, yet their reasoning behavior is typically evaluated without accounting for emotion as a source of representational variation. Prior work has largely treated emotion as a prediction target, for example in sentiment analysis or emotion classification. In contrast, we study emotion as a latent factor that shapes how models attend to and reason over text. We analyze how emotional tone systematically alters attention geometry in transformer models, showing that metrics such as locality, center-of-mass distance, and entropy vary across emotions and correlate with downstream question-answering performance. To facilitate controlled study of these effects, we introduce Affect-Uniform ReAding QA (AURA-QA), a question-answering dataset with emotionally balanced, human-authored context passages. Finally, an emotional regularization framework is proposed that constrains emotion-conditioned representational drift during training. Experiments across multiple QA benchmarks demonstrate that this approach improves reading comprehension in both emotionally-varying and non-emotionally varying datasets, yielding consistent gains under distribution shift and in-domain improvements on several benchmarks.
Large Language Models (LLMs) often produce emotionally flattened responses after alignment, limiting their effectiveness in affect-sensitive applications. In this paper, we propose EmoVec, a lightweight framework for controllable affective generation via latent vector steering. EmoVec extracts emotion-specific directions from paired neutral and emotion-conditioned responses using contrastive activation addition, and further refines them through task-specific debiasing and principal subspace removal. During inference, these vectors are injected into the final residual stream with static or scenario-adaptive scaling, enabling continuous control over emotional intensity without updating model weights. Experiments across three LLMs and eight emotions show that EmoVec consistently improves emotional salience while largely preserving semantic content, fluency, and coherence. Ablation studies and human evaluation further confirm the effectiveness of vector purification and adaptive scaling, establishing EmoVec as a practical inference-time method for affective control in deployed LLMs.
Xixian Yong, Siyuan Chang, Yingying Zhang +2
Gaoling School of Artificial Intelligence, Renmin University of China · Tencent Jarvis Lab · Beijing Key Laboratory of Research on Large Models and Intelligent Governance +1
Empathetic interaction involves two distinguishable capacities: representing another person's emotional state (cognitive empathy) and generating an appropriate response to that state (affective empathy). We apply this distinction to examine how emotion-related information is internally organized, selectively attended to, and carried through response generation in Llama-3.2-3B-Instruct across 10 emotion categories from the EmpatheticDialogues dataset. Layer-wise linear probing revealed a broad middle-layer region of peak emotion-category decodability, with the layer-11 L2 probe achieving 63.6% held-out accuracy against a 10% chance level and a 54.6% TF-IDF lexical baseline. During response generation, emotion-related user tokens received 1.89 times as much attention per token as other utterance tokens, concentrated across layers 10-15. Target-emotion alignment then decreased from a mean cosine similarity of 0.239 for input representations to 0.125 for generated responses, with positive drift in 77% of examples and the largest shifts for angry and furious. Emotion-category representation, attention allocation, and response-level alignment therefore emerge as distinguishable components of model processing that can be analyzed through a functional cognitive-affective empathy framework. These findings provide a foundation for monitoring emotion representations during generation and leveraging models' existing affective structure to support more naturally empathetic responses without relying on external prompting or explicit emotion conditioning.
Samhita Bollepally, Takashi Yamauchi, Marissa West