A Hormone-inspired Emotion Layer for Transformer language models (HELT)
Authors: Eslam Reda, Sara El-Metwally
Organizations: Artificial Intelligence Program Faculty of Computers and Information Mansoura University Mansoura 35516 Egypt · Computer Science Department Faculty of Computers and Information Mansoura University Mansoura 35516 Egypt
Large Language Models have demonstrated remarkable capabilities in generating contextually relevant and grammatically correct text. However, they fundamentally lack the ability to process and respond to emotional context in a manner analogous to human emotional cognition. Current approaches to emotion modeling in NLP systems rely primarily on discrete emotion classification or simplistic sentiment analysis, which fail to capture the continuous, multi-dimensional nature of human emotional states. In this paper, we introduce HormoneT5, a novel architecture that augments transformer language models with a biologically-inspired Hormone Emotion Block that simulates the human endocrine system's role in emotional processing. Our approach computes six continuous hormone-like values through specialized per-hormone attention heads, each with orthogonally initialized learnable queries, temperature-scaled attention mechanisms, and deep output projections. These hormone values are then transformed into an emotional embedding that modulates the encoder hidden states, enabling emotionally-appropriate response generation. We propose a multi-objective training framework combining sequence-to-sequence loss, hormone prediction loss with margin penalties, and diversity regularization to prevent attention collapse. Experimental results on our curated emotion-labeled dataset demonstrate that HormoneT5 achieves 85%+ per-hormone accuracy within a 0.15 tolerance threshold, with hormone differentiation ranges exceeding 0.85 across all six hormones between contrasting emotional tones. Human evaluation studies show significant preference (p < 0.01) for HormoneT5-generated responses in terms of emotional appropriateness and empathetic quality compared to baseline T5 outputs. Our work opens new directions for biologically-grounded affective computing and emotionally intelligent conversational agents.
Figures & tables
Figure 1: Hidden state modulation mechanism showing how the multi-dimensional emotional embedding modulates encoder representations through multiplicative gating.
Figure 2: HormoneT5 architecture overview showing the Hormone Emotion Block inserted between the T5 encoder and decoder. The block computes six continuous hormone values and uses them to modulate the encoder’s hidden states.
Tone
Neuroscientific Profile & Rationale
DOP
SER
COR
OXY
ADR
END
Friendly
Positive social reward yields high Dopamine and Endorphins. Serotonin indicates a stable, positive mood. High Oxytocin reflects social bonding, while low Cortisol and Adrenaline represent a calm, stress-free state.
0.95
0.90
0.05
0.90
0.10
0.95
Neutral
A baseline physiological state representing factual exchanges without significant emotional arousal, stress, or deep social bonding.
0.50
0.50
0.30
0.50
0.30
0.50
Rude
Fight-or-flight activation drives high Adrenaline and Cortisol (stress). Disrupted social bonds and unrewarding interactions result in severely depleted levels of Dopamine, Serotonin, Oxytocin, and Endorphins.
0.05
0.05
0.95
0.05
0.95
0.05
Sad
Absence of joy and depressed mood cause low Dopamine, Serotonin, and Endorphins. Oxytocin is remarkably high due to the psychological need for empathy and comfort. Adrenaline remains low (lethargy), with moderate Cortisol.
0.10
0.15
0.60
0.90
0.20
0.10
Excited
High anticipation and reward trigger peak Dopamine and Endorphins. Unlike a calm "friendly" state, Adrenaline is exceptionally high due to energetic arousal, while Cortisol remains low because the arousal is positive, not stress-induced.
0.95
0.85
0.05
0.70
0.90
0.95
Table 1: Tone-to-Hormone Mapping and Neuroscientific Rationale
Environment & Architecture
Optimization & Hyperparameters
GPU / Framework
NVIDIA CUDA / PyTorch 2.0+
Epochs / Batch Size
50 / 8
Transformers / Python
HuggingFace 4.30+ / Python 3.8+
Max Sequence Length
128
Random Seed
42 (Fixed for reproducibility)
Optimizer
AdamW (Weight Decay: 0.02)
Base Model
T5-small
Learning Rate
1×10−4
Hidden Dimension
512
Scheduler
CosineAnnealingWarmRestarts
Encoder Layers
6 (4 unfrozen)
Scheduler Params
T0=10 , Tmult=2
Table 2: Comprehensive Experimental Setup and Model Configurations
Figure 3: Training dynamics over 50 epochs showing loss curves, per-hormone accuracy, and differentiation range progression.
Hormone
MSE
MAE
Accuracy ( ± 0.15)
Diff. Range
Dopamine
0.024
0.098
87.2%
0.88
Serotonin
0.031
0.112
82.5%
0.81
Cortisol
0.019
0.087
91.3%
0.89
Oxytocin
0.038
0.124
78.4%
0.85
Adrenaline
0.026
0.102
85.7%
0.83
Endorphins
0.023
0.095
88.1%
0.86
Table 3: Hormone Prediction Performance
Figure 4: Hormone activations by emotional tone showing clear differentiation between contrasting emotions.
Metric
Baseline T5
HormoneT5
p-value
Effect Size (d)
Emotional Appropriateness
2.73±0.89
4.12±0.76
<0.001
1.68
Empathy Quality
2.45±1.02
3.98±0.82
<0.001
1.65
Fluency
4.21±0.65
4.18±0.71
0.782
0.04
Overall Preference
23%
77%
<0.001
—
Table 4: Human Evaluation Results comparing Baseline T5 and HormoneT5
Figure 5: Qualitative examples showing HormoneT5 responses with corresponding hormone activations for each emotional tone.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Category
Parameter
Value
Model
Base model
T5-small
Hidden dimension
512
Encoder layers
6
Decoder layers
6
Unfrozen encoder layers
4 (last)
Unfrozen decoder layers
4 (last)
Appendix
Table 5: Complete Hyperparameter Configuration
Tone
Input Prompt
Target Output
Friendly
“You’re so helpful, thank you!”
“Aww you’re so welcome! You’re literally the sweetest person!”
“You brighten my day”
“And you brighten mine! We’re like sunshine buddies!”
Rude
“THIS IS STUPID!”
“EXCUSE ME?! Don’t yell at me! YOU’RE being stupid right now!”
“You’re an idiot!”
“I’m an idiot?! Look in the mirror, buddy!”
Sad
“I’m feeling really sad today”
“Oh no… I’m so sorry. Come here, tell me what’s wrong. I’m here for you.”
“I feel like giving up”
“Please don’t give up. I know it’s hard. Let’s talk through this together.”
Appendix
Table 6: Representative Dataset Examples Across All Emotional Tones
Input
Expected Tone
Predicted
Issue
“Whatever, I guess it’s fine”
Neutral/Disappointed
Friendly
Sarcasm and passive-aggression not detected
“lol you’re so dumb haha”
Friendly (teasing)
Rude
Could not distinguish friendly teasing from hostility
“I’m SO happy right now…”
Potentially Sad (sarcastic)
Excited
Sarcasm with contradicting surface sentiment
“Great. Just great.”
Frustrated
Neutral
Flat sarcasm undetected
Appendix
Table 7: Examples of Incorrect Predictions and Underlying Issues
Gaoling School of Artificial Intelligence, Renmin University of China · Tencent Jarvis Lab · Beijing Key Laboratory of Research on Large Models and Intelligent Governance +1