We present EmBolic - a novel fully hyperbolic deep learning architecture for fine-grained emotion analysis from textual messages. The underlying idea is that hyperbolic geometry efficiently captures hierarchies between both words and emotions. In our context, these hierarchical relationships arise from semantic ambiguities. EmBolic aims to infer the curvature on the continuous space of emotions, rather than treating them as a categorical set without any metric structure. In the heart of our architecture is the attention mechanism in the hyperbolic disc. The model is trained to generate queries (points in the hyperbolic disc) from textual messages, while keys (points at the boundary) emerge automatically from the generated queries. Predictions are based on the Busemann energy between queries and keys, evaluating how well a certain textual message aligns with the class directions representing emotions. Our experiments demonstrate strong generalization properties and reasonably good prediction accuracy even for small dimensions of the representation space. Overall, this study supports our claim that affective computing is one of the application domains where hyperbolic representations are particularly advantageous.
Figures & tables
Figure 1: Circumplex model of affect by Russell [11] . Permissions obtained.
Figure 2: Visualization of the Poincaré disc. Two parallel geodesic lines (the shortest paths connecting points) in the left panel and visualization of distances in the right panel. The distance between points O and A is equal to the distance between B and C.
Figure 3: Embolic architecture.
Figure 4: Hyperbolic GloVe word embeddings in the Poincaré disc.
Figure 5: Randomly selected words with their representations in the Poincaré disc. This illustrates emotion-related semantic ambiguity of certain words.
Figure 6: Learned representations of textual messages for Desire, Love, and Nervousness (top row) and the corresponding representations after isometric (Möbius) correction (bottom row).
Figure 7: Visualization of the validation results. Representations of test messages for Desire, Love and Nervousness and the radii encoding these emotions. Correct predictions correspond to the points which are placed on the radii. The confidence levels are encoded by the moduli (distances from the center).
Simulation
Total samples
Top-1
Top-3
Top-5
Simulation 1
140
0.150
0.286
0.414
Simulation 2
140
0.143
0.343
0.457
Simulation 3
140
0.186
0.307
0.429
Table 1: Accuracy results for individual embeddings
Total samples
Top-1
Top-3
Top-5
140
0.264
0.464
0.579
Table 2: Cumulative accuracy results
Category
Count
Percentage (%)
Secure correct (Top-1 True, ≥ 20%)
17
12.14
Secure wrong (Top-1 False, ≥ 20%)
15
10.71
Insecure ( < 20%)
108
77.14
Total
140
100.00
Table 3: Confidence analysis of model predictions with a certainty threshold of 20%
Message
True label
Top prediction
Score (%)
True rank
I only use freedom units, sorry.
Approval
Remorse
75.41
–
I felt the same guilt when I stopped believing in [NAME].
Remorse
Caring
75.60
–
Ahhhh fond memories.
Joy
Sadness
53.94
Top-5 (4.83)
Yes, I’ve noticed it too. Or the photo of the item won’t load. This company is so shameful.
Embarrassment
Confusion
48.63
Top-2 (42.54)
Thanks for the laugh lol.
Gratitude
Amusement
33.13
Top-2 (13.04)
Table 4: High-confidence mispredictions of the emotion classification model
Message
True label
Top prediction
Score (%)
True rank
What a beautiful message—it’s all about love, and I hope it sinks in to some of the COB people who might see it.
Optimism
Admiration
21.05
Top-5 (4.52)
Sorry my teams gave you your only two losses this season ;)
Remorse
Embarrassment
25.85
Top-2 (22.78)
Here for [NAME], love this.
Love
Excitement
36.10
Top-2 (34.47)
Table 5: Examples where the true label appears in the top-5 predictions
Message
True label
Top prediction
Score (%)
True rank
[NAME] will be gone. Although I think end of the first is where he should go, he will go super early.
Optimism
Admiration
4.69
–
This is so scary. Keep safe! I would put fencing up and become a licensed carrier at that point. Sounds like crazy druggies!
Nervousness
Nervousness
5.99
Top-1 (5.99)
Table 6: Examples of uncertain model predictions
Figure 8: Emotion correlation matrix learned by EmBolic. Each cell (i,j) show how frequently emotion i appears together with j in top 5. Darker cells indicate stronger pairwise correlations.
Figure 9: Map of emotions in the Poincaré disc learned from the co-occurrence matrix of emotions depicted in Figure 8 .
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure B.1: Continuation of Figure 4 Hyperbolic GloVe word embeddings in the Poincaré disc for two additional simulations.
Figure B.2: Learned representations of textual messages for all 28 emotion categories. Each panel shows representations of messages annotated by the same emotion. The Möbius correction is further applied to these representations, in order to align fingers with radii/angles.
Figure B.3: Validation on the test set. Representations of test messages for 28 emotions and the radii encoding these emotions. Correct predictions correspond to the points lying on the radii. Representations shown for one embedding only. The final prediction is made based on scores obtained for three embeddings.
Emotion recognition uses language, voice, face, body movement, and physiological signals to infer emotional categories or dimensions such as valence and arousal.