Beyond a Single Sentiment Score: Topical and Affective Dimensions of Emoji Meaning on Twitter

Rohan James; Ioannis Christofilogiannis; Yuuv Jauhari; Shreya Shah; Gauri Pethkar

cs.CLOct 20, 2026

Beyond a Single Sentiment Score: Topical and Affective Dimensions of Emoji Meaning on Twitter

Authors: Rohan James, Ioannis Christofilogiannis, Yuuv Jauhari, Shreya Shah, Gauri Pethkar

Organizations: School of Computer Science and Statistics Trinity College Dublin, Ireland

Abstract

Emoji are widely treated as compact affective tokens whose meaning is summarised by a single sentiment score. We argue that this picture is incomplete. From a corpus of 16{,}575 English tweets and a per-emoji metric we call stability delta (within-class minus between-class cosine similarity of sentence embeddings), we show that emojis used in tightly clustered topical contexts also exhibit more polarised crowd-sourced sentiment, not less. Validated against the Kralj Novak et al.\ (2015) sentiment lexicon, the central correlation is Spearman $ρ= +0.361$ (Holm $p = 0.015$ , 95% bootstrap CI $[+0.10, +0.59]$ ); against the independent Ferré et al.\ (2025) Spanish 13-emotion ratings, $ρ= +0.509$ ( $p = 0.0003$ ). The result survives permutation testing, BCa bootstrap, leave-one-out jackknife, partial correlation controlling for emoji frequency, and Holm correction across a four-test family. Topical coherence and affective dispersion are distinct dimensions of emoji meaning. As a downstream check we report leakage-free five-fold cross-validation results for sarcasm classification on the Emoji Sarcasm Dataset; all three baselines (a gated multimodal architecture, fine-tuned BERTweet features, and fine-tuned RoBERTa features) score above 0.97 macro F1, indicating the dataset has saturating emoji-label cues. We frame the classifier as supporting evidence rather than the contribution.

Figures & tables

Figure 1: VADER compound score distributions by emoji position and role (substitutive vs. additive). Substitutive emojis produce more polarised distributions across all three positions.

Figure 2: Mean VADER compound score for the top-20 emojis across positional contexts (start, middle, end). Right column shows raw frequency. Individual emojis exhibit marked affective shifts depending on position; where an emoji sits predicts its sentiment as strongly as which emoji it is.

Emoji (description)	Within	Between	$\Delta$
Collision	0.409	0.172	0.237
Fire	0.332	0.140	0.192
Pointing-down	0.305	0.152	0.153
Astonished face	0.121	0.101	0.021
Loudly crying face	0.143	0.124	0.019
Confused face	0.124	0.109	0.014

Table 1: Top and bottom emojis ranked by stability delta. “Within” and “Between” are mean pairwise cosine similarities within and across emoji classes.

Test	$\rho$	Pct. CI (95%)	BCa CI (95%)	Perm. $p$	$\tau_{b}$	Partial $\rho$ (freq)	Holm $p$
$\Delta$ vs. Kralj Novak sentiment	$+0.408$	$[+0.16,+0.60]$	$[+0.16,+0.60]$	0.0008	$+0.258$	$+0.423$	0.005
$\Delta$ vs. Kralj Novak variance (primary)	$\mathbf{+0.361}$	$[+0.10,+0.59]$	$[+0.09,+0.58]$	0.005	$+0.261$	$+0.370$	0.015
Sarcasm rate vs. $p(\text{neg})$	$-0.286$	$[-0.57,+0.02]$	$[-0.55,+0.02]$	0.028	$-0.227$	$-0.280$	0.051
Sarcasm rate vs. Kralj Novak sentiment	$+0.291$	$[-0.01,+0.57]$	$[-0.01,+0.55]$	0.029	$+0.234$	$+0.289$	0.051

Table 2: Kralj Novak (2015) validation,

n=59

matched emojis. Holm correction applied across the family of four tests. Significance markers refer to Holm-corrected

p

-values.

Figure 3: Contextual stability is positively associated with affective dispersion. Left: stability

\Delta

vs. Kralj Novak sentiment variance (

n=59

, Spearman

\rho=+0.361

, BCa

{}_{95}=[+0.089,+0.581]

, Holm

p=.015

). Right: stability

\Delta

vs. number of strongly evoked emotions in Ferré et al. (

n=47

\rho=+0.509

p=.0003

). Emojis used in more topically coherent contexts evoke broader, not narrower, affective responses.

Figure 4: Stability delta versus Kralj Novak sentiment score (left) and sentiment variance (right). Red lines are OLS fits. Both relationships are positive and Holm-significant.

Likert threshold	$\rho$	$p$	Sig.
$\geq 2.0$	$+0.606$	$6.0\times 10^{-6}$
$\geq 2.5$ (primary)	$\mathbf{+0.509}$	$\mathbf{2.6\times 10^{-4}}$
$\geq 3.0$	$+0.458$	$1.2\times 10^{-3}$	**
$\geq 3.5$	$+0.415$	$3.7\times 10^{-3}$	**

Table 3: Stability delta vs. number of strongly evoked emotions in Ferré 2025, with threshold sensitivity.

n=47

matched emojis throughout.

Model	Macro F1	Sarc. F1
GEMA	$.985{\pm}.007$	$.982{\pm}.008$
BERTweet	$.974{\pm}.002$	$.967{\pm}.002$
RoBERTa	$.991{\pm}.001$	$.988{\pm}.001$

Table 4: Five-fold stratified CV on the English Twitter partition (

n=16{,}587

, mean

\pm

SD). Full 5-metric evaluation reported in Appendix F .

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Acc.	Macro F1	Sarc. F1	AUC	MCC
GEMA	$.986{\pm}.006$	$.985{\pm}.007$	$.982{\pm}.008$	$.999{\pm}.000$	$.970{\pm}.013$
BERTweet	$.975{\pm}.002$	$.974{\pm}.002$	$.967{\pm}.002$	$.997{\pm}.001$	$.948{\pm}.003$
RoBERTa	$.991{\pm}.001$	$.991{\pm}.001$	$.988{\pm}.001$	$.999{\pm}.000$	$.982{\pm}.002$

Appendix

Table 5: Full five-fold stratified cross-validation results on the English-filtered Twitter partition (

n=16{,}587

). Values show mean

\pm

standard deviation across folds.

Source	Test	$n$	$\rho$	$p$
Czestochowska	$\Delta$ vs. description entropy	59	$-0.087$	0.51
Czestochowska	$\Delta$ vs. unique words	59	$-0.029$	0.83
EmojiNet	$\Delta$ vs. sense count	62	$-0.119$	0.36
EmojiNet	$\Delta$ vs. keyword count	62	$+0.075$	0.56

Appendix

Table 6: Sanity checks against decontextualised ambiguity Czestochowska et al. (2022) and dictionary polysemy Wijeratne et al. (2017) .

Position	Count	VADER	TextBlob
Start	$2,188$	$+0.267$	$+0.190$
Middle	$4,728$	$+0.0667$	$+0.116$
End	$9,659$	$-0.018$	$+0.032$

Appendix

Table 7: Mean sentiment scores by emoji position (VADER and TextBlob, scored on emoji-free text).

Component / Parameter	Specification / Value
Architecture & Branches
Text Encoder	BiGRU over MiniLM embeddings, BERT [CLS]
Emoji Branch	Mean Kralj Novak sentiment, normalized emoji counts
Structural Branch (11-D)	VADER compound (emoji-free), polarity mismatch, caps ratio, punctuation density, norm. length, 5 emoji-position indicators, centroid distance *
Branch Fusion	Sigmoid gating
Training Hyperparameters

Appendix

Table 8: Architecture components and training hyperparameters for GEMA.

Social Media Analysis

Social media analysis studies online posts, media, users, and interactions to understand communication, communities, and public behavior.

See all 221 papers

Emoji-Emotion Ranking System Using Twitter Data
Oct 3, 2026Danila Khlebokazov, Nurkhan Tashimov, Pakizar ShamoiEmotion Recognition
School of Information Technology and Engineering, KBTU Almaty, Kazakhstan
Two Emojis of Difference: What Multilingual Affective Generation Benchmarks Actually Measure
Sep 24, 2026Fardeen Sadab, Adib SakhawatMultilingual Language Model Evaluation Language Model Generation Evaluation
Department of Computer Science and Engineering Islamic University of Technology, Dhaka, Bangladesh
Machine learning and digital pragmatics: Which word category influences emoji use most?
Aug 22, 2026Mohammed Q. Shormani, Yehia A. AlSohbani
Ibb University, Ibb, Yemen · Arab Open University, Riyadh, Saudi Arabia

Oct 3, 2026J/K move · Enter open · S save

Emoji-Emotion Ranking System Using Twitter Data

Danila Khlebokazov, Nurkhan Tashimov, Pakizar Shamoi

School of Information Technology and Engineering, KBTU Almaty, Kazakhstan

Emotion Recognition

Open paper AskPDF arXiv

Nowadays, emojis are often replacing words. Yet computational systems still oversimplify them. Most existing approaches treat emojis as static sentiment indicators and overlook their emotional distributions. In this study, we propose an emoji-aware emotion analysis framework based on a Twitter (X) dataset of 100,000 emoji-containing replies collected between 2020-2025. After preprocessing and text cleaning, we applied text-to-emotion classification to detect five primary emotions (Happy, Angry, Sad, Fear, and Surprise) for each message. By aggregating emotion scores across contexts in which each emoji appears, we estimate emoji-emotion association distributions and construct an emoji-emotion ranking system reflecting relative emotional dominance. Furthermore, we project emojis into the Russell valence-arousal space to enable continuous affective interpretation. Our results demonstrate that emojis exhibit probabilistic, context-sensitive emotional profiles rather than fixed sentiment polarities.

Abstract

Social Media Analysis

Explore similar work

Emoji-Emotion Ranking System Using Twitter Data

Two Emojis of Difference: What Multilingual Affective Generation Benchmarks Actually Measure

Machine learning and digital pragmatics: Which word category influences emoji use most?