Beyond a Single Sentiment Score: Topical and Affective Dimensions of Emoji Meaning on Twitter

Rohan James; Ioannis Christofilogiannis; Yuuv Jauhari; Shreya Shah; Gauri Pethkar

cs.CLOct 20, 2026

Beyond a Single Sentiment Score: Topical and Affective Dimensions of Emoji Meaning on Twitter

Authors: Rohan James, Ioannis Christofilogiannis, Yuuv Jauhari, Shreya Shah, Gauri Pethkar

Organizations: School of Computer Science and Statistics Trinity College Dublin, Ireland

Abstract

Emoji are widely treated as compact affective tokens whose meaning is summarised by a single sentiment score. We argue that this picture is incomplete. From a corpus of 16{,}575 English tweets and a per-emoji metric we call stability delta (within-class minus between-class cosine similarity of sentence embeddings), we show that emojis used in tightly clustered topical contexts also exhibit more polarised crowd-sourced sentiment, not less. Validated against the Kralj Novak et al.\ (2015) sentiment lexicon, the central correlation is Spearman $ρ= +0.361$ (Holm $p = 0.015$ , 95% bootstrap CI $[+0.10, +0.59]$ ); against the independent Ferré et al.\ (2025) Spanish 13-emotion ratings, $ρ= +0.509$ ( $p = 0.0003$ ). The result survives permutation testing, BCa bootstrap, leave-one-out jackknife, partial correlation controlling for emoji frequency, and Holm correction across a four-test family. Topical coherence and affective dispersion are distinct dimensions of emoji meaning. As a downstream check we report leakage-free five-fold cross-validation results for sarcasm classification on the Emoji Sarcasm Dataset; all three baselines (a gated multimodal architecture, fine-tuned BERTweet features, and fine-tuned RoBERTa features) score above 0.97 macro F1, indicating the dataset has saturating emoji-label cues. We frame the classifier as supporting evidence rather than the contribution.

Figures & tables

Figure 1: VADER compound score distributions by emoji position and role (substitutive vs. additive). Substitutive emojis produce more polarised distributions across all three positions.

Figure 2: Mean VADER compound score for the top-20 emojis across positional contexts (start, middle, end). Right column shows raw frequency. Individual emojis exhibit marked affective shifts depending on position; where an emoji sits predicts its sentiment as strongly as which emoji it is.

Emoji (description)	Within	Between	$\Delta$
Collision	0.409	0.172	0.237
Fire	0.332	0.140	0.192
Pointing-down	0.305	0.152	0.153
Astonished face	0.121	0.101	0.021
Loudly crying face	0.143	0.124	0.019
Confused face	0.124	0.109	0.014

Table 1: Top and bottom emojis ranked by stability delta. “Within” and “Between” are mean pairwise cosine similarities within and across emoji classes.

Test	$\rho$	Pct. CI (95%)	BCa CI (95%)	Perm. $p$	$\tau_{b}$	Partial $\rho$ (freq)	Holm $p$
$\Delta$ vs. Kralj Novak sentiment	$+0.408$	$[+0.16,+0.60]$	$[+0.16,+0.60]$	0.0008	$+0.258$	$+0.423$	0.005
$\Delta$ vs. Kralj Novak variance (primary)	$\mathbf{+0.361}$	$[+0.10,+0.59]$	$[+0.09,+0.58]$	0.005	$+0.261$	$+0.370$	0.015
Sarcasm rate vs. $p(\text{neg})$	$-0.286$	$[-0.57,+0.02]$	$[-0.55,+0.02]$	0.028	$-0.227$	$-0.280$	0.051
Sarcasm rate vs. Kralj Novak sentiment	$+0.291$	$[-0.01,+0.57]$	$[-0.01,+0.55]$	0.029	$+0.234$	$+0.289$	0.051

Table 2: Kralj Novak (2015) validation,

n=59

matched emojis. Holm correction applied across the family of four tests. Significance markers refer to Holm-corrected

p

-values.

Figure 3: Contextual stability is positively associated with affective dispersion. Left: stability

\Delta

vs. Kralj Novak sentiment variance (

n=59

, Spearman

\rho=+0.361

, BCa

{}_{95}=[+0.089,+0.581]

, Holm

p=.015

). Right: stability

\Delta

vs. number of strongly evoked emotions in Ferré et al. (

n=47

\rho=+0.509

p=.0003

). Emojis used in more topically coherent contexts evoke broader, not narrower, affective responses.

Figure 4: Stability delta versus Kralj Novak sentiment score (left) and sentiment variance (right). Red lines are OLS fits. Both relationships are positive and Holm-significant.

Likert threshold	$\rho$	$p$	Sig.
$\geq 2.0$	$+0.606$	$6.0\times 10^{-6}$
$\geq 2.5$ (primary)	$\mathbf{+0.509}$	$\mathbf{2.6\times 10^{-4}}$
$\geq 3.0$	$+0.458$	$1.2\times 10^{-3}$	**
$\geq 3.5$	$+0.415$	$3.7\times 10^{-3}$	**

Table 3: Stability delta vs. number of strongly evoked emotions in Ferré 2025, with threshold sensitivity.

n=47

matched emojis throughout.

Model	Macro F1	Sarc. F1
GEMA	$.985{\pm}.007$	$.982{\pm}.008$
BERTweet	$.974{\pm}.002$	$.967{\pm}.002$
RoBERTa	$.991{\pm}.001$	$.988{\pm}.001$

Table 4: Five-fold stratified CV on the English Twitter partition (

n=16{,}587

, mean

\pm

SD). Full 5-metric evaluation reported in Appendix F .

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Acc.	Macro F1	Sarc. F1	AUC	MCC
GEMA	$.986{\pm}.006$	$.985{\pm}.007$	$.982{\pm}.008$	$.999{\pm}.000$	$.970{\pm}.013$
BERTweet	$.975{\pm}.002$	$.974{\pm}.002$	$.967{\pm}.002$	$.997{\pm}.001$	$.948{\pm}.003$
RoBERTa	$.991{\pm}.001$	$.991{\pm}.001$	$.988{\pm}.001$	$.999{\pm}.000$	$.982{\pm}.002$

Appendix

Table 5: Full five-fold stratified cross-validation results on the English-filtered Twitter partition (

n=16{,}587

). Values show mean

\pm

standard deviation across folds.

Source	Test	$n$	$\rho$	$p$
Czestochowska	$\Delta$ vs. description entropy	59	$-0.087$	0.51
Czestochowska	$\Delta$ vs. unique words	59	$-0.029$	0.83
EmojiNet	$\Delta$ vs. sense count	62	$-0.119$	0.36
EmojiNet	$\Delta$ vs. keyword count	62	$+0.075$	0.56

Appendix

Table 6: Sanity checks against decontextualised ambiguity Czestochowska et al. (2022) and dictionary polysemy Wijeratne et al. (2017) .

Position	Count	VADER	TextBlob
Start	$2,188$	$+0.267$	$+0.190$
Middle	$4,728$	$+0.0667$	$+0.116$
End	$9,659$	$-0.018$	$+0.032$

Appendix

Table 7: Mean sentiment scores by emoji position (VADER and TextBlob, scored on emoji-free text).

Component / Parameter	Specification / Value
Architecture & Branches
Text Encoder	BiGRU over MiniLM embeddings, BERT [CLS]
Emoji Branch	Mean Kralj Novak sentiment, normalized emoji counts
Structural Branch (11-D)	VADER compound (emoji-free), polarity mismatch, caps ratio, punctuation density, norm. length, 5 emoji-position indicators, centroid distance *
Branch Fusion	Sigmoid gating
Training Hyperparameters

Appendix

Table 8: Architecture components and training hyperparameters for GEMA.

Social Media Analysis

Social media analysis studies online posts, media, users, and interactions to understand communication, communities, and public behavior.

See all 221 papers

Oct 3, 2026

Emoji-Emotion Ranking System Using Twitter Data

Nowadays, emojis are often replacing words. Yet computational systems still oversimplify them. Most existing approaches treat emojis as static sentiment indicators and overlook their emotional distributions. In this study, we propose an emoji-aware emotion analysis framework based on a Twitter (X) dataset of 100,000 emoji-containing replies collected between 2020-2025. After preprocessing and text cleaning, we applied text-to-emotion classification to detect five primary emotions (Happy, Angry, Sad, Fear, and Surprise) for each message. By aggregating emotion scores across contexts in which each emoji appears, we estimate emoji-emotion association distributions and construct an emoji-emotion ranking system reflecting relative emotional dominance. Furthermore, we project emojis into the Russell valence-arousal space to enable continuous affective interpretation. Our results demonstrate that emojis exhibit probabilistic, context-sensitive emotional profiles rather than fixed sentiment polarities.

Danila Khlebokazov, Nurkhan Tashimov, Pakizar Shamoi

School of Information Technology and Engineering, KBTU Almaty, Kazakhstan

Emotion Recognition

Ask PDF

Sep 24, 2026

Two Emojis of Difference: What Multilingual Affective Generation Benchmarks Actually Measure

We audit a multilingual affective generation benchmark eight instruction-tuned LLMs producing emoji summaries for 17,100 Bangla, English and Hindi sentences, with 6,960 human judgements and find its headline conclusions to be artefacts of the measurement instrument rather than properties of the systems. Treating annotators as a random rather than a fixed factor, no system differs significantly from any other (

F(7,14)=0.59

p=0.76

), although the conventional analysis declares 19 of 28 pairwise differences significant. Annotator identity explains far more rating variance than system identity, and the winning system changes whenever any single annotator is removed. The ordering that does emerge tracks output length: mean emoji count explains 78.7% of between-system variance, and a within-item length-matched comparison over 2,599 pairs reverses the leaderboard. We further show that cross-provider anisotropy differences vanish under mean-centring, that per-language token costs change sign with the normalising unit, and that multi-view row-wise splits inflate macro-F1 by

3.1

points and change the top-ranked system. In place of preference scoring we propose emoji-affect decodability, a reference-based probe whose rankings are stable to

\pm0.003

macro-F1 across seeds.

Fardeen Sadab, Adib Sakhawat

Department of Computer Science and Engineering Islamic University of Technology, Dhaka, Bangladesh

Multilingual Language Model Evaluation Language Model Generation Evaluation Emotion Representation in Language Models Text Generation

Ask PDF

Aug 22, 2026

Machine learning and digital pragmatics: Which word category influences emoji use most?

This study examines the performance of the state-of-the-art MARBERT model in identifying the lexical/pragmatic category associated with emoji use on X within a digital pragmatics approach (DPA). A net corpus of 15856 Colloquial Arabic (CA) posts containing emojis was collected from X using Python. The texts were tokenized and normalized into 4 lexical categories, namely noun_norm, verb_norm, adj_norm, and adverb_norm, and 2 pragmatic/structural categories, question_norm and exclamation_norm. MARBERT was finetuned and optimized to identify which category scores standard metrics more, hence associated with emoji use, while binary logistic regression was used to examine which category is statistically associated with emoji occurrence. Findings unveil that nouns dominate the corpus in normalized frequency (M = 0.675, SD = 0.161), followed by verbs (M = 0.083, SD = 0.100). However, verbs have the strongest influence of emoji use indicated by verb density (\b{eta} = 0.821, p = .001, 95% CI [0.332, 1.309]). The study concludes that in digital pragmatics of CA on X, emoji use association with lexical/pragmatic category can be explained by a hybrid approach of computational, statistical, and pragmatic methods, reflecting the interaction among machine learning, linguistic/lexical features, contextual representation, and pragmatic communication.

Mohammed Q. Shormani, Yehia A. AlSohbani

Ibb University, Ibb, Yemen · Arab Open University, Riyadh, Saudi Arabia

Topic pending

Ask PDF

Abstract

Social Media Analysis

Explore similar work

Emoji-Emotion Ranking System Using Twitter Data

Two Emojis of Difference: What Multilingual Affective Generation Benchmarks Actually Measure

Machine learning and digital pragmatics: Which word category influences emoji use most?