PERCEIVE: A Benchmark for Personalized Emotion and Communication Behavior Understanding on Social Media

Jian Liao; Yujin Zheng; Suge Wang; Jianxing Zheng; Deyu Li

cs.SIApr 10, 2026

PERCEIVE: A Benchmark for Personalized Emotion and Communication Behavior Understanding on Social Media

Authors: Jian Liao, Yujin Zheng, Suge Wang, Jianxing Zheng, Deyu Li

Organizations: School of Computer and Information Technology, Shanxi University, China · Joint Laboratory of Tourism Big Data in Shanxi Province, China · Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education, Shanxi University, China

Abstract

Current emotion analysis in social media is predominantly author-centric, failing to capture the subjective nature of emotional responses across diverse readers. This paradigm overlooks the crucial link between individual perception, communication behavior, and the underlying social network. To bridge this gap, we introduce PERCEIVE, a novel bilingual (English and Chinese) large-scale benchmark that, to the best of our knowledge, is the first to integrate five critical dimensions for social perception: author-created content, genuine readers' emotional feedback (derived from their comments), communication behavior, user attributes, and the social graph. This benchmark enables a paradigm shift towards truly personalized, reader-centric analysis, where different readers' emotional responses to the same content are naturally captured through their real-world interactions. By annotating emotions from reader comments and synchronously capturing communication intent, PERCEIVE provides a unique resource to model the intrinsic coupling between emotion and behavior, grounded in social context. We establish a comprehensive evaluation protocol, testing state-of-the-art methods, including large language models (LLMs) with advanced reasoning enhancement. Our findings reveal significant shortcomings in existing approaches when handling this multifaceted, user-aware task. PERCEIVE offers a foundational resource and clear direction for future research in socially-intelligent NLP, pushing models towards a more unified understanding of emotion on social media.

Figures & tables

Figure 1: Illustration of personalized emotions and differential communication behaviors evoked by the same social media content from different reader perspectives.

Dataset	Emotion Subject	Domain	Language	Behavior	User Profile	Social Graph	Scale	Applicable Tasks
SemEval-14	author	product reviews	EN	$\times$	$\times$	$\times$	7.6k	ABSA
ECC	author	conversation	EN	$\times$	$\times$	$\times$	2.4k	emotion cause extraction
SemEval-17	author	Twitter	EN, AR	$\times$	$\times$	$\times$	72k	ABSA
GoEmotions	author	Reddit	EN	$\times$	$\times$	$\times$	58k	ABSA
OATS-ABSA	author	social media	EN	$\times$	$\times$	$\times$	44k	open-target ABSA
M-ABSA	author	social media	EN	$\times$	$\times$	$\times$	14k	multimodal ABSA

Table 1: Comparison of PERCEIVE with existing sentiment analysis datasets.

	Communication Behavior
	R&C		R		L		Total
Twitter	22580		22233		21975		66788
Weibo	15663		15358		15639		46660
	Emotion Distribution of Comments
	Ha	An	Sa	Fe	Su	Di	Ne	Total
Twitter	12338	1645	947	446	1862	614	4728	22580

Table 2: Distribution of behaviors and emtoions.

	Profiles		History		Total	E f	D/U
	w/	w/o	w/	w/o	Total	E f	D/U
Twitter	18331	5142	8552	14921	23473	47127	2.01
Weibo	12139	3737	9946	5930	15876	540530	34.05

Table 3: Statistics of user information

	Twitter		Weibo
models	M-F1	Acc	M-F1	Acc
GLM	0.5346	0.7436	0.4684	0.6437
Qwen	0.5429	0.7387	0.5205	0.7056
DeepSeek	0.5334	0.7520	0.4916	0.6756
GLM_TOC	0.5096	0.7201	0.4430	0.6341
Qwen_TOC	0.5186	0.7161	0.4856	0.6584

Table 4: Overall performance on Task A

	Twitter		Weibo
models	M-F1	Acc	M-F1	Acc
GLM	0.4907	0.7152	0.4471	0.6073
Qwen	0.5456	0.7436	0.4952	0.6628
DeepSeek	0.5226	0.7484	0.4525	0.6424
GLM_TOC	0.5126	0.7219	0.4318	0.6258
Qwen_TOC	0.5085	0.7046	0.4841	0.6533

Table 5: Overall performance on Task B

	Twitter		Weibo
models	M-F1	Acc	M-F1	Acc
GLM	0.3003	0.4047	0.3701	0.4896
Qwen	0.3738	0.5320	0.2678	0.3167
DeepSeek	0.3500	0.4745	0.2654	0.3384
GLM_TOC	0.3112	0.4304	0.3660	0.4879
Qwen_TOC	0.3622	0.4874	0.2955	0.4227

Table 6: Overall performance on Task C

Appendix figures & tables31 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 2: Prompt for LLM-based pre-annotation.

Models	Happy	Sad	Anger	Surprise	Disgust	Fear	Neutral	M-F1	Acc
GLM	0.8661	0.7296	0.4455	0.5648	0.3733	0.6842	0.6137	0.5346	0.7436
Qwen	0.8550	0.6832	0.5188	0.5955	0.4556	0.7308	0.5043	0.5429	0.7387
DeepSeek	0.8687	0.7219	0.6473	0.5250	0.2963	0.6250	0.5833	0.5334	0.7520
GLM_TOC	0.8411	0.6447	0.6424	0.5567	0.3636	0.6111	0.4170	0.5096	0.7201
Qwen_TOC	0.8391	0.7421	0.5420	0.5816	0.4645	0.6279	0.3516	0.5186	0.7161
DeepSeek_TOC	0.8575	0.7052	0.4416	0.5872	0.4277	0.6750	0.4958	0.5238	0.7334

Appendix

Table 7: Detailed results for Task A in Twitter dataset

Models	Happy	Sad	Anger	Surprise	Disgust	Fear	Neutral	M-F1	Acc
GLM	0.7317	0.5165	0.5670	0.4571	0.4359	0.3729	0.6660	0.4684	0.6437
Qwen	0.7930	0.6250	0.5571	0.4252	0.4800	0.5882	0.6953	0.5205	0.7056
DeepSeek	0.7711	0.5976	0.5263	0.5000	0.4828	0.4068	0.6486	0.4916	0.6756
GLM_TOC	0.7528	0.5670	0.5399	0.4545	0.2581	0.3793	0.5922	0.4430	0.6341
Qwen_TOC	0.7813	0.5647	0.5124	0.4521	0.4565	0.5600	0.5575	0.4856	0.6584
DeepSeek_TOC	0.7735	0.6140	0.4455	0.4828	0.4444	0.4571	0.6279	0.4807	0.6724

Appendix

Table 8: Detailed results for Task A in Weibo dataset

Panel of Figure 3 — Figure 3: Comparison of high- and low- activity users for Task B.

Models	Happy	Sad	Anger	Surprise	Disgust	Fear	Neutral	M-F1	Acc
GLM	0.8521	0.671	0.5045	0.3962	0.5455	0.4186	0.5373	0.4907	0.7152
Qwen	0.8562	0.7273	0.5493	0.5814	0.5126	0.4564	0.6818	0.5456	0.7436
DeepSeek	0.8618	0.6871	0.5464	0.6714	0.5215	0.3471	0.5455	0.5226	0.7484
GLM_TOC	0.8384	0.6667	0.4023	0.6782	0.5563	0.3918	0.5672	0.5126	0.7219
Qwen_TOC	0.8327	0.6753	0.299	0.5737	0.565	0.5342	0.5882	0.5085	0.7046
DeepSeek_TOC	0.8469	0.6795	0.4897	0.4502	0.5643	0.3822	0.5143	0.4909	0.7183

Appendix

Table 9: Detailed results for Task B in Twitter dataset

Models	Happy	Sad	Anger	Surprise	Disgust	Fear	Neutral	M-F1	Acc
GLM	0.7111	0.5591	0.6342	0.5161	0.4328	0.4928	0.2308	0.4471	0.6073
Qwen	0.7643	0.6027	0.6434	0.5455	0.3652	0.449	0.5915	0.4952	0.6628
DeepSeek	0.7464	0.5973	0.6062	0.5179	0.438	0.3929	0.3214	0.4525	0.6424
GLM_TOC	0.7456	0.5385	0.6059	0.4883	0.4094	0.3279	0.339	0.4318	0.6258
Qwen_TOC	0.774	0.6128	0.5718	0.5289	0.4203	0.4096	0.5556	0.4841	0.6533
DeepSeek_TOC	0.7583	0.6161	0.6294	0.4044	0.4615	0.4407	0.4839	0.4743	0.6571

Appendix

Table 10: Detailed results for Task B in Weibo dataset

	Twitter					Weibo
models	R&C	R	L	M-F1	Acc	R&C	R	L	M-F1	Acc
GLM	0.3949	0.4303	0.3760	0.3003	0.4047	0.3411	0.4884	0.6511	0.3701	0.4896
Qwen	0.2801	0.5066	0.7084	0.3738	0.5320	0.4629	0.2788	0.3295	0.2678	0.3167
DeepSeek	0.4432	0.3474	0.6093	0.3500	0.4745	0.248	0.4055	0.4080	0.2654	0.3384
GLM_TOC	0.5340	0.284	0.4269	0.3112	0.4304	0.5129	0.3179	0.6332	0.3660	0.4879
Qwen_TOC	0.3923	0.4104	0.6463	0.3622	0.4874	0.4738	0.1468	0.5612	0.2955	0.4227

Appendix

Table 11: Detailed results for Task C

Panel of Figure 4 — Figure 4: Comparison of high- and low- activity users for Task C.

Panel of Figure 5 — Figure 5: Confusion matrices of emotion-behavior joint prediction using GLM in Twitter.

Panel of Figure 6 — Figure 6: Confusion matrices of emotion-behavior joint prediction using GLM in Weibo.

Panel of Figure 7 — Figure 7: Confusion matrices of emotion-behavior joint prediction using Qwen in Twitter.

Panel of Figure 8 — Figure 8: Confusion matrices of emotion-behavior joint prediction using Qwen in Weibo.

Panel of Figure 9 — Figure 9: Confusion matrices of emotion-behavior joint prediction using DeepSeek in Twitter.

Panel of Figure 10 — Figure 10: Confusion matrices of emotion-behavior joint prediction using DeepSeek in Weibo.

Panel of Figure 11 — Figure 11: BLEU and Rouge-L of Task E based on different LLMs.

Panel of Figure 12 — Figure 12: Comparison of UIC, PFC, and EEC based on different LLMs.

Figure 13: Prompt for Task A (basic LLM).

Figure 14: Prompt for Task A (LLM + THOR).

Figure 15: Prompt for Task A (LLM + TOC).

Figure 16: Prompt for Task A (LLM + Debate).

Figure 17: Prompt for Task B (basic LLM).

Figure 18: Prompt for Task B (LLM + THOR).

Figure 19: Prompt for Task B (LLM + TOC).

Figure 20: Prompt for Task B (LLM + Debate).

Figure 21: Prompt for Task C (basic LLM).

Figure 22: Prompt for Task C (LLM + THOR).

Figure 23: Prompt for Task C (LLM + TOC).

Figure 24: Prompt for Task C (LLM + Debate).

Figure 26: Prompt of in-context comment generation for Task E.

Figure 27: Prompt of UIC, PFC, EEC measurements for Task E.

Social Media Analysis

Social media analysis studies online posts, media, users, and interactions to understand communication, communities, and public behavior.

See all 221 papers

Persona-E $^2$ : A Human-Grounded Dataset for Personality-Shaped Emotional Responses to Textual Events
Apr 10, 2026Yuqin Yang, Haowu Zhou, Haoran Tu +7Affective Computing Emotion Recognition
School of Future Technology, South China University of Technology, Guangzhou, China
Emotion Experience, Expression, and Perception: Emotion Analysis on Multimodal Social Media Posts
Sep 16, 2026Christopher Bagdon, Carina Silberer, Roman KlingerSocial Media Analysis Multimodal Emotion Recognition
Fundamentals of Natural Language Processing, University of Bamberg, Germany · Institut für Maschinelle Sprachverarbeitung, University of Stuttgart, Germany
APRiL: An Open-ended Benchmark for Chinese Social Awareness in Large Language Models
Oct 14, 2026Tian Lan, Han Zhang, Shanshan Wang +3LLM Evaluation Social Reasoning
Kyoto University · Shanghai Jiao Tong University · University of Macau +2

Apr 10, 2026J/K move · Enter open · S save

Persona-E $^2$ : A Human-Grounded Dataset for Personality-Shaped Emotional Responses to Textual Events

Yuqin Yang, Haowu Zhou, Haoran Tu, Zhiwen Hui+6

School of Future Technology, South China University of Technology, Guangzhou, China

Affective Computing Emotion Recognition Personality Modeling in Language Models LLM Evaluation

Open paper AskPDF arXiv

Most affective computing research treats emotion as a static property of text, focusing on the writer's sentiment while overlooking the reader's perspective. This approach ignores how individual personalities lead to diverse emotional appraisals of the same event. Although role-playing Large Language Models (LLMs) attempt to simulate such nuanced reactions, they often suffer from "personality illusion'' -- relying on surface-level stereotypes rather than authentic cognitive logic. A critical bottleneck is the absence of ground-truth human data to link personality traits to emotional shifts. To bridge the gap, we introduce Persona-E

^2

(Persona-Event2Emotion), a large-scale dataset grounded in annotated MBTI and Big Five traits to capture reader-based emotional variations across news, social media, and life narratives. Extensive experiments reveal that state-of-the-art LLMs struggle to capture precise appraisal shifts, particularly in social media domains. Crucially, we find that personality information significantly improves comprehension, with the Big Five traits alleviating "personality illusion.'

Abstract

Social Media Analysis

Explore similar work

Persona-E2^22: A Human-Grounded Dataset for Personality-Shaped Emotional Responses to Textual Events

Emotion Experience, Expression, and Perception: Emotion Analysis on Multimodal Social Media Posts

APRiL: An Open-ended Benchmark for Chinese Social Awareness in Large Language Models

Persona-E $^2$ : A Human-Grounded Dataset for Personality-Shaped Emotional Responses to Textual Events