PERCEIVE: A Benchmark for Personalized Emotion and Communication Behavior Understanding on Social Media

Jian Liao; Yujin Zheng; Suge Wang; Jianxing Zheng; Deyu Li

cs.SIApr 10, 2026

PERCEIVE: A Benchmark for Personalized Emotion and Communication Behavior Understanding on Social Media

Authors: Jian Liao, Yujin Zheng, Suge Wang, Jianxing Zheng, Deyu Li

Organizations: School of Computer and Information Technology, Shanxi University, China · Joint Laboratory of Tourism Big Data in Shanxi Province, China · Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education, Shanxi University, China

Abstract

Current emotion analysis in social media is predominantly author-centric, failing to capture the subjective nature of emotional responses across diverse readers. This paradigm overlooks the crucial link between individual perception, communication behavior, and the underlying social network. To bridge this gap, we introduce PERCEIVE, a novel bilingual (English and Chinese) large-scale benchmark that, to the best of our knowledge, is the first to integrate five critical dimensions for social perception: author-created content, genuine readers' emotional feedback (derived from their comments), communication behavior, user attributes, and the social graph. This benchmark enables a paradigm shift towards truly personalized, reader-centric analysis, where different readers' emotional responses to the same content are naturally captured through their real-world interactions. By annotating emotions from reader comments and synchronously capturing communication intent, PERCEIVE provides a unique resource to model the intrinsic coupling between emotion and behavior, grounded in social context. We establish a comprehensive evaluation protocol, testing state-of-the-art methods, including large language models (LLMs) with advanced reasoning enhancement. Our findings reveal significant shortcomings in existing approaches when handling this multifaceted, user-aware task. PERCEIVE offers a foundational resource and clear direction for future research in socially-intelligent NLP, pushing models towards a more unified understanding of emotion on social media.

Figures & tables

Figure 1: Illustration of personalized emotions and differential communication behaviors evoked by the same social media content from different reader perspectives.

Dataset	Emotion Subject	Domain	Language	Behavior	User Profile	Social Graph	Scale	Applicable Tasks
SemEval-14	author	product reviews	EN	$\times$	$\times$	$\times$	7.6k	ABSA
ECC	author	conversation	EN	$\times$	$\times$	$\times$	2.4k	emotion cause extraction
SemEval-17	author	Twitter	EN, AR	$\times$	$\times$	$\times$	72k	ABSA
GoEmotions	author	Reddit	EN	$\times$	$\times$	$\times$	58k	ABSA
OATS-ABSA	author	social media	EN	$\times$	$\times$	$\times$	44k	open-target ABSA
M-ABSA	author	social media	EN	$\times$	$\times$	$\times$	14k	multimodal ABSA

Table 1: Comparison of PERCEIVE with existing sentiment analysis datasets.

	Communication Behavior
	R&C		R		L		Total
Twitter	22580		22233		21975		66788
Weibo	15663		15358		15639		46660
	Emotion Distribution of Comments
	Ha	An	Sa	Fe	Su	Di	Ne	Total
Twitter	12338	1645	947	446	1862	614	4728	22580

Table 2: Distribution of behaviors and emtoions.

	Profiles		History		Total	E f	D/U
	w/	w/o	w/	w/o	Total	E f	D/U
Twitter	18331	5142	8552	14921	23473	47127	2.01
Weibo	12139	3737	9946	5930	15876	540530	34.05

Table 3: Statistics of user information

	Twitter		Weibo
models	M-F1	Acc	M-F1	Acc
GLM	0.5346	0.7436	0.4684	0.6437
Qwen	0.5429	0.7387	0.5205	0.7056
DeepSeek	0.5334	0.7520	0.4916	0.6756
GLM_TOC	0.5096	0.7201	0.4430	0.6341
Qwen_TOC	0.5186	0.7161	0.4856	0.6584

Table 4: Overall performance on Task A

	Twitter		Weibo
models	M-F1	Acc	M-F1	Acc
GLM	0.4907	0.7152	0.4471	0.6073
Qwen	0.5456	0.7436	0.4952	0.6628
DeepSeek	0.5226	0.7484	0.4525	0.6424
GLM_TOC	0.5126	0.7219	0.4318	0.6258
Qwen_TOC	0.5085	0.7046	0.4841	0.6533

Table 5: Overall performance on Task B

	Twitter		Weibo
models	M-F1	Acc	M-F1	Acc
GLM	0.3003	0.4047	0.3701	0.4896
Qwen	0.3738	0.5320	0.2678	0.3167
DeepSeek	0.3500	0.4745	0.2654	0.3384
GLM_TOC	0.3112	0.4304	0.3660	0.4879
Qwen_TOC	0.3622	0.4874	0.2955	0.4227

Table 6: Overall performance on Task C

Appendix figures & tables31 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 2: Prompt for LLM-based pre-annotation.

Models	Happy	Sad	Anger	Surprise	Disgust	Fear	Neutral	M-F1	Acc
GLM	0.8661	0.7296	0.4455	0.5648	0.3733	0.6842	0.6137	0.5346	0.7436
Qwen	0.8550	0.6832	0.5188	0.5955	0.4556	0.7308	0.5043	0.5429	0.7387
DeepSeek	0.8687	0.7219	0.6473	0.5250	0.2963	0.6250	0.5833	0.5334	0.7520
GLM_TOC	0.8411	0.6447	0.6424	0.5567	0.3636	0.6111	0.4170	0.5096	0.7201
Qwen_TOC	0.8391	0.7421	0.5420	0.5816	0.4645	0.6279	0.3516	0.5186	0.7161
DeepSeek_TOC	0.8575	0.7052	0.4416	0.5872	0.4277	0.6750	0.4958	0.5238	0.7334

Appendix

Table 7: Detailed results for Task A in Twitter dataset

Models	Happy	Sad	Anger	Surprise	Disgust	Fear	Neutral	M-F1	Acc
GLM	0.7317	0.5165	0.5670	0.4571	0.4359	0.3729	0.6660	0.4684	0.6437
Qwen	0.7930	0.6250	0.5571	0.4252	0.4800	0.5882	0.6953	0.5205	0.7056
DeepSeek	0.7711	0.5976	0.5263	0.5000	0.4828	0.4068	0.6486	0.4916	0.6756
GLM_TOC	0.7528	0.5670	0.5399	0.4545	0.2581	0.3793	0.5922	0.4430	0.6341
Qwen_TOC	0.7813	0.5647	0.5124	0.4521	0.4565	0.5600	0.5575	0.4856	0.6584
DeepSeek_TOC	0.7735	0.6140	0.4455	0.4828	0.4444	0.4571	0.6279	0.4807	0.6724

Appendix

Table 8: Detailed results for Task A in Weibo dataset

Panel of Figure 3 — Figure 3: Comparison of high- and low- activity users for Task B.

Models	Happy	Sad	Anger	Surprise	Disgust	Fear	Neutral	M-F1	Acc
GLM	0.8521	0.671	0.5045	0.3962	0.5455	0.4186	0.5373	0.4907	0.7152
Qwen	0.8562	0.7273	0.5493	0.5814	0.5126	0.4564	0.6818	0.5456	0.7436
DeepSeek	0.8618	0.6871	0.5464	0.6714	0.5215	0.3471	0.5455	0.5226	0.7484
GLM_TOC	0.8384	0.6667	0.4023	0.6782	0.5563	0.3918	0.5672	0.5126	0.7219
Qwen_TOC	0.8327	0.6753	0.299	0.5737	0.565	0.5342	0.5882	0.5085	0.7046
DeepSeek_TOC	0.8469	0.6795	0.4897	0.4502	0.5643	0.3822	0.5143	0.4909	0.7183

Appendix

Table 9: Detailed results for Task B in Twitter dataset

Models	Happy	Sad	Anger	Surprise	Disgust	Fear	Neutral	M-F1	Acc
GLM	0.7111	0.5591	0.6342	0.5161	0.4328	0.4928	0.2308	0.4471	0.6073
Qwen	0.7643	0.6027	0.6434	0.5455	0.3652	0.449	0.5915	0.4952	0.6628
DeepSeek	0.7464	0.5973	0.6062	0.5179	0.438	0.3929	0.3214	0.4525	0.6424
GLM_TOC	0.7456	0.5385	0.6059	0.4883	0.4094	0.3279	0.339	0.4318	0.6258
Qwen_TOC	0.774	0.6128	0.5718	0.5289	0.4203	0.4096	0.5556	0.4841	0.6533
DeepSeek_TOC	0.7583	0.6161	0.6294	0.4044	0.4615	0.4407	0.4839	0.4743	0.6571

Appendix

Table 10: Detailed results for Task B in Weibo dataset

	Twitter					Weibo
models	R&C	R	L	M-F1	Acc	R&C	R	L	M-F1	Acc
GLM	0.3949	0.4303	0.3760	0.3003	0.4047	0.3411	0.4884	0.6511	0.3701	0.4896
Qwen	0.2801	0.5066	0.7084	0.3738	0.5320	0.4629	0.2788	0.3295	0.2678	0.3167
DeepSeek	0.4432	0.3474	0.6093	0.3500	0.4745	0.248	0.4055	0.4080	0.2654	0.3384
GLM_TOC	0.5340	0.284	0.4269	0.3112	0.4304	0.5129	0.3179	0.6332	0.3660	0.4879
Qwen_TOC	0.3923	0.4104	0.6463	0.3622	0.4874	0.4738	0.1468	0.5612	0.2955	0.4227

Appendix

Table 11: Detailed results for Task C

Panel of Figure 4 — Figure 4: Comparison of high- and low- activity users for Task C.

Panel of Figure 5 — Figure 5: Confusion matrices of emotion-behavior joint prediction using GLM in Twitter.

Panel of Figure 6 — Figure 6: Confusion matrices of emotion-behavior joint prediction using GLM in Weibo.

Panel of Figure 7 — Figure 7: Confusion matrices of emotion-behavior joint prediction using Qwen in Twitter.

Panel of Figure 8 — Figure 8: Confusion matrices of emotion-behavior joint prediction using Qwen in Weibo.

Panel of Figure 9 — Figure 9: Confusion matrices of emotion-behavior joint prediction using DeepSeek in Twitter.

Panel of Figure 10 — Figure 10: Confusion matrices of emotion-behavior joint prediction using DeepSeek in Weibo.

Panel of Figure 11 — Figure 11: BLEU and Rouge-L of Task E based on different LLMs.

Panel of Figure 12 — Figure 12: Comparison of UIC, PFC, and EEC based on different LLMs.

Figure 13: Prompt for Task A (basic LLM).

Figure 14: Prompt for Task A (LLM + THOR).

Figure 15: Prompt for Task A (LLM + TOC).

Figure 16: Prompt for Task A (LLM + Debate).

Figure 17: Prompt for Task B (basic LLM).

Figure 18: Prompt for Task B (LLM + THOR).

Figure 19: Prompt for Task B (LLM + TOC).

Figure 20: Prompt for Task B (LLM + Debate).

Figure 21: Prompt for Task C (basic LLM).

Figure 22: Prompt for Task C (LLM + THOR).

Figure 23: Prompt for Task C (LLM + TOC).

Figure 24: Prompt for Task C (LLM + Debate).

Figure 26: Prompt of in-context comment generation for Task E.

Figure 27: Prompt of UIC, PFC, EEC measurements for Task E.

Social Media Analysis

Social media analysis studies online posts, media, users, and interactions to understand communication, communities, and public behavior.

See all 221 papers

Apr 10, 2026

Persona-E $^2$ : A Human-Grounded Dataset for Personality-Shaped Emotional Responses to Textual Events

Most affective computing research treats emotion as a static property of text, focusing on the writer's sentiment while overlooking the reader's perspective. This approach ignores how individual personalities lead to diverse emotional appraisals of the same event. Although role-playing Large Language Models (LLMs) attempt to simulate such nuanced reactions, they often suffer from "personality illusion'' -- relying on surface-level stereotypes rather than authentic cognitive logic. A critical bottleneck is the absence of ground-truth human data to link personality traits to emotional shifts. To bridge the gap, we introduce Persona-E

^2

(Persona-Event2Emotion), a large-scale dataset grounded in annotated MBTI and Big Five traits to capture reader-based emotional variations across news, social media, and life narratives. Extensive experiments reveal that state-of-the-art LLMs struggle to capture precise appraisal shifts, particularly in social media domains. Crucially, we find that personality information significantly improves comprehension, with the Big Five traits alleviating "personality illusion.'

Yuqin Yang, Haowu Zhou, Haoran Tu +7

School of Future Technology, South China University of Technology, Guangzhou, China

Affective Computing Emotion Recognition Personality Modeling in Language Models LLM Evaluation

Ask PDF

Sep 16, 2026

Emotion Experience, Expression, and Perception: Emotion Analysis on Multimodal Social Media Posts

Emotions are an essential aspect of human communication, particularly on social media, where authors frequently combine text and images to convey their emotions. Yet prior work on emotion analysis of social media posts has overlooked two important aspects in regard to measuring how well readers can reconstruct the authors' intent: (1)~the image modality, with most work focusing solely on text, and (2)~the real-world events that trigger the expressed emotions, and their relationship to the post content. We therefore study the relation between (a) the author's experience of the event that caused them to write a social media post and (b) the content of the post, with a focus on readers' capability to reconstruct that emotion expression. To do that, we introduce the Multimodal Multi-Emotion-Model dataset Mult2EMo, created by collecting annotations from both authors and readers on the posts and their triggering events. We find that reconstruction is possible but challenging for both human readers and computational models. We show that understanding the triggering event is crucial for accurate reconstruction, and that reconstruction is particularly challenging when posts rely heavily on the image to express emotion.

Christopher Bagdon, Carina Silberer, Roman Klinger

Fundamentals of Natural Language Processing, University of Bamberg, Germany · Institut für Maschinelle Sprachverarbeitung, University of Stuttgart, Germany

Social Media Analysis Multimodal Emotion Recognition Emotion Recognition

Ask PDF

Oct 14, 2026

APRiL: An Open-ended Benchmark for Chinese Social Awareness in Large Language Models

Large language models are increasingly used in social interactions, where appropriate responses require recognizing implicit cues, inferring mental states, identifying conflicts, and considering potential consequences. However, existing benchmarks often focus on specific social abilities, constrained task formats, or particular interaction settings, offering limited coverage of social awareness across diverse contexts. To address this gap, we introduce APRiL, an open-ended benchmark for Chinese social awareness with instance-specific rubrics. APRiL contains 2,196 Social Awareness Evaluation Instances across six categories and two complementary tasks: Social Scenario Analysis evaluates models' understanding of social cues, mental states, and conflict structures, while Socially Aware Response assesses whether they can translate this understanding into appropriate, natural responses. Experiments on 12 models from six families reveal a consistent understanding-to-response gap. Higher-quality scenario analyses improve responses, while weak analyses may reduce response quality. APRiL provides a systematic resource for advancing socially aware language models in Chinese contexts. Our data and code are available at https://anonymous.4open.science/r/APRiL-05DE/.

Tian Lan, Han Zhang, Shanshan Wang +3

Kyoto University · Shanghai Jiao Tong University · University of Macau +2

LLM Evaluation Social Reasoning Language Model Generation Evaluation

Ask PDF

Abstract

Social Media Analysis

Explore similar work

Persona-E2^22: A Human-Grounded Dataset for Personality-Shaped Emotional Responses to Textual Events

Emotion Experience, Expression, and Perception: Emotion Analysis on Multimodal Social Media Posts

APRiL: An Open-ended Benchmark for Chinese Social Awareness in Large Language Models

Persona-E $^2$ : A Human-Grounded Dataset for Personality-Shaped Emotional Responses to Textual Events