CFMS: Towards Explainable and Fine-Grained Chinese Multimodal Sarcasm Detection Benchmark

Junzhao Zhang; Hsiu-Yuan Huang; Chenming Tang; Yutong Yang; Yunfang Wu

cs.CLMar 23, 2026

CFMS: Towards Explainable and Fine-Grained Chinese Multimodal Sarcasm Detection Benchmark

Authors: Junzhao Zhang, Hsiu-Yuan Huang, Chenming Tang, Yutong Yang, Yunfang Wu

Organizations: National Key Laboratory for Multimedia Information Processing, Peking University · School of Computer Science, Peking University · School of Software and Microelectronics, Peking University

Abstract

Multimodal sarcasm detection has recently garnered significant attention. However, existing benchmarks suffer from coarse-grained annotations and limited cultural coverage, which hinder research into fine-grained semantic understanding. To address this, we construct CFMS, the first fine-grained multimodal sarcasm dataset tailored for Chinese social media. It comprises 2,796 high-quality image-text pairs and provides a triple-level annotation framework: sarcasm identification, target recognition, and explanation generation. We find that the fine-grained explanation annotations effectively guide AI in generating images with explicit sarcastic intent. Furthermore, we curate a high-consistency parallel Chinese-English metaphor subset (200 entries each), revealing significant limitations of current models in metaphoric reasoning. To overcome the constraints of traditional retrieval methods, we propose a Reinforcement Learning-augmented In-Context Learning strategy (PGDS) to dynamically optimize exemplar selection. Extensive experiments demonstrate that CFMS provides a solid foundation for building reliable multimodal sarcasm understanding systems, and the PGDS method significantly outperforms existing baselines on key tasks. Our data and code are available at https://anonymous.4open.science/r/CFMS-E8F9.

Figures & tables

Figure 1: A representative instance of Chinese multimodal sarcasm from the CFMS dataset. The neutral text “装上全部一致的脑袋” (lit. “Install fully consistent heads”) contrasts with the visual of a conveyor belt, satirizing ideological conformity via culturally-grounded semantic inversion.

Figure 2: The human-in-the-loop annotation pipeline for the CFMS dataset, integrating GPT-4o Pre-annotation, Human Verification, and Interactive Error Correction.

Category	Avg. Length (Chars)
Text Context	18.17
Sarcasm Target	5.44
Sarcasm Explanation	61.69

Table 1: Text length distribution in CFMS (N=2,796).

Split	Positive	Negative	Total
Training	872	1,084	1,956
Validation	187	233	420
Test	187	233	420
Total	1,246	1,550	2,796

Table 2: Statistics and distribution of the CFMS dataset.

Case 1: Modern Human Activity
	Text : 现代人类活动(Modern Human Activity) Target : 手机依赖 (Mobile device dependency) Explanation : 通过人与狗在相似场景中的对比，讽刺人类对手机及充电设备的过度依赖。反映了科技在提供便利的同时，也带来了生理与精神上的束缚。 (Translation: By contrasting humans and dogs in similar scenarios, this sample satirizes excessive reliance on mobile devices, reflecting how technology limits freedom while providing convenience.)
Case 2: The Role Model
	Text : 《榜样》(《The Role Model》) Target : 家长榜样作用缺失 (Lack of parental role modeling) Explanation : 左边的家长使用手机，孩子随之模仿，右边的家长在读书，孩子也读书。通过对比揭示了一些家长以身作则不足的问题。 (Translation: This case contrasts two parental behaviors—using phones vs. reading—and the child’s subsequent imitation, revealing the failure of parental role modeling in fostering habits.)
Case 3: Rich vs. Poor
	Text : 穷人富人(Rich Poor) Target : 贫富差距 (Wealth disparity) Explanation : 通过对比穷人负重弯曲的脊背与富人鼓胀的肚子，反映富人通过压榨穷人积累财富的现象，批判社会的贫富不均。 (Translation: By comparing the hunched back of the poor carrying heavy loads with the bloated stomach of the rich, it reflects the accumulation of wealth through exploitation and criticizes social inequality.)

Table 3: Fine-grained annotations of samples from the CFMS dataset.

Figure 3: Word cloud of sarcasm targets in the Chinese CFMS dataset. Larger fonts indicate more frequently identified targets, such as “social phenomena” and “individual behavior.”

Figure 4: Word cloud of sarcasm targets in the English parallel subset. It reveals a higher concentration of targets related to public policy and institutional efficiency compared to the Chinese subset.

Type (Chinese)	Positive	Negative
Metaphor	73	127
Sarcasm	71	129

Table 4: Distribution of the Chinese Sarcasm-Metaphor subset (N=200).

Type (English)	Positive	Negative
Metaphor	100	100
Sarcasm	100	100

Table 5: Distribution of the English parallel subset (N=200).

Annotator	Sarcastic Effect Met	Ratio (%)
Annotator A	70	70.0
Annotator B	82	82.0
Average	76	76.0

Table 6: Evaluation of AI-generated images based on CFMS annotations (N=100).

Closed-Source
Model	Method	Acc $\uparrow$	P	R	F1 $\uparrow$	Target Acc $\uparrow$	BLEU-4 ( $\%$ )	BERTScore ( $\%$ )
GPT-4o	Zero-shot	70.24	60.67	96.30	74.44	71.03	8.07	72.81
	Random 1-shot	76.36	70.08	86.41	77.39	61.17	8.39	72.59
	RAG 1-shot	77.62	68.20	94.18	79.10	73.02	10.60	73.84
Gemini-2.5-Flash	Zero-shot	71.19	60.31	95.24	74.84	82.54	6.30	72.41
	Random 1-shot	75.71	67.61	88.36	76.61	76.19	6.88	72.55

Table 7: Main results on the CFMS dataset (excluding LoRA fine-tuning). Zero-shot , Random 1-shot , RAG 1-shot are baselines; PGDS (Ours) is our proposed dynamic demonstration selection strategy. Best results per column (across all models/methods) are in bold . ”-” indicates no corresponding experimental results.

Model	Method	Acc $\uparrow$	P	R	F1 $\uparrow$	Target Acc $\uparrow$	BLEU-4 ( $\%$ )	BERTScore ( $\%$ )
Qwen2.5-VL-7B-Instruct	LoRA FT	82.30	81.22	78.61	79.89	49.73	7.51	71.24
InternVL2.5-8B	LoRA FT	83.25	79.70	83.96	81.77	53.48	9.35	72.36
LLaVA-1.5-7B	LoRA FT	80.95	84.71	70.37	76.88	36.51	5.71	69.32

Table 8: Experimental results of LoRA fine-tuning (LoRA FT) on the CFMS dataset. Best results per column are in bold .

Model	Chinese Subset (N=200)					English Subset (N=200)
	Sarcasm		Metaphor		$\Delta_{\text{F1}}$	Sarcasm		Metaphor		$\Delta_{\text{F1}}$
	Acc.	F1	Acc.	F1	$\Delta_{\text{F1}}$	Acc.	F1	Acc.	F1	$\Delta_{\text{F1}}$
GPT-4o	77.00	75.53	41.50	55.51	-20.02	79.00	81.25	58.00	69.78	-11.47
Gemini-2.5-Flash	77.00	75.27	47.50	57.49	-17.78	72.00	74.31	64.50	72.58	-1.73
Qwen2.5-VL-7B-Instruct	84.50	78.77	60.50	59.49	-19.28	67.00	66.67	55.00	66.17	-0.50
InternVL2.5-8B	81.50	79.10	63.75	64.02	-15.08	70.00	63.41	54.00	58.68	-4.73

Table 9: Binary classification performance (%) comparison between Sarcasm and Metaphor identification across Chinese and English subsets.

\Delta_{\text{F1}}

denotes the F1-score drop from sarcasm to metaphor.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Examples of AI-generated images under sarcastic prompts.

Task	Original Chinese Prompt	English Translation
Zero-shot	给你一张图片，图片配文为:{text}。分析该图文对是否含有讽刺，并给出讽刺对象和解释…	Given an image with caption: {text}. Analyze whether this pair contains sarcasm, and provide the target and explanation…
Few-shot	Example: 输入：配文:{text}; 图片:{img}. 输出：是否讽刺:{y/n}; 讽刺对象:{target}; 讽刺解释:{exp}. Test: 给你一张图片,分析该图片是否含有讽刺…	Example: Input: {text/img}; Output: {y/n; target; exp}. Test: Given an image, analyze whether it contains sarcasm and provide the target/explanation.
AI Gen	请生成一幅讽刺图片，图片描述是：{描述}；讽刺对象为：{对象}；讽刺解释为：{解释}。	Please generate a sarcastic image. Description: {desc}; Target: {target}; Explanation: {exp}.

Appendix

Table 10: Standardized prompts for model evaluation and generation tasks.

Figure 6: Snapshot of the custom-developed CFMS annotation platform.

Sarcasm Detection

See all 21 papers

GRASP: Grounded CoT Reasoning with Dual-Stage Optimization for Multimodal Sarcasm Target Identification
Apr 10, 2026Faxian Wan, Xiaocui Yang, Yifan Cao +3Sarcasm Detection Multimodal CoT Reasoning
School of Computer Science and Engineering, Northeastern University, Shenyang, China
Evaluating Multimodal Large Language Models on Spoken Sarcasm Detection
Sep 18, 2025Zhu Li, Xiyuan Gao, Yuqing Zhang +2Sarcasm Detection Multimodal Model Evaluation
University of Groningen, The Netherlands
ProCrit: Self-Elicited Multi-Perspective Reasoning with Critic-Guided Revision for Multimodal Sarcasm Detection
May 20, 2026Yingjia Xu, Jiulong Wu, Bowen Zhang +3LLM Self-Correction Multimodal CoT Reasoning
School of Computer Science and Technology, Soochow University, Suzhou, China · Baidu Inc., Beijing, China · Zhipu AI, Beijing, China

Apr 10, 2026J/K move · Enter open · S save

GRASP: Grounded CoT Reasoning with Dual-Stage Optimization for Multimodal Sarcasm Target Identification

Faxian Wan, Xiaocui Yang, Yifan Cao, Shi Feng+2

School of Computer Science and Engineering, Northeastern University, Shenyang, China

Sarcasm Detection Multimodal CoT Reasoning

Open paper AskPDF arXiv

Moving beyond the traditional binary classification paradigm of Multimodal Sarcasm Detection, Multimodal Sarcasm Target Identification (MSTI) presents a more formidable challenge, requiring precise localization of fine-grained targets such as textual phrases and visual regions. Existing approaches predominantly rely on implicit cross-modal alignment, offering limited interpretability and suboptimal fine-grained localization. To address these limitations, we propose GRASP, Grounded Chain-of-Thought ReAsoning with Dual-Stage Optimization for Multimodal Sarcasm Prediction and Target Identification, a framework that integrates visual grounding with explicit Chain-of-Thought (CoT) reasoning to move beyond black-box MSTI. Specifically, we curate MSTI-MAX, a refined dataset that mitigates class imbalance and enriches multimodal sarcasm cues. We introduce Grounded CoT reasoning, which explicitly anchors sarcasm-related visual regions within the reasoning trajectory and prompts the model to articulate rationales before predicting the final classification labels and sarcasm targets. Furthermore, we employ a dual-stage outcome-supervised joint optimization strategy: Supervised Fine-Tuning with a coordinate-aware weighted loss, followed by Fine-Grained Target Policy Optimization. Extensive experiments demonstrate that GRASP outperforms existing baselines in fine-grained sarcasm target identification across modalities, and an LLM-as-a-Judge evaluation quantitatively measures the quality of internal reasoning chains. Our dataset and source code will be released on GitHub.

Abstract

Sarcasm Detection

Explore similar work

GRASP: Grounded CoT Reasoning with Dual-Stage Optimization for Multimodal Sarcasm Target Identification

Evaluating Multimodal Large Language Models on Spoken Sarcasm Detection

ProCrit: Self-Elicited Multi-Perspective Reasoning with Critic-Guided Revision for Multimodal Sarcasm Detection