GRASP: Grounded CoT Reasoning with Dual-Stage Optimization for Multimodal Sarcasm Target Identification

Faxian Wan; Xiaocui Yang; Yifan Cao; Shi Feng; Daling Wang; Yifei Zhang

cs.CLApr 10, 2026

GRASP: Grounded CoT Reasoning with Dual-Stage Optimization for Multimodal Sarcasm Target Identification

Authors: Faxian Wan, Xiaocui Yang, Yifan Cao, Shi Feng, Daling Wang, Yifei Zhang

Organizations: School of Computer Science and Engineering, Northeastern University, Shenyang, China

Abstract

Moving beyond the traditional binary classification paradigm of Multimodal Sarcasm Detection, Multimodal Sarcasm Target Identification (MSTI) presents a more formidable challenge, requiring precise localization of fine-grained targets such as textual phrases and visual regions. Existing approaches predominantly rely on implicit cross-modal alignment, offering limited interpretability and suboptimal fine-grained localization. To address these limitations, we propose GRASP, Grounded Chain-of-Thought ReAsoning with Dual-Stage Optimization for Multimodal Sarcasm Prediction and Target Identification, a framework that integrates visual grounding with explicit Chain-of-Thought (CoT) reasoning to move beyond black-box MSTI. Specifically, we curate MSTI-MAX, a refined dataset that mitigates class imbalance and enriches multimodal sarcasm cues. We introduce Grounded CoT reasoning, which explicitly anchors sarcasm-related visual regions within the reasoning trajectory and prompts the model to articulate rationales before predicting the final classification labels and sarcasm targets. Furthermore, we employ a dual-stage outcome-supervised joint optimization strategy: Supervised Fine-Tuning with a coordinate-aware weighted loss, followed by Fine-Grained Target Policy Optimization. Extensive experiments demonstrate that GRASP outperforms existing baselines in fine-grained sarcasm target identification across modalities, and an LLM-as-a-Judge evaluation quantitatively measures the quality of internal reasoning chains. Our dataset and source code will be released on GitHub.

Figures & tables

Figure 1. An illustrative example of MSTI. The sarcasm arises from the contradiction between the specific textual span “soooo expected. not” and the localized visual region (depicting a perilously low-flying airplane), which collectively conveys the sarcastic intent. Our GRASP framework not only outputs fine-grained multimodal targets directly but also yields explicit grounding chain-of-thought reasoning. An example of multimodal sarcasm. The text expresses surprise with heavy irony, while the image shows a dangerously low-flying aircraft over highway traffic, creating a strong text-image semantic contradiction.

Dataset	Split	Sarcasm	Non-sarcasm	Total
MSTI 2.0	Train	3,500	46	3,546
	Val	711	16	727
	Test	729	13	742
	Total	4,940	75	5,015
MSTI-MAX	Train	3,500	3,046	6,546
	Val	711	516	1,227

Table 1. Statistics of MSTI 2.0 and MSTI-MAX datasets.

Figure 2. Comparison of the fine-grained distribution of sarcasm targets across different modalities in the MSTI-MAX and MSTI 2.0 datasets. To reflect the diverse nature of real-world sarcastic expressions, the targets within each instance are categorized into three types: exclusively in the text (Text-only), exclusively in the visual modality (Visual-only), and present in both modalities (Text & Visual). Proportions of different datasets, Fine-grained Sarcasm Modality Distribution of MSTI-MAX.

Figure 3. Overview of the GRASP training pipeline. The framework consists of two stages: (1) Supervised Fine-Tuning (SFT) with coordinate-aware weighted loss employing structured grounded CoT reasoning; (2) Fine-Grained Target Policy Optimization (FTPO) with multidimensional rewards, which further strengthens reasoning quality and visual grounding precision. Overview of the GRASP training pipeline.

Text-only Baselines
Methods	Sarcasm Detection				Text Target		Visual Target
Methods	Acc	Precision	Recall	F1	EM	F1	AP	AP50	AP75
BERT-base	80.99	87.86	78.46	82.89	17.23	39.20	–	–	–
BERT-large	81.59	85.98	79.97	82.87	21.74	42.53	–	–	–
Task-specific Multimodal Model
CofiPara	80.20	73.52	85.32	78.98	49.52	48.74	9.79	23.18	8.00

Table 2. Main results on the MSTI-MAX dataset. The best results in each column are highlighted in bold , and the second-best are underlined. “–” indicates the metric is not applicable. “*” means the reproduced result.

Figure 4. LLM-as-a-Judge evaluation results. Higher values indicate better performance on a 1–5 scale. LLM-as-a-Judge evaluation results.

Ablation Study of GRASP Based on Qwen3-VL-4B
Variants	Sarcasm Detection				Text Target		Visual Target
Variants	Acc	Precision	Recall	F1	EM	F1	AP	AP50	AP75
GRASP (Qwen3-VL-4B)	78.50	77.11	90.12	83.11	43.16	35.04	41.31	45.84	40.74
w/o FTPO (SFT only)	73.26	83.02	68.44	75.03	46.45	28.88	34.21	38.25	33.70
w/o SFT (FTPO only)	72.71	77.69	75.03	76.34	45.11	7.62	12.94	17.24	12.58
w/o Bbox-Weight	72.79	75.16	80.11	77.56	41.14	29.39	30.03	33.70	29.38

Table 3. Ablation study of the proposed components based on Qwen3-VL-4B and generalization performance across various foundational MLLMs on the MSTI-MAX dataset. The best results within each group are highlighted in bold .

Method	Acc	Precision	Recall	F1
LLaVA (Zero-Shot)	51.06	40.09	46.40	43.02
Qwen-VL (Zero-Shot)	40.63	32.44	35.53	33.63
GPT-4o (Zero-Shot)	71.07	79.52	71.07	70.24
LLaVA (CoT)	48.69	40.93	65.17	50.28
Qwen-VL (CoT)	58.86	56.82	58.67	57.26
GPT-4o (CoT)	74.26	65.81	72.68	68.92

Table 4. Generalization results on the MMSD 2.0 benchmark. Results of baselines are taken from GDCNet ( Zhang et al., 2026 ) .

Figure 5. A case study of GRASP on the MSTI task. A case study showing the complete GRASP reasoning and target identification process on a ride-hailing app sarcasm example.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	SFT	FTPO
LoRA Rank / Alpha	4 / 16	8 / 32
Learning Rate	3.0e-5	1.0e-5
Warmup	80 steps	5% ratio
Num Epochs	2	1
Effective Batch Size	8	16
Max Sequence Length	2048	2048

Appendix

Table 5. Training hyperparameters for SFT and FTPO stages.

$\lambda_{\text{coord}}$	Sarcasm Detection				Text Target		Visual Target
$\lambda_{\text{coord}}$	Acc	Prec.	Rec.	F1	EM	F1	AP	AP50	AP75
1	72.79	75.16	80.11	77.56	41.14	29.39	30.03	33.70	29.38
5	77.54	82.61	78.19	80.34	46.14	30.06	37.62	41.82	37.06
10	78.50	77.11	90.12	83.11	43.16	35.04	41.31	45.84	40.74
15	69.00	67.77	89.99	77.31	46.38	11.90	22.92	25.39	22.70

Appendix

Table 6. Effect of the coordinate weight

\lambda_{\text{coord}}

in the SFT stage on GRASP (Qwen3-VL-4B). Results for

\lambda_{\text{coord}}\in\{1,10\}

are reported in the main paper; the additional settings

\lambda_{\text{coord}}\in\{5,15\}

are supplemented here.

Setting	Sarcasm Detection				Text Target		Visual Target
Setting	Acc	Prec.	Rec.	F1	EM	F1	AP	AP50	AP75
GRASP	78.50	77.11	90.12	83.11	43.16	35.04	41.31	45.84	40.74
Equal-weight	78.66	79.67	85.46	82.46	43.32	32.66	39.81	43.83	39.36
Acc-heavy	75.12	77.78	80.66	79.19	42.67	31.20	36.46	40.30	35.85
Format-enhanced	76.73	76.51	87.11	81.46	40.18	32.29	39.22	43.33	38.88

Appendix

Table 7. Effect of FTPO reward weight allocation on GRASP (Qwen3-VL-4B). Larger weights on the grounding-oriented rewards

R^{\text{box}}

and

R^{\text{txt}}

yield the best overall trade-off across MSD and MSTI metrics.

Sarcasm Detection

See all 21 papers

SarcasmMiner: A Dual-Track Post-Training Framework for Robust Audio-Visual Sarcasm Reasoning
Mar 5, 2026Zhu Li, Yongjian Chen, Huiyuan Lai +3Sarcasm Detection Multimodal Reasoning
Speech Technology Lab, University of Groningen, The Netherlands · Center for Language and Cognition, University of Groningen, The Netherlands
Dynamic Gated Cross-Modal Fusion with Sarcastic-aware Contrastive Regularization for Multimodal Sarcasm Detection
Aug 20, 2026Hao Guo, Subin Huang, Junjie Chen +3Multimodal Fusion
School of Computer and Information Anhui Polytechnic University Wuhu, China · Bigdata and Responsible Artificial Intelligence for National Governance, Renmin University of China, Beijing 100872, China
Focus on What You Need: Global-to-Local Dynamic Attention for Multimodal Sarcasm Detection
Oct 13, 2026Dehao Wu, Xingjie ZhuangSarcasm Detection Cross-Modal Attention
Xiamen University

Mar 5, 2026J/K move · Enter open · S save

SarcasmMiner: A Dual-Track Post-Training Framework for Robust Audio-Visual Sarcasm Reasoning

Zhu Li, Yongjian Chen, Huiyuan Lai, Xiyuan Gao+2

Speech Technology Lab, University of Groningen, The Netherlands · Center for Language and Cognition, University of Groningen, The Netherlands

Sarcasm Detection Multimodal Reasoning Multimodal Robustness RL for Language Model Reasoning Multimodal Reward Modeling

Open paper AskPDF arXiv

Multimodal sarcasm detection requires resolving pragmatic incongruity across textual, acoustic, and visual cues through cross-modal reasoning. To enable robust sarcasm reasoning with foundation models, we propose SarcasmMiner, a reinforcement learning based post-training framework that resists hallucination in multimodal reasoning. We reformulate sarcasm detection as structured reasoning and adopt a dual-track distillation strategy: high-quality teacher trajectories initialize the student model, while the full set of trajectories trains a generative reward model (GenRM) to evaluate reasoning quality. The student is optimized with group relative policy optimization (GRPO) using decoupled rewards for accuracy and reasoning quality. On MUStARD++, SarcasmMiner increases F1 from 59.83% (zero-shot), 68.23% (supervised finetuning) to 70.22%. These findings suggest that reasoning-aware reward modeling enhances both performance and multimodal grounding.

Abstract

Sarcasm Detection

Explore similar work

SarcasmMiner: A Dual-Track Post-Training Framework for Robust Audio-Visual Sarcasm Reasoning

Dynamic Gated Cross-Modal Fusion with Sarcastic-aware Contrastive Regularization for Multimodal Sarcasm Detection

Focus on What You Need: Global-to-Local Dynamic Attention for Multimodal Sarcasm Detection