More Than Sum of Its Parts: Deciphering Intent Shifts in Multimodal Hate Speech Detection

Runze Sun; Yu Zheng; Zexuan Xiong; Zhongjin Qu; Lei Chen; Jie Zhou; Jiwen Lu

cs.CLMar 22, 2026

More Than Sum of Its Parts: Deciphering Intent Shifts in Multimodal Hate Speech Detection

Authors: Runze Sun, Yu Zheng, Zexuan Xiong, Zhongjin Qu, Lei Chen, Jie Zhou, Jiwen Lu

Organizations: Department of Automation, Tsinghua University

Abstract

Combating hate speech on social media is critical for securing cyberspace, yet relies heavily on the efficacy of automated detection systems. As content formats evolve, hate speech is transitioning from solely plain text to complex multimodal expressions, making implicit attacks harder to spot. Current systems, however, often falter on these subtle cases, as they struggle with multimodal content where the emergent meaning transcends the aggregation of individual modalities. To bridge this gap, we move beyond binary classification to characterize semantic intent shifts where modalities interact to construct implicit hate from benign cues or neutralize toxicity through semantic inversion. Guided by this fine-grained formulation, we curate the Hate via Vision-Language Interplay (H-VLI) benchmark where the true intent hinges on the intricate interplay of modalities rather than overt visual or textual slurs. To effectively decipher these complex cues, we further propose the Asymmetric Reasoning via Courtroom Agent DEbate (ARCADE) framework. By simulating a judicial process where agents actively argue for accusation and defense, ARCADE forces the model to scrutinize deep semantic cues before reaching a verdict. Extensive experiments demonstrate that ARCADE significantly outperforms state-of-the-art baselines on H-VLI, particularly for challenging implicit cases, while maintaining competitive performance on established benchmarks. Our code and data are available at: https://github.com/Sayur1n/H-VLI

Figures & tables

Panel of Figure 1 — Figure 1: Examples of implicit multimodal hate speech.

Figure 2: The construction pipeline of our H-VLI dataset. This process employs a hybrid strategy that combines consensus filtering of real-world samples with a generative injection path to ensure diversity. It utilizes the SMI paradigm to systematically categorize samples into eight distinct interaction patterns, which are further stratified into three difficulty levels (Easy, Normal, and Hard) based on the semantic interplay between modalities.

Difficulty	Pattern ( $y^{Text},y^{Image}\to y^{Combined}$ )	Interaction Mechanism
Easy	000, 011, 101, 111	Semantic Consistency
Normal	100, 010	Contextual Neutralization
Hard	001, 110	Implicit Emergence / Inversion

Table 1: Difficulty Stratification of Multimodal Interaction Patterns (0: NotHate, 1: Hate).

Dataset	Source	# Samples	# Cls	SMI	Diff.	Agreem. ( $\kappa$ )
MMHS150K Gomez et al. (2020)	Img+Txt	149,823	6	✗	✗	0.15 †
FHM Kiela et al. (2020)	Meme	10,000	2	✗	✗	0.68 ∗
MultiOFF Suryawanshi et al. (2020)	Meme	743	2	✗	✗	0.4 – 0.5 †
Harm-C Pramanick et al. (2021a)	Meme	3,544	3/4	✗	✗	0.68 / 0.78 ∗
MAMI Fersini et al. (2022a)	Meme	11,000	2/5	✗	✗	0.58 / 0.34 †
H-VLI (Raw)	Img+Txt	8,438	6	✓	✓	0.59 †

Table 2: of our dataset (H-VLI) with existing multimodal hate speech datasets. H-VLI (Raw) represents data before consistency filtering.

Figure 3: The architecture of the ARCADE framework. The model simulates an asymmetric courtroom debate involving Prosecutor and Defender agents. It features a Gated Dual-Track mechanism that routes samples to either a Fast-Track for explicit hate or a Deep-Dive trial for implicit reasoning, allowing the Judge agent to synthesize the debate history and render a final verdict.

Step	Filter	Discarded	Remaining
–	Initial Candidates	–	8,802
1	Quality Control	364	8,438
2	Consensus Filtering	2,593	5,845
3	Intent Alignment	276	5,569

Table 3: Post-annotation filtering statistics.

Split	Total	Hate	Non-Hate
Train	4,391	1,533(34.9%)	2,858(65.1%)
Test	1,178	462(39.2%)	716(60.8%)
Overall	5,569	1,995(35.8%)	3,574(64.2%)

Table 4: Distribution of hate vs. non-hate samples in the H-VLI benchmark.

Model	Training	Task 1: Fine-grained Hate Categorization						Task 2: Binary Hate Detection
		Easy	Normal	Hard	Overall			Easy	Normal	Hard	Overall
		Acc	Acc	Acc	Acc	Mac-F1	W-F1	Acc	Acc	Acc	Acc	Recall	F1
BERT Devlin et al. (2019)	✓	65.83	72.66	29.59	62.31	57.40	63.03	73.33	85.47	39.64	71.48	68.83	65.43
ViT-b-16 Dosovitskiy (2021)	✓	41.94	68.86	37.28	47.88	14.52	43.32	48.89	61.94	55.62	53.06	40.69	40.47
BERT+ViT	✓	68.33	77.51	33.73	65.62	61.25	66.19	75.83	84.78	38.46	72.67	71.00	67.08
MMBT Kiela et al. (2019)	✓	68.61	80.28	33.14	66.38	62.17	66.99	74.44	88.58	36.69	72.50	68.18	66.04

Table 5: Results on our H-VLI dataset. Bold indicates the best, underline indicates the second best. (Gray rows indicate trained models; bold/underline highlighting only applies to training-free models. Rows highlighted in color indicate models enhanced with our framework.)

Model	Training	Acc	Recall	F1
Text BERT Devlin et al. (2019)	✓	57.12	-	-
Image Region Kiela et al. (2020)	✓	52.34	-	-
CLIP-BERT Pramanick et al. (2021b)	✓	58.28	-	-
DisMultiHate Pramanick et al. (2021a)	✓	62.42	-	-
ViLBERT CC Sharma et al. (2018)	✓	64.70	-	-
MMBT-Region Kiela et al. (2019)	✓	65.06	-	-

Table 6: Results on the FHM Dataset for Binary Hate Detection. Bold and underline indicate the best and the second best results. Gray rows indicate training-based models. Rows highlighted in blue indicate models enhanced with our framework. Models marked with † utilize additional knowledge via Retrieval-Augmented Generation (RAG) in a few-shot setting.

Model	Training	Acc	Recall	F1
Qwen3-VL-Plus	✗	69.46	67.92	68.56
Qwen3-VL-Plus w/ Ours	✗	73.82	77.87	74.45
GPT-5-mini	✗	75.80	66.33	72.87
GPT-5-mini w/ Ours	✗	77.12	82.29	77.91

Table 7: Results on the FHM test set for Binary Hate Detection.

Method	On H-VLI		On FHM
Method	Acc	Mac-F1	Acc	F1
Baseline	70.61	69.42	69.77	64.56
Multiround †	74.47	71.43	72.23	74.47
Ours ( $K=1$ )	78.03	73.80	73.28	73.98
Ours ( $K=2$ )	77.71	73.07	-	-
Ours ( $K=3$ )	-	-	71.93	74.67

Table 8: Results of ablation studies. Bold indicates the best, underlined indicates the second best. Note that for our default settings: † Multiround uses

K=3

for Our Dataset and

K=2

for FHM. ‡ Ours uses

K=3

for Our Dataset and

K=2

for FHM. The notation ‘Model A

\to

Model B’ signifies using Model A as the Auxiliary (Prosecutor/Defender) and Model B as the Judge.

Setting	Pre-Filter	Post-Filter
With hints	$\kappa=0.59$	$\kappa=0.94$
Without hints	$\kappa=0.57$	$\kappa=0.94$

Table 9: Inter-annotator agreement comparison.

Type	Total	Track I	Track II	Dismiss
000	361	31(8.6%)	156(43.2%)	174(48.2%)
001	99	60(60.6%)	39(39.4%)	0(0.0%)
010	169	149(88.2%)	20(11.8%)	0(0.0%)
011	40	39(97.5%)	1(2.5%)	0(0.0%)
100	119	98(82.4%)	21(17.6%)	0(0.0%)
101	126	117(92.9%)	9(7.1%)	0(0.0%)

Table 10: Sample routing statistics of

\Phi(S)

on H-VLI using Qwen3-VL-Plus.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Label	Hate Category
0	NotHate
1	Racist
2	Sexist
3	Homophobic
4	Religious Hate
5	OtherHate (e.g., disability, age)

Appendix

Table 11: Hate category definitions

Panel of Figure 4 — Figure 4: The prompt template used for generating contextual inversion samples.

Panel of Figure 5 — Figure 5: Showcase of representative samples from the H-VLI dataset. Type 001 represents implicit hate where benign components construct malice. Type 110 shows benign intent despite toxic modalities. Type 010/100 demonstrate contextual neutralization where text or image corrects the other’s toxicity.

Model	Fine-grained Task		Binary Task
Model	Ref.#	Ref.%	Ref.#	Ref.%
Qwen-VL-Plus Bai et al. (2023b)	4	0.34	6	0.51
Qwen-VL-Plus w/ Ours	10	0.85	11	0.93
Qwen-VL-Max Bai et al. (2023b)	4	0.34	6	0.51
Qwen-VL-Max w/ Ours	10	0.85	8	0.68
GPT-4.1-mini OpenAI et al. (2024)	0	0.00	0	0.00

Appendix

Table 12: Safety refusal analysis of MLLMs. Ref.# denotes the number of refused samples, and Ref.% denotes the refusal rate (percentage) over the test set.

Category	SMI	Baseline	Ours	$\boldsymbol{\Delta}$
Overall Statistics
Total Count	–	261	77	-184
Refusal Rate	–	22.16%	6.54%	-15.62%
Detailed Statistics
Easy	000	1	1	0
	011	4	2	-2

Appendix

Table 13: Comparison of refusal counts between the Baseline and our ARCADE framework with the backbone model GPT-5 OpenAI (2024) on Task I.

	Case I: Contextual Neutralization	Case II: Implicit Metaphor
Multimodal Input	Text: “THIS ISN’T ‘PROTECTION,’ YOU MORONS. IT’S JUST VIOLENCE WRAPPED IN HATE SPEECH… #TransRightsAreHumanRights” GT Label: NotHate	Text: “Crying inside… weeds in the white roses again. emoji(sob) emoji(herb)” GT Label: Hate Speech (Racist) (Metaphorical insinuations of racial superiority)
Baseline (GPT-4o)	Prediction: Hate Speech (Homophobic) Reasoning: The tweet and image imply violence towards individuals … specifically targeting transgender individuals under the guise of protecting women.	Prediction: NotHate Reasoning: The tweet appears to be expressing a metaphorical sentiment about roses and weeds without indicating hate speech.
Ours (ARCADE)	Gatekeeper: Explicit threat cues detected (“he’ll crawl out…”) $\rightarrow$ Trigger Fast-Track Trial . Prosecutor (Cue): The phrase equates trans women with predatory males… creating a dangerous narrative. Defender (Rebuttal): The tweet explicitly calls this “VIOLENCE WRAPPED IN HATE SPEECH”. The user is quoting the meme to condemn it. Judge Verdict: NotHate . Judge Reason: The tweet explicitly condemns the hateful rhetoric and violence depicted in the meme , labeling it as ’VIOLENCE WRAPPED IN HATE SPEECH’ and … The hashtag ’#TransRightsAreHumanRights’ further reinforces the author’s support for transgender rights…	Gatekeeper: No explicit slurs detected $\rightarrow$ Trigger Deep-Dive Trial . Prosecutor (R1): Metaphor: The tweet equates non-white people with invasive, undesirable ’weeds’ disrupting the purity or beauty of ’white roses’ … Defender (R1): The tweet contains no explicit reference to race, ethnicity, or skin color. The text is a generic botanical metaphor; the image shows happy, harmonious people. (…Debate continues for 2 more rounds…) Judge Verdict: Hate Speech (Racist) . Judge Reason: “White roses” symbolize racial purity ; “weeds” imply unwanted elements, creating a dehumanizing analogy. The juxtaposition signals implicit hate speech through cultural coding.

Appendix

Table 14: Qualitative Comparison between Baseline and ARCADE. Both cases are from H-VLI. Red text indicates incorrect predictions or reasoning; Green text indicates correct reasoning.

	Case III: Knowledge Gap (Slang)	Case IV: Misaligned Reasoning
Multimodal Input	Text: “someone added to much soap to the dishwasher again” GT Label: Hate Speech (Sexist) (Implies women are objects/dishwashers)	Text: “This might be appealing to certain Muslims out there. emoji(earth)” GT Label: Hate Speech (Religious) (Insinuations of bestiality)
Baseline (GPT-4o)	Prediction: NotHate Reasoning: The model fails to identify the offensive metaphor, interpreting the statement literally as a complaint regarding a household appliance.	Prediction: Hate Speech (Religious) Reasoning: The tweet associates Muslims with camels in a way that could be interpreted as stereotyping or demeaning based on religion. (Vague reasoning)
Ours (ARCADE)	Gatekeeper: No explicit slurs detected $\rightarrow$ Trigger Deep-Dive Trial . Prosecutor (Investigation): [Failed to identify implicit mapping] . The agent failed to recognize “dishwasher” as a derogatory slang for women in this context. Verdict: NotHate (Summary Dismissal).	Gatekeeper: No explicit slurs detected $\rightarrow$ Trigger Deep-Dive Trial . Prosecutor (R1): Orientalist Trope: The tweet implies Muslims are best understood through primitive or animalistic symbols (camels) , reducing identity to desert fauna. Defender (R1): The image presents biological facts about camels; “certain Muslims” is speculative and contextually neutral. (…Debate continues for 2 more rounds…) Judge Verdict: Hate Speech (Religious) . Judge Reason: The content constructs a dehumanizing narrative by equating Muslims with backwardness and primitive symbols .

Appendix

Table 15: Failure Case Analysis. We analyze typical error patterns: Case III (from FHM) shows a False Negative where the model missed specific internet slang (Objectification); Case IV (from H-VLI) shows a case with the Correct Label but Misaligned Reasoning . Red text indicates incorrect predictions or reasoning; Green text indicates correct reasoning.

Figure 6: The user interface of our custom-built annotation platform. The layout is designed to strictly follow the SMI paradigm, guiding annotators from unimodal model-assisted suggestions (on the right) to final multimodal categorization.

Hate Speech Detection

Hate Speech Detection identifies hateful or abusive language, its targets and intent, to support safer and more accountable content moderation.

See all 98 papers

Beyond Hate: Differentiating Uncivil and Intolerant Speech in Multimodal Content Moderation
Mar 24, 2026Nils A. Herrmann, Tobias Eder, Jingyi He +1Multimodal Content Moderation Hate Speech Detection
Technical University of Munich
Is AI Ready for Multimodal Hate Speech Detection? A Comprehensive Dataset and Benchmark Evaluation
Mar 23, 2026Rui Xing, Qi Chai, Jie Ma +5Hate Speech Detection Harmful Meme Detection
MOE KLINNS Lab, Xi’an Jiaotong University · The Hong Kong University of Science and Technology (Guangzhou) · School of Cyber Science and Engineering, Xi’an Jiaotong University +1
CLARA: Clip-Level Multimodal Alignment with VLM-Derived Rationales for Hateful Video Detection
Aug 16, 2026Yuchen Zhang, Shuang Dai, Zeyu Fu +3Hate Speech Detection Video Content Moderation
Institute for Analytics and Data Science, University of Essex, Colchester, United Kingdom · Department of Engineering, University of Exeter, Exeter, United Kingdom · Department of Computer Science, University of Exeter, Exeter, United Kingdom +1

Mar 24, 2026J/K move · Enter open · S save

Beyond Hate: Differentiating Uncivil and Intolerant Speech in Multimodal Content Moderation

Nils A. Herrmann, Tobias Eder, Jingyi He, Georg Groh

Technical University of Munich

Multimodal Content Moderation Hate Speech Detection Harmful Meme Detection

Open paper AskPDF arXiv

Current multimodal toxicity benchmarks typically use a single binary hatefulness label. This coarse approach conflates two fundamentally different characteristics of expression: tone and content. Drawing on communication science theory, we introduce a fine-grained annotation scheme that distinguishes two separable dimensions: incivility (rude or dismissive tone) and intolerance (content that attacks pluralism and targets groups or identities) and apply it to 2,030 memes from the Hateful Memes dataset. We evaluate different vision-language models under coarse-label training, transfer learning across label schemes and a joint learning approach that combines the coarse hatefulness label with our fine-grained annotations. Our results show that fine-grained annotations complement existing coarse labels and, when used jointly, improve overall model performance. Moreover, models trained with the fine-grained scheme exhibit more balanced moderation-relevant error profiles and are less prone to under-detection of harmful content than models trained on hatefulness labels alone (FNR-FPR, the difference between false negative and false positive rates: 0.74 to 0.42 for LLaVA-1.6-Mistral-7B; 0.54 to 0.28 for Qwen2.5-VL-7B). This work contributes to data-centric approaches in content moderation by improving the reliability and accuracy of moderation systems through enhanced data quality. Overall, combining both coarse and fine-grained labels provides a practical route to more reliable multimodal moderation.

Abstract

Hate Speech Detection

Explore similar work

Beyond Hate: Differentiating Uncivil and Intolerant Speech in Multimodal Content Moderation

Is AI Ready for Multimodal Hate Speech Detection? A Comprehensive Dataset and Benchmark Evaluation

CLARA: Clip-Level Multimodal Alignment with VLM-Derived Rationales for Hateful Video Detection