Beyond Hate: Differentiating Uncivil and Intolerant Speech in Multimodal Content Moderation

Nils A. Herrmann; Tobias Eder; Jingyi He; Georg Groh

cs.CLMar 24, 2026

Beyond Hate: Differentiating Uncivil and Intolerant Speech in Multimodal Content Moderation

Authors: Nils A. Herrmann, Tobias Eder, Jingyi He, Georg Groh

Organizations: Technical University of Munich

Abstract

Current multimodal toxicity benchmarks typically use a single binary hatefulness label. This coarse approach conflates two fundamentally different characteristics of expression: tone and content. Drawing on communication science theory, we introduce a fine-grained annotation scheme that distinguishes two separable dimensions: incivility (rude or dismissive tone) and intolerance (content that attacks pluralism and targets groups or identities) and apply it to 2,030 memes from the Hateful Memes dataset. We evaluate different vision-language models under coarse-label training, transfer learning across label schemes and a joint learning approach that combines the coarse hatefulness label with our fine-grained annotations. Our results show that fine-grained annotations complement existing coarse labels and, when used jointly, improve overall model performance. Moreover, models trained with the fine-grained scheme exhibit more balanced moderation-relevant error profiles and are less prone to under-detection of harmful content than models trained on hatefulness labels alone (FNR-FPR, the difference between false negative and false positive rates: 0.74 to 0.42 for LLaVA-1.6-Mistral-7B; 0.54 to 0.28 for Qwen2.5-VL-7B). This work contributes to data-centric approaches in content moderation by improving the reliability and accuracy of moderation systems through enhanced data quality. Overall, combining both coarse and fine-grained labels provides a practical route to more reliable multimodal moderation.

Figures & tables

Figure 1: Example memes from the hateful meme dataset that illustrate the concept of ’benign confounders’. Image above is a compilation of assets, including ©Getty Images.

Annotation type	Labels	Count	% of original dataset
Coarse single-label	Hateful vs. Neutral	9664	100%
Fine-grained multi-label	Intolerant vs. Tolerant Uncivil vs. Civil	2030	21%

Table 1: Two annotation schemes used in this study, with their respective coverage in the dataset.

	Incivility	Intolerance
Sr. 1 vs. Sr. 2	0.88 (0.77)	0.90 (0.78)
Sr. 1 vs. Jr. 1	0.71 (0.39)	0.67 (0.37)
Sr. 2 vs. Jr. 1	0.69 (0.38)	0.69 (0.42)

Table 2: Agreement between senior annotators (Sr. 1 and Sr. 2) and junior annotator (Jr. 1). Table shows share of labels that agree between annotators with Cohen’s

\kappa

in brackets.

	Hateful	Intolerant	Uncivil
Original (coarse)	0.37	-	-
Our (fine-grained)	0.35	0.37	0.44

Table 3: Marginal distributions in the original dataset and our fine-grained subset.

	Tolerant	Intolerant
Civil	0.50	0.06
Uncivil	0.13	0.31

Table 4: Joint distribution of incivility and intolerance labels in our fine-grained subset.

Figure 2: Evaluation pipeline. A meme–text pair is passed to the VLM (e.g. LLaVA-1.6) along with a prompt asking whether the content is “Hateful” or “Neutral”. The model generates a binary classification based on both visual and textual input. Image above is a compilation of assets, including ©Getty Images.

			LLaVA-1.6-Mistral-7B		Qwen2.5-VL-7B		GPT-5.1
Setup	Finetune Split	Evaluation Split	Acc.	F1	Acc.	F1	Acc.	F1
Zero-shot	-	Hateful	0.68	0.68	0.68	0.68	0.80	0.80
	-	Uncivil	0.61	0.60	0.56	0.51	0.74	0.74
	-	Intolerant	0.49	0.46	0.58	0.56	0.81	0.81
Baseline	Hateful	Hateful	0.73	0.67	0.77	0.74	-	-
	Uncivil/Intolerant	Uncivil	0.78	0.77	0.77	0.76	-	-

Table 5: Multi-model comparison for accuracy (Acc.) and weighted F1 score (F1) across all learning setups. Best results per model and evaluation split are highlighted in bold.

Model	Task	FPR	FNR	FNR-FPR
GPT-5.1	Hateful	0.22	0.17	-0.05
GPT-5.1	Intolerance	0.21	0.16	-0.05
LLaVA-1.6-Mistral-7B	Hateful	0.01	0.75	0.74
LLaVA-1.6-Mistral-7B	Intolerance	0.09	0.52	0.42
Qwen2.5-VL-7B	Hateful	0.04	0.58	0.54
Qwen2.5-VL-7B	Intolerance	0.09	0.36	0.28

Table 6: False positive rate (FPR) and false negative rate (FNR) across models and label granularity when detecting harmful content. Error asymmetry (FNR-FPR) is smaller with the fine-grained annotation scheme.

Operationalization	Match rate	Correlation
Hateful = Intolerant	0.89	0.76
Hateful = Intolerant AND Uncivil	0.88	0.73
Hateful = Intolerant OR Uncivil	0.81	0.65
Hateful = Uncivil	0.80	0.59

Table 7: Comparison of alternative operationalizations relating hatefulness to incivility and intolerance. Match rate indicates the proportion of cases where the label implied by a given operationalization matches the original hatefulness label. Correlation is the phi coefficient (equivalent to Pearson’s

r

for binary variables). All associations are statistically significant according to a chi-squared test of independence at

\alpha=0.001

Category	n	P(hateful)	$\phi$
Incivility
Attacks	106	0.91	0.27
Vulgar	537	0.60	0.31
Aspersions	45	0.56	0.06
Civil	1140	0.10	-0.59
Intolerance

Table 8: Empirical hatefulness probability conditioned on fine-grained categories and

\phi

coefficient (majority vote aggregation).

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Nr.	Form of Incivility	Text	Visual
0	Civil	Absence of incivility.	Absence of offensive, hostile, or degrading imagery.
1	Profane or vulgar	Use of explicit profanities or vulgarities, regardless of whether they are directed at a person or entity.	Images containing profanity, obscene gestures, or crude/vulgar symbolism. Sexual or violent images aimed at intimidating, shocking, or degrading public discourse.
2	Attacks	Derogatory or pejorative language directed at specific individuals or groups. These can focus on personal characteristics (e.g., appearance), traits, character, or choices.	Images targeting identifiable individuals or demographic groups with pejorative, mocking, or dehumanising intent.
3	Aspersions	Attacks targeting groups, organizations, or institutions aiming to undermine credibility, legitimacy, or moral standing.	Images delegitimising institutions, policies, or parties through ridicule, slurs, or hostile visual metaphors.

Appendix

Table 9: Subtypes of Incivility in Text and Visual Content. Used in the annotation scheme.

Nr.	Form of Intolerance	Description
0	Tolerant / Neutral	Absence of expressions of intolerance.
1	Threats to Individual Rights	Claims or images where certain groups do not have equal civil, political, or human rights.
2	Intolerance Toward Political Positions	Delegitimizing opposing political views or calling for elimination of political actors.
3	Racism	Discriminatory, stereotypical, hateful, or prejudicial speech toward racial minorities.
4	Social/Economic Intolerance	Discriminatory, stereotypical, hateful, or prejudicial speech toward others based on education level, social status, or income.
5	Gender and Sexual Intolerance	Discriminatory, stereotypical, hateful, or prejudicial speech toward women and/or LGBTQ+ individuals based on gender status or sexual orientation.

Appendix

Table 10: Subtypes of Intolerance in Text Content. Used in the annotation scheme.

Figure 3: Validation accuracy for hatefulness prediction across different shares of annotated training data.

Figure 4: Frequency of fine-grained incivility (left) and intolerance (right) labels across all annotations before aggregation. A single meme may receive multiple labels.

Setup	Model	LoRA r	LoRA $\alpha$	Dropout	Learning Rate	Weight Decay	Epochs
Coarse	Qwen2.5-VL-7B	64	128	0.0	2e-3	0.05	2
Coarse	LLaVA-v1.6-Mistral-7B	64	128	0.05	2e-4	0.05	2
Fine	Qwen2.5-VL-7B	128	256	0.1	2e-3	0.05	2
Fine	LLaVA-v1.6-Mistral-7B	64	128	0.05	5e-4	0.0	2
Joint	Qwen2.5-VL-7B	64	128	0.0	2e-3	0.05	3
Joint	LLaVA-v1.6-Mistral-7B	128	256	0.05	5e-4	0.1	2

Appendix

Table 11: Hyperparameters for all models across training setups.

			LLaVA-1.6-Mistral-7B		Qwen2.5-VL-7B
Setup	Finetune Split	Evaluation Split	Acc.	IPR	Acc.	IPR
Coarse	Hateful	Hateful	0.64 ( $\pm$ 0.05)	0.00 ( $\pm$ 0.00)	0.65 ( $\pm$ 0.15)	0.15 ( $\pm$ 0.02)
Fine	Uncivil/Intolerant	Uncivil	0.75 ( $\pm$ 0.02)	0.00 ( $\pm$ 0.00)	0.81 ( $\pm$ 0.03)	0.05 ( $\pm$ 0.07)
Fine	Uncivil/Intolerant	Intolerant	0.77 ( $\pm$ 0.02)	0.00 ( $\pm$ 0.00)	0.76 ( $\pm$ 0.08)	0.05 ( $\pm$ 0.07)
Joint	Uncivil/Intolerant/Hateful	Hateful	0.73 ( $\pm$ 0.03)	0.01 ( $\pm$ 0.01)	0.68 ( $\pm$ 0.09)	0.12 ( $\pm$ 0.01)
		Uncivil	0.75 ( $\pm$ 0.04)		0.77 ( $\pm$ 0.02)

Appendix

Table 12: Average validation accuracy (Acc.) and invalid prediction rate (IPR) across models and training setups using 5 different seeds. Standard deviation reported in brackets.

Category	Count	Share
Civil	3552	0.58
Vulgar	1623	0.27
Attacks	603	0.10
Aspersions	323	0.05
Total	6101	1.00

Appendix

Table 13: Distribution of incivility categories in the dataset.

Category	Count	Share
Tolerant	3463	0.52
Threats to Rights	56	0.01
Political Intolerance	255	0.04
Racism	815	0.12
Social Intolerance	25	0.00
Gender Intolerance	615	0.09

Appendix

Table 14: Distribution of intolerance categories in the dataset.

Figure 5: Hatefulness empirical probability conditioned on each category. Error bars represent the Wilson Score Interval at 0.05 significance level.

Figure 6: Odds ratios from a logistic regression of the hateful label on fine-grained incivility and intolerance category probabilities. Points to the right of the dashed line ( $\mathrm{OR}>1$ ) indicate a positive association with hatefulness. Error bars denote 95% confidence intervals.

Multimodal Content Moderation

See all 21 papers

Mar 23, 2026

Is AI Ready for Multimodal Hate Speech Detection? A Comprehensive Dataset and Benchmark Evaluation

Hate speech online targets individuals or groups based on identity attributes and spreads rapidly, posing serious social risks. Memes, which combine images and text, have emerged as a nuanced vehicle for disseminating hate speech, often relying on cultural knowledge for interpretation. However, existing multimodal hate speech datasets suffer from coarse-grained labeling and a lack of integration with surrounding discourse, leading to imprecise and incomplete assessments. To bridge this gap, we propose an agentic annotation framework that coordinates seven specialized agents to generate hierarchical labels and rationales. Based on this framework, we construct M^3 (Multi-platform, Multi-lingual, and Multimodal Meme), a dataset of 2,455 memes collected from X, 4chan, and Weibo, featuring fine-grained hate labels and human-verified rationales. Benchmarking state-of-the-art Multimodal Large Language Models reveals that these models struggle to effectively utilize surrounding post context, which often fails to improve or even degrades detection performance. Our finding highlights the challenges these models face in reasoning over memes embedded in real-world discourse and underscores the need for a context-aware multimodal architecture. Our dataset and code are available at https://github.com/mira-ai-lab/M3.

Rui Xing, Qi Chai, Jie Ma +5

MOE KLINNS Lab, Xi’an Jiaotong University · The Hong Kong University of Science and Technology (Guangzhou) · School of Cyber Science and Engineering, Xi’an Jiaotong University +1

Hate Speech Detection Harmful Meme Detection Multimodal Model Evaluation Multimodal Large Language Models

Ask PDF

Mar 22, 2026

More Than Sum of Its Parts: Deciphering Intent Shifts in Multimodal Hate Speech Detection

Combating hate speech on social media is critical for securing cyberspace, yet relies heavily on the efficacy of automated detection systems. As content formats evolve, hate speech is transitioning from solely plain text to complex multimodal expressions, making implicit attacks harder to spot. Current systems, however, often falter on these subtle cases, as they struggle with multimodal content where the emergent meaning transcends the aggregation of individual modalities. To bridge this gap, we move beyond binary classification to characterize semantic intent shifts where modalities interact to construct implicit hate from benign cues or neutralize toxicity through semantic inversion. Guided by this fine-grained formulation, we curate the Hate via Vision-Language Interplay (H-VLI) benchmark where the true intent hinges on the intricate interplay of modalities rather than overt visual or textual slurs. To effectively decipher these complex cues, we further propose the Asymmetric Reasoning via Courtroom Agent DEbate (ARCADE) framework. By simulating a judicial process where agents actively argue for accusation and defense, ARCADE forces the model to scrutinize deep semantic cues before reaching a verdict. Extensive experiments demonstrate that ARCADE significantly outperforms state-of-the-art baselines on H-VLI, particularly for challenging implicit cases, while maintaining competitive performance on established benchmarks. Our code and data are available at: https://github.com/Sayur1n/H-VLI

Runze Sun, Yu Zheng, Zexuan Xiong +4

Department of Automation, Tsinghua University

Hate Speech Detection Multimodal Reasoning

Ask PDF

Jun 27, 2026

Majority Vote Silences Minority Values: Annotator Disagreement at the Hate/Offensive Boundary in HateXplain

Hate speech annotation pipelines routinely collapse annotator disagreement into majority vote labels before training. We show that this aggregation is not neutral: 42.6% of all annotator disagreement in HateXplain concentrates specifically at the hate/offensive boundary, a pattern consistent with annotators applying different thresholds for where hate begins (chi-squared = 135.199, df = 2, p < 0.0001). Both a hard-label BERT model (Model A) and a soft-label model (Model B) drop 22 percentage points in accuracy from agreed posts (~80%) to disagreement posts (~58%), confirmed at p < 0.0001. A per-annotator multi-head model (Model C) widens this gap further to 28 points while collapsing offensive disagreement accuracy to 0.245. Critically, Model A expresses significantly higher confidence on boundary case errors than Model C (0.710 vs. 0.495, p < 0.0001), meaning standard evaluation metrics will not detect the failure. Three downstream interventions of increasing sophistication all fail to recover boundary accuracy. We argue the problem is structural. Majority vote presents a contested judgment as ground truth, and models inherit that false certainty. The intervention must be upstream in annotation design.

Joshua Muhumuza, Joab Ezra Agaba, Mercy Amiyo

Makerere University, Kampala, Uganda.

Annotator Disagreement Hate Speech Detection Majority Voting

Ask PDF

Abstract

Multimodal Content Moderation

Explore similar work

Is AI Ready for Multimodal Hate Speech Detection? A Comprehensive Dataset and Benchmark Evaluation

More Than Sum of Its Parts: Deciphering Intent Shifts in Multimodal Hate Speech Detection

Majority Vote Silences Minority Values: Annotator Disagreement at the Hate/Offensive Boundary in HateXplain