MexHat: A Dataset for Hate Speech Detection in Mexican Spanish Videos

Itzel Tlelo-Coyotecatl; Hugo Jair Escalante

cs.CLSep 25, 2026

MexHat: A Dataset for Hate Speech Detection in Mexican Spanish Videos

Authors: Itzel Tlelo-Coyotecatl, Hugo Jair Escalante

Organizations: INAOE, Puebla, Mexico · The University of Texas at El Paso, El Paso, TX, USA

Abstract

Ensuring online safety through content monitoring had raised Hate Speech Detection as a crucial task to be addressed. By essence the task demands the capture of contextual cues, which are essential for a precise understanding of the content's intent. Although automated detection approaches for the task have advanced significantly, the scarcity of non-English resources persists, limiting the ability of models to adapt to the subtle, context-dependent, and culturally related nature of multimodal content. In this paper, we introduce MexHat, a video dataset designed to capture the linguistic and cultural cues for the hate-speech detection task in a Mexican Spanish context. Our dataset comprises around 1k video clips annotated across two tasks: a three-way class evaluation (no negative content, offensive content and hate-speech content), and a fine-grained class evaluation including three hate-speech sub-categories. The dataset statistics and the baseline results highlight the inherent challenges associated with the task. Disclaimer: This paper contains sensitive content that may be disturbing to some readers.

Figures & tables

Panel of Figure 1 — Figure 1: Overview of the annotation process

Panel of Figure 2 — Figure 2: Annotator demographics.

Figure 3: Number of videos with disagreement at annotation stage.

Panel of Figure 4 — Figure 4: Wordclouds from test partition transcriptions.

Panel of Figure 5 — Figure 5: Examples of videos from classes NN : no negative content, OF : offensive content, and HS : hate-speech content.

Modality	$n_{min}$	$n_{max}$	$n_{avg}$	$n_{median}$	$n_{std}$	$D$
Audio	2	69	59.27	62	10	128
Text	2	488	177.36	186	73.23	768
Video	0	31	24.88	27	5.25	1024+1024

Table 1: Statistics for each modality. Each video corresponds to a

n\times D

representation.

Partition	Number of video clips	NN	OF	HS	hs1	hs2	hs3
test	278	116	123	39	6	15	18
train	824	394	314	116	28	47	41
val	276	130	99	47	10	9	28
ALL	1378	640	536	202	44	71	87

Table 2: Final dataset partitions. For the three-class labeling column NN : no negative content, OF : offensive content and HS : hate-speech content. For the fine-grained labeling, if a video clip belongs to the HS class the subgroup is delimited by hs1 : racism, xenophobic factors, hs2 : sexism and LGBTQ+ related factors; and hs3 : others.

Panel of Figure 6 — Figure 6: T-SNE visualization of data points for the test partition. Each point is a vector from each video representation and the color represents its associated class where NN: green, OF: yellow and HS: red; and $+$ refers to concatenation.

Modality	Model	Acc	F1-macro	F1-NN	F1-OF	F1-HS
T	MLP	0.543	0.391	0.599	0.574	0
V	MLP	0.468	0.363	0.533	0.475	0.080
A	MLP	0.450	0.329	0.470	0.516	0
T + A	MLP	0.489	0.353	0.531	0.528	0
V + T	MLP	0.435	0.335	0.523	0.407	0.075
A + T	MLP	0.568	0.423	0.603	0.616	0.049

Table 3: Result from evaluating the unimodal, bimodal and multimodal representiations for the three-class classification.

Modality	Model	Acc	F1-macro	F1-NN	F1-OF	F1-hs1	F1-hs2	F1-hs3
T	MLP	0.518	0.223	0.586	0.527	0	0	0
V	MLP	0.442	0.227	0.525	0.419	0	0.105	0.087
A	MLP	0.475	0.225	0.461	0.554	0	0.111	0
T + A	MLP	0.522	0.247	0.568	0.549	0	0.118	0
V + A	MLP	0.482	0.242	0.527	0.516	0	0	0.167
A + T	MLP	0.496	0.214	0.536	0.531	0	0	0

Table 4: Result from evaluating the unimodal, bimodal and multimodal representiations for the fine-grained class classification.

Figure 7: Video instance (0tZ5s2W6nZY.05) wrongly classified.

Figure 8: Video instance (0tZ5s2W6nZY.05) wrongly classified.

Hate Speech Detection

Hate Speech Detection identifies hateful or abusive language, its targets and intent, to support safer and more accountable content moderation.

See all 100 papers

Is AI Ready for Multimodal Hate Speech Detection? A Comprehensive Dataset and Benchmark Evaluation
Mar 23, 2026Rui Xing, Qi Chai, Jie Ma +5Hate Speech Detection Harmful Meme Detection
MOE KLINNS Lab, Xi’an Jiaotong University · The Hong Kong University of Science and Technology (Guangzhou) · School of Cyber Science and Engineering, Xi’an Jiaotong University +1
ParsHate: A Benchmark Dataset for Hate and Target Detection in Persian
Sep 14, 2026Zahra Bokaei, Walid Magdy, Bonnie WebberHate Speech Detection
School of Informatics University of Edinburgh
AHA-Memes: A Fine-Grained Multimodal Benchmark for Understanding Hate in Arabic Memes
Jul 29, 2026Mohamed Bayan Kmainasi, Ali Ezzat Shahroor, Abul Hasnat +3Arabic NLP Hate Speech Detection
Qatar Computing Research Institute, Qatar · 2APAVI.AI, France · 3Hamad Bin Khalifa University, Qatar +1

Mar 23, 2026J/K move · Enter open · S save

Is AI Ready for Multimodal Hate Speech Detection? A Comprehensive Dataset and Benchmark Evaluation

Rui Xing, Qi Chai, Jie Ma, Jing Tao+4

MOE KLINNS Lab, Xi’an Jiaotong University · The Hong Kong University of Science and Technology (Guangzhou) · School of Cyber Science and Engineering, Xi’an Jiaotong University+1

Hate Speech Detection Harmful Meme Detection Multimodal Model Evaluation Multimodal Large Language Models

Open paper AskPDF arXiv

Hate speech online targets individuals or groups based on identity attributes and spreads rapidly, posing serious social risks. Memes, which combine images and text, have emerged as a nuanced vehicle for disseminating hate speech, often relying on cultural knowledge for interpretation. However, existing multimodal hate speech datasets suffer from coarse-grained labeling and a lack of integration with surrounding discourse, leading to imprecise and incomplete assessments. To bridge this gap, we propose an agentic annotation framework that coordinates seven specialized agents to generate hierarchical labels and rationales. Based on this framework, we construct M^3 (Multi-platform, Multi-lingual, and Multimodal Meme), a dataset of 2,455 memes collected from X, 4chan, and Weibo, featuring fine-grained hate labels and human-verified rationales. Benchmarking state-of-the-art Multimodal Large Language Models reveals that these models struggle to effectively utilize surrounding post context, which often fails to improve or even degrades detection performance. Our finding highlights the challenges these models face in reasoning over memes embedded in real-world discourse and underscores the need for a context-aware multimodal architecture. Our dataset and code are available at https://github.com/mira-ai-lab/M3.

Abstract

Hate Speech Detection

Explore similar work

Is AI Ready for Multimodal Hate Speech Detection? A Comprehensive Dataset and Benchmark Evaluation

ParsHate: A Benchmark Dataset for Hate and Target Detection in Persian

AHA-Memes: A Fine-Grained Multimodal Benchmark for Understanding Hate in Arabic Memes