When Does Hate Transfer? A Large-Scale Study of Cross-Lingual Hate Speech Detection

Zahra Bokaei; Walid Magdy; Bonnie Webber

cs.CLOct 19, 2026

When Does Hate Transfer? A Large-Scale Study of Cross-Lingual Hate Speech Detection

Authors: Zahra Bokaei, Walid Magdy, Bonnie Webber

Organizations: School of Informatics, University of Edinburgh

Abstract

Effective detection of hate content is essential for maintaining healthy online communities, yet performance can vary substantially across languages. This work investigates a core practical question: Can training data from one language support hate-speech detection in another language with comparable performance? To answer this question, we conduct a large-scale study across 14 languages. \change{We observe consistent patterns in which certain language pairs transfer more effectively than others. Among the factors we measure, linguistic similarity is the strongest predictor of these patterns, with temporal overlap and dataset-size difference playing smaller roles. These measurable factors do not fully explain the observed patterns; a complementary qualitative analysis suggests that shared cultural cues and communicative norms may be associated with some of the remaining cases, particularly within several clusters. We present this cultural interpretation as a hypothesis from qualitative inspection rather than a measured cause.}{} Our analysis includes low-resource languages, offering practical insights into when cross-lingual transfer can support them and where its limitations emerge. These findings inform the design of more reliable multilingual hate-detection systems.

Figures & tables

Language	Data Size	Train	Test	Train Hate	Test Hate	Source	Period	Ref.
Bengali (BN)	30000	9007	1002	4504	501	YT, FB	2017–2020	Romim et al. (2020)
Urdu (UR)	10041	9000	1000	4500	300	X	–	Rizwan et al. (2020)
Hindi (HI)	8192	4171	467	2222	241	X, FB, WA	–	Bhardwaj et al. (2020)
Persian (FA)	7000	2060	230	1002	117	X	2020–2022	Delbari et al. (2024)
Indonesian (IN)	13169	9005	1003	4502	502	X	2018	Ibrohim and Budi (2019)
Malay (MS)	5014	3403	378	1701	189	X	2022–2023	Maity et al. (2023)

Table 1: Hate language detection datasets. FB = Facebook, YT = YouTube, X = Twitter, WA: WhatsApp, IG = Instagram, RD = Reddit. The definition of Hate and more details for each dataset are shown in Table 4

Figure 1: Cross-lingual F1 scores across 14 languages.

	TR	UR	FA	AR	HI	BN	IN	MS	EN	ES	FR	DE	KO	ZH
Solo-Source TL	90 $\pm$ 0.6	85 $\pm$ 0.6	80 $\pm$ 0.8	91 $\pm$ 0.6	85 $\pm$ 0.7	79 $\pm$ 0.7	82 $\pm$ 0.6	78 $\pm$ 0.7	96 $\pm$ 0.5	90 $\pm$ 0.6	94 $\pm$ 0.5	90 $\pm$ 0.6	75 $\pm$ 0.7	75 $\pm$ 0.7
Global-Imb	92 $\pm$ 0.3	88 $\pm$ 0.3	64 $\pm$ 0.4	87 $\pm$ 0.3	73 $\pm$ 0.4	66 $\pm$ 0.4	84 $\pm$ 0.3	62 $\pm$ 0.4	98 $\pm$ 0.2	95 $\pm$ 0.3	96 $\pm$ 0.3	94 $\pm$ 0.3	71 $\pm$ 0.4	72 $\pm$ 0.4
	[+2]	[+3]	[-16]	[-4]	[-12]	[-13]	[+2]	[-16]	[+2]	[+5]	[+2]	[+4]	[-4]	[-3]
Global-bal	91 $\pm$ 0.5	87 $\pm$ 0.5	67 $\pm$ 0.6	89 $\pm$ 0.5	81 $\pm$ 0.6	69 $\pm$ 0.6	83 $\pm$ 0.5	63 $\pm$ 0.6	96 $\pm$ 0.4	92 $\pm$ 0.5	94 $\pm$ 0.4	93 $\pm$ 0.5	73 $\pm$ 0.6	76 $\pm$ 0.6
	[+1]	[+2]	[-13]	[-2]	[-4]	[-10]	[+1]	[-15]	[0]	[+2]	[0]	[+3]	[-2]	[+1]
Observed Cluster	90 $\pm$ 0.4	90 $\pm$ 0.4	83 $\pm$ 0.5	93 $\pm$ 0.4	90 $\pm$ 0.5	83 $\pm$ 0.5	85 $\pm$ 0.4	80 $\pm$ 0.5	97 $\pm$ 0.3	96 $\pm$ 0.4	96 $\pm$ 0.4	95 $\pm$ 0.4	75 $\pm$ 0.5	78 $\pm$ 0.5

Table 2: Performance comparison across different fine-tuning strategies. Results are reported as mean

\pm

standard deviation over five random seeds. Bracketed numbers indicate improvement or decline relative to Solo-Source TL.

Factor	Spearman $\rho$	$p$ -value
Dataset size difference	-0.19	0.011
Temporal overlap	0.35	$1.6\times 10^{-4}$
Linguistic similarity	0.46	$<10^{-9}$

Table 3: Spearman correlations between transfer performance and dataset properties.

Lang	Reference	Labels	Hate definition as stated in reference
BN	Romim et al. (2020)	Hate, Not hate	A sentence that dehumanizes one or multiple persons or a community (e.g., comparing them to insects, objects, criminals, or targeting race, gender, disability).
UR	Rizwan et al. (2020)	Hate-Offensive, Normal, Abusive, Offensive, Sexism, Religious Hate, Profane	Language that expresses hatred, abuse, or offensive content such as profanity, vulgar or insulting language targeting individuals or groups; includes gender-based hatred and religion-based hatred.
HI	Bhardwaj et al. (2020)	Hostile, Non-hostile, Fake News, Hate, Offensive, Defamation, Normal	A post targeting a specific group of people based on ethnicity, religious beliefs, geographical belonging, race, etc., with malicious intentions of spreading hate or encouraging violence.
FA	Delbari et al. (2024)	Normal, Violence, Hate, Vulgar	Communication targeting individuals or groups with the intention of attacking, dehumanizing, inciting violence, promoting discrimination, or fostering hostility based on characteristics such as race, ethnicity, religion, gender, sexual orientation, political beliefs, social class, disability, or appearance.
IN	Ibrohim and Budi (2019)	Hate, Not hate	Communication that disparages a person or a group based on characteristics such as race, ethnicity, gender, sexual orientation, nationality, religion, or other attributes.
MS	Maity et al. (2023)	Hate, Not hate	Communication intended to attack the dignity of a group on the basis of characteristics such as race, gender, ethnicity, sexual orientation, nationality, religion, or other characteristics.

Table 4: Comparison of hate-speech definitions and annotation schemes across languages.

Method	Model	TR	UR	FA	AR	HI	BN	IN	MS	EN	ES	FR	DE	KO	ZH
Zero-shot	GPT-4o	71	45	55	64	67	53	65	56	82	75	77	75	60	67
	LLaMA-3-8B-Instruct	70	52	45	72	70	58	64	51	82	79	76	74	49	66
	Gemma-3-4B-Instruct	46	38	58	59	56	54	40	51	85	67	69	70	43	52
	Gemma-4-4B-Instruct	58	50	64	67	64	60	55	58	87	76	78	78	55	64
Fine-tuning	LLaMA-3-8B	90	85	80	91	85	79	82	78	96	90	94	90	75	75
	Gemma-3-4B	88	85	75	90	75	70	75	71	94	88	90	94	76	82

Table 5: F1 scores of different models for multilingual hate-speech detection.

Table 6: Hate samples correctly predicted via transfer learning across languages.

Table 7: Case study of Persian hate-speech instances correctly predicted through transfer learning from cluster languages, or misclassified in all experimental settings.

Strategy	TR	UR	FA	AR	HI	BN	IN	MS	EN	ES	FR	DE	KO	ZH
Solo-Source TL	88	85	75	90	75	70	75	71	94	88	90	94	76	82
Global-Imb	89 [+1]	87 [+2]	62 [-13]	86 [-4]	65 [-10]	60 [-10]	77 [+2]	57 [-14]	95 [+1]	92 [+4]	92 [+2]	96 [+2]	73 [-3]	79 [-3]
Global-Bal	88 [0]	86 [+1]	64 [-11]	88 [-2]	71 [-4]	62 [-8]	76 [+1]	58 [-13]	94 [0]	90 [+2]	90 [0]	95 [+1]	74 [-2]	81 [-1]
Observed Cluster	88 [0]	89 [+4]	78 [+3]	92 [+2]	79 [+4]	73 [+3]	78 [+3]	73 [+2]	95 [+1]	93 [+5]	92 [+2]	96 [+2]	76 [0]	84 [+2]

Table 8: Performance comparison across fine-tuning strategies using Gemma-3. Bracketed numbers indicate change relative to Solo-Source TL.

Strategy	TR	UR	FA	AR	HI	BN	IN	MS	EN	ES	FR	DE	KO	ZH
Solo-Source TL	92	88	82	93	84	78	84	79	97	92	95	96	81	86
Global-Imb	94 [+2]	90 [+2]	70 [-12]	90 [-3]	75 [-9]	69 [-9]	86 [+2]	67 [-12]	98 [+1]	95 [+3]	96 [+1]	97 [+1]	79 [-2]	84 [-2]
Global-Bal	93 [+1]	89 [+1]	73 [-9]	92 [-1]	81 [-3]	71 [-7]	85 [+1]	68 [-11]	97 [0]	94 [+2]	95 [0]	97 [+1]	80 [-1]	86 [0]
Observed Cluster	92 [0]	92 [+4]	85 [+3]	95 [+2]	87 [+3]	81 [+3]	86 [+2]	81 [+2]	98 [+1]	96 [+4]	96 [+1]	98 [+2]	81 [0]	88 [+2]

Table 9: Performance comparison across fine-tuning strategies using Gemma-4. Bracketed numbers indicate change relative to Solo-Source TL.

Test $\downarrow$ / Train $\rightarrow$	TR	UR	FA	AR	HI	BN	IN	MS	EN	ES	FR	DE	KO	ZH
TR	88	64	65	56	56	47	39	36	64	55	61	68	49	53
UR	63	85	66	65	67	69	41	35	53	49	49	41	39	38
FA	63	65	75	73	50	47	73	53	55	41	43	41	51	51
AR	54	67	67	90	51	47	49	45	55	47	43	49	53	47
HI	57	82	65	51	75	58	53	41	60	54	43	39	43	41
BN	45	76	39	31	65	70	53	47	55	49	39	31	35	33

Table 10: Cross-lingual F1 scores across 14 languages using Gemma-3. Rows: test language. Columns: train language. Diagonal (matched train/test) in bold.

Test $\downarrow$ / Train $\rightarrow$	TR	UR	FA	AR	HI	BN	IN	MS	EN	ES	FR	DE	KO	ZH
TR	92	68	69	60	59	51	41	41	67	59	64	70	53	57
UR	67	88	70	69	71	73	45	39	56	53	53	45	43	41
FA	69	70	82	75	55	51	75	56	60	47	47	47	55	55
AR	59	71	70	93	55	51	53	49	60	51	47	53	57	51
HI	62	84	69	55	84	71	57	45	64	58	47	43	47	45
BN	47	80	43	35	71	78	57	51	60	53	43	35	39	37

Table 11: Cross-lingual F1 scores across 14 languages using Gemma-4. Rows: test language. Columns: train language. Diagonal (matched train/test) in bold.

Cluster	Hate Train Size (per language)	Platform	Time Period
EN–FR–ES–DE	EN: 4500, FR: 3739, ES: 4731, DE: 4503	X	2016–2023
UR–HI–BN	UR: 4500, HI: 2222, BN: 4504	X, FB, WA, YT	BN: 2017–2020; UR, HI: unspecified
FA–AR–UR–IN	FA: 1002, AR: 4503, UR: 4500, IN: 4502	X	FA: 2020–2022; AR: 2016–2018; IN: 2018; UR: unspecified
IN–MS	IN: 4502, MS: 1701	X	IN: 2018; MS: 2022–2023
KO–ZH	KO: 1520, ZH: 2470	X / Web	KO: 2018–2020; ZH: unspecified

Table 12: Summary of dataset characteristics across the five empirically observed transfer clusters.

Held-out	Solo F1	LOLO Cluster F1	$\Delta$ vs Solo	Original $\Delta$
FA	80 $\pm$ 0.8	83 $\pm$ 0.5	+3	+3
AR	91 $\pm$ 0.6	93 $\pm$ 0.4	+2	+2
UR	85 $\pm$ 0.6	90 $\pm$ 0.4	+5	+5
IN	82 $\pm$ 0.6	85 $\pm$ 0.4	+3	+3

Table 13: LOLO results for Cluster 1: Persian-Arabic-Urdu-Indonesian. “Original

\Delta

” refers to the gain reported in Table 2

Held-out	Solo F1	LOLO Cluster F1	$\Delta$ vs Solo	Original $\Delta$
UR	85 $\pm$ 0.6	88 $\pm$ 0.5	+3	+5
HI	80 $\pm$ 0.7	83 $\pm$ 0.5	+3	+5
BN	70 $\pm$ 0.7	73 $\pm$ 0.5	+3	+4

Table 14: LOLO results for Cluster 2: Urdu–Hindi–Bengali.

Held-out	Solo F1	LOLO Cluster F1	$\Delta$ vs Solo	Original $\Delta$
EN	96 $\pm$ 0.5	96 $\pm$ 0.4	0	+1
ES	90 $\pm$ 0.6	94 $\pm$ 0.4	+4	+6
FR	94 $\pm$ 0.5	95 $\pm$ 0.4	+1	+2
DE	90 $\pm$ 0.6	93 $\pm$ 0.4	+3	+5

Table 15: LOLO results for Cluster 3: English–Spanish–French–German.

source	target	F1	size_diff	ling_similarity	temporal_overlap
TR	FA	66	1.5	0.7	0.29
TR	MS	39	0.97	0.46	0.14
TR	ES	57	0.05	0.47	1
TR	FR	62	0.19	0.48	1
FA	TR	48	1.5	0.7	0.29
FA	AR	63	1.5	0.78	0

Table 16: Cross-lingual transfer cases with corresponding dataset properties for a representative sample.

Hate Speech Detection

Hate Speech Detection identifies hateful or abusive language, its targets and intent, to support safer and more accountable content moderation.

See all 100 papers

Toward Generalized Cross-Lingual Hateful Language Detection with Web-Scale Data and Ensemble LLM Annotations
Mar 18, 2026Dang H. Dang, Jelena Mitrovi, Michael GranitzerHate Speech Detection Low-Resource Language Processing
University of Passau, Chair of Data Science
Comparison of Modern Multilingual Text Embedding Techniques for Hate Speech Detection Task
Apr 16, 2026Evaldas Vaiciukynas, Paulius Danenas, Linas Ablonskis +5Sentence Embeddings Text Embedding Evaluation
Department of Information Systems, Faculty of Informatics, Kaunas University of Technology, Kaunas, Lithuania · Centre of Information Systems Design Technologies, Faculty of Informatics, Kaunas University of Technology, Kaunas, Lithuania
Multilingual Hate Speech Detection and Counterspeech Generation: A Comprehensive Survey and Practical Guide
Mar 1, 2026Zahra Safdari Fesaghandis, Suman Kalyan MaityHate Speech Detection Text Generation
Bilkent University, Ankara, Turkey · Missouri University of Science and Technology, Rolla, Missouri, USA

Mar 18, 2026J/K move · Enter open · S save

Toward Generalized Cross-Lingual Hateful Language Detection with Web-Scale Data and Ensemble LLM Annotations

Dang H. Dang, Jelena Mitrovi, Michael Granitzer

University of Passau, Chair of Data Science

Hate Speech Detection Low-Resource Language Processing Synthetic Data Generation Language Model Ensembles Continued Pretraining

Open paper AskPDF arXiv

We study whether large-scale unlabelled web data and LLM-based synthetic annotations can improve multilingual hate speech detection. Starting from texts crawled via OpenWebSearch.eu~(OWS) in four languages (English, German, Spanish, Vietnamese), we pursue two complementary strategies. First, we apply continued pre-training to BERT models by continuing masked language modelling on unlabelled OWS texts before supervised fine-tuning, and show that this yields an average macro-F1 gain of approximately 3% over standard baselines across sixteen benchmarks, with stronger gains in low-resource settings. Second, we use four open-source LLMs (Mistral-7B, Llama3.1-8B, Gemma2-9B, Qwen2.5-14B) to produce synthetic annotations through three ensemble strategies: mean averaging, majority voting, and a LightGBM meta-learner. The LightGBM ensemble consistently outperforms the other strategies. Fine-tuning on these synthetic labels substantially benefits a small model (Llama3.2-1B: +11% pooled F1), but provides only a modest gain for the larger Qwen2.5-14B (+0.6%). Our results indicate that the combination of web-scale unlabelled data and LLM-ensemble annotations is the most valuable for smaller models and low-resource languages.

Test $\downarrow$ / Train $\rightarrow$	TR	UR	FA	AR	HI	BN	IN	MS	EN	ES	FR	DE	KO	ZH
TR	88	64	65	56	56	47	39	36	64	55	61	68	49	53
UR	63	85	66	65	67	69	41	35	53	49	49	41	39	38
FA	63	65	75	73	50	47	73	53	55	41	43	41	51	51
AR	54	67	67	90	51	47	49	45	55	47	43	49	53	47
HI	57	82	65	51	75	58	53	41	60	54	43	39	43	41
BN	45	76	39	31	65	70	53	47	55	49	39	31	35	33

Test $\downarrow$ / Train $\rightarrow$	TR	UR	FA	AR	HI	BN	IN	MS	EN	ES	FR	DE	KO	ZH
TR	92	68	69	60	59	51	41	41	67	59	64	70	53	57
UR	67	88	70	69	71	73	45	39	56	53	53	45	43	41
FA	69	70	82	75	55	51	75	56	60	47	47	47	55	55
AR	59	71	70	93	55	51	53	49	60	51	47	53	57	51
HI	62	84	69	55	84	71	57	45	64	58	47	43	47	45
BN	47	80	43	35	71	78	57	51	60	53	43	35	39	37

Test $\downarrow$ / Train $\rightarrow$	TR	UR	FA	AR	HI	BN	IN	MS	EN	ES	FR	DE	KO	ZH
TR	88	64	65	56	56	47	39	36	64	55	61	68	49	53
UR	63	85	66	65	67	69	41	35	53	49	49	41	39	38
FA	63	65	75	73	50	47	73	53	55	41	43	41	51	51
AR	54	67	67	90	51	47	49	45	55	47	43	49	53	47
HI	57	82	65	51	75	58	53	41	60	54	43	39	43	41
BN	45	76	39	31	65	70	53	47	55	49	39	31	35	33

Test $\downarrow$ / Train $\rightarrow$	TR	UR	FA	AR	HI	BN	IN	MS	EN	ES	FR	DE	KO	ZH
TR	92	68	69	60	59	51	41	41	67	59	64	70	53	57
UR	67	88	70	69	71	73	45	39	56	53	53	45	43	41
FA	69	70	82	75	55	51	75	56	60	47	47	47	55	55
AR	59	71	70	93	55	51	53	49	60	51	47	53	57	51
HI	62	84	69	55	84	71	57	45	64	58	47	43	47	45
BN	47	80	43	35	71	78	57	51	60	53	43	35	39	37

Abstract

Hate Speech Detection

Explore similar work

Toward Generalized Cross-Lingual Hateful Language Detection with Web-Scale Data and Ensemble LLM Annotations

Comparison of Modern Multilingual Text Embedding Techniques for Hate Speech Detection Task

Multilingual Hate Speech Detection and Counterspeech Generation: A Comprehensive Survey and Practical Guide

Test $\downarrow$ / Train $\rightarrow$	TR	UR	FA	AR	HI	BN	IN	MS	EN	ES	FR	DE	KO	ZH
TR	88	64	65	56	56	47	39	36	64	55	61	68	49	53
UR	63	85	66	65	67	69	41	35	53	49	49	41	39	38
FA	63	65	75	73	50	47	73	53	55	41	43	41	51	51
AR	54	67	67	90	51	47	49	45	55	47	43	49	53	47
HI	57	82	65	51	75	58	53	41	60	54	43	39	43	41
BN	45	76	39	31	65	70	53	47	55	49	39	31	35	33

Test $\downarrow$ / Train $\rightarrow$	TR	UR	FA	AR	HI	BN	IN	MS	EN	ES	FR	DE	KO	ZH
TR	92	68	69	60	59	51	41	41	67	59	64	70	53	57
UR	67	88	70	69	71	73	45	39	56	53	53	45	43	41
FA	69	70	82	75	55	51	75	56	60	47	47	47	55	55
AR	59	71	70	93	55	51	53	49	60	51	47	53	57	51
HI	62	84	69	55	84	71	57	45	64	58	47	43	47	45
BN	47	80	43	35	71	78	57	51	60	53	43	35	39	37