GENDIS: Gender-Related Discourse and Stance in Multilingual YouTube Comments from India, Kenya, and Nigeria

Luise Koch; Manuel Tonneau; Niyati Malhotra; Sunny Rai; Ana Maria Munoz Boudet; Victor Orozco-Olvera; Samuel P. Fraiberger; Sharath Chandra Guntuku

cs.CLOct 13, 2026

GENDIS: Gender-Related Discourse and Stance in Multilingual YouTube Comments from India, Kenya, and Nigeria

Authors: Luise Koch, Manuel Tonneau, Niyati Malhotra, Sunny Rai, Ana Maria Munoz Boudet, Victor Orozco-Olvera, Samuel P. Fraiberger, Sharath Chandra Guntuku

Organizations: World Bank · Technical University of Munich · University of Oxford · New York University ·

Abstract

Research on gender-related online content is often operationalized through harm-centered tasks that collapse non-harmful gender-related and unrelated content, while existing datasets remain concentrated in high-resource languages and Global North settings. To bridge these gaps, we introduce GENDIS, a dataset of 9,600 multilingual, human-annotated YouTube comments from India, Kenya, and Nigeria (3,200 per country), annotated using a two-level framework that first identifies whether gender is referenced and then classifies gender-related content as negative, positive, or neutral. This separation provides a denominator for gender-related discourse independent of harm, enabling both prevalence estimation and characterization of its evaluative range. In the uniform random stratum, 23.8-35.0% of comments are gender-related, and among those with a resolved stance, 82% are neutral or positive. Across models, gender-reference detection is substantially easier than stance classification, especially for positive and neutral speech. Lightly supervised classifiers match frontier zero-shot large language models at substantially lower cost and outperform generic toxicity detection, while cross-country transfer remains limited. GENDIS provides a locally grounded resource for measuring when gender is invoked online and how that reference is framed.

Figures & tables

Figure 1: Gendis consists of three annotated datasets ( $N=3{,}200$ each) sampled from country-associated channel corpora in India, Kenya, and Nigeria. The dataset is strongly multilingual and code-mixed, with English alongside languages and varieties such as Hindi, Swahili, Sheng, Nigerian Pidgin, Yoruba, Hausa, and Igbo.

Dataset	Platform	Region	#Lang	Gender ref. beyond harm	Size
EDOS ( Kirk et al., 2023 )	Gab/Reddit	Unspecified	1	No	20,000
XtremeSpeech ( Maronikolakis et al., 2022 )	Multiple	BR/DE/IN/KE	5	No	20,297
NaijaHate ( Tonneau et al., 2024b )	Twitter	NG	5	No	35,976
Uli ( Arora et al., 2024 )	Twitter	IN	3	No	23,266
AfriHate ( Muhammad et al., 2025 )	Twitter	Africa	15	No	90,437
EXIST 2023 ( Plaza et al., 2023 )	Twitter	Unspecified	2	No	10,034

Table 1: Prior (gendered) harm datasets and Gendis compared across platform, region, number of languages, whether gender is referenced beyond harm, and size. Gendis uniquely treats gender reference beyond harm as a first-class annotation target, capturing negative, positive, and neutral evaluative stance across multiple Global South contexts. Further datasets are discussed in § 2 .

Panel of Figure 2 — Figure 2: The two-level annotation scheme, with illustrative example comments in the shaded boxes. Level 1 detects whether a comment is gender-related, sending comments with no gender reference (such as general insults) to a leaf and gender-related ones to Level 2, which labels how gender is used. Further examples per category and country are in Appendix A.1 .

		Baseline	Supervised (CV)		Zero-shot LLMs
		Toxicity	XLM-R	SetFit	Llama-3.1-8B	Gemma-2-9B	gpt-5-nano	gpt-5.4
Level 1 (detection)		.46	.82	.83	.76	.79	.81	.84
Level 2	negative	.35	.71	.76	.70	.71	.71	.77
	neutral	.59	.55	.66	.28	.17	.45	.65
	positive	.00	.56	.60	.24	.34	.53	.50
	macro	.31	.61	.67	.41	.41	.56	.64

Table 2: F1 on the same

N=9{,}600

comments for every system, using out-of-fold predictions for the supervised models. Level 1 and End-to-end report macro-F1. The Level 2 rows report per-class F1 and their macro-average, conditioned on gold gender-related comments. Supervised models use pooled cross-validation (CV), with other regimes in Table 10 . Best per row in bold.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Country	Label	Original comment	English translation
Kenya	Positive	hongera sana dada umeongeza heshima kwa wanawake, kuwa wanaweza	Congratulations, sister. You have increased respect for women by showing that women are capable.
	Positive	#endfemicideke…a perfect song for the movement..stop killing women	–
	Neutral	I didn’t know she’s the singer of this song , I also like her teachings pale TikTok	I didn’t know she was the singer of this song. I also like her teachings on TikTok.
	Neutral	cute babies,wanakaa nicholas but rengi ni ya mama yao	Cute babies, they look like Nicholas, but their complexion comes from their mother.
	Negative	kuna baadh ya wanawake wana makalio makubwa ila brain hakuna kitu	Some women have big buttocks, but no brains.
	Negative	wanawake wenye sauti kama hiyo are very rudy	Women with voices like that are very rude.

Appendix

Table 3: Additional verbatim examples of gender-positive, gender-neutral, and gender-negative comments from Kenya, Nigeria, and India.

Country	Principal languages and varieties represented	Entries
India	English, Hindi, Bengali, Marathi, Tamil, Telugu, Hinglish, Sylheti, Urdu-linked expressions, and other code-mixed forms	778
Nigeria	English, Nigerian Pidgin, Hausa, Igbo, Yoruba, and code-mixed forms	386
Kenya	English, Swahili, Sheng, Luo, Kikuyu, Meru, Embu, Kalenjin, and code-mixed forms	366
Total		1,530

Appendix

Table 4: Principal language and variety coverage of the final country-specific keyword lists. The lists also include hashtags, abbreviations, emojis, transliterations, slang, and other platform-specific or hybrid forms.

Language / category	% of comments
India
English	58
Hindi	46
Tamil	11
Bengali	5
Telugu	3

Appendix

Table 5: Language composition of the annotation sample, as a share of each country’s 1,000 comments. Values are presence rates (a comment can contribute to more than one language, so they sum to over 100%). Code-mixed is the share of comments mixing two or more languages, and Other the share in languages outside the country’s language list. Languages present in under 1.5% of comments are omitted. Labels are automatic and approximate (Appendix C ).

Context	Responses	Reported languages and varieties	Relevant contextual familiarity
India	3/3	English, Hindi, and Bengali, with additional partial familiarity with other Indian languages reported.	Respondents reported familiarity with Indian social-media discourse, gendered cultural references, and relevant stereotypes.
Nigeria	3/3	English, Hausa, and Yoruba.	Respondents reported familiarity with Nigerian social-media discourse and locally used gendered expressions.
Kenya	2/3	English, Swahili/Kiswahili, Sheng, Kikuyu, Kamba, and Luo.	Respondents reported familiarity with Kenyan social-media discourse, code-switching, and locally used gendered expressions.

Appendix

Table 6: Aggregated linguistic and contextual expertise reported by the eight annotators who completed the optional questionnaire. Sensitive demographic and professional characteristics are omitted because the country-level panels are small.

Country	Not gender-related	GENNEG	GENPOS	Neutral	Unsure	No L2 majority
India	1,730	631	113	704	8	14
Kenya	1,850	503	125	680	0	42
Nigeria	1,577	734	337	237	9	306
All	5,157	1,868	575	1,621	17	362

Appendix

Table 7: Distribution of majority labels by country (

N=3{,}200

each). GENNEG denotes gender-negative and GENPOS gender-positive comments. A Level 2 label is assigned when at least two of the three annotators agree on it. Unsure counts comments whose agreed Level 2 label is unsure, and No L2 majority counts gender-related comments on which no two annotators agree on any Level 2 label. The two columns are reported separately and under the same rule for all three countries.

Figure 3: Distribution of aggregated labels within each country’s 3,200-comment annotated sample. Each sample comprises 1,500 keyword-enriched comments, 1,500 comments drawn uniformly at random from the full corpus, and 200 keyword-free comments. Because the full sample is partly keyword-enriched, these percentages do not estimate prevalence in the broader corpus; prevalence estimates use the uniform random stratum only. Exact counts are reported in Table 7 .

Country	Level 1	Level 2	Combined
India	.80 / .80	.82 / .83	.77 / .77
Kenya	.79 / .79	.71 / .72	.72 / .72
Nigeria	.55 / .54	.49 / .49	.47 / .47
All	.71 / .71	.70 / .68	.66 / .65

Appendix

Table 8: Inter-annotator agreement by country, reported as Fleiss’

\kappa

/ Krippendorff’s

\alpha

and calculated from the individual annotations before majority aggregation, over all 3,200 comments per country and all 9,600 comments for the All row. Level 1 distinguishes gendered from not-gendered comments. Level 2 distinguishes negative, positive, neutral, and unsure gendered speech. The combined label treats not-gendered comments as one category and combines gendered comments with their Level 2 label.

	SetFit
Country	Within	CV	LOCO	gpt-5-nano
	L1 / L2	L1 / L2	L1 / L2	L1 / L2
Kenya	.86 / .64	.84 / .64	.75 / .48	.84 / .56
Nigeria	.86 / .69	.86 / .68	.81 / .41	.80 / .57
India	.79 / .64	.79 / .64	.69 / .48	.78 / .55

Appendix

Table 9: Per-country macro-F1 for SetFit under the three training regimes, with zero-shot gpt-5-nano for comparison. Within-country and pooled CV are within two points of each other in every country and both far exceed leave-one-country-out, whose drop is steepest for India, so the degradation reflects cross-cultural difference rather than training-set size.

System	Regime	L1	L2	E2E
SetFit	Within	.84	.70	.65
	CV	.83	.67	.63
	LOCO	.75	.47	.46
XLM-R	Within	.82	.58	.55
	CV	.82	.61	.58
	LOCO	.77	.49	.48

Appendix

Table 10: Supervised models by training regime: within-country, pooled cross-validation (CV), and leave-one-country-out (LOCO), reported as overall macro-F1. Best regime per model and metric in bold: within-country and CV trade places and are always close, while LOCO is never best, the signature of a transfer gap. Per-country SetFit results are in Table 9 .

System	L1	L2
SetFit (CV)	.84	.64
gpt-5.4	.84	.53
XLM-R (CV)	.83	.58
gpt-5-nano	.80	.51
Gemma-2-9B	.76	.31
Llama-3.1-8B	.73	.35

Appendix

Table 11: Macro-F1 on the random stratum only (

n=4{,}500

, of which 1,226 gender-related with a resolved Level-2 label), a keyword-independent sample. Level 1 scores stay within a few points of the full evaluation set (Table 2 ), so keyword enrichment does not overstate detection, while the toxicity baseline collapses. With round-2 data, Level 2 is reportable here for the first time (previously

n=94

): SetFit keeps its lead, while gpt-5.4’s Level 2 score drops well below its keyword-enriched score, widening the gap with SetFit on this more representative sample.

	Level 1 (detection)				Level 2 (stance)
System	Eng	Local	Mixed	Other	Eng	Local	Mixed	Other
SetFit (CV)	.90	.73	.79	.71	.71	.62	.65	.61
gpt-5-nano	.87	.76	.75	.72	.62	.51	.55	.36
Llama-3.1-8B	.83	.69	.71	.70	.47	.34	.40	.26
Gemma-2-9B	.85	.73	.73	.73	.44	.40	.38	.29
XLM-R (CV)	.89	.71	.77	.68	.62	.56	.58	.58

Appendix

Table 12: Macro-F1 by language group for Level 1 (gender-related or not) and Level 2 (negative, positive, or neutral, conditioned on gold gender-related comments). Groups are derived from the automatic per-comment language label: Eng lish, Local (non-English monolingual), Mixed (code-mixed), and Other (languages outside the country’s language list). Group sizes: English 3,509, Local 2,078, Code-mixed 3,287, Other 726. Every system peaks on English and the gap widens at Level 2, while SetFit holds up best on the Other group.

Figure 4: End-to-end four-way confusion for SetFit (CV), row-normalized (%, so each row is recall for a gold class). Neutral gender-related speech is the hardest class, most often misread as negative or not gender-related.

Social Media Analysis

Social media analysis studies online posts, media, users, and interactions to understand communication, communities, and public behavior.

See all 221 papers

Ben-Misandry: A Bengali Dataset and Late-Fusion Transformer Framework for Detecting Anti-Male Gender Hate
Oct 17, 2026Md Mehedi Hasan Nipu, Chinmoy Mitra, Tarannum Ahmed Nowshin +1Hate Speech Detection Bengali NLP
Department of Electrical and Computer Engineering, North South University, Dhaka, Bangladesh · Department of Computer Science & Engineering, Rajshahi University of Engineering & Technology, Rajshahi, Bangladesh · Department of Computer Science and Engineering, BRAC University, Dhaka, Bangladesh +1
ProText: A benchmark dataset for measuring (mis)gendering in long-form texts
Mar 29, 2026Hadas Kotek, Margit Bowler, Patrick Sonnenberg +1Gender Bias in Language Models Language Model Bias Evaluation
Apple, USA
Harm Laundering in GPT Models: Evidence That Gender Discrimination Is Transformed Rather Than Reduced Across Safety-Trained Generations
Sep 17, 2026Sarah Wyer, Sue Black, Noura Al MoubayedGender Bias in Language Models Language Model Safety Evaluation
Durham University

Oct 17, 2026J/K move · Enter open · S save

Ben-Misandry: A Bengali Dataset and Late-Fusion Transformer Framework for Detecting Anti-Male Gender Hate

Md Mehedi Hasan Nipu, Chinmoy Mitra, Tarannum Ahmed Nowshin, Israt Moyeen Noumi

Department of Electrical and Computer Engineering, North South University, Dhaka, Bangladesh · Department of Computer Science & Engineering, Rajshahi University of Engineering & Technology, Rajshahi, Bangladesh · Department of Computer Science and Engineering, BRAC University, Dhaka, Bangladesh+1

Hate Speech Detection Bengali NLP

Open paper AskPDF arXiv

Gender based hate speech research in Bengali has focused almost exclusively on misogyny, leaving anti male hate, or misandry, entirely unstudied for this language and creating a systematic blind spot in moderation systems trained to catch attacks on women but not on men. We introduce Ben-Misandry, the first Bengali misandry detection dataset: 5,000 comments across six taxonomy classes, including a novel category for mocking male emotional vulnerability with no direct analogue in existing misogyny taxonomies. The corpus is taxonomy-controlled synthetic data generated with GPT-4.1, spot-checked by four human reviewers on a 500-comment sample who confirmed the large majority of assigned labels; a full inter-annotator- agreement study is left to future work. We benchmark it with a classical pipeline (seven classifiers over four text representations) and a transformer pipeline (BanglaBERT, XLM-RoBERTa, and two fusion strategies). The best classical model, a linear SVM over character-level TF-IDF, reaches 91.8% accuracy; a late fusion ensemble averaging BanglaBERT and XLM-RoBERTa softmax outputs reaches 96.6% accuracy and macro ROC-AUC above 99%, nominally ahead of every single model and early fusion, though we have not run significance testing to confirm these margins are meaningful. We relate this to classifier combination theory, under which averaging encoders with decorrelated errors is expected to reduce variance, and corroborate the decision boundary with Shapley explanations that recover the same class- discriminative vocabulary found in our exploratory analysis, while noting this cannot rule out the model partly keying on synthetic-generation artifacts. Our results indicate misandry detection in Bengali is feasible and measurable on this synthetic corpus, offered as an initial public resource pending validation on real-world data.

Abstract

Social Media Analysis

Explore similar work

Ben-Misandry: A Bengali Dataset and Late-Fusion Transformer Framework for Detecting Anti-Male Gender Hate

ProText: A benchmark dataset for measuring (mis)gendering in long-form texts

Harm Laundering in GPT Models: Evidence That Gender Discrimination Is Transformed Rather Than Reduced Across Safety-Trained Generations