Ben-Misandry: A Bengali Dataset and Late-Fusion Transformer Framework for Detecting Anti-Male Gender Hate
Authors: Md Mehedi Hasan Nipu, Chinmoy Mitra, Tarannum Ahmed Nowshin, Israt Moyeen Noumi
Organizations: Department of Electrical and Computer Engineering, North South University, Dhaka, Bangladesh · Department of Computer Science & Engineering, Rajshahi University of Engineering & Technology, Rajshahi, Bangladesh · Department of Computer Science and Engineering, BRAC University, Dhaka, Bangladesh · Department of Computer Science and Engineering, Ahsanullah University of Science and Technology, Dhaka, Bangladesh
Gender based hate speech research in Bengali has focused almost exclusively on misogyny, leaving anti male hate, or misandry, entirely unstudied for this language and creating a systematic blind spot in moderation systems trained to catch attacks on women but not on men. We introduce Ben-Misandry, the first Bengali misandry detection dataset: 5,000 comments across six taxonomy classes, including a novel category for mocking male emotional vulnerability with no direct analogue in existing misogyny taxonomies. The corpus is taxonomy-controlled synthetic data generated with GPT-4.1, spot-checked by four human reviewers on a 500-comment sample who confirmed the large majority of assigned labels; a full inter-annotator- agreement study is left to future work. We benchmark it with a classical pipeline (seven classifiers over four text representations) and a transformer pipeline (BanglaBERT, XLM-RoBERTa, and two fusion strategies). The best classical model, a linear SVM over character-level TF-IDF, reaches 91.8% accuracy; a late fusion ensemble averaging BanglaBERT and XLM-RoBERTa softmax outputs reaches 96.6% accuracy and macro ROC-AUC above 99%, nominally ahead of every single model and early fusion, though we have not run significance testing to confirm these margins are meaningful. We relate this to classifier combination theory, under which averaging encoders with decorrelated errors is expected to reduce variance, and corroborate the decision boundary with Shapley explanations that recover the same class- discriminative vocabulary found in our exploratory analysis, while noting this cannot rule out the model partly keying on synthetic-generation artifacts. Our results indicate misandry detection in Bengali is feasible and measurable on this synthetic corpus, offered as an initial public resource pending validation on real-world data.
Figures & tables
Class
Definition
Discredit
Belittles men’s competence or worth as a gender.
Stereotype & Objectification
Reduces men to a provider, emotionless, or aggressive trope.
Mockery & Emotional Dismissal *
Mocks male emotional expression or dismisses male suffering.
Threats & Violence
Explicit or implicit threats, or endorsement of violence, against men.
Dominance & Derailing
Female-supremacist framing, or derailing discussion of men’s issues.
Non-misandric
Neutral, factual, or pro-equality content; control class.
TABLE I: Ben-Misandry taxonomy
Statistic
Value
Total comments
5,000
Classes
6 (imbalance ratio 1.002)
Unique texts
5,000 (0% duplicates)
Bengali script coverage
100%
Rows containing any emoji
683 (13.7%)
Mean words / comment
10.2 (median 10.0)
TABLE II: Ben-Misandry corpus statistics
Fig. 1: Top fifteen content words per class after stopword removal. Vocabulary is visibly disjoint: discredit centers on capability words, stereotype/objectification on reductive nouns, mockery/emotional dismissal on emotion and suffering words.
Fig. 2: Frequency of male referent tokens by class. Every class, including the control, is dense in male referents. Purushjati (male race) occurs only in threats_violence.
Family
Model (best feature)
Acc.
F1 w
F1 m
Prec. w
Rec. w
κ
Transformer
Late Fusion (Averaging)
0.966
0.9660
0.9660
0.9663
0.966
0.9592
Transformer
Late Fusion (Max-Conf. Voting)
0.966
0.9660
0.9660
0.9663
0.966
0.9592
Transformer
Early Fusion
0.960
0.9601
0.9601
0.9607
0.960
0.9520
Transformer
BanglaBERT (single)
0.958
0.9579
0.9579
0.9581
0.958
0.9496
Transformer
XLM-R-base (single)
0.950
0.9500
0.9500
0.9506
0.950
0.9400
Classical
Linear SVM (char_wb 3-5)
0.918
0.9181
0.9181
0.9192
0.918
0.9016
TABLE III: Test set results, all transformer configurations and the best classical configuration per model family
Class
Prec.
Rec.
F1
Supp.
Discredit
0.9518
0.9518
0.9518
83
Dominance & Derailing
0.9405
0.9518
0.9461
83
Mockery & Emo. Dismissal
0.9875
0.9518
0.9693
83
Non-misandric
0.9540
0.9881
0.9708
84
Stereotype & Objectification
0.9643
0.9643
0.9643
84
Threats & Violence
1.0000
0.9880
0.9939
83
TABLE IV: Per-class test results, Late Fusion (Averaging)
Fig. 3: Confusion matrix, Late Fusion (Averaging), held-out test set.
Fig. 4: ROC and precision-recall curves, Late Fusion (Averaging). Macro ROC-AUC = 0.999, macro AP = 0.995.
Fig. 5: SHAP token attribution for Late Fusion on a held-out dominance_derailing comment. Green pushes toward the predicted class, red away from it.
Lexicon-driven misogyny detectors cannot, by construction, distinguish a slur used against a woman from the same slur mentioned in counter-speech ("don't call her that") -- yet exactly this distinction governs whether moderation protects or silences the people discussing abuse. We study this problem in code-mixed Hinglish and make three contributions. First, we diagnose two evaluation artifacts on a publicly available redacted corpus: category-encoding anonymization placeholders leak the label (a no-learning rule scores 1.000), and even after they are neutralized misogynistic and benign comments occupy lexically disjoint registers, so bag-of-words reaches macro-F1 approximately 1.00 under random cross-validation but collapses under template-disjoint evaluation. Second, we release Hinglish-MGY-Diag, a deterministic generator and a 416-item / 163-minimal-pair contrast-set diagnostic across five linguistically motivated categories in which slur presence and gendered register are decorrelated from the label by construction. Third, we introduce a strict pair-consistency metric that credits a model only when both members of a minimal pair are correctly labelled. Five from-scratch classical baselines evaluated under construction-disjoint five-fold cross-validation reveal that the strongest model reaches 0.93 accuracy on the cleanest use-mention subset but only 0.82 consistency -- it still mislabels roughly one counter-speech pair in five. A frontier LLM used as an author-model ceiling attains 1.000 on all metrics, doubling as independent label validation and confirming the benchmark is a capability gradient rather than an adversarial wall. We release all code, data, the generator, and an arms-length LLM harness for reproducing every number.
Ashanvi Yadav, Shubham Bhardwaj
Manipal University Jaipur Rajasthan, India · BITS Pilani, Hyderabad Campus Telangana, India
We introduce ProText, a dataset for measuring gendering and misgendering in stylistically diverse long-form English texts. ProText spans three dimensions: Theme nouns (names, occupations, titles, kinship terms), Theme category (stereotypically male, stereotypically female, gender-neutral/non-gendered), and Pronoun category (masculine, feminine, gender-neutral, none). The dataset is designed to probe (mis)gendering in text transformations such as summarization and rewrites using state-of-the-art Large Language Models, extending beyond traditional pronoun resolution benchmarks and beyond the gender binary. We validated ProText through a mini case study, showing that even with just two prompts and two models, we can draw nuanced insights regarding gender bias, stereotyping, misgendering, and gendering. We reveal systematic gender bias, particularly when inputs contain no explicit gender cues or when models default to heteronormative assumptions.
Transformer-based transfer models now dominate Bangla sentiment classification, yet their adversarial robustness remains largely unexamined, and no prior study pairs a Bangla attack suite with a defense that measurably recovers robustness. We address this gap with destroR, a unified pipeline for evaluating and hardening Bangla text classifiers. First, we introduce three meaning-preserving Bangla attack recipes a paraphrase attack, a back-translation attack, and a one-hot word-swap attack that perturb inputs while regenerating fluent, semantically faithful sentences, inducing model prediction perplexity rather than input noise. Second, we construct a robustness benchmark that evaluates five transfer models (BanglaBERT, BanglishBERT, XLM-RoBERTa, MuRIL, and IndicBERTv2) across four datasets against five attacks, placing our recipes against two strong word-substitution baselines, TextFooler and BAE, under an identical protocol. Third, we harden every model through adversarial training and report a full robustness matrix. Our analysis yields three findings: word-substitution baselines are more potent than semantically constrained recipes (BAE reaches a 54.2% attack success rate); adversarial training on the union of all attack families lowers residual attack success for every attack; and, contrary to expectation, the Indic-multilingual MuRIL backbone is markedly more robust than the Bangla-dedicated models. All models, adversarial data, and code are released for full reproducibility.