RAGing Against Misogyny: Improving Implicit Misogyny Explanations

Ivana Crescenzi; Arianna Muti; Maarten Sap; Tommaso Caselli; Debora Nozza

cs.CLOct 26, 2026

RAGing Against Misogyny: Improving Implicit Misogyny Explanations

Authors: Ivana Crescenzi, Arianna Muti, Maarten Sap, Tommaso Caselli, Debora Nozza

Organizations: Department of Computing Sciences, Bocconi University, Milan, Italy · Language Technologies Institute, Carnegie Mellon University, Pittsburgh, USA · Center for Language and Cognition Groningen, University of Groningen, The Netherlands

Abstract

Implicit misogyny often requires background knowledge to be understood. To address this challenge, we propose a Retrieval-Augmented Generation (RAG) framework for detecting and explaining implicit misogyny in English and Italian, leveraging encyclopedic, slang and feminist knowledge sources. Using the ImplicIT-Mis and SBIC+ datasets, we evaluate the contribution of each knowledge source to detection and explanation quality, investigating also whether RAG addresses known LLM failure modes in implicit misogyny. Results show that RAG improves recall on ImplicIT-Mis, but does not improve detection performance on SBIC+. However, human evaluation shows that RAG-based explanations are preferred over no-RAG explanations in both languages. Feminist literature emerges as the most consistently useful source for detection and explanation.

Figures & tables

Panel of Figure 1 — Figure 1: Example RAG pipeline. Among the candidate sources, the relevant Wikipedia page is retrieved; compression retrieves only the key sentences within the full article, enabling the model to draw the correct implied assumption (unlike the baseline without RAG).

KB	IT	EN
Wikipedia	87.4%	3.4%
Slang	9.7%	57.1%
Literature	29.6%	40.8%

Table 1: Retrieval coverage for each knowledge base.

Dataset	Prompt setting	Recall
Implicit-MIS	text-only - IT	0.838
	text-only - EN	0.771
	+ headline - IT	0.865
	+ headline - EN	0.891

Table 2: Impact of context (headline) and prompt language (IT vs . EN) on misogyny detection.

Dataset	System	Prompt	Recall
Implicit-MIS	+ headline	EN	0.891
	Wikipedia	IT	0.881
	Wikipedia	EN	0.853
	Slang	IT	0.916
	Slang	EN	0.895
	Literature	IT	0.923

Table 3: Comparison between baselines ( + headline for ImplicIT-Mis and baseline for SBIC+ ) and different retrieval sources for misogyny detection.

System	N	Sum	Mean	Std.	N	Sum	Mean	Std.
	Implicit-MIS				SBIC+
Wikipedia	194	128	0.659	0.774	36	39	1.083	0.841
Slang	94	72	0.765	0.835	184	240	1.304	0.833
Literature	174	161	0.925	0.826	186	253	1.360	0.802
no RAG	200	118	0.590	0.745	200	269	1.345	0.774

Table 4: Faithfulness scores for generated explanations, aggregated across two annotators and 100 samples per language. N indicates the number of scores, i.e. twice the number of evaluated explanations, Sum is the total faithfulness score, and Mean reports the average faithfulness score.

Dataset	N. of texts	RAG	Baseline
Implicit-MIS	200	121	51
SBIC+	200	154	95

Table 5: Aggregate preference across two annotators. A text can count in both columns in case of ties or disagreements.

Dataset	RAG Wins	Baseline Wins	Tie	Neither
Implicit-MIS	88	18	33	61
SBIC+	86	27	68	19

Table 6: Mutually exclusive breakdown of the comparison in Table 5 . Neither indicates that no explanation was selected.

Dataset	System	Preferred	Normalized
Implicit-MIS	Wikipedia	46/194	0.237
	Slang	24/94	0.255
	Literature	71/174	0.408
	+ headline	43/200	0.215
SBIC+	Wikipedia	13/36	0.361
	Slang	92/184	0.500

Table 7: Preference by knowledge source. Preferred reports how often each system’s explanation was preferred, over the number of explanations produced.

#	Failure mode	RAG source	Input text	Baseline explanation	RAG explanation
(1)	Lack of Reference	Wikipedia	“Dignified is silence, not the spectacularization of an announced tragedy. You are killing again the other murdered women; remain in the dignified silence of relatives. But the show must go on, right?” Context: a post suggesting Elena Cecchettin as “woman of the year” for speaking up about femicides.	Women are exploited to attract audiences and attention.	Women who speak publicly about their suffering are not dignified.
(2)	Wrong inference	Literature	“If only patriarchy existed” (translated from IT), in response to a headline about Elena Cecchettin’s speech on femicide.	Patriarchy protects women from situations such as femicide.	Patriarchy is considered positive and desirable.
(3)	Sarcasm /Irony	Literature	“The female superiority (translated from IT)”	Women have greater awareness and autonomy than their parents.	Female superiority is viewed with suspicion or as a negative idea.
(4)	Figurative language	Slang	“"makeup tell me about roasties, why do they wear the mask?"”	women (specifically those who wear ’roasties’ or possibly those who wear makeup) are hiding or disguising their true appearance.	women wear makeup to hide their appearance.

Table 8: Examples of high-quality RAG explanations addressing known failure modes in implicit misogyny.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Data	KB	Setting	RAG	No-RAG
1B	IT	Wikipedia	few-shot	0.949	0.754
1B	EN	Wikipedia	zero-shot	0.887	0.882
3B	IT	Wikipedia	few-shot	0.893	0.868
3B	EN	Wikipedia	zero-shot	0.964	0.965
1B	EN	Slang	few-shot	0.946	0.944
3B	IT	Slang	few-shot	0.712	0.666

Appendix

Table 9: Results for smaller models.

Subset	Italian	English
	Cohen’s $\kappa$	Cohen’s $\kappa$
General	0.505	0.454
Wiki	0.710	0.452
Slang	0.680	0.504
Literature	0.650	0.421
Baseline	0.700	0.437

Appendix

Table 10: Inter-annotator agreement on Faithfulness: Cohen’s

\kappa

across subsets and languages.

Implicit Bias

Implicit bias studies how training algorithms and model structure favor particular solutions, representations, and generalization behaviors without explicit penalties.

See all 30 papers

Used, Mentioned, or Condemned? A Controlled Contrast-Set Diagnostic for the Use-Mention Distinction in Code-Mixed Hinglish Misogyny Detection
Sep 6, 2026Ashanvi Yadav, Shubham BhardwajHate Speech Detection Content Moderation
Manipal University Jaipur Rajasthan, India · BITS Pilani, Hyderabad Campus Telangana, India
E-MIA: Exam-Style Black-Box Membership Inference Attacks against RAG Systems
May 1, 2026Zelin Guan, Shengda Zhuo, Zeyan Li +4Membership Inference Attacks Privacy Leakage in Language Models
College of Cyber Security, Jinan University, Guangzhou, China · School of Computer Science, Shanghai Jiaotong University, Shanghai, China · Institute of Artificial Intelligence, Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, Beihang University, Beijing 100191, China +1
When Detection Does Not Guarantee Resistance: Reasoning and Poisoned Context in RAG
Aug 17, 2026Mehrdad Ghassabi, Audrina Ebrahimi, Sadra Hakim +3Knowledge Conflicts in Language Models RAG Poisoning Attacks
University of Isfahan · University of Texas at Dallas · University of Windsor

Sep 6, 2026J/K move · Enter open · S save

Used, Mentioned, or Condemned? A Controlled Contrast-Set Diagnostic for the Use-Mention Distinction in Code-Mixed Hinglish Misogyny Detection

Ashanvi Yadav, Shubham Bhardwaj

Manipal University Jaipur Rajasthan, India · BITS Pilani, Hyderabad Campus Telangana, India

Hate Speech Detection Content Moderation Automated Evaluation

Open paper AskPDF arXiv

Lexicon-driven misogyny detectors cannot, by construction, distinguish a slur used against a woman from the same slur mentioned in counter-speech ("don't call her that") -- yet exactly this distinction governs whether moderation protects or silences the people discussing abuse. We study this problem in code-mixed Hinglish and make three contributions. First, we diagnose two evaluation artifacts on a publicly available redacted corpus: category-encoding anonymization placeholders leak the label (a no-learning rule scores 1.000), and even after they are neutralized misogynistic and benign comments occupy lexically disjoint registers, so bag-of-words reaches macro-F1 approximately 1.00 under random cross-validation but collapses under template-disjoint evaluation. Second, we release Hinglish-MGY-Diag, a deterministic generator and a 416-item / 163-minimal-pair contrast-set diagnostic across five linguistically motivated categories in which slur presence and gendered register are decorrelated from the label by construction. Third, we introduce a strict pair-consistency metric that credits a model only when both members of a minimal pair are correctly labelled. Five from-scratch classical baselines evaluated under construction-disjoint five-fold cross-validation reveal that the strongest model reaches 0.93 accuracy on the cleanest use-mention subset but only 0.82 consistency -- it still mislabels roughly one counter-speech pair in five. A frontier LLM used as an author-model ceiling attains 1.000 on all metrics, doubling as independent label validation and confirming the benchmark is a capability gradient rather than an adversarial wall. We release all code, data, the generator, and an arms-length LLM harness for reproducing every number.

Abstract

Implicit Bias

Explore similar work

Used, Mentioned, or Condemned? A Controlled Contrast-Set Diagnostic for the Use-Mention Distinction in Code-Mixed Hinglish Misogyny Detection

E-MIA: Exam-Style Black-Box Membership Inference Attacks against RAG Systems

When Detection Does Not Guarantee Resistance: Reasoning and Poisoned Context in RAG