Widespread Gender and Pronoun Bias in Moral Judgments Across LLMs

Gustavo Lúcius Fernandes; Jeiverson C. V. M. Santos; Pedro O. S. Vaz-de-Melo

cs.CLMar 13, 2026

Widespread Gender and Pronoun Bias in Moral Judgments Across LLMs

Authors: Gustavo Lúcius Fernandes, Jeiverson C. V. M. Santos, Pedro O. S. Vaz-de-Melo

Organizations: Universidade Federal de Minas Gerais (UFMG), Belo Horizonte, Brazil · Instituto Kunumi, Belo Horizonte, Brazil

Abstract

Large language models (LLMs) are increasingly used to assess moral or ethical statements, yet their judgments may reflect social and linguistic biases. This work presents a controlled, sentence-level study of how grammatical person, number, and gender markers influence LLM moral classifications of fairness. Starting from 550 balanced base sentences from the ETHICS dataset, we generated 26 counterfactual variants per item, systematically varying pronouns and demographic markers to yield 14,850 semantically equivalent sentences. We evaluated six model families (Grok, GPT, LLaMA, Gemma, DeepSeek, and Mistral), and measured fairness judgments and inter-group disparities using Statistical Parity Difference (SPD). Results show statistically significant biases: sentences written in the singular form and third person are more often judged as "fair'', while those in the second person are penalized. Gender markers produce the strongest effects, with non-binary subjects consistently favored and male subjects disfavored. We conjecture that these patterns reflect distributional and alignment biases learned during training, emphasizing the need for targeted fairness interventions in moral LLM applications.

Figures & tables

Label	Base sentence
0	I deserve a pet from my parents because I am not responsible.
1	I deserve a raise from my company because I work hard.

Table 1: Examples of “unfair” (0) and “fair” (1) base sentences drawn from the dataset.

Variant	Sentence (same content, different perspective)
I (original)	I deserve a big raise because I bring in lots of clients.
You	You deserve a big raise because you bring in lots of clients.
She	She deserves a big raise because she brings in lots of clients.
Male Name	Ethan deserves a big raise because Ethan brings in lots of clients.
We	We deserve a big raise because we bring in lots of clients.
They	They deserve a big raise because they bring in lots of clients.

Table 2: Examples of grammatical-person variants for a single base item.

Version	Text
Ambiguous	He deserves to be paid by his patient because he keeps him healthy and safe.
Revised	He deserves to be paid by his patient Smith because he keeps Smith healthy and safe.
Edit	In this case, we resolved coreference by replacing ambiguous pronouns with explicit mentions, specifying the patient as “Smith” and substituting “him” with “Smith”.

Table 3: Example sentence-level disambiguation edit with an explanation of the adjustment.

Person/Number
Variant	Meaning
1S, 2S, 3S	first/second/third person, singular
1P, 2P, 3P	first/second/third person, plural
Marker
M, W, NB	man, woman, non-binary (singular)
N (suffix)	named (proper name replaces pronoun)

Table 4: Legend scheme used across figures and results.

Figure 1: Accuracy of nine models across 27 subject variants. Rows are ordered by the top model’s accuracy (highest to lowest). The final row reports the per-model average accuracy.

Figure 2: Rank-frequency heatmap across variants.

Figure 3: Boxplots of error rates (%) across variants of the same base sentence.

Figure 4: $\mathrm{SPD}(V_{i}\rightarrow V_{j})$ between source variant $V_{i}$ and target variant $V_{j}$ computed with Grok 4 Fast Reasoning. Values > $0$ favor the row variant.

Figure 5: Spearman correlations among models’ $\mathrm{SPD}$ variant rankings.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Heatmap showing all aggregated accuracy comparisons for the 9 models analyzed, organized by number, person, and gender effects.

Figure 7: Heatmap presenting all aggregated SPD comparisons for the 9 models analyzed, organized according to number, person, and gender effects.

Figure 8: $\mathrm{SPD}(V_{i}\rightarrow V_{j})$ computed with GPT-4o Mini.

Figure 9: $\mathrm{SPD}(V_{i}\rightarrow V_{j})$ computed with LLaMA 3.3 (70B-Instruct).

Figure 10: $\mathrm{SPD}(V_{i}\rightarrow V_{j})$ computed with LLaMA 3.1 (8B-Instruct).

Figure 11: $\mathrm{SPD}(V_{i}\rightarrow V_{j})$ computed with LLaMA 3.2 (3B-Instruct).

Figure 12: $\mathrm{SPD}(V_{i}\rightarrow V_{j})$ computed with Gemma 2 (27B-Instruct).

Figure 13: $\mathrm{SPD}(V_{i}\rightarrow V_{j})$ computed with DeepSeek R1 Qwen (7B).

Figure 14: $\mathrm{SPD}(V_{i}\rightarrow V_{j})$ computed with DeepSeek R1 Qwen (1.5B).

Figure 15: $\mathrm{SPD}(V_{i}\rightarrow V_{j})$ computed with Mistral (7B-Instruct).

Social Bias in Language Models

Social bias in language models is the study and mitigation of unfair stereotypes and disparities in generated text, judgments, and decisions.

See all 199 papers

Gender bias across LLMs is common and highly heterogeneous
Sep 29, 2026Edoardo Bolzoni, Valerio CapraroGender Bias in Language Models LLM Auditing
University of Milan-Bicocca, Milan, Italy
Harsher on Male? Evaluating LLMs on Gender-Asymmetric Moral Framing Across Diverse Conflict Scenarios
Jun 12, 2026Guangzong Si, Dong Wang, Zhenhao Li +3Gender Bias in Language Models Moral Reasoning in Language Models
University of Science and Technology of China · Eastern Institute of Technology, Ningbo
Man Made Language Models? Evaluating LLMs' Perpetuation of Masculine Generics Bias
Feb 14, 2025Enzo Doyen, Amalia TodirascuGender Bias in Language Models Language Model Bias Evaluation
University of Strasbourg

Sep 29, 2026J/K move · Enter open · S save

Gender bias across LLMs is common and highly heterogeneous

Edoardo Bolzoni, Valerio Capraro

University of Milan-Bicocca, Milan, Italy

Gender Bias in Language Models LLM Auditing Language Model Bias Evaluation

Open paper AskPDF arXiv

Understanding gender biases in large language models (LLMs) is increasingly important as these systems become embedded in decision-support tools with real consequences. Prior research has focused only on a small set of models, leaving open the extent to which gender biases are common and heterogeneous across LLMs. We address this gap across ten models released between April 2025 and June 2026, spanning nine vendors, using two paradigms: gender attribution to stereotyped phrases (Study 1) and moral judgment of abuse or torture against a woman or a man to prevent a catastrophic outcome (Study 2). In Study 1, two of ten models attributed masculine-stereotyped phrases to female writers more often than the reverse, while three models showed the opposite pattern. In Study 2, several models converged on a male-disadvantaging asymmetry that was directionally consistent with a documented human tendency to protect female targets from harm, though the specific conditions under which this asymmetry emerged varied by model; three other models, by contrast, showed no variation across conditions. These results indicate that gender-related biases are common in LLMs. Their direction and magnitude, however, are highly heterogeneous, to the point that some models behave in diametrically opposite ways to others. Bias auditing should therefore be treated as an ongoing, multi-vendor process, rather than a one-time assessment.

Abstract

Social Bias in Language Models

Explore similar work

Gender bias across LLMs is common and highly heterogeneous

Harsher on Male? Evaluating LLMs on Gender-Asymmetric Moral Framing Across Diverse Conflict Scenarios

Man Made Language Models? Evaluating LLMs' Perpetuation of Masculine Generics Bias