LLMs Cannot Be Neutral: Gender Sensitivity Across Candidates, Evaluators, and Models
Authors: Danial Amin, Joni Salminen, Bernard J. Jansen
Organizations: University of Vaasa, Vaasa, Finland · Otherkind Tech, Helsinki, Finland · Peking University, Beijing, China · Nankai University, Tianjin, China
Large language models (LLMs) are increasingly used to evaluate candidates for professional roles, raising concerns about their sensitivity to demographic identities. We examine how candidate and evaluator gender assignments affect decisions by LLM-as-a-Judge (LaJ) models and decision language models (DLMs). In a hypothetical city council voting task, eight models evaluated 240 synthetic professional narratives from NVIDIA's US and French Nemotron-Personas datasets under independently varied gender assignments, producing 30,720 judgments. Explicit candidate gender labels increased Yes-vote probability by 1.05-1.73 percentage points relative to unspecified gender, while only the nonbinary evaluator assignment showed a significant increase (1.26 points). Identity matching showed no significant overall association. Yes-vote rates varied across models (81.3-97.2%), and 2.5-3.3% of decisions reversed relative to the unspecified-identity baseline. Under direct substitutions between explicit gender identities, DLMs exhibited fewer decision reversals than LaJ models (1.18% vs. 3.38%), indicating lower observed decision-level gender sensitivity among the tested DLMs. These findings demonstrate that gender sensitivity depends on candidate and evaluator identities and the evaluation model, while highlighting the importance of examining individual decision consistency alongside aggregate differences.
Figures & tables
Figure 1. Three sources of variation in AI-mediated voting decisions. Each panel changes one element of the evaluation while the candidate’s professional narrative stays fixed. Left: Evaluated persona effect. The assigned gender identity of the evaluated persona varies. Center: Evaluator persona effect. The assigned gender identity of the AI evaluator varies. Right: Evaluation system effect. General-purpose LLM judges and decision language models evaluate the same persona under equivalent conditions. Three side-by-side illustrations showing the evaluated persona effect, evaluator persona effect, and evaluation system effect in AI-mediated voting decisions.
Concept
Definition
Evaluated persona
Synthetic representation of a hypothetical candidate, derived from the Nemotron-Personas datasets.
Candidate gender identity
Gender identity assigned to the evaluated persona.
Evaluator persona
Role assigned to an AI evaluator through prompting.
Evaluator gender identity
Gender identity assigned to the evaluator persona.
Evaluation model
AI system responsible for producing the judgment.
LLM-as-a-Judge (LaJ)
General-purpose LLM prompted to evaluate a candidate against specified criteria.
Table 1. Key concepts used in the study.
Figure 2. Overview of the experimental procedure. Each of the 240 prepared narratives was combined with all 16 candidate gender identity and evaluator gender identity assignments and evaluated by four LaJ models and four DLMs, producing 30,720 binary voting decisions. Narratives were prepared once, before identities were assigned, so the professional information was identical across conditions.
Component
Levels
Description
Candidate gender identity
4
Man, woman, nonbinary, unspecified
Evaluator gender identity
4
Man, woman, nonbinary, unspecified
Evaluation model
8
Four LaJ models and four DLMs
Identity combinations
16
4×4 per narrative and model
Explicit-identity combinations
9
3 matching and 6 nonmatching
Combinations with unspecified identity
7
At least one identity unspecified
Table 2. Experimental design and number of observations. Each of the 240 narratives was evaluated under all 16 combinations of candidate gender identity and evaluator gender identity by all eight models, yielding 30,720 valid votes. The nine combinations with explicit identities on both sides form the basis of the matching analysis.
Source
Occupation category
Eligible records by age
18–39
40–64
US
Secondary school teacher
1,652
2,290
Construction laborer
3,708
3,735
France
Cadres
37,853
58,244
Ouvriers
57,652
62,831
Total eligible
100,865
127,100
Table 3. Stratified sampling of professional narratives. Note. Of 227,965 eligible records, 30 narratives were sampled from each of the eight strata ( N=240 ). Equal allocation does not reflect national population distributions. Occupation categories are not directly comparable across countries. Cadres refers to higher-level professional and intellectual occupations; Ouvriers refers to manual-worker occupations.
Type
Company
Model
OpenRouter tag
LaJ
Anthropic
Claude Sonnet 5.5
anthropic/claude-sonnet-5.5
OpenAI
GPT-6.1-sol
openai/gpt-6.1-sol
Google
Gemini 4 Argon
google/gemini-4-argon
DeepSeek
DeepSeek V4 Pro
deepseek/deepseek-v4-pro-0813
DLM
Typesafe
Jev
typesafe/jev-1.14
Microsoft
Microsoft-Decision-1
microsoft/microsoft-decision-1
Table 4. Evaluation models included in the experiment. Note. All models were accessed through OpenRouter. LaJ models used the chat interface at temperature zero, whereas DLMs used a structured decision interface without an equivalent temperature setting.
Measure
Definition
Use
Binary vote
Yes = 1; No = 0
Primary outcome
Yes-vote rate
Yes votes divided by valid votes in a condition
Descriptive
Identity difference
Difference in Yes-vote probability between an explicit identity and its unspecified reference, for the same narratives
H1, H2
Identity matching
Additional association of matching explicit identities with a Yes vote after accounting for the two identity labels
H3
Model differences
Differences in voting outcomes among the eight models and between LaJ and DLM categories
H4
Model-specific identity effects
Differences between models in the effects of candidate gender identity, evaluator gender identity, or identity matching
H5
Table 5. Voting outcome and derived measures. The binary vote is the primary outcome. Identity differences, identity matching, and model differences address H1–H5, while decision reversals and sampling-group differences are exploratory. Reversals are defined against the same narrative and model with both identities unspecified.
Test
Statistical comparison
Reported effect
H1
Candidate gender identity versus unspecified, adjusting for evaluator gender identity and evaluation model
Matching versus non-matching explicit identities, adjusting for both labels and model
Adjusted matching association
H4
Joint Wald test across eight models; separate LaJ–DLM category contrast
Model and category differences
H5
Model interactions with candidate gender identity, evaluator gender identity, and identity matching (matching tested separately)
Differences in identity effects by model
Table 6. Statistical analyses used to test the hypotheses. All analyses use binomial models with standard errors clustered by narrative. Candidate gender identity and evaluator gender identity are each compared with the unspecified condition (H1, H2) before matching is tested beyond both labels (H3). Model differences (H4) and model-specific identity effects (H5) are tested with joint Wald tests.
Factor
Label
Δ Yes (pp), 95% CI
OR [95% CI]
Evaluated
Man
1.05 [0.59, 1.52]
1.10 [1.06, 1.15]
Woman
1.41 [0.90, 1.92]
1.14 [1.09, 1.19]
Nonbinary
1.73 [1.17, 2.29]
1.17 [1.12, 1.23]
Evaluator
Man
0.44 [0.02, 0.87]
1.04 [1.00, 1.08]
Woman
−0.07 [ −0.45 , 0.32]
0.99 [0.96, 1.03]
Nonbinary
1.26 [0.78, 1.75]
1.13 [1.08, 1.18]
Table 7. Gender contrasts relative to the unspecified condition. All three candidate gender identity contrasts were significant (1.05–1.73 percentage points; odds ratios 1.10–1.17), whereas among evaluator gender identities only nonbinary was (1.26 points; odds ratio 1.13). The evaluator-man contrast did not remain significant after Holm adjustment.
Figure 3. Change in Yes-vote probability when an explicit gender label replaced the unspecified condition. Every explicit candidate gender identity increased Yes votes (1.05–1.73 percentage points, all adjusted p<.001 ), whereas only the nonbinary evaluator gender identity did so (1.26 points); the man and woman evaluator gender identities were not significant after Holm adjustment. Points show within-narrative estimates and bars show 95% confidence intervals; filled markers indicate Holm-adjusted p<.05 .
Figure 4. Yes-vote rates for the eight evaluation models. Rates ranged from 81.3% for Gemini 4 Argon to 97.2% for OpenAI Decisions, a spread larger than any gender effect. DLMs returned Yes more often than LaJ models on average (88.9% vs. 86.1%), but rates varied widely within both groups. Each line runs from the group mean (dashed) to the model’s rate; each model contributed 3,840 judgments.
Figure 5. Candidate gender identity effects by evaluation model. Gemini 4 Argon showed the largest effects, with all three labels increasing Yes votes by 3.54–4.38 percentage points, whereas no label had a significant effect for Claude Sonnet 5.5, GPT-6.1-sol, or Microsoft-Decision-1. Estimates are changes in Yes-vote probability relative to the unspecified candidate gender identity; bars show 95% confidence intervals, marker shape indicates gender, and filled markers indicate Holm-adjusted p<.05 within the model.
Figure 6. Evaluator gender identity effects by evaluation model. The pattern differed from the candidate gender identity effects: a nonbinary evaluator gender identity increased Yes votes for Claude Sonnet 5.5 (3.02 percentage points) and DeepSeek V4 Pro (2.40), while no evaluator gender identity had a significant effect for Gemini 4 Argon. DeepSeek V4 Pro also showed the only significant negative contrast ( −1.46 points for a woman evaluator gender identity). Estimates are relative to the unspecified evaluator gender identity; encoding as in Figure 5 .
Figure 7. Decision reversals relative to the unspecified/unspecified baseline for the same narrative and evaluation model. Most reversals were from No to Yes. Gemini 4 Argon reversed 7.36% of its votes when only the candidate gender identity changed, compared with 0.28% for Microsoft-Decision-1, and DLMs reversed less often than LaJ models under all three manipulations ( −2.30 to −2.81 percentage points). Bars left of zero show Yes-to-No changes and bars right of zero show No-to-Yes changes; panel headers give the pooled reversal rate and the DLM–LaJ difference.
Understanding gender biases in large language models (LLMs) is increasingly important as these systems become embedded in decision-support tools with real consequences. Prior research has focused only on a small set of models, leaving open the extent to which gender biases are common and heterogeneous across LLMs. We address this gap across ten models released between April 2025 and June 2026, spanning nine vendors, using two paradigms: gender attribution to stereotyped phrases (Study 1) and moral judgment of abuse or torture against a woman or a man to prevent a catastrophic outcome (Study 2). In Study 1, two of ten models attributed masculine-stereotyped phrases to female writers more often than the reverse, while three models showed the opposite pattern. In Study 2, several models converged on a male-disadvantaging asymmetry that was directionally consistent with a documented human tendency to protect female targets from harm, though the specific conditions under which this asymmetry emerged varied by model; three other models, by contrast, showed no variation across conditions. These results indicate that gender-related biases are common in LLMs. Their direction and magnitude, however, are highly heterogeneous, to the point that some models behave in diametrically opposite ways to others. Bias auditing should therefore be treated as an ongoing, multi-vendor process, rather than a one-time assessment.
Large language models are increasingly used in value-sensitive decision settings, where irrelevant demographic cues should not alter judgments. We construct the Realistic Value Decision Benchmark (RVDB), a controlled benchmark that varies only the role-gender configuration while holding the scenario, ordered value pair, roles, candidate decisions, Value Distance, and Decision Severity fixed. Using a position-balanced evaluation across seven models, we test whether models preserve decision invariance under gender perturbations and whether their self-attributions reflect observed behavioral changes. We find that explicit gender cues induce bounded but systematic decision flips, including under an explicit gender-attribution prompt that asks models to report whether gender influenced their choice. Cross-gender role swaps reveal a consistent female-proposed-decision asymmetry, while models often attribute flipped decisions to No Influence or other non-gender factors. Further analysis shows that gender effects concentrate near less determinate value boundaries and under more severe decision contexts, suggesting that gender cues act as local boundary-shifting factors rather than global overrides of value reasoning. Value rankings remain largely stable, but ordered value-pair trade-offs shift unevenly across role-gender configurations. These results show that gender can enter LLM value trade-offs behaviorally while remaining obscured in self-attribution, motivating controlled behavioral audits beyond explanation-based evaluation.
Whether a language model looks demographically biased can depend on how the audit asks its question. A charitable-aid benchmark reports that the same models favor minority applicants when rating requests one at a time and penalize some when ranking side by side. We test whether that reversal generalizes to hiring, lending, and medical triage: 40,726 requests to five models, applications differing only in the applicant's name, and a primary test fixed before collection. It does not. None of 36 planned contrasts survives correction. The rating advantage keeps its sign at roughly half the published size, and a precision extension bounds any hiring ranking penalty below the published effect, though the lending and triage ranking floors sit above that margin, so the exclusion is conclusive for hiring ranking and for rating in all three domains only. Planted disparities tracking their injected sizes and a directional replication on the original aid materials bound these nulls. The audit is livelier than the demographics: models recognize transparent audits nearly always, tie every identical-content comparison whether the varying detail is race or a hobby, and reward first-listed candidates as much as any demographic effect we measure. Audit verdicts reflect audit construction more than demographic bias.
Siddharth Vohra, Manikandan Ravikiran
Carnegie Mellon University Amazon Web Services AI Native Pittsburgh, PA, USA · Indian Institute of Technology Mandi, India