LLMs Cannot Be Neutral: Gender Sensitivity Across Candidates, Evaluators, and Models
Authors: Danial Amin, Joni Salminen, Bernard J. Jansen
Organizations: University of Vaasa, Vaasa, Finland · Otherkind Tech, Helsinki, Finland · Peking University, Beijing, China · Nankai University, Tianjin, China
Large language models (LLMs) are increasingly used to evaluate candidates for professional roles, raising concerns about their sensitivity to demographic identities. We examine how candidate and evaluator gender assignments affect decisions by LLM-as-a-Judge (LaJ) models and decision language models (DLMs). In a hypothetical city council voting task, eight models evaluated 240 synthetic professional narratives from NVIDIA's US and French Nemotron-Personas datasets under independently varied gender assignments, producing 30,720 judgments. Explicit candidate gender labels increased Yes-vote probability by 1.05-1.73 percentage points relative to unspecified gender, while only the nonbinary evaluator assignment showed a significant increase (1.26 points). Identity matching showed no significant overall association. Yes-vote rates varied across models (81.3-97.2%), and 2.5-3.3% of decisions reversed relative to the unspecified-identity baseline. Under direct substitutions between explicit gender identities, DLMs exhibited fewer decision reversals than LaJ models (1.18% vs. 3.38%), indicating lower observed decision-level gender sensitivity among the tested DLMs. These findings demonstrate that gender sensitivity depends on candidate and evaluator identities and the evaluation model, while highlighting the importance of examining individual decision consistency alongside aggregate differences.
Figures & tables
Figure 1. Three sources of variation in AI-mediated voting decisions. Each panel changes one element of the evaluation while the candidate’s professional narrative stays fixed. Left: Evaluated persona effect. The assigned gender identity of the evaluated persona varies. Center: Evaluator persona effect. The assigned gender identity of the AI evaluator varies. Right: Evaluation system effect. General-purpose LLM judges and decision language models evaluate the same persona under equivalent conditions. Three side-by-side illustrations showing the evaluated persona effect, evaluator persona effect, and evaluation system effect in AI-mediated voting decisions.
Concept
Definition
Evaluated persona
Synthetic representation of a hypothetical candidate, derived from the Nemotron-Personas datasets.
Candidate gender identity
Gender identity assigned to the evaluated persona.
Evaluator persona
Role assigned to an AI evaluator through prompting.
Evaluator gender identity
Gender identity assigned to the evaluator persona.
Evaluation model
AI system responsible for producing the judgment.
LLM-as-a-Judge (LaJ)
General-purpose LLM prompted to evaluate a candidate against specified criteria.
Table 1. Key concepts used in the study.
Figure 2. Overview of the experimental procedure. Each of the 240 prepared narratives was combined with all 16 candidate gender identity and evaluator gender identity assignments and evaluated by four LaJ models and four DLMs, producing 30,720 binary voting decisions. Narratives were prepared once, before identities were assigned, so the professional information was identical across conditions.
Component
Levels
Description
Candidate gender identity
4
Man, woman, nonbinary, unspecified
Evaluator gender identity
4
Man, woman, nonbinary, unspecified
Evaluation model
8
Four LaJ models and four DLMs
Identity combinations
16
4×4 per narrative and model
Explicit-identity combinations
9
3 matching and 6 nonmatching
Combinations with unspecified identity
7
At least one identity unspecified
Table 2. Experimental design and number of observations. Each of the 240 narratives was evaluated under all 16 combinations of candidate gender identity and evaluator gender identity by all eight models, yielding 30,720 valid votes. The nine combinations with explicit identities on both sides form the basis of the matching analysis.
Source
Occupation category
Eligible records by age
18–39
40–64
US
Secondary school teacher
1,652
2,290
Construction laborer
3,708
3,735
France
Cadres
37,853
58,244
Ouvriers
57,652
62,831
Total eligible
100,865
127,100
Table 3. Stratified sampling of professional narratives. Note. Of 227,965 eligible records, 30 narratives were sampled from each of the eight strata ( N=240 ). Equal allocation does not reflect national population distributions. Occupation categories are not directly comparable across countries. Cadres refers to higher-level professional and intellectual occupations; Ouvriers refers to manual-worker occupations.
Type
Company
Model
OpenRouter tag
LaJ
Anthropic
Claude Sonnet 5.5
anthropic/claude-sonnet-5.5
OpenAI
GPT-6.1-sol
openai/gpt-6.1-sol
Google
Gemini 4 Argon
google/gemini-4-argon
DeepSeek
DeepSeek V4 Pro
deepseek/deepseek-v4-pro-0813
DLM
Typesafe
Jev
typesafe/jev-1.14
Microsoft
Microsoft-Decision-1
microsoft/microsoft-decision-1
Table 4. Evaluation models included in the experiment. Note. All models were accessed through OpenRouter. LaJ models used the chat interface at temperature zero, whereas DLMs used a structured decision interface without an equivalent temperature setting.
Measure
Definition
Use
Binary vote
Yes = 1; No = 0
Primary outcome
Yes-vote rate
Yes votes divided by valid votes in a condition
Descriptive
Identity difference
Difference in Yes-vote probability between an explicit identity and its unspecified reference, for the same narratives
H1, H2
Identity matching
Additional association of matching explicit identities with a Yes vote after accounting for the two identity labels
H3
Model differences
Differences in voting outcomes among the eight models and between LaJ and DLM categories
H4
Model-specific identity effects
Differences between models in the effects of candidate gender identity, evaluator gender identity, or identity matching
H5
Table 5. Voting outcome and derived measures. The binary vote is the primary outcome. Identity differences, identity matching, and model differences address H1–H5, while decision reversals and sampling-group differences are exploratory. Reversals are defined against the same narrative and model with both identities unspecified.
Test
Statistical comparison
Reported effect
H1
Candidate gender identity versus unspecified, adjusting for evaluator gender identity and evaluation model
Matching versus non-matching explicit identities, adjusting for both labels and model
Adjusted matching association
H4
Joint Wald test across eight models; separate LaJ–DLM category contrast
Model and category differences
H5
Model interactions with candidate gender identity, evaluator gender identity, and identity matching (matching tested separately)
Differences in identity effects by model
Table 6. Statistical analyses used to test the hypotheses. All analyses use binomial models with standard errors clustered by narrative. Candidate gender identity and evaluator gender identity are each compared with the unspecified condition (H1, H2) before matching is tested beyond both labels (H3). Model differences (H4) and model-specific identity effects (H5) are tested with joint Wald tests.
Factor
Label
Δ Yes (pp), 95% CI
OR [95% CI]
Evaluated
Man
1.05 [0.59, 1.52]
1.10 [1.06, 1.15]
Woman
1.41 [0.90, 1.92]
1.14 [1.09, 1.19]
Nonbinary
1.73 [1.17, 2.29]
1.17 [1.12, 1.23]
Evaluator
Man
0.44 [0.02, 0.87]
1.04 [1.00, 1.08]
Woman
−0.07 [ −0.45 , 0.32]
0.99 [0.96, 1.03]
Nonbinary
1.26 [0.78, 1.75]
1.13 [1.08, 1.18]
Table 7. Gender contrasts relative to the unspecified condition. All three candidate gender identity contrasts were significant (1.05–1.73 percentage points; odds ratios 1.10–1.17), whereas among evaluator gender identities only nonbinary was (1.26 points; odds ratio 1.13). The evaluator-man contrast did not remain significant after Holm adjustment.
Figure 3. Change in Yes-vote probability when an explicit gender label replaced the unspecified condition. Every explicit candidate gender identity increased Yes votes (1.05–1.73 percentage points, all adjusted p<.001 ), whereas only the nonbinary evaluator gender identity did so (1.26 points); the man and woman evaluator gender identities were not significant after Holm adjustment. Points show within-narrative estimates and bars show 95% confidence intervals; filled markers indicate Holm-adjusted p<.05 .
Figure 4. Yes-vote rates for the eight evaluation models. Rates ranged from 81.3% for Gemini 4 Argon to 97.2% for OpenAI Decisions, a spread larger than any gender effect. DLMs returned Yes more often than LaJ models on average (88.9% vs. 86.1%), but rates varied widely within both groups. Each line runs from the group mean (dashed) to the model’s rate; each model contributed 3,840 judgments.
Figure 5. Candidate gender identity effects by evaluation model. Gemini 4 Argon showed the largest effects, with all three labels increasing Yes votes by 3.54–4.38 percentage points, whereas no label had a significant effect for Claude Sonnet 5.5, GPT-6.1-sol, or Microsoft-Decision-1. Estimates are changes in Yes-vote probability relative to the unspecified candidate gender identity; bars show 95% confidence intervals, marker shape indicates gender, and filled markers indicate Holm-adjusted p<.05 within the model.
Figure 6. Evaluator gender identity effects by evaluation model. The pattern differed from the candidate gender identity effects: a nonbinary evaluator gender identity increased Yes votes for Claude Sonnet 5.5 (3.02 percentage points) and DeepSeek V4 Pro (2.40), while no evaluator gender identity had a significant effect for Gemini 4 Argon. DeepSeek V4 Pro also showed the only significant negative contrast ( −1.46 points for a woman evaluator gender identity). Estimates are relative to the unspecified evaluator gender identity; encoding as in Figure 5 .
Figure 7. Decision reversals relative to the unspecified/unspecified baseline for the same narrative and evaluation model. Most reversals were from No to Yes. Gemini 4 Argon reversed 7.36% of its votes when only the candidate gender identity changed, compared with 0.28% for Microsoft-Decision-1, and DLMs reversed less often than LaJ models under all three manipulations ( −2.30 to −2.81 percentage points). Bars left of zero show Yes-to-No changes and bars right of zero show No-to-Yes changes; panel headers give the pooled reversal rate and the DLM–LaJ difference.