When the Top Answer Decides: Gender Bias in Decision Models

Yeb Havinga

cs.CLOct 29, 2026

When the Top Answer Decides: Gender Bias in Decision Models

Authors: Yeb Havinga

Organizations: Independent Researcher

Abstract

Decision models are language models built to route, classify and triage: they return option probabilities from one forward pass, and an application acts on the top answer. They are offered off the shelf, trained for the decision format rather than for fairness or any one application, and we found no bias measurement in their cards. We test TypeSafe's closed Jev and five open decision models on gender-swapped yes/no decisions about people (Tamkin et al., 2023), BBQ, StereoSet and the gender-bias studies of Fulgu and Capraro (2024). Five of the six give women significantly more favourable answers than men (+0.05 to +0.10 in log-odds), and a gender swap changes 1-3.5% of top answers (Jev's partly through its call-to-call variation). The makers' recommended confidence gate removes many biased commitments but raises the stereotyped share of those it keeps in ambiguous BBQ gender questions (granite-switch 0.76 to 0.85, pplx 0.81 to 1.00). Probes show gender preferences in the same models: all six rate violence for a greater good as more acceptable against a man than against a woman, and two default to "girl" when forced to guess a child's gender without a cue. Decision-level and probability-level measures differ, so audits should read a model as its application does.

Figures & tables

Decision model	Control	Precision
Jev 1.14	Jev 1.13; a second pass	API
pplx-decider 27B	Qwen3.8-27B a	Q8_0 b
clef-flash 9B	Qwen3.5-9B	bf16
granite-switch 8B	its text interface	bf16
Kev 4B	Qwen3.5-4B-Base	bf16
d1 3B	none	bf16

Table 1: The decision models and their controls (base model, teacher or interface). Makers: TypeSafe AI (Jev), Perplexity ( pplx-decider-v1.1-27b ), Cloudflare, IBM ( granite-switch-4.1-8b ), Jared Palmer (Kev), Liquid AI (d1; its base LFM2.5-VL-3B was not run) and AutoTrust AI (JEV). Precision: the released 16-bit, except the 27B models, whose 16-bit weights do not fit in two 24 GB GPUs; GGUF sources in Appendix A . a Also with generated answers. b Our conversion. c Released in bf16.

Test	What the model is asked	Requests
Study 1	Who wrote a child’s phrase: the original’s 40 phrases and 8 without a gender cue; female, male or non-binary; 6 wordings $\times$ 3 option orders	864
Study 1u	The same with “can’t tell” as a fourth option, the abstention option; 6 wordings $\times$ 4 orders	1,152
Study 2a/2b	How acceptable it is to insult, harass, abuse or torture a woman or a man, or to sacrifice one person, to stop a nuclear apocalypse and without a stake; a 1–7 rating listed both ways, and yes/no; 10 wordings	540
Study 2e	The four acts under four more stakes: a pandemic, hostages, a bomb, a kidnapped child	960
Study 2c, 2d	The original’s explicit ranking of violations and its Neo Metropolis vignette	216
BBQ	Which of two people fits a stereotyped question, with and without the evidence: all 50 gender-identity templates (1,152); eight more categories, up to 4 fillings per template (3,580)	4,732

Table 2: The tests: 19,405 requests per model (9,955 for the JEV and base models, which did not run the decision task). Study 1 follows the original’s Studies 1a–1c, which asked GPT-3.5, GPT-4 and GPT-4o; we compare with GPT-4 (Study 1b). Studies 2a–2d follow the original’s; 1u, 2e, the insult item, the no-stake control and the pairwise comparisons in 2c are ours. Decision task: Tamkin et al. (2023) .

Figure 1: Swapping only the gender moves P(yes) and changes some decisions. Left: woman (dot) or non-binary person (diamond) minus man in logit P(yes), in the same decision, age and race, averaged per decision, with 95% intervals over the 70 decisions; positive favours the woman or the non-binary person. Right: share of the 3,150 woman–man pairs whose top answer differs, split into “woman yes, man no” (dark) and the reverse (light), with the count. Grey: Jev’s own variation, the share of 9,450 identical requests whose top answer changed in a second pass; the local models, sent one request at a time, repeat their answers exactly.

Figure 2: The decision-level measure shows more disparity than the probability-level one. Ring (probability-level): mean probability of “girl” rather than “boy” for the eight uncued phrases, of the stereotyped person among the two people in BBQ gender questions without evidence, and of the stereotyped sentence among the two meaningful ones in StereoSet. Dot (decision-level): share of top (argmax) answers that choose it. 0.5 is no preference. BBQ: Jev’s top answers name no person and JEV-27B’s two. JEV models: Appendix G .

Figure 3: For every act, each model rates the man’s case as more acceptable; the vignette contrast is small. Acceptability against a man minus against a woman, in points on the 1–7 scale. Dot: nuclear apocalypse, 95% interval over 20 variants (10 wordings, both scale directions). Diamond: mean over five stakes (apocalypse, pandemic, hostages, bomb, kidnapped child), resampling stakes and wordings together. Intervals shorter than their marker are hidden. Ring: GPT-4 under the apocalypse, from sampled chat answers ( Fulgu and Capraro, 2024a , Studies 2a, 2b and 2d) ; the original has no insult item, and its torture gap (0.49) is not significant. Vignette: Amanda against Adam minus Andrew against Anna (Study 2d, one scenario). JEV models: Appendix G .

Figure 4: Without a gender cue, two models default to “girl” when forced to choose; most choose “can’t tell” when offered. Top answers for the eight phrases without a gender cue, over wordings and option orders. Upper bar: forced choice among girl, boy and non-binary (hatched; Study 1, 18 variants per phrase). Lower bar: with “can’t tell” offered (Study 1u, 24 variants per phrase). Right: share answering “girl”.

Figure 5: The models differ most in how often they name a person; most of those that do name the stereotyped one. Gender items. Left: StereoSet, share of the 497 dev-set gender items in which the stereotyped sentence is preferred, over three option orders, with a 95% interval over items; 50 is no preference. Middle: share of the 570 BBQ gender questions without evidence in which the top answer names a person rather than “unknown”, the right answer. Right: share of those committed answers that name the stereotyped person, with a 95% interval over templates; a count where fewer than 10 answers commit. BBQ’s score

s_{\mathrm{AMB}}

(1-\mathrm{accuracy})\times(2\times\mathrm{share}-1)

, so a model that seldom commits scores near 0.

Figure 6: The stereotype direction holds in all nine BBQ categories. Ambiguous questions: share of committed answers that name the stereotyped person; 0.5 is no preference. Bold: the 95% interval over templates excludes 0.5. Hatched: fewer than 10 committed answers, given as a count. The printed values carry the result; the colour repeats them. Commitment rates and intervals: Appendix F .

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Maker	License	Weights as run (revision)	Backbone	Decision output	Precision	Serving
Jev 1.14, 1.13	TypeSafe AI	closed, API terms	versioned ids	not disclosed	not disclosed	–	TypeSafe API
pplx-decider-v1.1-27b	Perplexity	Apache-2.0	our Q8_0 of the release @14c9d90	Qwen3.8-27B	decision head (llama.cpp type pplx-decider )	Q8_0	llama.cpp, 2 GPUs
clef-flash	Cloudflare	Apache-2.0	bartowski GGUF @5fcdd9b	Qwen3.5-9B	decision blocks (type clef )	bf16	llama.cpp
granite-switch-4.1-8b	IBM	Apache-2.0	IBM GGUF @9581e33	Granite 4.1 8B	decision aLoRA, label probabilities	bf16	llama.cpp, slot erased
Kev 4B	Jared Palmer	Apache-2.0	ggml-org GGUF @f4624b6 of the release @6cfce5c	Qwen3.5-4B-Base	LoRA adapter and pointer head	bf16	llama.cpp
d1 3B	Liquid AI	LFM 1.0	Liquid AI GGUF @bb1e436	LFM2.5-VL-3B	decision head (type lfm2-d1 )	bf16	llama.cpp

Appendix

Table 3: The decision models as run. Licenses, backbones and decision outputs as stated in the model cards and GGUF headers; repositories, revisions and server arguments in results/servers-z390.json . llama.cpp runs sent one request at a time, vLLM and API runs eight at once.

Category	Jev	pplx	clef-flash	granite-switch	Kev	d1
Age	– (2%)	0.94 (58%)	0.81 (81%)	0.77 (96%)	0.88 (33%)	0.83 (57%)
Disability status	– (3%)	0.96 (47%)	0.82 (82%)	0.86 (96%)	0.78 (31%)	0.86 (39%)
Nationality	– (4%)	0.73 (22%)	0.66 (69%)	0.71 (76%)	0.67 (30%)	0.55 (32%)
Physical appearance	– (1%)	0.91 (35%)	0.87 (77%)	0.89 (86%)	0.82 (26%)	0.90 (44%)
Race and ethnicity	– (0%)	0.91 (6%)	0.68 (50%)	0.65 (62%)	0.70 (15%)	0.61 (18%)
Religion	1.00 (6%)	0.76 (24%)	0.76 (50%)	0.75 (78%)	0.66 (22%)	0.75 (28%)

Appendix

Table 4: BBQ, ambiguous questions: stereotype-consistent share of committed answers (share of questions in which the model commits to a person); –: fewer than 10 committed answers.

Against Jev 1.14’s first pass: the noise floor
Model	Study 1	BBQ	StereoSet
Jev 1.14, second pass	0.003	0.001	0.002
Against Jev 1.13
Jev 1.14	0.029	0.005	0.018
JEV-27B (card 0.025)	0.120	0.068	0.119
JEV-9B (card 0.028)	0.250	0.172	0.168

Appendix

Table 5: Distance to Jev 1.13 on choice questions: mean KL(Jev 1.13

\|

model) per request, the direction the JEV cards use (0 = the same answers; probabilities floored at 0.005, since Jev reports steps of 0.01). Card: AutoTrust’s value for choice questions on its own test set. The first row compares two calls of Jev 1.14; all others compare with Jev 1.13.

	Qwen3.8-27B			Qwen3.5-9B		Qwen3.5-4B		granite-switch 8B
Measure	pplx	base, letters	base, gen.	clef- flash	base	Kev	base	deci- sion	text
Study 1 asymmetry $I_{m}-I_{f}$ (GPT-4 $+0.27$ )	$+0.11$	$+0.16$	$+0.15$	$+0.05$	$+0.10$	$-0.08$	$+0.13$	$+0.17$	$+0.14$
Uncued phrases labelled a girl’s	47%	42%	44%	85%	51%	57%	74%	82%	49%
Harassment gap, apocalypse (GPT-4 $+2.34$ )	$+2.76$	$+1.38$	$+1.80$	$+1.30$	$+0.64$	$+0.93$	$+0.46$	$+1.51$	$+0.69$
Harassment gap, five stakes	$+1.98$	$+1.49$		$+1.25$	$+0.90$	$+1.09$	$+0.59$	$+1.26$	$+1.08$
Torture gap, apocalypse (GPT-4 $+0.49$ )	$+1.15$	$+1.50$	$+0.59$	$+0.85$	$+0.47$	$+0.74$	$+0.34$	$+1.42$	$+0.46$

Appendix

Table 6: Decision models against their controls, grouped by backbone. Bases: Qwen3.8-27B (read through option letters, and as five generated answers per request), Qwen3.5-9B and Qwen3.5-4B-Base (option letters); granite-switch through its decision interface and its own text interface (option letters). Empty cells: Study 2e, BBQ and StereoSet were not run with generated answers.

Check	Jev	pplx	clef-flash	granite-switch	Kev	d1	JEV-27B	JEV-9B
Median time per answer	186 ms a	138 ms b	68 ms	62 ms	31 ms	16 ms	192 ms b	91 ms
The same request repeated: largest probability difference	0.11	0	0	0	0	0	0.008	0.015
Four requests at once: largest probability difference	0.16	0.028	0	n/a c	0	0.011	0.008	0.012
Study 1: option order changes the top answer	11.1%	14.9%	0%	7.3%	29.9%	22.9%	4.5%	4.9%
StereoSet: option order changes the top answer	8.7%	26.2%	0%	12.9%	24.7%	13.9%	11.1%	11.5%
Ten wordings of one Study 2 question: SD of the mean rating	0.24	0.25	0.34	0.43	0.18	0.38	0.14	0.23

Appendix

Table 7: Reliability checks of the main runs, each local model on one RTX 3090 unless noted. All llama.cpp runs sent one request at a time; the JEV runs in vLLM sent eight at once. a TypeSafe’s API, including the network from the Netherlands. b Two RTX 3090s. c Not comparable: granite-switch’s slot is emptied before every request (Appendix A ), and the erase calls of parallel requests interleave.

Model, 4-bit file	KL	harass	abuse	torture	“girl”	commits	stereotyped	ss
		Study 2 gap, apocalypse			Uncued	BBQ, no evidence		StereoSet
pplx, Q4_K_S	0.006	$+2.76$ / $+2.62$	$+1.78$ / $+1.41$	$+1.15$ / $+1.21$	47/46	8/6	0.81/0.92	76.5/76.5
clef-flash, Q4_K_S	0.016	$+1.30$ / $+1.27$	$+0.44$ / $+0.36$	$+0.85$ / $+0.90$	85/90	33/31	0.76/0.73	76.5/75.9
granite, Q4_K_M	0.017	$+1.51$ / $+1.37$	$+0.59$ / $+0.46$	$+1.42$ / $+1.25$	82/81	61/67	0.76/0.75	77.1/76.7
Kev, Q4_K_M	0.005	$+0.93$ / $+0.90$	$+0.44$ / $+0.41$	$+0.74$ / $+0.72$	57/44	13/15	0.59/0.54	74.4/75.5
d1, Q4_K_M	0.008	$+0.85$ / $+0.75$	$+0.43$ / $+0.50$	$+0.40$ / $+0.37$	56/60	22/20	0.72/0.72	64.0/64.2

Appendix

Table 8: Main run (16-bit; pplx and Qwen3.8-27B 8-bit) against the 4-bit run of the same model: mean KL from the main run’s answers to the 4-bit run’s, and each measure as main / 4-bit (“girl” and commits in %; gen.: five generated answers per request). Stereotyped: stereotype-consistent share of committed answers. Source: results/summary.md , “Quantization check”.

	granite-switch (label logprobs)			Kev (decision head)
Comparison	median	max	changed	median	max	changed
GPU against CPU, bf16	0.0004	0.013	0 of 41	0.0008	0.008	1 of 41
GPU against CPU, Q4_K_M	0.0056	0.226	3 of 41	0.0069	0.080	2 of 122
Batch of 2048 against 128 tokens, bf16	0.0000	0.024	1 of 122	0.0000	0.004	0 of 122
Batch of 2048 against 128 tokens, Q4_K_M	0.0000	0.096	0 of 122	0.0000	0.024	1 of 122
Batch of 2048 against 1 token, Q4_K_M	0.0052	0.216	3 of 122	not possible

Appendix

Table 9: Precision against numeric path: largest probability difference per request (median and maximum), and the number of requests whose top answer changes. CPU runs cover 41 of the 122 requests, except Kev in Q4_K_M. Kev cannot run with one token per batch, because a question and its options must fit in one batch.

Run	insult	harass	abuse	torture
Jev 1.14	$+1.08$ [0.90, 1.25]	$+1.65$ [1.47, 1.83]	$+1.44$ [1.26, 1.62]	$+1.08$ [0.91, 1.26]
pplx 27B	$+1.64$ [1.31, 1.96]	$+1.99$ [1.66, 2.31]	$+1.07$ [0.75, 1.40]	$+1.15$ [0.83, 1.48]
clef-flash	$+0.87$ [0.37, 1.37]	$+1.25$ [0.75, 1.75]	$+0.75$ [0.25, 1.25]	$+0.79$ [0.29, 1.29]
granite-switch	$+1.59$ [0.92, 2.26]	$+1.26$ [0.59, 1.93]	$+0.54$ [ $-$ 0.13, 1.21]	$+1.08$ [0.41, 1.75]
Kev	$+0.96$ [0.70, 1.22]	$+1.09$ [0.83, 1.35]	$+0.55$ [0.29, 0.81]	$+0.58$ [0.32, 0.84]
d1	$+0.46$ [0.20, 0.72]	$+0.52$ [0.27, 0.78]	$+0.27$ [0.01, 0.53]	$+0.23$ [ $-$ 0.03, 0.49]

Appendix

Table 10: Study 2, mixed-effects victim effect on the mean rating (1–7 scale): acceptability against the man minus against the woman, with crossed random intercepts for wording and stake; 95% Wald intervals. Logit P(yes): results/mixed.md .

Run	female	non-binary	age per decade	race (range of four)
Jev 1.14	$+0.050$ [0.036, 0.064]	$+0.046$ [0.017, 0.075]	$-0.009$ [ $-$ 0.011, $-$ 0.008]	$+0.001$ to $+0.109$
pplx 27B	$+0.080$ [0.041, 0.120]	$+0.210$ [0.143, 0.278]	$-0.008$ [ $-$ 0.011, $-$ 0.005]	$-0.012$ to $+0.196$
clef-flash	$+0.097$ [0.055, 0.139]	$+0.145$ [0.060, 0.231]	$-0.022$ [ $-$ 0.025, $-$ 0.020]	$+0.057$ to $+0.158$
granite-switch	$+0.082$ [0.054, 0.109]	$+0.126$ [0.077, 0.175]	$-0.040$ [ $-$ 0.042, $-$ 0.037]	$+0.090$ to $+0.134$
Kev	$+0.058$ [0.038, 0.078]	$+0.060$ [0.026, 0.093]	$-0.004$ [ $-$ 0.005, $-$ 0.003]	$+0.016$ to $+0.079$
d1	$+0.018$ [ $-$ 0.000, 0.036]	$+0.024$ [ $-$ 0.008, 0.056]	$-0.001$ [ $-$ 0.002, $-$ 0.000]	$+0.009$ to $+0.048$

Appendix

Table 11: Decision task, mixed-effects estimates on logit P(yes) against a 60-year-old white man, with a random intercept and random gender slopes per decision; 95% Wald intervals. Race: Black, Hispanic, Asian and Native American, each against white.

Run	asymmetry	shift	female	non-binary
	Study 1 (16 tests)		Decision task (12 tests)
Jev 1.14	.005 / .075	.028 / .360	${<}.001$ / ${<}.001$	.002 / .006
pplx 27B	.050 / .447	.038 / .416	${<}.001$ / ${<}.001$	${<}.001$ / ${<}.001$
clef-flash	.473 / 1	.691 / 1	${<}.001$ / ${<}.001$	${<}.001$ / .002
granite-switch	.045 / .447	.532 / 1	${<}.001$ / ${<}.001$	${<}.001$ / ${<}.001$
Kev	.157 / .783	.048 / .447	${<}.001$ / ${<}.001$	${<}.001$ / .004

Appendix

Table 12: Raw / Holm-adjusted p-values (two-sided permutation tests). Study 1: the attribution asymmetry

I_{m}-I_{f}

and the shift, how far feminine phrases move the answer toward “girl” against masculine phrases toward “boy”. Decision task: woman and non-binary person against man. Source: results/summary.md , “Multiple comparisons”.

Gender Bias in Language Models

Gender bias in language models examines how systems encode and express gender stereotypes and how to evaluate and reduce their effects.

See all 67 papers

Gender bias across LLMs is common and highly heterogeneous
Sep 29, 2026Edoardo Bolzoni, Valerio CapraroGender Bias in Language Models LLM Auditing
University of Milan-Bicocca, Milan, Italy
Do Gender Cues Affect LLM Value Trade-offs? Evidence from a Controlled Decision Benchmark
Jun 1, 2026Yangyang Liu, Dong Yu, Pengyuan LiuGender Bias in Language Models Moral Reasoning in Language Models
Beijing Language and Culture University
Are AI Decision Models Biased? A Case Study of Jev
Oct 14, 2026Ilaria Imperatrice, Taisia Kameneva, Adam Eric Greenberg +1Cognitive Bias in Language Models LLM Decision-Making
Bocconi University

Sep 29, 2026J/K move · Enter open · S save

Gender bias across LLMs is common and highly heterogeneous

Edoardo Bolzoni, Valerio Capraro

University of Milan-Bicocca, Milan, Italy

Gender Bias in Language Models LLM Auditing Language Model Bias Evaluation

Open paper AskPDF arXiv

Understanding gender biases in large language models (LLMs) is increasingly important as these systems become embedded in decision-support tools with real consequences. Prior research has focused only on a small set of models, leaving open the extent to which gender biases are common and heterogeneous across LLMs. We address this gap across ten models released between April 2025 and June 2026, spanning nine vendors, using two paradigms: gender attribution to stereotyped phrases (Study 1) and moral judgment of abuse or torture against a woman or a man to prevent a catastrophic outcome (Study 2). In Study 1, two of ten models attributed masculine-stereotyped phrases to female writers more often than the reverse, while three models showed the opposite pattern. In Study 2, several models converged on a male-disadvantaging asymmetry that was directionally consistent with a documented human tendency to protect female targets from harm, though the specific conditions under which this asymmetry emerged varied by model; three other models, by contrast, showed no variation across conditions. These results indicate that gender-related biases are common in LLMs. Their direction and magnitude, however, are highly heterogeneous, to the point that some models behave in diametrically opposite ways to others. Bias auditing should therefore be treated as an ongoing, multi-vendor process, rather than a one-time assessment.

Abstract

Gender Bias in Language Models

Explore similar work

Gender bias across LLMs is common and highly heterogeneous

Do Gender Cues Affect LLM Value Trade-offs? Evidence from a Controlled Decision Benchmark

Are AI Decision Models Biased? A Case Study of Jev