Effects of Answer Format Variation on Gender Bias in Large Language Models

Ksenia Merzlyakova; Sebastian Padó; Franziska Weeber

cs.CLAug 18, 2026

Effects of Answer Format Variation on Gender Bias in Large Language Models

Authors: Ksenia Merzlyakova, Sebastian Padó, Franziska Weeber

Organizations: Institute for Natural Language Processing, University of Stuttgart

Abstract

Gender bias or other social biases in large language models (LLMs) are frequently evaluated with question answering or survey benchmarks where the LLM needs to give a response in a predefined answer format. It is well known in survey science that the answer format has a substantial impact on answers, just as LLMs are sensitive to the prompt wording. However, to our knowledge it has not been studied yet how changes in answer format impact the measurement of gender bias in LLMs and their alignment with human response distributions. We evaluate three instruction-tuned models on the BBQ benchmark and OpinionQA survey data across closed-ended, Likert-scaled and open-ended formats, comparing bias measurement and distributional alignment under otherwise identical conditions. We find that answer format does substantially alter measured outcomes, including reversals in order rankings. These differences arise because each format elicits distinct response behaviours, such as forced-choice selection, scale-based distributions and refusal in free-text generation. Our findings highlight the importance of treating answer format as a substantive component of LLM evaluation and motivate multi-format designs for more robust model assessment.

Figures & tables

Figure 1: Example of different responses to the same disambiguated, negative, non-target-first BBQ item (blue) across one closed-ended (green), one Likert-scaled (purple) and one open-ended (orange) answer format variant.

Figure 2: Overview of methodological setup. We evaluate all combinations of three answer formats (top), two datasets (center) and three models (bottom).

Figure 3: Example of an OpinionQA item (blue) with one closed-ended (green), one Likert-scaled (purple) and one open-ended (orange) answer format variant.

Figure 4: Overall bias scores for the modified gender identity BBQ questions across three queried models, shown separately for closed-ended, Likert-scaled and open-ended formats. To align with Parrish et al. (2022) , scores are reported as percentages derived from the original $[-1,+1]$ bias scale and stratified by score type ( $s_{\text{DIS}}$ and $s_{\text{AMB}}$ ). Results are further divided into evaluations of the gender identity label subset, the proper name subset, and both subsets combined. Asterisks (*) denote bias scores statistically significant from zero ( $\alpha=0.05$ ).

Figure 5: Overall distributional distance scores for the OpinionQA responses across three queried models, shown separately for closed-ended, Likert-scaled and open-ended formats. Results are reported using three metrics: Wasserstein distance (WDist), Jensen-Shannon distance (JSDist) and the composite metric $D_{\alpha}$ . No statistical significance markers are shown, all measured distances differ significantly from zero.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Question polarity	Response aligns with	Bias direction
Negative	Stereotyped group	Reinforcing ( $+$ )
Negative	Non-stereotyped group	Countering ( $-$ )
Non-negative	Stereotyped group	Countering ( $-$ )
Non-negative	Non-stereotyped group	Reinforcing ( $+$ )

Appendix

Table 1: Bias direction assignment by question polarity and response target. This logic applies across all formats (closed-ended, open-ended and Likert-scaled). In the closed-ended and open-ended conditions, the magnitude is

\pm 1

for a discrete choice of a group and 0 for ‘UNKNOWN’. In the Likert condition, the magnitude is the normalised agreement strength

\in[-1,+1]

Model	Subset	Format	$s_{\text{DIS}}$	$s_{\text{AMB}}$	Acc.
Mistral7B	Labels	Closed	$+$ 8.0*	$+$ 16.6*	71.2
		Likert	$+$ 3.2*	$+$ 1.7	18.9
		Open	$+$ 0.4	$+$ 0.1*	99.9
	Names	Closed	$+$ 8.5*	$+$ 16.2*	56.2
		Likert	$+$ 4.2*	$+$ 3.9*	24.5
		Open	$+$ 1.2	$+$ 2.7*	95.1

Appendix

Table 2: Cross-format summary of

s_{\text{DIS}}

s_{\text{AMB}}

and accuracy for all models and subset variations (identity labels, proper names, combined). Bias scores are expressed as changes in percentage points (range

-

100 to

+

100). Accuracy (Acc.) is expressed as percentage. Asterisks (*) denote scores whose 95% bootstrap CI excludes zero.

Figure 6: Results for the closed-ended BBQ questions across answer order permutations (MFU, FMU, MUF, FUM, UMF, UFM; M = male, F = female, U = ‘UNKNOWN’) and their aggregate (Overall), shown by score type ( $s_{\text{DIS}}$ , $s_{\text{AMB}}$ ) and subset (identity labels, proper names, combined). Colours and significance markers follow Figure 4 . Two point estimates ( Gemma12B $s_{\text{DIS}}$ proper name scores, MFU and FMU) exhibit a near-zero-width CI ( $<1e-6$ ); this is not visually marked in the figure.

Figure 7: Results for the Likert-scaled BBQ questions across scale lengths and polarity (2/4/5/9/10-point; ‘pos’ = highest value indicates agreement, ‘neg’ = highest value indicates disagreement), including an aggregate overall condition. Scores are shown by type ( $s_{\text{DIS}}$ , $s_{\text{AMB}}$ ) and subset (identity labels, proper names, combined). Colours and significance markers follow Figure 4 . Daggers (†) mark $s_{\text{AMB}}$ scores in 2-scale conditions: accuracy = 0 by design; $s_{\text{AMB}}$ is unweighted and not directly comparable to other conditions.

Figure 8: Results for the open-ended BBQ questions, comparing responses without and with reasoning, along with an aggregate overall condition. Scores are shown by type ( $s_{\text{DIS}}$ , $s_{\text{AMB}}$ ) and subset (identity labels, proper names, combined). Colours and significance markers follow Figure 4 . One point estimate ( Mistral7B with reasoning on identity-label questions) exhibits a degenerate CI ( $<1e-6$ ); this is not visually marked in the figure.

Model	Format	Metric	Mean	95% CI
Mistral7B	Closed	WDist	0.2034	[0.1994, 0.2074]
		JSDist	0.4081	[0.4038, 0.4126]
		Comp.	0.3058	[0.3019, 0.3097]
	Likert	WDist	0.1611	[0.1578, 0.1642]
		JSDist	0.3046	[0.3016, 0.3075]
		Comp.	0.2328	[0.2299, 0.2358]

Appendix

Table 3: Bootstrap 95% confidence intervals for overall metrics (Wasserstein, Jensen-Shannon, composite distance) by model and format. All CIs are based on 1000 resamples at the response level. Likert values are computed using per-response bootstrapping, not the scaled baseline employed for within-format robustness checks.

Figure 9: Results for the closed-ended OpinionQA questions across systematic answer-format permutations (24 total), grouped by sequence length (3/4/5/6-item reorderings), and their aggregate (Overall). Metrics and colour mapping follow Figure 5 . No statistical significance markers are shown; cells marked d indicate degenerate confidence intervals (CI width $<1e-6$ ).

Figure 10: Results for the Likert-scaled OpinionQA questions across scale lengths and polarity (2/4/5/9/10-point; ‘pos’ = highest value indicates agreement, ‘neg’ = highest value indicates disagreement), including an aggregate overall condition. Metrics and colour mapping follow Figure 5 .

Figure 11: Results for the open-ended OpinionQA questions, comparing responses without and with reasoning, along with an aggregate overall condition. Metrics and colour mapping follow Figure 5 .

Language Model Bias Evaluation

Language model bias evaluation tests how models’ outputs, decisions, and representations vary across social groups, cultures, languages, and contexts.

See all 493 papers

Gender bias across LLMs is common and highly heterogeneous
Sep 29, 2026Edoardo Bolzoni, Valerio CapraroGender Bias in Language Models LLM Auditing
University of Milan-Bicocca, Milan, Italy
BiAxisAudit: A Novel Framework to Evaluate LLM Bias Across Prompt Sensitivity and Response-Layer Divergence
May 9, 2026Jialing Gan, Junhao Dong, Songze LiPrompt Sensitivity LLM Auditing
Southeast University, China · Nanyang Technological University, Singapore
Alignment Reduces Expressed but Not Encoded Gender Bias: A Unified Framework and Study
Mar 25, 2026Nour Bouchouchi, Thibault Laugel, Xavier Renard +3Gender Bias in Language Models LLM Alignment
Sorbonne Université, CNRS, LIP6, F-75005 Paris, France · AXA, Paris, France · Polish Academy of Science, IBS PAN, Warsaw, Poland

Sep 29, 2026J/K move · Enter open · S save

Gender bias across LLMs is common and highly heterogeneous

Edoardo Bolzoni, Valerio Capraro

University of Milan-Bicocca, Milan, Italy

Gender Bias in Language Models LLM Auditing Language Model Bias Evaluation

Open paper AskPDF arXiv

Understanding gender biases in large language models (LLMs) is increasingly important as these systems become embedded in decision-support tools with real consequences. Prior research has focused only on a small set of models, leaving open the extent to which gender biases are common and heterogeneous across LLMs. We address this gap across ten models released between April 2025 and June 2026, spanning nine vendors, using two paradigms: gender attribution to stereotyped phrases (Study 1) and moral judgment of abuse or torture against a woman or a man to prevent a catastrophic outcome (Study 2). In Study 1, two of ten models attributed masculine-stereotyped phrases to female writers more often than the reverse, while three models showed the opposite pattern. In Study 2, several models converged on a male-disadvantaging asymmetry that was directionally consistent with a documented human tendency to protect female targets from harm, though the specific conditions under which this asymmetry emerged varied by model; three other models, by contrast, showed no variation across conditions. These results indicate that gender-related biases are common in LLMs. Their direction and magnitude, however, are highly heterogeneous, to the point that some models behave in diametrically opposite ways to others. Bias auditing should therefore be treated as an ongoing, multi-vendor process, rather than a one-time assessment.

Abstract

Language Model Bias Evaluation

Explore similar work

Gender bias across LLMs is common and highly heterogeneous

BiAxisAudit: A Novel Framework to Evaluate LLM Bias Across Prompt Sensitivity and Response-Layer Divergence

Alignment Reduces Expressed but Not Encoded Gender Bias: A Unified Framework and Study