Monocultural Biases: Correlated biases in large language models lead to unequal systemic exclusion rates in hiring

Matthew Bone; Fabian Stephany; Maria del Rio-Chanona

cs.CLAug 26, 2026

Monocultural Biases: Correlated biases in large language models lead to unequal systemic exclusion rates in hiring

Authors: Matthew Bone, Fabian Stephany, Maria del Rio-Chanona

Organizations: University of Oxford · Burning Glass Institute · University College London

Abstract

Employers are increasingly using large language models (LLMs) to automate their hiring process. This paper investigates the risk of monocultural biases, in which the widespread deployment of large language models homogenizes biases across the labor market, leading to greater systemic exclusion for certain demographic groups. For ten LLMs, we measure hiring biases across their base and post-trained versions to identify which stage, pre-training or post-training, lead to monocultural biases. We find that, compared to their base models, post-trained models are 3.6% less likely to callback older applicants. This negative shift occurs in eight of the ten models that we evaluate. Post-trained models have much more correlated decisions than base models which is likely driven by human capital traits like skills or college major. However, greater consensus among models increases global systemic exclusion rates from 5.6% to 17.3% and exacerbates demographic inequalities, with intersectional systemic exclusion rates ranging from 12.2% to 21.7% for post-trained models. We find that this inequality is primarily driven by age-based discrimination that is exacerbated in post-training. These results indicate that while post-training techniques may improve models' abilities to select the best applicants, they may raise systemic inequality risks for those at the margin by uniformly introducing new biases.

Figures & tables

Figure 1: This diagram displays the experiment creation and implementation pipeline. First, we generate a base prompt following the sampling procedures described in Section 3.1 and then we generate 8 demographic variations following the process in Section 3.2 . We have the LLM evaluate each experiment-variant 20 times, 10 times with the perturbed candidate first and 10 times with the perturbed candidate second, to remove order bias. Our outcome of interest is the specific callback rate for each variant $p$ .

Name	# Parameters	Provider	Citation
Nemotron Nano 3	30B	Nvidia	NVIDIA et al. (2025)
Granite 4	32B	IBM	Granite Team (2024)
Gemma 3	27B	Google	Gemma Team et al. (2025)
Olmo 3	32B	AllenAI	Olmo et al. (2025)
Qwen 3.5	35B	Alibaba	Yang et al. (2025)
GLM 4	32B	Zhipu	Zeng et al. (2025)

Table 1: This table includes metadata on each of the models we evaluated. The number of parameters is in billions (B).

Figure 2: (A) Each point is a bias estimate for a model with 95% confidence intervals. The value can be interpreted as the percent increase in getting a callback for being black, female, or over 40, relative to white, male, or being under 40. (B) Each point shows the aggregate shift in model biases from base to post-trained models with standard error bars. The value can be interpreted as the percent change in callback probability for post-trained models, compared to their base model.

Figure 3: (A) The first row shows the shift in bias correlation from within base models to within post-trained models. The second row shows the shift in residual correlation .

\Delta\bar{r}

is the average shift in pairwise correlation. (B) Each point shows the proportion of experiments, for that demographic variant, that are systemically excluded. The bars around it are bootstrapped 95% confidence intervals. The shaded region is the 95% confidence interval for the global systemic exclusion rate.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

SOC	Occupation	uniq. employers	vacancy titles	vacancy skills	vacancy salaries	profile titles	profile degrees	uniq. profile skills
11-2022	Sales Managers	1266	1265	2092	88161	33016	353	8063
11-3031	Financial Managers	942	634	1885	68975	47134	340	7227
11-3051	Industrial Production…	653	381	1307	38525	13218	327	5540
11-3061	Purchasing Managers	397	277	868	25306	5255	285	2340
11-3071	Transportation, Storag…	423	361	1062	31231	13204	320	4292
11-3111	Compensation and Benef…	101	133	433	9065	2839	213	1437

Appendix

Table 2: This table displays the number of unique human capital traits are being sampled from for each occupation. It showcases the diversity of realistic profiles that can be generated.

Group	First name	Last name
White-Male	Adam, Brad, Bradley, Brendan, Brett, Chad, Geoffrey, Greg, Jacob, Jason, Jay, Jeremy, Joshua, Justin, Matthew, Nathan, Neil, Scott, Todd	Bauer, Becker, Burkholder, Byler, Carlson, Erickson, Gallagher, Graber, Hershberger, Hostetler, Klein, Kramer, Larson, Mast, Meyer, Mueller, Olson, Roush, Schmidt, Schneider, Schroeder, Schultz, Schwartz, Stoltzfus, Troyer, Yoder
White-Female	Allison, Amanda, Amy, Anne, Carrie, Emily, Erin, Heather, Jennifer, Jill, Julie, Kristen, Laurie, Lori, Meredith, Misty, Rebecca, Sarah, Susan	Bauer, Becker, Burkholder, Byler, Carlson, Erickson, Gallagher, Graber, Hershberger, Hostetler, Klein, Kramer, Larson, Mast, Meyer, Mueller, Olson, Roush, Schmidt, Schneider, Schroeder, Schultz, Schwartz, Stoltzfus, Troyer, Yoder
Black-Male	Antwan, Darnell, Donnell, Hakim, Jamal, Jermaine, Kareem, Lamar, Lamont, Leroy, Marquis, Maurice, Rasheed, Reginald, Roderick, Terrance, Terrell, Tremayne, Tyrone	Alston, Battle, Bethea, Bolden, Booker, Braxton, Chatman, Diggs, Felder, Francois, Hairston, Hollins, Jean, Jefferson, Lockett, Louis, McCray, Muhammad, Myles, Pierre, Randle, Ruffin, Smalls, Washington, Winston, Witherspoon
Black-Female	Aisha, Ebony, Keisha, Kenya, Lakeisha, Lakesha, Lakisha, Lashonda, Latasha, Latisha, Latonya, Latoya, Lawanda, Patrice, Tameka, Tamika, Tanisha, Tawanda, Tomeka	Alston, Battle, Bethea, Bolden, Booker, Braxton, Chatman, Diggs, Felder, Francois, Hairston, Hollins, Jean, Jefferson, Lockett, Louis, McCray, Muhammad, Myles, Pierre, Randle, Ruffin, Smalls, Washington, Winston, Witherspoon

Appendix

Table 3: First and last names used to signal race and gender in the experimental perturbations. Each cell lists the unique names drawn from the source name dataset.

Panel of Figure 4 — Figure 4: This prompt is based on the SOC Occupation ”Architectural and Engineering Managers”. The second applicant is perturbed to signal an older white male.

Panel of Figure 5 — Figure 5: This prompt is based on the SOC Occupation ”Cost Estimators”. The first applicant is perturbed to signal an older black female.

Model	Base N/A	Base Rate	Post-Train Rate
Ernie 4.5	0	0.00%	0.00%
Falcon H1	0	0.00%	0.00%
Gemma 3	1	0.00%	0.00%
GLM 4	0	0.00%	0.00%
Granite 4	94	0.08%	0.00%
Ministral 3	0	0.00%	0.00%

Appendix

Table 4: Every model was evaluated on (76 Occupations)

\times

(10 experiments)

\times

(8 variants)

\times

(20 trials)

=

121,600 prompts. These are the N/A rates over all all prompts.

Model	Black	Female	Over 40	$q_{t}$	$q_{s}$	$q_{d}$	$R^{2}$
Base
Ernie 4.5	0.03*	0.03*	-0.00	0.21*	1.71*	0.27*	0.277
Falcon H1	0.00	0.06*	-0.04*	0.28*	1.92*	0.27*	0.266
Gemma 3	-0.07*	0.01*	-0.01	0.00	0.30*	-0.02	0.073
GLM 4	0.04*	0.00	-0.06*	0.19*	2.22*	0.26*	0.236
Granite 4	-0.06*	0.00	0.00	-0.00	0.03	-0.02	0.060

Appendix

Table 5: Model-specific hiring regressions from the full specification. Deeper green cells indicate larger positive coefficients and deeper red cells indicate larger negative coefficients within each column. Asterisks mark coefficients with

p<0.05

Model	Black (b)	Female (f)	Over 40 (o)	$q_{t}$	$q_{s}$	$q_{d}$	b $\times$ f	b $\times$ o	f $\times$ o	b $\times$ f $\times$ o	$R^{2}$
Base
Ernie 4.5	0.03*	0.03*	-0.00	0.21*	1.71*	0.27*	-0.00	-0.00	0.00	-0.00	0.277
Falcon H1	0.01*	0.06*	-0.03*	0.28*	1.92*	0.27*	-0.01	-0.02*	-0.00	0.00	0.267
Gemma 3	-0.05*	0.04*	-0.01	0.00	0.30*	-0.02	-0.05*	0.01	-0.00	-0.02	0.080
GLM 4	0.08*	0.03*	-0.05*	0.19*	2.22*	0.26*	-0.05*	-0.03*	-0.00	0.00	0.238
Granite 4	-0.02	0.05*	-0.00	-0.00	0.03	-0.02	-0.10*	-0.00	0.00	0.02	0.070

Appendix

Table 6: Model-specific hiring regressions from the demographic-interactions specification. Deeper green cells indicate larger positive coefficients and deeper red cells indicate larger negative coefficients within each column. Asterisks mark coefficients with

p<0.05

Term	Coef.	Std. Err.
Black	-0.003*	0.001
Female	0.020*	0.001
Over 40	-0.021*	0.001
Post-train	0.008	0.006
Black $\times$ Post-train ( $\lambda_{b}$ )	0.015*	0.001
Female $\times$ Post-train ( $\lambda_{f}$ )	0.011*	0.002

Appendix

Table 7: This is a summary of the pooled regression which has it’s

\lambda

values displayed in Figure 2 , Panel B. Reported terms show the demographic main effects, post-training shift terms, and quality controls. Occupation and model-family fixed effects are included in the pooled model but omitted from the table. Asterisks mark coefficients with

p<0.05

Figure 6: These heatmaps show the correlations from Figure 3 . In particular, you can see how the Moonlight model is an outlier when it comes to matching on residual correlation.

Figure 7: The color of this heatmap represents the global systemic exclusion rate for the base and post-train models. It shows this across a full range of values for parameters $\rho$ and $\tau$ .

Figure 8: The color of this heatmap represents the range of systemic exclusion ranges across demographic groups (i.e. the highest rate minus the lowest). It is a proxy for demographic inequality. It shows this across a full range of values for parameters $\rho$ and $\tau$ .

Variable	OLMo 3 Base	OLMo 3.1 SFT	OLMo 3.1 DPO	OLMo 3.1 Instruct
Black	0.020*	-0.004	-0.002	-0.001
	(0.004)	(0.004)	(0.003)	(0.003)
Female	0.026*	0.026*	0.023*	0.024*
	(0.003)	(0.004)	(0.004)	(0.004)
Over 40	-0.051*	-0.101*	-0.094*	-0.093*
	(0.004)	(0.004)	(0.004)	(0.005)

Appendix

Table 8: We fit the model from Equation 2 onto each checkpoint in Olmo’s post-training process. Reported terms show the demographic coefficients and quality-control coefficients for each OLMo model version, with standard errors reported in parentheses below each coefficient. Occupation fixed effects are included in each model-specific regression but omitted from the table. Asterisks mark coefficients with

p<0.05

Figure 9: We fit the model from Equation 2 but with only demographic variables (

\sum_{k}\beta_{k}X_{ivk}

) and human-capital controls (

\sum_{c}\omega_{c}Q_{ic}

), independently. For each of these models we plot the explained variance (

R^{2}

) values.

Panel of Figure 10 — Figure 10: This prompt is based on the SOC Occupation ”Architectural and Engineering Managers”. The applicant is perturbed to signal an older white male.

Figure 11: (A) Each point is a bias estimate for a model with 95% confidence intervals. The value can be interpreted as the increase in rating for being black, female, or over 40, relative to white, male, or being under 40. (B) Each point shows the aggregate shift in model biases from base to post-trained models with standard error bars. The value can be interpreted as the change in rating for post-trained models, compared to their base model.

Figure 12: (A) The first row shows the shift in bias correlation from within base models to within post-trained models. The second row shows the shift in residual correlation .

\Delta\bar{r}

Homogenization in Language Models

See all 8 papers

Jun 27, 2026

Can LLMs Hire Fairly? Racial Bias in Resume Screening

We audit fourteen mainstream large language models (LLMs) for hiring discrimination using the paired-resume methodology of Kline, Rose, and Walters (2022). The sole 2023-vintage model reproduces the pro-White callback gap documented in field experiments on labor market discrimination (

+2.12

pp, significant at the 1% level). Every model released in 2024 or after shows either a null gap or a significant pro-Black reversal (up to

-3.01

pp). The same pattern holds on the gender axis. Based on 24,024 paired postings per model across 14 models, our results document a reversal in the direction of algorithmic hiring bias across model generations.

Zhenyu Gao, Wenxi Jiang, Yutong Yan

Department of Finance, CUHK Business School, The Chinese University of Hong Kong

Gender Bias in Language Models Social Bias in Language Models Algorithmic Fairness Language Model Bias Evaluation AI and Labor

Ask PDF

Sep 16, 2026

Linguistic Triggers of Gender and Racial Bias in Open-Weight LLMs Applied to Recruitment

Open-weight large language models are rapidly entering hiring pipelines, yet their discriminatory failure modes -- and the regulatory exposure these create under the EU AI Act high-risk classification (Annex III) and U.S. EEOC adverse-impact analysis -- remain poorly understood. We present the first systematic, multi-model audit of open-weight LLMs that treats job-posting language as the primary experimental variable, evaluating six models (Llama 3.2, Mistral, Gemma 3, Qwen 3, Phi 3, DeepSeek-R1) across four controlled experiments that jointly probe recruiter-simulation and job-seeker-simulation tasks. We find that (1) agentic posting language depresses recruiter recommendation scores for female candidates (r_rb = 0.309, p_Bonf = 7x10^-5; model-fixed-effects r_rb = 0.448), while communal language partially reverses the penalty; and (2) coded-exclusion language suppresses non-White recruiter scores at large effect sizes (r_rb = 0.646-0.758) and, on the job-seeker side, selectively deters non-White personas from expressing interest -- operationalizing a chilling-effect mechanism at scale. A label-ablation experiment isolates the explicit demographic persona label as the primary causal driver, and Word Embedding Association Tests corroborate these findings at the representational level (d = 1.01-1.45 under Caliskan et al.'s multi-word gender attribute lists). We translate these results into a concrete pre-deployment audit protocol -- posting-vocabulary scoring, persona-conditioned LLM probing, and adverse-impact flagging against the four-fifths threshold -- that operationalizes the documentation and risk-management obligations Annex III imposes on high-risk AI in recruitment.

Kosuke Kitahara, Nobuhiro Yamaguchi

Recruit Co., Ltd. 1-9-2 Marunouchi Chiyoda-ku, Tokyo 100-6640 Japan

Gender Bias in Language Models AI-Assisted Decision Making Open-Weight Language Models Language Model Bias Evaluation Linguistic Bias in Language Models Demographic Bias in Language Models

Ask PDF

Mar 10, 2026

Quantifying Gender Bias in Large Language Models: When ChatGPT Becomes a Hiring Manager

The growing prominence of large language models (LLMs) in daily life has heightened concerns that LLMs exhibit many of the same gender-related biases as their creators. In the context of hiring decisions, we quantify the degree to which LLMs perpetuate societal biases and investigate prompt engineering as a bias mitigation technique. Our findings suggest that for a given resumé, an LLM is more likely to hire a female candidate and perceive them as more qualified, but still recommends lower pay relative to male candidates.

Nina Gerszberg, Janka Hamori, Andrew Lo

Massachusetts Institute of Technology Cambridge, Massachusetts, USA

Gender Bias in Language Models Algorithmic Bias Language Model Bias Evaluation LLM Prompting

Ask PDF

Abstract

Homogenization in Language Models

Explore similar work

Can LLMs Hire Fairly? Racial Bias in Resume Screening

Linguistic Triggers of Gender and Racial Bias in Open-Weight LLMs Applied to Recruitment

Quantifying Gender Bias in Large Language Models: When ChatGPT Becomes a Hiring Manager