cs.CYMar 10, 2026
SaveQuantifying Gender Bias in Large Language Models: When ChatGPT Becomes a Hiring Manager
Organizations: Massachusetts Institute of Technology Cambridge, Massachusetts, USA
Abstract
The growing prominence of large language models (LLMs) in daily life has heightened concerns that LLMs exhibit many of the same gender-related biases as their creators. In the context of hiring decisions, we quantify the degree to which LLMs perpetuate societal biases and investigate prompt engineering as a bias mitigation technique. Our findings suggest that for a given resumé, an LLM is more likely to hire a female candidate and perceive them as more qualified, but still recommends lower pay relative to male candidates.
Figures & tables
Figure 1. This diagram describes our full prompting procedure used across each of the LLMs. The top left depicts our two name datasets. The left one contains names that sound like they are from a particular gender, our “gendered names" dataset. The right dataset contains names that sound gender-neutral but are followed by a gendered pronoun: either “(he/him)", “(she/her)", or “(they/them)". Each of these names are used on each of the 120 resumés beneath the name datasets. To the right of the resumés is a box containing the prompts. Each prompt is formed by combining each of the Baseline Prompts with each of the Perturbations. Although arrows are only drawn for the top question, all questions are combined with all perturbations. After that, each prompt is fed into each of the LLMs. In short, we take all possible combinations of names, resumés, prompts, perturbations, and LLMs to form the full test set. We refer to each combination as an experiment.
| Feature | Statistical Test |
|---|---|
| {Yes, No} | Chi-Square |
| 1-10 | Wilcoxon |
| Kolmogorov-Smirnov |
Table 1. Statistical Tests Used for Feature Evaluation
Figure 2. Baseline Hire and Qualified results. Darker colors indicate statistically significant differences in distributions.
Figure 3. Percentage Differences in Compensations above the Mean Between Males and Other Genders
Figure 4. Bias Mitigation Results
| Prompt Type | Percentage Decrease in Hiring | |
|---|---|---|
| Gendered Names | Pronouns | |
| Baseline vs Reasoning | 15% | 26% |
| DEI vs DEI with Reasoning | 40% | 34% |
Table 2. Percentage decrease in hiring likelihood across datasets and prompt types.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5. This plot displays a 2D representation of the names embedded with Google’s "models/embedding-001". The divide between the pink dots and blue dots demonstrates that the model is able to differentiate between male-sounding names and female-sounding names. The purple dots are more scattered throughout the image, revealing that the model does not consistently associate gender-neutral-sounding names with a given gender when they do not have pronouns afterward.
| Model Name | Model Version |
|---|---|
| GPT-3.5 | gpt-3.5-turbo-0125 |
| GPT-4 | gpt-4-0125-preview |
| LLaMA 7B | meta-llama/Llama-2-7b-chat-hf |
| LLaMA 13B | meta-llama/Llama-2-13b-chat-hf |
| LLaMA 70B | meta-llama/Llama-2-13b-chat-hf |
| Mistral | mistralai/Mistral-7B-Instruct-v0.1 |
Table 3. Full list of models and versions used for testing
Figure 6. This graph demonstrates the distribution of compensations for different fields. The distribution across fields varies significantly. This holds true across other models and other prompts as well, and demonstrates the need for a standardization process.
Figure 7. Both of these graphs depict the distribution of salaries across different genders from the Llama 7B test using the pronouns dataset with a DEI statement and reasons for all 5 resumés in the Information Technology Sector. The left figure depicts the distribution before standardization and the right figure depicts the distribution after standardization.
Figure 8. This graph depicts the baseline standardized compensation distributions.
Figure 13. Effectiveness of Bias Mitigation Prompts. Each of these graphs was calculated using our definition of bias. Taller bars indicate more bias, meaning our bias mitigation techniques were often ineffective.
More in this topic
Gender Bias in Language Models
Gender bias in language models examines how systems encode and express gender stereotypes and how to evaluate and reduce their effects.
See all 64 papers