Names carry demographic indicators such as age, gender, and race, which makes them a subtle but strong target of bias in large language models (LLMs). In this work, we present an evaluation of name-based bias across 6 local large language models spanning multiple providers and parameter scales. Using a fill-in-the-blank task, we prompt these models to choose between pairs of first names drawn from U.S. Social Security Administration datasets and a race classification dataset by Rosenman et al.. Our experimental de- sign isolates age and gender effects through balanced name pairings and sentence templates that share a fixed syntactic structure. Across 300,000 local LLM responses, we find consistent and measurable name bias. Models prefer names that are less stereotypically male, older, popular, and white. Furthermore, we find that name-bias is highly sentence-dependent, but not more than associated stereotypes, and gender is the most significant bias, followed by age, name popularity, and race. Our results demonstrate that name-based bias remains pervasive even in local LLMs, highlighting the necessity for standardized evaluation methods and mitigation strategies.
Figures & tables
Model
Params
Iters
Provider
Llama 3.2 1B
Instruct
1B
2,000
Meta
Gemma 2 2B
Instruct
2B
2,000
Google
Phi-3.5 Mini
Instruct
3.8B
2,000
Microsoft
Table 1: Model configurations and iteration counts. Each iteration generates 30 responses (one per sentence template), yielding 60,000 responses per model.
Figure 1: Figure showing how the two largest sources of name-bias change for names ranked by how often LLMs choose them. As shown, generally, the LLMs favor more female and older names.
Figure 2: Figure showing how the three smallest sources of name-bias change for names ranked by how often LLMs choose them. As shown, generally, the LLMs favor more popular, whiter names. Token count does not seem to affect the names they choose.
Figure 3: Figure showing how the perceived gender and age of the names change depending on the sentence. As shown, the Mistral LLM potentially promotes harmful biases, like saying female-sounding names are more likely to get in a fight with their spouse or older-sounding names are more likely to be a director at a company.
LLM
Bias
Δ White
∑ White
Δ Older
∑ Older
Δ Male
∑ Male
Δ Pop
∑ Pop
Sent.
Phi3.5-mini
-3.0945
0.1838
-0.0846
0.6433
-0.4683
-0.4353
0.0496
0.5816
-0.2515
-0.4686
Qwen2.5-7b
-2.3552
0.1942
-0.1106
0.3186
-0.3781
-0.4902
-0.0219
0.1813
-0.1527
-0.5052
Llama3.1-8b
-0.9119
-0.0631
0.0233
-0.0698
0.005
-0.2471
-0.034
0.0335
-0.0472
-1.6091
Gemma2-9b
-1.5614
-0.022
0.0421
0.3424
-0.0544
-0.5787
0.2117
0.2695
-0.0933
-0.0185
Mistral-7b
-1.205
0.1127
-0.2682
0.3449
-0.117
-0.7903
-0.0968
0.4091
-0.053
-0.0454
Table 2: Table showing the output of the mixed effects modeling. As shown, the different sources of bias contribute in this order (high-to-low contribution): order of the names shown to the LLM (bias column), gender, age, popularity, and race.
Figure 4: The correlations between biases within each model. As shown, some models’ biases do not correlate, while other models show correlation between all of the biases or between specific biases.
Large language models (LLMs) are increasingly used for automated text annotation in tasks ranging from academic research to content moderation and hiring. Across 19 LLMs and two experiments totaling more than 4 million annotation judgments, we show that subtle identity cues embedded in text systematically bias annotation outcomes in ways that mirror racial stereotypes. In a names-based experiment spanning 39 annotation tasks, texts containing names associated with Black individuals are rated as more aggressive by 18 of 19 models and more gossipy by 18 of 19. Asian names produce a bamboo-ceiling profile: 17 of 19 models rate individuals as more intelligent, while 18 of 19 rate them as less confident and less sociable. Arab names elicit cognitive elevation alongside interpersonal devaluation, and all four minority groups are consistently rated as less self-disciplined. In a matched dialect experiment, the same sentence is judged significantly less professional (all 19 models, mean gap −0.774), less indicative of an educated speaker (−0.688), more toxic (18/19), and more angry (19/19) when written in African American Vernacular English rather than Standard American English. A notable exception occurs for name-based hireability, where fine-tuning appears to overcorrect, systematically favoring minority-named applicants. These findings suggest that using LLMs as automated annotators can embed socially patterned biases directly into the datasets and measurements that increasingly underpin research, governance, and decision-making.
Petter Törnberg
Institute for Logic, Language and Computation (ILLC), University of Amsterdam„ Science Park 107, 1098 XG Amsterdam, Netherlands
Names are personal identifiers, but they also carry social meaning and are widely used to evaluate how language models treat different people. Such evaluations typically assume that matched names are comparable model inputs. We show that this assumption often fails at the lexical interface: matched names are not necessarily matched inputs. Some names receive direct single-token access, while others are assembled from multiple subwords, creating unequal name-surface support. Across nearly half a million first names and 12 LLM-associated tokenizers, direct lexical access is highly selective, model dependent, and uneven across race- and gender-associated name metadata. We introduce NameTrace, a model-native, fine-grained, pre-behavioral framework for measuring whether unequal name-surface support remains a vocabulary property or becomes visible in task-relevant internal representations. NameTrace measures concept accessibility from the model's own probabilities over task-specific adjective axes with continuous task-aligned weights. On matched atomic and short-fragmented names within the same race/ethnicity--gender-associated strata, support predicts systematic differences in concept accessibility across fellowship, hiring, clinical assessment, and lending. These differences persist across all eight matched strata, extend across model families, and transfer to unseen names. Hidden-state interventions further show that the measured task directions have downstream leverage, shifting later constrained choices. Unequal lexical support is therefore demographically structured at the input and remains visible in task-relevant model computation. NameTrace makes lexical comparability measurable, supporting a broader principle: behavioral comparability begins with lexical comparability.
Mir Tafseer Nayeem, Davood Rafiei
Department of Computing Science University of Alberta
Large language models (LLMs) reproduce homogeneity bias -- the tendency to portray marginalized groups as more internally similar than dominant groups -- but whether this bias is stable or an artifact of inference settings has only been studied in single proprietary models. We map homogeneity bias across a 5x5 temperature-by-top-p grid in seven open-weight instruction-tuned LLMs (7-20B parameters). Hispanic and Asian Americans are portrayed as more homogeneous than White Americans in at least 18 of 20 hyperparameter configurations across six of seven models, including at extreme sampling settings. African American and gender bias show model-specific variation in direction. A conservative cell-level re-analysis confirms Hispanic and Asian homogeneity as robust, while weaker African American and gender signals largely do not survive, establishing group-specific robustness. We also apply the same grid to a names-based paradigm in which group identity is signaled via racially distinctive surnames rather than explicit labels. The names paradigm corroborates Hispanic and Asian homogeneity bias, but Black-coded surnames elicit robustly less homogeneous outputs than White-coded names in every model tested -- a reversal absent from the label paradigm -- showing that how group identity is operationalized shapes which biases surface and in which direction.