Language models are aligned to avoid using protected attributes such as race, ethnicity, and gender to inform their responses. Therefore, they increasingly refuse or hedge when such attributes appear in a prompt. However, if a system must decide how much to learn from what people tell it, it is costly to refuse valuable information because of its source. In this article, we measure the cost of declining information in a two-urn game with an exact Bayesian answer. We present leading LLM models from two developers (OpenAI and Anthropic) with a prompt that requires them to decide how far to trust an unverifiable report from a participant whose description we randomize across ten configurations (6,600 sessions). We show that leading models often decline to assess the reporter, treating information as worthless at rates set by the description rather than by the evidence. At its widest, models decline a Black-described reporter 46% of the time and a white-described reporter 9%; yet, when they do assess the information, the Black-described reporter receives slightly more trust. The bias is invisible to audits that compare average trust or look only at assessed cases. It grows with reasoning effort, follows words rather than categories (models invoke race for "Black" but not for its Spanish translation negro, and never for "Indigenous"), and worsens under warning instructions a practitioner would try first. We also present a scoring option that reduces the refusal rate bias. When a model's trust is scored against revealed truth over repeated rounds, it credits identical records equally. In four of five configurations from Anthropic and OpenAI, the racial gap in trust falls below 0.02 by the last rounds.
Figures & tables
Figure 1: What a report is worth. The log-likelihood ratio of a report, ℓ(q^) in Eq. 2 , against the reliability assigned to the reporter, for three urn compositions p . Declining ( q^=21 , dashed line) deletes the report; values in the shaded region discount it.
Configuration
judged
declined
full trust (1.0)
refused
unparsed
luna, low
0.63
0.31
0.06
—
—
luna, high
0.63
0.30
0.07
—
—
terra, low
0.02
0.39
0.59
—
—
terra, high
0.01
0.33
0.66
—
—
Haiku 4.5
0.27
0.73
0.01
—
—
Sonnet 5, low
0.92
0.08
0.01
—
—
Table 1: Decision to judge or to decline by type of configuration. Share of 660 sessions.
Figure 2: Rate of decline by trait and configuration. Fable 5.1 also includes a separate refuse rate at the API.
Figure 3: Estimated selection effect γ^ and evaluation effect δ^ of a Black versus white description, factorial cells, with 95% intervals. For Fable 5.1 we show the difference in refusal rates.
π(B)
π(W)
d(B)
d(W)
Δ(B)
Δ(W)
ι^
naive gap
luna, low
0.34
0.13
0.84
0.81
0.116
0.042
0.075
−0.044
luna, high
0.46
0.09
0.85
0.81
0.162
0.027
0.135
−0.094
terra, low
0.35
0.47
0.99
1.00
0.174
0.231
−0.057
+0.055
terra, high
0.30
0.33
1.00
1.00
0.149
0.165
−0.017
+0.017
Haiku 4.5
0.76
0.73
0.93
0.93
0.330
0.313
0.017
−0.014
Sonnet 5, low
0.15
0.13
0.79
0.77
0.044
0.034
0.011
+0.013
Table 2: Components of Eq. 5 by configuration, Black- ( B ) and white-described ( W ) sources pooled over five cells each ( n=150 ).
Figure 4: The institution. (A) Round-1 decline rate for Black- and white-described elementary-school pupils at Luna High, unscored and scored with revealed truth; 60 sessions per bar. (B) Mean q^ by round under the three sets of rules, luna low. (C) Learning curves for Black- and white-described pupils under scoring with revealed truth, on identical records.
Demographic bias in language models studies how identity cues shape model outputs, decisions, and representations, and how to detect and reduce resulting disparities.
Large language models are increasingly used as decision aids whose probability judgments shape downstream choices. Whether those judgments carry a systematic directional tilt has been hard to detect: calibration metrics aggregate unsigned errors, and naturalistic uncertainty offers no ground-truth probability. When an LLM rates a startup's success at 70% but its failure at 15%, the missing 15 points expose a distortion no aggregate score flags. We introduce OptimismBench, which detects directional bias with inverted pairs: each scenario elicits both P(success) and P(failure), and asymmetry between the two framings yields a signed bias score without ground truth. Across 16 models from 8 providers, fourteen are optimistic; pessimism appears only in Anthropic's frontier tier. Eleven matched base-versus-chat pairs across four families show post-training sets the sign of the bias, with opposite shifts in different families. The pattern survives prompt, temperature, perspective, and self-debiasing ablations. A seventeen-model six-language comparison further shows model identity dominates language, with inter-model variance at 4.7x inter-language variance. We release 3,870 items across 10 languages for per-model directional-bias auditing. When alignment makes a model more helpful, it also tilts its probabilities; downstream pipelines inherit the tilt by default.
Large Language Models (LLMs) have been shown to contain biases in the process of integrating conflicting information when answering questions. Here we ask whether such biases also exist with respect to which language is used for each conflicting piece of information. To answer this question, we extend the conflicting needles in a haystack paradigm to a multilingual setting and perform a comprehensive set of evaluations with naturalistic news domain data in five different languages, for a range of multilingual LLMs of different sizes. We find that all LLMs tested, including GPT-5.2, ignore the conflict and confidently assert only one of the possible answers in the large majority of cases. Furthermore, there is a consistent bias across models and prompting languages in which languages are preferred, with a general bias against Russian and, for the longest context lengths, in favor of Chinese. The language preferences are consistent between models trained inside and outside of mainland China, though somewhat stronger in the former category. There is also a general tendency among models to prioritize information that matches the language used for prompting. We hope to make users and developers of multilingual LLMs aware of this category of biases, to spur further research on their causes and possible mitigation.
Robert Östling, Murathan Kurfalı
Stockholm University Sweden · RISE Research Institutes of Sweden Sweden
Large Language Model (LLM) assistants routinely decide whether they can trust users and third parties whose competence, intentions, and integrity they cannot verify. This uncertainty matters for safety, as trusting the wrong party can lead an agent to comply with harmful requests or act on malicious instructions encountered during tool use. To study this problem, we define trust as an assistant's willingness to accept vulnerability to the actions of another party and ask whether such behavior can be causally controlled through model activations. We build 2,000 contrastive conversations spanning ability, benevolence, and integrity, where paired responses complete the same request but differ in whether the assistant trusts the user. From these pairs, we learn steering matrices while keeping the model parameters frozen and test them across six instruction-tuned models from three families, finding that steering changes trust decisions monotonically in both directions. We then ask whether this effect extends to several safety-related agent settings involving harmful requests, prompt injections, and insider threats, while using benign-task and reasoning as controls. Our findings provide evidence that trust in the user can be causally controlled along linear directions in model activations and provide a way to study how trust shapes safety-relevant behavior in language models.
Théo Lasnier, Romain Froger, Maxence Lasbordes +1
Inria Paris · Sorbonne Université · Meta SuperIntelligence Labs +1