Failure of contextual invariance in large language models

Sagar Kumar; Ariel Flint; Luca Maria Aiello; Andrea Baronchelli

cs.CLMar 24, 2026

Failure of contextual invariance in large language models

Authors: Sagar Kumar, Ariel Flint, Luca Maria Aiello, Andrea Baronchelli

Organizations: Network Science Institute, Northeastern University, USA · Center for Health Informatics Program, Boston Children’s Hospital, USA · Dept. of Mathematics, City St George’s, University of London, UK · IT University of Copenhagen, DK

Abstract

Standard evaluation practices assume that large language model (LLM) outputs are stable when prompts are embedded in contextually equivalent discourses. Here, we test this assumption in the setting of gender inference. Using a controlled pronoun selection task, we introduce minimal, theoretically uninformative discourse context and find that this induces large, systematic shifts in model outputs. Correlations with cultural gender stereotypes, present in decontextualized settings, weaken or disappear once context is introduced, while theoretically irrelevant features, such as the gender of a pronoun for an unrelated referent, become the most informative predictors of model behavior. A Contextuality-by-Default analysis reveals that, in 19--52% of cases across models, this dependence persists after accounting for all marginal effects of context on individual outputs and cannot be attributed to simple pronoun repetition. These findings show that LLM outputs violate contextual invariance even under near-identical syntactic formulations, with implications for bias benchmarking and deployment in high-stakes settings.

Figures & tables

Figure 1: Discourse context destabilizes pronoun generation probabilities across all models. (a-e) Distribution, over all templates, of empirical generation probabilities. The height of the curve at the left extrema of each plot indicates the fraction of templates for which, in that context setting, $\hat{p}(f|c)=0$ (the model always generates the masculine pronoun in context setting $c$ ). The inverse is true for the right extrema, indicating the fraction of templates for which $\hat{p}(f|c)=1$ . (f-j) Average Kullback-Leibler Divergence, $\langle\mathrm{KL}\left(c\,\|\,c_{\emptyset}\right)\rangle$ (in bits), between the distribution of responses for each template in the unprimed setting, $c_{\emptyset}$ , and each of the primed settings tested, $c\in\{c_{f},c_{m},c_{0,1},c_{0,2}\}$ .

Figure 2: Correlations with cultural stereotypes vanish once discourse context is introduced. Spearman Correlation between the cultural gender stereotypes as measured by Misersky et al. (2014) and empirical generation probabilities in the unprimed (

c_{\emptyset})

, masculine-primed (

c_{m}

), and feminine-primed (

c_{f}

) context settings measured in our experiment, with significance (

***:=p<.001

**:=p<.01

*:=p<.05

Figure 3: Discourse context shifts predictive dominance towards the priming pronoun, away from cultural and syntactic features. Average mutual information (in bits) between features of the prompt and the pronoun generated by the model in unprimed settings and primed settings with standard error. Prompt features: ‘Role Type’ refers to the type of role of the antecedent of the target pronoun, ‘Stereotype’ refers to cultural stereotypes about the antecedent. ‘Case’ refers to the grammatical case of the pronoun, ‘Order’ refers to the linear order of options presented in the prompt, and ‘Priming Pronoun’ refers to the gender of the priming pronoun (present only in the primed settings).

Figure 4: Contextual effects vary across models and are sensitive to the linear order of pronoun options. (a) Fraction of template pairs which exhibited contextual measurements, disambiguated by the linear order of pronoun options presented in the prompt. (b) Fraction of template pairs which exhibited contextual measurements in both members of each model pair.

Model	% Contextual Measurements
GPT	27%
Phi-4	52%
Gemma	39%
Llama	22%
Qwen	19%

Table 1: Total fraction of the 180 template pairs tested in each model that exhibited contextuality in their pronoun generation probabilities in either option ordering.

Model	Params.
gpt-oss-20b	22B
Phi-4	14B
gemma-3-12b-it	12B
Llama-3.1-8B-Instruct	8B
Qwen2.5-7B-Instruct	7.6B

Table 2: Model name (per Hugging Face convention) and reported number of parameters for all models tested.

Figure 5: Example set of templates. The template shown in the unprimed setting is modified by adding a priming sentence, as shown in the Discourse Primed and Null Primed examples. Each of these is tested by being substituted for the $template$ variable in Box 9 .

	Unprimed		Masc Primed		Fem Primed
	MF	FM	MF	FM	MF	FM
GPT	$\frac{144800}{144800}$	$\frac{144800}{144800}$	$\frac{72400}{72400}$	$\frac{72400}{72400}$	$\frac{72400}{72400}$	$\frac{72400}{72400}$
Phi	$\frac{126966}{127200}$	$\frac{126860}{127200}$	$\frac{63324}{63600}$	$\frac{63256}{63600}$	$\frac{63316}{63600}$	$\frac{63178}{63600}$
Gemma	$\frac{107170}{107200}$	$\frac{106800}{107200}$	$\frac{53500}{53600}$	$\frac{53396}{53600}$	$\frac{53476}{53600}$	$\frac{53238}{53600}$
Llama-8B	$\frac{181000}{181000}$	$\frac{181000}{181000}$	$\frac{90500}{90500}$	$\frac{90500}{90500}$	$\frac{90500}{90500}$	$\frac{90500}{90500}$
Qwen	$\frac{173600}{173600}$	$\frac{173600}{173600}$	$\frac{86800}{86800}$	$\frac{86800}{86800}$	$\frac{86800}{86800}$	$\frac{86800}{86800}$

Table 3: Fraction of valid measurements out of total number of measurements made by each model in each context setting.

	Anaphora	PoS	Tracking
GPT	0.62	1.00	0.85
Phi	0.74	1.00	0.91
Gemma	0.74	1.00	0.73
Llama-8B	0.58	0.94	0.96
Qwen	0.45	0.99	0.36

Table 4: Model accuracy in the three metaprompting tasks.

Figure 8: Statistical stability across temperatures. (a-c) Distribution of $p_{i}(f|c)$ in GPT for the three context settings shown across three generation temperatures. As before, the left extrema represents the case where $p_{i}(f|c)=0$ (and therefore, $p_{i}(m|c)=1$ ) while the right is the case where $p_{i}(f|c)=1$ . (d-f) Correlations between pronoun generation probabilities in each context setting and empirical femininity ratings, measured across three temperatures. (g-i) Average mutual information between generation probabilities at each temperature and the feature specified, with bars depicting standard error.

Figure 9: Contextuality across temperatures. (a) Fraction of templates that exhibited contextual measurements in both linear orderings in GPT across three generation temperatures. (b) Fraction of templates which measured as contextual in row temperature that were also measured as contextual by the column temperature.

Language Model Bias Evaluation

Language model bias evaluation tests how models’ outputs, decisions, and representations vary across social groups, cultures, languages, and contexts.

See all 492 papers

Gender bias across LLMs is common and highly heterogeneous
Sep 29, 2026Edoardo Bolzoni, Valerio CapraroGender Bias in Language Models LLM Auditing
University of Milan-Bicocca, Milan, Italy
Man Made Language Models? Evaluating LLMs' Perpetuation of Masculine Generics Bias
Feb 14, 2025Enzo Doyen, Amalia TodirascuGender Bias in Language Models Language Model Bias Evaluation
University of Strasbourg
Anchoring LLM Gender Bias to Human Baselines: A Cross-Lingual Audit
May 29, 2026Jiwoo Choi, Seonwoo Ahn, Tongxin Zhang +1Gender Bias in Language Models Multilingual Language Model Evaluation
School of Digital Humanities and Computational Social Sciences, KAIST, South Korea

Sep 29, 2026J/K move · Enter open · S save

Gender bias across LLMs is common and highly heterogeneous

Edoardo Bolzoni, Valerio Capraro

University of Milan-Bicocca, Milan, Italy

Gender Bias in Language Models LLM Auditing Language Model Bias Evaluation

Open paper AskPDF arXiv

Understanding gender biases in large language models (LLMs) is increasingly important as these systems become embedded in decision-support tools with real consequences. Prior research has focused only on a small set of models, leaving open the extent to which gender biases are common and heterogeneous across LLMs. We address this gap across ten models released between April 2025 and June 2026, spanning nine vendors, using two paradigms: gender attribution to stereotyped phrases (Study 1) and moral judgment of abuse or torture against a woman or a man to prevent a catastrophic outcome (Study 2). In Study 1, two of ten models attributed masculine-stereotyped phrases to female writers more often than the reverse, while three models showed the opposite pattern. In Study 2, several models converged on a male-disadvantaging asymmetry that was directionally consistent with a documented human tendency to protect female targets from harm, though the specific conditions under which this asymmetry emerged varied by model; three other models, by contrast, showed no variation across conditions. These results indicate that gender-related biases are common in LLMs. Their direction and magnitude, however, are highly heterogeneous, to the point that some models behave in diametrically opposite ways to others. Bias auditing should therefore be treated as an ongoing, multi-vendor process, rather than a one-time assessment.

Abstract

Language Model Bias Evaluation

Explore similar work

Gender bias across LLMs is common and highly heterogeneous

Man Made Language Models? Evaluating LLMs' Perpetuation of Masculine Generics Bias

Anchoring LLM Gender Bias to Human Baselines: A Cross-Lingual Audit