Revealing the Biases of Large Language Models By Inferring Prior Distributions
Authors: Joshua Ryo Cheng, Thomas L. Griffiths, Jake C. Snell
Organizations: Department of Mathematics, Princeton University · Department of Computer Science, Princeton University · Department of Psychology, Princeton University
Large Language Models (LLMs) have been known to exhibit a wide range of social biases including name-nationality associations, occupation-gender associations and racial stereotypes. Existing approaches to evaluating these biases typically involve measuring surface-level behavior where a model is fed a dataset of prompts and its outputs are scored relative to ground-truth answers. Inspired by prior elicitation in cognitive science, we propose a different perspective that reconstructs latent bias as a prior underlying the model's behavior. We formulate the problem as a Bayesian inverse problem where, given prompt-response observations on a bias benchmark dataset, we infer a low-dimensional latent distribution that summarizes the model's stereotyping tendencies. This paper develops the mathematical foundation, a practical reconstruction algorithm, and experiments using the Bias Benchmark for QA (BBQ) benchmark to infer the prior distributions of LLMs over responses in ambiguous social contexts.
Figures & tables
Figure 1: Recovered prior distributions over shifts for Gemma 3 1B IT, Gemma 3 4B IT, and Llama 3.2 3B IT. Error bars denote 95% credible intervals obtained via Laplace approximation. Even in the absence of input signals, all models exhibit non-uniform prior distributions, with notable differences across architectures and scales.
Figure 2: Recovered latent priors over the reduced answer space {S,A,U} for each BBQ social dimension, with 95% credible intervals derived from the Laplace approximation. Each model exhibited largely different latent priors across the different social dimensions.
Figure 3: BBQ bias scores for each model and social dimension. Positive values indicate bias towards the stereotype-consistent answer S , negative values indicate bias towards the stereotype-inconsistent answer A , and values near zero indicate no systematic bias in either direction.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: The recovered prior using our reconstruction algorithm much more closely resembles the control prior distribution. Meanwhile, the naive empirical mean method is biased by the varying prompt evidence.
Figure 5: Training loss history for the Adam-driven MAP optimization for Gemma 3 1B IT, Gemma 3 4B IT, and Llama 3.2 3B IT on the ciphertext problem. The smooth loss curves indicate a stable training objective.
Figure 6: Training loss history for the Adam-driven MAP optimization for Gemma 3 1B IT, Gemma 3 4B IT, and Llama 3.2 3B IT. The training curves are shown separately for each social dimension. We note that each training run exhibits stable convergence.