Political Bias in Language Models

Follow this topic Ask about this topic

Researchers measure political bias with ideological questionnaires, issue and election benchmarks, paired counterfactual prompts, news summaries, and multi-turn role-play, often comparing languages, personas, or user identities. These tests find no single stable political position: outputs can shift with wording, retrieved material, language, source cues, and inferred user ideology, while patterns also vary by model, topic, and task. Other work traces disparities to pretraining and post-training, examines internal representations and refusal behavior, and tests mitigations such as consistency training and targeted fine-tuning. A central open question is how to distinguish baseline tendencies from elicitation and sycophancy, then reduce unfair or manipulative behavior without sacrificing accuracy, useful adaptation, or reliability across contexts.

Latest papers

62 papers5 papers in the last four weeks, down 44% on the four weeks before. 0.0% of all new papers.

What Is The Political Content in LLMs' Pre- and Post-Training Data?
Sep 26, 2025Tanise Ceron, Dmitry Nikolaev, Dominik Stammbach +1LLM Auditing Political Bias in Language Models
Bocconi University, Italy · University of Manchester, UK · Princeton University, USA
Political Ideology Shifts in Large Language Models
Aug 22, 2025Pietro Bernardelle, Stefano Civelli, Leon Fröhling +3Political Bias in Language Models Language Model Bias Evaluation
The University of Queensland, Brisbane, Australia · GESIS, Cologne, Germany · University of Udine, Udine, Italy

Sep 26, 2025J/K move · Enter open · S save

What Is The Political Content in LLMs' Pre- and Post-Training Data?

Tanise Ceron, Dmitry Nikolaev, Dominik Stammbach, Debora Nozza

Bocconi University, Italy · University of Manchester, UK · Princeton University, USA

LLM Auditing Political Bias in Language Models

Open paper AskPDF arXiv

Large language models (LLMs) reflect politically-slanted opinions in their generated text. Even though it is widely assumed that model behavior stem from training data, there has been no study quantifying the extent to which political content is part of the training data. To bridge this gap, we aim to directly estimate (1)~the proportion of politically engaged texts in training data, (2)~respective data imbalance, (3)~cross-dataset similarity, and (4)~correlations between data composition and model behaviour. We analyze the political content of pre- and post-training datasets of open-source LLMs, combining large-scale sampling, political-leaning classification, and stance detection. We find that all LLM training datasets are systematically skewed towards left-leaning content, with pre-training containing more politically engaged than post-training corpora. We further observe a strong correlation between political stances in training data and model behavior, which is present already in most base models and persists across post-training stages. These findings highlight the role of data composition in correlating with model behavior and motivate the need for greater data transparency as a means to understand and monitor model behavior.

Latest papers

What Is The Political Content in LLMs' Pre- and Post-Training Data?

Political Ideology Shifts in Large Language Models