Large Language Models as Implicit Sociological Models: Reconstructing Voting Behaviour from Sociodemographic Profiles
Authors: Roman Neruda, Martin Bakoš, Josef Šlerka, Vít Tuček, Petra Vidnerová, Gabriela Kadlecová
Organizations: Institute of Computer Science, Czech Academy of Sciences, Prague, Czech Republic. · Faculty of Arts, Charles University, Prague, Czech Republic. · distil labs, Berlin, Germany.
Large language models (LLMs) trained on large-scale internet corpora encode extensive statistical regularities about social identities, attitudes, and political behaviour. This paper introduces and evaluates a methodological framework that leverages these latent representations to reconstruct aggregate voting behaviour from individual-level sociodemographic profiles. We operationalize LLMs as implicit sociological models by conditioning them on demographic descriptions, eliciting probabilistic turnout and party preferences, and aggregating individual outputs via a soft voting procedure. Using the 2021 Czech parliamentary election as a validation case, we demonstrate that contemporary LLMs reproduce official election outcomes with low mean absolute error, recover known political bloc structures, and align with independently established sociodemographic gradients. The contribution of this work is methodological rather than predictive: we show how LLMs can be systematically interrogated as compressed representations of social reality, offering a novel exploratory instrument for computational social science while clearly delineating its epistemic and ethical limits.
Figures & tables
Model
Context Window
Max Output Tokens
Knowledge Cutoff
GPT-4o-mini
128,000
16,384
Oct 2023
GPT-4o
128,000
16,384
Oct 2023
GPT-4.1-nano
1,047,576
32,768
Jun 2024
GPT-4.1
1,047,576
32,768
Jun 2024
Table 1: Model Specifications
Parameter
Value
temperature
0
top_p
1
parallel_tool_calls
true
truncation
disabled
service_tier
auto
store
true
Table 2: OpenAI API Configuration Parameters
Model
P(voted)=0
Undefined Preference
Overlap (Undefined ∩P=0 )
GPT-4o-mini
2.65%
2.65%
100.0%
GPT-4o
0.00%
0.00%
–
GPT-4.1-nano
61.37%
62.14%
95.0%
GPT-4.1
0.00%
0.00%
–
Table 3: Distribution of Absolute Turnout Predictions and Undefined Party Preferences
Figure 1: Voting simulation results using soft voting, for the GPT-4o-mini model (left) and the GPT-4o model (right).
Figure 2: Voting simulation results using soft voting, for the GPT-4.1-nano model (left) and the GPT-4.1 model (right).
Figure 3: Mean Absolute Error (MAE) between Predicted, Claimed, and Actual voting behaviour in Czechia and across all 14 Czech regions for GPT-4o-mini model.
Figure 4: Mean Absolute Error (MAE) between Predicted, Claimed, and Actual voting behaviour in Czechia and across all 14 Czech regions for GPT-4o.
Figure 5: Correlation matrix of predicted vote shares for the GPT-4o-mini model.
Figure 6: Correlation matrix of predicted vote shares for the GPT-4o model.
Figure 7: The values are Pillai’s trace statistics, interpreted here as indicators of explained multivariate variance by each variable in MANOVA that was performed for each model. The values in the brackets are the p-values.
Figure 8: Mean absolute error to the real election outcomes. Columns Kantar CZ, Median and STEM stand for latest pre-election polls of three major Czech polling agencies. The column “claimed” is given by the self-reported values from the panel dataset.
Figure 9: MAE for the latent knowledge which was extracted by two different prompts.
Figure 10: Best 2D alignment between the inter-party distances in the CHES dataset and the LLMs output.
As users increasingly turn to Large Language Models (LLMs) for information and advice on political matters, particularly during election periods, the political preferences expressed by these systems have become a matter of public interest. Prior research has shown that interactions with LLMs can influence users' political attitudes and choices, raising questions about how these models themselves evaluate political actors. In this paper, we investigate whether and how LLMs express preferences toward political parties and political leaders. We introduce a systematic and reproducible auditing framework in which multiple LLMs are prompted to evaluate parties and leaders across nine criteria. Rather than attempting to infer the models' "true" political beliefs, we focus on their observable behavior, examining consistency across evaluations, differences between models, refusal rates, and sensitivity to prompt formulation. We further investigate how these evaluations vary when models are instructed to adopt different personas. We demonstrate the framework through an Italian case study, providing a systematic analysis of LLM-generated political evaluations on italian parties and leaders.
Large language models (LLMs) exhibit systematic political biases in voter simulations, but their underlying mechanisms and cross-lingual generalizations remain poorly understood. We introduce Dynamo-K, a census-grounded simulation framework evaluating Korean-language LLM political behavior across four models on six Korean elections (2017-2025). Using this framework, we identify three systematic failure modes: (1) progressive bias in moderate agents, where explicit mitigation reduces Mean Absolute Error (MAE) by 5.2 times; (2) model-dependent third-party salience collapse, distinguishing between salience failure and decision bias; and (3) regional polarization collapse, where models bidirectionally under-predict historical party strongholds. To address these failures, we demonstrate that scenario reframing recovers 62% of 2017 MAE by restoring third-party visibility. Furthermore, we introduce a learned reweighting adapter that successfully calibrates opposing-valence models without relying on candidate names at train or test time. Validating our diagnostic framework, Dynamo-K accurately predicts 3/3 presidential winners - including a 2.1%p MAE on the highly contested 0.73%p-margin 2022 race - and correctly identifies the dominant party in a held-out local election. The pipeline is open-source and provides a scalable, cost-effective method for diagnosing LLM political behavior.
Sungwoo Kang
Department of Electrical and Computer Engineering Korea University, Seoul, Republic of Korea
Prior research has established that instruction-tuned large language models exhibit left-of-center political bias, measured exclusively through abstract political questionnaires. We show that this finding does not generalize to concrete policy decisions. We introduce a dual-instrument methodology grounded in Swiss democratic reality. The Smartvote questionnaire (75 abstract policy questions) is administered to 66 LLMs from 27 model families and compared to 184 elected members of the Swiss National Council, replicating the established leftward convergence (Cohen's d = 3.64, p = 0.0002). Then, novel to this work, 9 flagship LLMs are confronted with 48 real federal referenda (Volksabstimmungen) in four national languages (German, French, Italian, Romansh) under three information conditions, comparing votes to actual outcomes and party recommendations (Parolen). Three findings challenge the prevailing narrative. (1) Abstract questionnaires do not predict concrete behavior: the left-to-right agreement gradient on Smartvote shifts from left-peaked to center-peaked on Volksabstimmungen, where models align most with centrist Die Mitte and FDP rather than leftist SP and Gruene (Wilcoxon p = 0.008). (2) For some models, the language of a political question changes the answer more than the political content does: cross-linguistic consistency ranges from 50% (Mistral) to 98% (GPT-5.4). (3) Two models exhibit systematic change-aversion rather than political bias, voting Nein on 83-94% of referenda regardless of direction (binomial p < 0.0001). What prior work measured as "leftward bias" may not generalize beyond abstract instruments. On concrete policy decisions, LLMs behave less like coalition partners of the left and more like cautious civil servants: centrist, status-quo-favoring, and inconsistent across languages.