Framing the Narrative: Ideological Mimicry in Large Language Models
Organizations: Centre for AI, Department of Computer Science, University College London · Public Policy Group, ETH Zürich · Department of Political Science, University College London · Department of Computer Science and Engineering, University of Bologna
Abstract
Large language models (LLMs) are increasingly used to answer questions about politically contentious issues, yet evaluations typically treat a model's stance as a relatively stable property. Real users, however, communicate political signals through their terminology, assumptions, and personal context. We investigate whether such signals produce ideological mimicry: systematic shifts in the political stance expressed by an LLM toward the position conveyed by the interaction. If LLMs adapt their responses to these signals, they risk creating personalised political information environments in which users with opposing views receive systematically different accounts of the same issue, potentially reinforcing existing divisions. We build the Poli-SHIFT dataset and evaluation framework and assess seven open-weight LLMs across ten contentious political topics in the United States, United Kingdom, and Australia, systematically manipulating contested terminology, politically valenced premises, and user information, and eliciting responses in both multiple-choice and open-text formats. Across models, we find robust evidence that prompt framing shapes the political stance of LLM outputs. Changing terminology alone reverses which side of an issue a model supports in 16.9% of matched comparisons. Stated political ideology also systematically shifts responses toward the user's position. These findings show that political stance is not a fixed property of LLMs; the views expressed are conditional on the interaction with the user. As LLMs become increasingly personalised sources of information, such interaction-dependent adaptation could contribute to political information environments that reinforce users' existing perspectives.
Figures & tables
| Topic | AU | UK | US |
|---|---|---|---|
| Climate Change Policy | ✓ | ✓ | ✓ |
| Immigration & Asylum Policy | ✓ | ✓ | ✓ |
| LGBT+ Rights | ✓ | ✓ | ✓ |
| Healthcare & Public Services | ✓ | ✓ | ✓ |
| Israel-Palestine War | ✓ | ✓ | ✓ |
| Russia-Ukraine War | ✓ | ✓ | ✓ |
Appendix figures & tables36 assets
Supplementary material from the paper’s appendix.
Appendix
| Pro-side | The UK should provide military assistance to the Ukrainian government in response to Russia’s unprovoked invasion . |
| Anti-side | The UK should provide military assistance to the Kiev regime in response to Russia’s special military operation . |
| Pro-side | Immigrants contribute more in taxes than they receive in public benefits in the U.S. |
| Anti-side | Immigration depresses wages and takes jobs from American workers. |
| Originally specified | Collected |
|---|---|
| Gemma 2 | Gemma 2 27B & Gemma 3 27B |
| Llama 3 | Llama 3.1 8B & Llama 4 Scout |
| Falcon 3 | Falcon 3 10B |
| DeepSeek Qwen | DeepSeek V4 Flash & Qwen 3.6 27B |
| Command R+ | Command A |
| Mistral | Mistral Large |
| Model | Format | Treatment | Effect | 95% CI | ||
|---|---|---|---|---|---|---|
| Qwen 3.6 27B | MCQ | premise | +1.26 | [+1.10, +1.43] | 16.17 | 0.001* |
| Falcon 3 10B | MCQ | premise | +1.06 | [+0.86, +1.25] | 11.04 | 0.001* |
| Gemma 3 27B | MCQ | premise | +0.95 | [+0.68, +1.22] | 7.30 | 0.001* |
| Llama 4 Scout | MCQ | premise | +0.63 | [+0.41, +0.86] | 5.92 | 0.001* |
| Mistral Large | MCQ | premise | +0.59 | [+0.29, +0.89] | 4.04 | 0.001* |
| DeepSeek V4 Flash | MCQ | premise | +0.46 | [+0.23, +0.69] | 4.20 | 0.001* |
| Model | Attribute | Partial | df | ||||
|---|---|---|---|---|---|---|---|
| Command A | Ideology | 0.151 | 2 | 25634.8 | 0.001* | 288,341 | 0.24 |
| Gender | 0.019 | 2 | 2840.4 | 0.001* | |||
| Race | 0.010 | 9 | 311.6 | 0.001* | |||
| Education | 0.004 | 4 | 301.2 | 0.001* | |||
| Age | 0.001 | 5 | 76.1 | 0.001* | |||
| DeepSeek V4 Flash | Ideology | 0.124 | 2 | 22107.8 | 0.001* | 312,304 | 0.18 |
| Model | Conservative | Liberal |
|---|---|---|
| Command A | -0.24* [-0.25, -0.23] | +0.59* [+0.58, +0.59] |
| DeepSeek V4 Flash | -0.63* [-0.64, -0.62] | +0.52* [+0.51, +0.53] |
| Gemma 3 27B | -0.57* [-0.58, -0.57] | +0.90* [+0.89, +0.90] |
| Llama 4 Scout | -0.61* [-0.61, -0.60] | +0.51* [+0.50, +0.52] |
| Mistral Large | -0.26* [-0.27, -0.26] | +0.71* [+0.71, +0.72] |
| Qwen 3.6 27B | -0.60* [-0.61, -0.60] | +0.87* [+0.86, +0.88] |
| Model | 20 | 30 | 50 | 60 | 70 |
|---|---|---|---|---|---|
| Command A | +0.06* [+0.04, +0.07] | +0.03* [+0.02, +0.04] | -0.02* [-0.03, -0.01] | -0.03* [-0.04, -0.02] | -0.02* [-0.03, -0.01] |
| DeepSeek V4 Flash | +0.05* [+0.03, +0.06] | +0.03* [+0.01, +0.04] | +0.00 [-0.01, +0.02] | -0.01 [-0.03, +0.00] | -0.01 [-0.02, +0.01] |
| Gemma 3 27B | +0.05* [+0.04, +0.06] | +0.03* [+0.02, +0.04] | -0.03* [-0.03, -0.02] | -0.04* [-0.05, -0.03] | -0.05* [-0.06, -0.04] |
| Llama 4 Scout | +0.03* [+0.02, +0.04] | +0.02* [+0.01, +0.03] | -0.01* [-0.02, -0.00] | -0.01 [-0.02, +0.00] | -0.03* [-0.04, -0.01] |
| Mistral Large | +0.03* [+0.02, +0.03] | +0.02* [+0.01, +0.02] | -0.01 [-0.02, +0.00] | -0.02* [-0.02, -0.01] | -0.02* [-0.03, -0.01] |
| Qwen 3.6 27B | +0.06* [+0.05, +0.07] | +0.02* [+0.01, +0.04] | -0.02* [-0.04, -0.01] | -0.05* [-0.06, -0.04] | -0.07* [-0.08, -0.06] |
| Model | No Formal Education | Technical Certification | Undergraduate Degree | Postgraduate Degree |
|---|---|---|---|---|
| Command A | -0.00 [-0.01, +0.01] | +0.05* [+0.04, +0.06] | +0.08* [+0.07, +0.09] | +0.14* [+0.13, +0.15] |
| DeepSeek V4 Flash | -0.04* [-0.05, -0.02] | +0.07* [+0.06, +0.09] | +0.09* [+0.08, +0.11] | +0.13* [+0.11, +0.14] |
| Gemma 3 27B | -0.12* [-0.13, -0.11] | -0.02* [-0.03, -0.01] | +0.00 [-0.01, +0.01] | +0.03* [+0.02, +0.04] |
| Llama 4 Scout | -0.07* [-0.08, -0.06] | +0.09* [+0.08, +0.10] | +0.04* [+0.03, +0.05] | +0.08* [+0.07, +0.09] |
| Mistral Large | -0.03* [-0.03, -0.02] | +0.04* [+0.03, +0.05] | +0.05* [+0.04, +0.06] | +0.09* [+0.08, +0.10] |
| Qwen 3.6 27B | +0.01 [-0.01, +0.02] | +0.06* [+0.05, +0.07] | +0.04* [+0.03, +0.06] | +0.09* [+0.08, +0.10] |
| Model | Woman | Non-Binary Person |
|---|---|---|
| Command A | +0.08* [+0.08, +0.09] | +0.28* [+0.27, +0.28] |
| DeepSeek V4 Flash | +0.11* [+0.10, +0.12] | +0.19* [+0.18, +0.21] |
| Gemma 3 27B | +0.15* [+0.15, +0.16] | +0.27* [+0.26, +0.27] |
| Llama 4 Scout | +0.11* [+0.10, +0.11] | +0.19* [+0.18, +0.20] |
| Mistral Large | +0.12* [+0.12, +0.13] | +0.24* [+0.23, +0.24] |
| Qwen 3.6 27B | +0.09* [+0.08, +0.10] | +0.16* [+0.16, +0.17] |
| Model | Arab | Black | Central Asian | East Asian | Hispanic | Indigenous | Jewish | South Asian | Mixed Race |
|---|---|---|---|---|---|---|---|---|---|
| Command A | -0.07* [-0.08, -0.06] | +0.08* [+0.07, +0.10] | -0.17* [-0.19, -0.16] | -0.02* [-0.04, -0.01] | -0.03* [-0.04, -0.01] | +0.07* [+0.06, +0.09] | +0.11* [+0.10, +0.12] | +0.03* [+0.02, +0.05] | +0.06* [+0.05, +0.08] |
| DeepSeek V4 Flash | -0.04* [-0.06, -0.02] | +0.13* [+0.11, +0.15] | -0.07* [-0.09, -0.05] | +0.05* [+0.03, +0.07] | +0.03* [+0.01, +0.05] | +0.08* [+0.06, +0.10] | +0.22* [+0.20, +0.24] | +0.06* [+0.04, +0.08] | +0.10* [+0.08, +0.12] |
| Gemma 3 27B | +0.02* [+0.01, +0.03] | +0.13* [+0.12, +0.15] | -0.07* [-0.08, -0.05] | +0.04* [+0.03, +0.05] | -0.02* [-0.04, -0.01] | +0.14* [+0.12, +0.15] | +0.17* [+0.16, +0.19] | +0.08* [+0.07, +0.09] | +0.09* [+0.08, +0.10] |
| Llama 4 Scout | +0.14* [+0.13, +0.15] | +0.15* [+0.13, +0.16] | +0.01 [-0.00, +0.03] | +0.09* [+0.07, +0.10] | +0.02* [+0.01, +0.03] | +0.21* [+0.19, +0.22] | +0.21* [+0.19, +0.22] | +0.12* [+0.11, +0.14] | +0.03* [+0.02, +0.05] |
| Mistral Large | +0.10* [+0.08, +0.11] | +0.11* [+0.10, +0.12] | -0.03* [-0.04, -0.02] | +0.05* [+0.04, +0.06] | +0.04* [+0.02, +0.05] | +0.16* [+0.15, +0.17] | +0.21* [+0.20, +0.22] | +0.11* [+0.10, +0.13] | +0.08* [+0.07, +0.09] |
| Qwen 3.6 27B | -0.02* [-0.04, -0.01] | +0.08* [+0.06, +0.09] | -0.08* [-0.10, -0.06] | +0.03* [+0.02, +0.05] | +0.02* [+0.00, +0.03] | +0.13* [+0.11, +0.14] | +0.16* [+0.14, +0.17] | +0.07* [+0.06, +0.09] | +0.06* [+0.04, +0.07] |
| Model | Format | Treatment | Effect | Primary | Clustered SE | Clustered |
|---|---|---|---|---|---|---|
| Qwen 3.6 27B | MCQ | premise | +1.26 | 0.001* | 0.083 | 0.001* |
| Falcon 3 10B | MCQ | premise | +1.06 | 0.001* | 0.125 | 0.001* |
| Gemma 3 27B | MCQ | premise | +0.95 | 0.001* | 0.197 | 0.001* |
| Llama 4 Scout | MCQ | premise | +0.63 | 0.001* | 0.147 | 0.001* |
| Mistral Large | MCQ | premise | +0.59 | 0.001* | 0.227 | 0.009* |
| DeepSeek V4 Flash | MCQ | premise | +0.46 | 0.001* | 0.187 | 0.014* |
| Attribute | Command A | DeepSeek V4 Flash | Gemma 3 27B | Llama 4 Scout | Mistral Large | Qwen 3.6 27B |
|---|---|---|---|---|---|---|
| Ideology | 2/2 of 2 | 2/2 of 2 | 2/2 of 2 | 2/2 of 2 | 2/2 of 2 | 2/2 of 2 |
| Race | 5/9 of 9 | 7/9 of 9 | 5/9 of 9 | 6/8 of 9 | 7/9 of 9 | 6/9 of 9 |
| Gender | 2/2 of 2 | 2/2 of 2 | 2/2 of 2 | 2/2 of 2 | 2/2 of 2 | 2/2 of 2 |
| Age | 4/5 of 5 | 2/2 of 5 | 5/5 of 5 | 1/4 of 5 | 2/4 of 5 | 5/5 of 5 |
| Education | 3/3 of 4 | 4/4 of 4 | 2/3 of 4 | 4/4 of 4 | 4/4 of 4 | 3/3 of 4 |
| Topic | MCQ | OT | Pooled |
|---|---|---|---|
| Gaza | +1.70 | +1.06 | +1.38 |
| Gun control | +1.73 | +1.01 | +1.37 |
| Ukraine | +1.07 | +0.96 | +1.01 |
| Climate | +1.34 | +0.64 | +0.99 |
| Healthcare | +1.13 | +0.70 | +0.92 |
| LGBT+ rights | +1.19 | +0.42 | +0.80 |
| Format | Australia | United Kingdom | United States |
|---|---|---|---|
| MCQ | +1.06 | +1.10 | +1.20 |
| OT | +0.65 | +0.63 | +0.68 |
| Model | Format | Treatment | Mean range | Contradiction rate |
|---|---|---|---|---|
| Command A | MCQ | premise | 0.017 | 0.0% |
| MCQ | term | 0.004 | 0.0% | |
| OT | premise | 0.346 | 0.9% | |
| OT | term | 0.385 | 0.4% | |
| DeepSeek V4 Flash | MCQ | premise | 0.743 | 5.7% |
| MCQ | term | 1.104 | 8.7% |
| Judge pair | Exact | Within 1 | Side agreement | ||
|---|---|---|---|---|---|
| Gemma 3 27B vs. Command A | 12,454 | 0.77 | 58.5% | 92.3% | 76.4% |
| Gemma 3 27B vs. Llama 4 Scout | 9,339 | 0.79 | 63.6% | 92.3% | 80.8% |
| Command A vs. Llama 4 Scout | 8,915 | 0.85 | 75.0% | 95.0% | 84.0% |
| Model | OT judge (all 3) | Survey | Bio |
|---|---|---|---|
| Command A | 100%/99% | 97% | 92% |
| DeepSeek V4 Flash | 100%/99% | 99% | 100% |
| Falcon 3 10B | 96%/98% | 98% | 4% |
| Gemma 2 27B | 98%/97% | 97% | 97% |
| Gemma 3 27B | 98%/99% | 100% | 100% |
| Llama 3.1 8B | 98%/99% | 55% | 70% |
| Model | Overall | Most-refused topic | Widest attribute | Highest value | Lowest value |
|---|---|---|---|---|---|
| Llama 3.1 8B | 29.6% | Gaza (59%) | Race (17pp) | Indigenous 42% | Mixed race 25% |
| Llama 4 Scout | 5.8% | Abortion (21%) | Ideology (8pp) | Conservative 10% | Liberal 2% |
| Qwen 3.6 27B | 1.5% | Abortion (3%) | no attribute clears a 5pp spread | ||
| Topic | Indigenous | Hispanic | South Asian | East Asian | White |
|---|---|---|---|---|---|
| Gaza | 71.8% | 60.9% | 63.1% | 61.2% | 59.2% |
| Ukraine | 60.2% | 38.8% | 44.9% | 39.5% | 31.9% |
| Indigenous rights | 43.8% | 56.6% | 56.4% | 52.1% | 47.9% |
| Immigration | 46.2% | 33.1% | 28.4% | 29.4% | 29.3% |
| Abortion | 60.3% | 49.2% | 45.4% | 46.1% | 46.1% |
| LGBT+ rights | 39.3% | 28.1% | 27.5% | 24.1% | 21.3% |
| Comparison | Exact | Within 1 | Side agreement | ||
|---|---|---|---|---|---|
| Human consensus vs. Gemma 3 27B | 200 | 0.79 | 62.0% | 94.0% | 75.5% |
| Human consensus vs. Command A | 198 | 0.87 | 63.1% | 93.9% | 79.8% |
| Human consensus vs. Llama 4 Scout | 198 | 0.80 | 64.6% | 92.9% | 80.8% |
| Human consensus vs. jury median | 200 | 0.84 | 63.5% | 94.5% | 80.5% |
| Group | Mean | Mean | Mean exact | Mean within 1 | Mean side agreement |
|---|---|---|---|---|---|
| Human annotators (3 pairs) | 199 | 0.77 | 54.5% | 90.6% | 75.9% |
| AI judges (3 pairs) | 197 | 0.79 | 64.7% | 91.9% | 78.8% |
Framing Effects in Language Models
Framing effects in language models study how wording, context, perspective, and presentation alter model judgments and outputs.
See all 45 papers