Stance Drift: How AI-mediated Communication Distorts Our Message
Authors: Lingchong Liu, Yanfei Zhou, Jacob Bien, Y. X. Rachel Wang, Lucy Xia, Xin Tong
Organizations: Department of Information Systems, Business Statistics and Operations Management Hong Kong University of Science and Technology Hong Kong SAR · Department of Data Sciences and Operations University of Southern California Los Angeles, CA, USA · School of Mathematics and Statistics University of Sydney Sydney, Australia · Faculty of Business and Economics University of Hong Kong, Hong Kong SAR
Large language models (LLMs) increasingly mediate human communication, from drafting emails to summarizing scientific reports, yet whether they faithfully preserve a speaker's position remains largely untested. We model AI-mediated communication as a two-step generation-extraction pipeline: one LLM produces an argument from a specified stance, and a second LLM extracts the stance from that argument. We represent the pipeline as a probabilistic state transition over five Likert-type stance categories and define the stance preservation rate (SPR) as the average probability that the extracted stance matches the initial stance. Across 112 debate propositions, none of the nine LLMs tested exceeded an SPR of 0.7 under the default configuration. Three drift patterns accounted for most of the drift: polarization, deviation from neutrality, and flipping. Among the mitigation strategies tested, including in-context learning, multiple extraction with shuffled options, assertion, and reflection, only adding medium reasoning effort to a reflection prompt for GPT-5.4 substantially improved the SPR, to 0.775, yet polarization remained the largest pattern, with 0.119 of the transition mass. An exploratory comparison with human extraction on a single proposition suggests that drift arises at both the generation and the extraction stage. These results point to a fidelity gap in AI-mediated communication, with implications for journalism, policy deliberation, scientific communication, and other domains where opinion-laden messages pass through language models.
Figures & tables
Figure 1: AI-mediated communication model. A sender generates an argument with LLM assistance from an initial stance; a receiver extracts the stance from the argument with LLM assistance.
Figure 2: Example of empirical transition matrices deviating from the ideal identity matrix. Rows are the initial stances and columns the extracted stances; the (i,j) entry is the probability, in percent, of extracting the j th choice given the i th choice as the initial stance. ( A ) GPT-5.4 , default configuration, proposition “We should increase fuel tax”. ( B ) GPT-5.4 , default configuration, proposition “The use of AI should be abandoned”. ( C ) The identity matrix: perfect preservation of every stance.
Figure 3: The two-step generation and extraction framework. An initial stance on a proposition is expanded into an argument, and the stance is extracted back from the argument.
Figure 4: SPR across models under the default configuration. The solid rule marks perfect fidelity (SPR =1 ) and the dashed rule the uniform random baseline (SPR =0.2 ).
Model
Agree strongly
Agree
Neutral
Disagree
Disagree strongly
GPT-4o-mini
.933 (.021)
.108 (.016)
.717 (.026)
.310 (.035)
.784 (.032)
GPT-4.1
.962 (.014)
.599 (.031)
.800 (.026)
.168 (.027)
.903 (.016)
GPT-5.4
.996 (.002)
.147 (.022)
.703 (.032)
.136 (.023)
.982 (.010)
GPT-3.5-turbo
.621 (.031)
.484 (.031)
.313 (.023)
.244 (.027)
.527 (.036)
gemma-3n-E4B
.595 (.040)
.348 (.034)
.920 (.019)
.114 (.022)
.617 (.041)
Llama-3.3-70B
.954 (.015)
.505 (.036)
.851 (.023)
.161 (.026)
.942 (.017)
Table 1: Stance preservation by initial stance for the nine models under the default configuration. ( A ) Success rates. ( B ) Proportion of propositions whose one-sided 99% CI of the success rate, with Bonferroni correction, includes 1.
Model
SPR
Polarization
Moderation
Moderation to neutral
Flipping
Deviation from neutrality
GPT-4o-mini
.570 (.009)
.246 (.011)
.022 (.004)
.009 (.002)
.097 (.012)
.057 (.005)
GPT-4.1
.686 (.007)
.244 (.005)
.024 (.004)
.001 (.000)
.005 (.001)
.040 (.005)
GPT-5.4
.593 (.008)
.328 (.006)
.002 (.001)
.000 (.000)
.018 (.004)
.059 (.006)
GPT-3.5-turbo
.438 (.008)
.158 (.009)
.087 (.006)
.000 (.000)
.180 (.014)
.137 (.005)
gemma-3n-E4B
.519 (.013)
.165 (.010)
.041 (.006)
.082 (.010)
.178 (.015)
.016 (.004)
Llama-3.3-70B
.683 (.011)
.191 (.007)
.014 (.003)
.015 (.003)
.069 (.009)
.030 (.005)
Table 2: SPR and the components of the SDR by drift pattern for the nine models under the default configuration. Mean (SE) over the 112 propositions of the per-proposition values averaged over the five initial stances; each row sums to 1 up to rounding. Per-stance values and all tested models: Tables S5 – S10 in the appendix.
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S1: Individual example illustrating stance drift in the two-step framework. Given an initial “Agree strongly” stance on the proposition “The use of AI should be abandoned” an LLM agent is first prompted, in the generation stage, to produce an argument consistent with the specified stance. In the subsequent extraction stage, a separate LLM agent—with no access to the prior interaction—is prompted to infer the stance expressed in the generated argument relative to the proposition. In this case, the model outputs a “Disagree strongly” stance, diverging from the initial input. This discrepancy indicates that a stance drift has occurred.
Figure S2: Individual example illustrating stance preservation in the two-step framework. Starting from an initial “Agree” stance on the proposition “Casinos should be banned”, the generation stage produces an argument reflecting this stance. The extraction stage subsequently identifies the stance as “Agree”, aligning with the initial input and indicating that the stance was preserved.
Figure S3: Empirical transition matrices with standard errors. The rows and columns represent the initial stances (rows) and the extracted stances (columns). The (i,j) entry indicates the probability of returning to the j th choice given the i th choice as the initial stance. The color intensity reflects the transition probabilities. Each cell gives the mean transition probability with its standard error over the 100 trials, in percent. ( A ) Empirical transition matrix for the proposition “We should increase fuel tax”, derived from experiments with GPT-5.4 . The matrix reveals large deviations from the ideal identity matrix, with a clear pattern of polarization. ( B ) Empirical transition matrix for another proposition “The use of AI should be abandoned”, based on experiments with GPT-5.4 . The matrix shows substantial deviations from the ideal identity matrix, including patterns of deviation from neutrality and flipping.
Figure S4: SPR for all models tested. The solid rule marks perfect fidelity (SPR =1 ) and the dashed rule the uniform random baseline (SPR =0.2 ).
Model
Agree strongly
Agree
Neutral
Disagree
Disagree strongly
GPT-4o-mini
.933 (.021)
.108 (.016)
.717 (.026)
.310 (.035)
.784 (.032)
GPT-4.1
.962 (.014)
.599 (.031)
.800 (.026)
.168 (.027)
.903 (.016)
GPT-5.4
.996 (.002)
.147 (.022)
.703 (.032)
.136 (.023)
.982 (.010)
GPT-3.5-turbo
.621 (.031)
.484 (.031)
.313 (.023)
.244 (.027)
.527 (.036)
gemma-3n-E4B
.595 (.040)
.348 (.034)
.920 (.019)
.114 (.022)
.617 (.041)
Llama-3.3-70B
.954 (.015)
.505 (.036)
.851 (.023)
.161 (.026)
.942 (.017)
Appendix
Table S1: Average success rates with respect to initial stance for all tested models.
Model
Agree strongly
Agree
Neutral
Disagree
Disagree strongly
GPT-4o-mini
.223 (.040)
.000 (.000)
.402 (.047)
.116 (.030)
.536 (.047)
GPT-4.1
.920 (.026)
.286 (.043)
.616 (.046)
.054 (.021)
.732 (.042)
GPT-5.4
.991 (.009)
.027 (.015)
.527 (.047)
.018 (.013)
.973 (.015)
GPT-3.5-turbo
.000 (.000)
.045 (.020)
.018 (.013)
.000 (.000)
.080 (.026)
gemma-3n-E4B
.348 (.045)
.098 (.028)
.500 (.047)
.009 (.009)
.464 (.047)
Llama-3.3-70B
.920 (.026)
.295 (.043)
.714 (.043)
.054 (.021)
.902 (.028)
Appendix
Table S2: Proportion of propositions having 1 included in the 99% confidence interval (1 sided, with Bonferroni correction) of the mean diagonal probabilities, with respect to initial stance, for all tested models.
Model
Agree strongly
Agree
Neutral
Disagree
Disagree strongly
GPT-4o-mini
.107 (.029)
.000 (.000)
.250 (.041)
.071 (.024)
.375 (.046)
GPT-4.1
.893 (.029)
.179 (.036)
.491 (.047)
.027 (.015)
.670 (.045)
GPT-5.4
.973 (.015)
.009 (.009)
.402 (.047)
.000 (.000)
.955 (.020)
GPT-3.5-turbo
.000 (.000)
.018 (.013)
.000 (.000)
.000 (.000)
.018 (.013)
gemma-3n-E4B
.259 (.042)
.062 (.023)
.339 (.045)
.009 (.009)
.312 (.044)
Llama-3.3-70B
.902 (.028)
.223 (.040)
.562 (.047)
.036 (.018)
.848 (.034)
Appendix
Table S3: Proportion of propositions having 1 included in the 99% confidence interval (1 sided) of the mean diagonal probabilities, with respect to initial stance, for all tested models.
Model
Agree strongly
Agree
Neutral
Disagree
Disagree strongly
GPT-4o-mini
.884 (.030)
.000 (.000)
.277 (.042)
.098 (.028)
.598 (.047)
GPT-4.1
.902 (.028)
.188 (.037)
.491 (.047)
.036 (.018)
.661 (.045)
GPT-5.4
.973 (.015)
.009 (.009)
.375 (.046)
.000 (.000)
.955 (.020)
GPT-3.5-turbo
.241 (.041)
.071 (.024)
.009 (.009)
.009 (.009)
.250 (.041)
gemma-3n-E4B
.429 (.047)
.071 (.024)
.804 (.038)
.009 (.009)
.464 (.047)
Llama-3.3-70B
.893 (.029)
.223 (.040)
.536 (.047)
.036 (.018)
.848 (.034)
Appendix
Table S4: Pass rate at threshold 0.95 with respect to initial stance, for all tested models. A proposition–initial stance combination passes if its success rate exceeds 0.95; each cell is the proportion of the 112 propositions that pass.
Model
SPR
Polarization
Moderation
Moderation to neutral
Flipping
Deviation from neutrality
GPT-4o-mini
.570 (.009)
.246 (.011)
.022 (.004)
.009 (.002)
.097 (.012)
.057 (.005)
GPT-4.1
.686 (.007)
.244 (.005)
.024 (.004)
.001 (.000)
.005 (.001)
.040 (.005)
GPT-5.4
.593 (.008)
.328 (.006)
.002 (.001)
.000 (.000)
.018 (.004)
.059 (.006)
GPT-3.5-turbo
.438 (.008)
.158 (.009)
.087 (.006)
.000 (.000)
.180 (.014)
.137 (.005)
gemma-3n-E4B
.519 (.013)
.165 (.010)
.041 (.006)
.082 (.010)
.178 (.015)
.016 (.004)
Llama-3.3-70B
.683 (.011)
.191 (.007)
.014 (.003)
.015 (.003)
.069 (.009)
.030 (.005)
Appendix
Table S5: SPR and the components of the stance drift rate (SDR) by drift pattern for all tested models. As Table 2 in the main text: overall values with a uniform prior over initial stances, as fractions of the transition mass; the five pattern columns are the components of the SDR, one minus SPR, and sum to it; each row sums to 1 up to rounding; mean (SE) over the 112 propositions, with the SE computed on the per-proposition values averaged over the five initial stances.
Model
Agree strongly
Agree
Neutral
Disagree
Disagree strongly
GPT-4o-mini
—
.589 (.040)
—
.642 (.037)
—
GPT-4.1
—
.397 (.031)
—
.825 (.028)
—
GPT-5.4
—
.777 (.028)
—
.863 (.023)
—
GPT-3.5-turbo
—
.326 (.035)
—
.462 (.035)
—
gemma-3n-E4B
—
.234 (.034)
—
.592 (.041)
—
Llama-3.3-70B
—
.137 (.024)
—
.817 (.028)
—
Appendix
Table S6: Mass of polarization with respect to initial stance, for all tested models. Fraction of the transition mass of the initial stance; mean (SE) over the 112 propositions. A dash marks an initial stance for which the pattern is empty by definition.
Model
Agree strongly
Agree
Neutral
Disagree
Disagree strongly
GPT-4o-mini
.000 (.000)
—
—
—
.108 (.021)
GPT-4.1
.038 (.014)
—
—
—
.081 (.013)
GPT-5.4
.002 (.001)
—
—
—
.008 (.004)
GPT-3.5-turbo
.294 (.027)
—
—
—
.140 (.021)
gemma-3n-E4B
.140 (.023)
—
—
—
.063 (.017)
Llama-3.3-70B
.046 (.015)
—
—
—
.022 (.006)
Appendix
Table S7: Mass of moderation with respect to initial stance, for all tested models. Fraction of the transition mass of the initial stance; mean (SE) over the 112 propositions. A dash marks an initial stance for which the pattern is empty by definition.
Model
Agree strongly
Agree
Neutral
Disagree
Disagree strongly
GPT-4o-mini
.000 (.000)
.039 (.009)
—
.004 (.002)
.000 (.000)
GPT-4.1
.000 (.000)
.003 (.001)
—
.002 (.001)
.000 (.000)
GPT-5.4
.000 (.000)
.000 (.000)
—
.000 (.000)
.000 (.000)
GPT-3.5-turbo
.000 (.000)
.000 (.000)
—
.001 (.000)
.000 (.000)
gemma-3n-E4B
.027 (.009)
.315 (.035)
—
.043 (.014)
.023 (.010)
Llama-3.3-70B
.000 (.000)
.073 (.013)
—
.001 (.001)
.000 (.000)
Appendix
Table S8: Mass of moderation to neutral with respect to initial stance, for all tested models. Fraction of the transition mass of the initial stance; mean (SE) over the 112 propositions. A dash marks an initial stance for which the pattern is empty by definition.
Model
Agree strongly
Agree
Neutral
Disagree
Disagree strongly
GPT-4o-mini
.067 (.021)
.264 (.034)
—
.045 (.016)
.108 (.027)
GPT-4.1
.000 (.000)
.002 (.001)
—
.005 (.002)
.016 (.005)
GPT-5.4
.002 (.001)
.076 (.018)
—
.001 (.001)
.011 (.009)
GPT-3.5-turbo
.085 (.018)
.190 (.028)
—
.293 (.031)
.332 (.033)
gemma-3n-E4B
.237 (.035)
.103 (.022)
—
.251 (.039)
.296 (.040)
Llama-3.3-70B
.000 (.000)
.286 (.038)
—
.021 (.010)
.036 (.014)
Appendix
Table S9: Mass of flipping with respect to initial stance, for all tested models. Fraction of the transition mass of the initial stance; mean (SE) over the 112 propositions. A dash marks an initial stance for which the pattern is empty by definition.
Model
Agree strongly
Agree
Neutral
Disagree
Disagree strongly
GPT-4o-mini
—
—
.283 (.026)
—
—
GPT-4.1
—
—
.200 (.026)
—
—
GPT-5.4
—
—
.297 (.032)
—
—
GPT-3.5-turbo
—
—
.687 (.023)
—
—
gemma-3n-E4B
—
—
.080 (.019)
—
—
Llama-3.3-70B
—
—
.149 (.023)
—
—
Appendix
Table S10: Mass of deviation from neutrality with respect to initial stance, for all tested models. Fraction of the transition mass of the initial stance; mean (SE) over the 112 propositions. A dash marks an initial stance for which the pattern is empty by definition.
Figure S5: GPT-5.4 across the four configurations, with respect to initial stance. ( A ) Success rate. ( B ) Mass of polarization. The configurations are the default configuration, reflection, third-person reflection, and third-person reflection with medium reasoning effort; the three later configurations each add one change to the previous one. Bars are means over the 112 propositions and error bars are standard errors. The values are listed in Tables S1 and S6 .
Initial Stance
Agree strongly
Agree
Neutral
Disagree
Disagree strongly
Generation-stage drift
0
20
5
0
1
Extraction-stage drift
0
19
13
20
1
Appendix
Table S11: Argument counts by drift stage and initial stance for the human-extraction study. Each cell records the number of arguments (out of 20 per initial stance) attributable to each drift stage, for proposition “The use of AI should be abandoned” generated by GPT-5.4 . Generation-stage drift: the human-majority label differs from the known initial stance, indicating that the generated argument itself does not carry the intended stance signal. Extraction-stage drift: the AI-majority label differs from the human-majority label, attributing the discrepancy to the LLM extractor rather than the generated content. The two rows are not mutually exclusive.
Initial stance
Generated argument
Human-majority label
AI-majority label
Drift pattern
Drift stage
Disagree
Abandoning AI entirely would throw away major benefits in medicine, education, accessibility, scientific research, and public services. Like other powerful technologies, AI can be misused, but that argues for strong rules, transparency, and accountability—not total rejection. The United States would also harm its competitiveness and innovation capacity by stepping back while other countries continue developing it. A better approach is to limit dangerous applications, require human oversight where stakes are high, and enforce ethical standards so society can capture AI’s advantages while reducing risks.
Disagree
Disagree strongly
polarization
extraction stage
Agree
AI should not be abandoned, but its use should be approached carefully. In the United States, AI already helps doctors detect diseases earlier, assists businesses with routine tasks, and supports scientific research. Rejecting it entirely would mean giving up major benefits and global competitiveness. The real issue is not whether AI should exist, but how to govern it responsibly. Strong rules on privacy, transparency, bias, and accountability can reduce harm while preserving innovation. With thoughtful oversight, AI can remain a valuable tool rather than something society must completely discard.
Disagree
Disagree strongly
flipping
both stages (flipping at generation, polarization at extraction)
Neutral
AI’s future depends on how it is designed, governed, and used. It can improve medical diagnosis, expand access to education, and automate dangerous tasks, yet it also raises serious concerns about bias, privacy, job disruption, and misuse. Because both benefits and harms are significant, abandoning it entirely may be premature, but embracing it without safeguards would be reckless. A balanced approach would focus on strict oversight, transparency, accountability, and limits on high-risk applications while society continues evaluating whether its advantages truly outweigh its costs.
Neutral
Disagree
deviation from neutrality
extraction stage
Appendix
Table S12: One example argument per drift pattern from the human-extraction study. All examples come from the single proposition of the study, “The use of AI should be abandoned”, with arguments generated by GPT-5.4 and AI labels extracted by GPT-5.4 ; they illustrate the patterns on this proposition and do not estimate their frequency. Drift pattern: the pattern of the transition from the initial stance to the AI-majority label, as defined in the main text. Drift stage: generation stage if the human-majority label differs from the initial stance, extraction stage if the AI-majority label differs from the human-majority label, both stages if both differ. For each pattern the argument with the largest sum of the human-majority and AI-majority vote shares is shown, in full; four arguments share the largest sum for flipping and three for deviation from neutrality, and the first in the row order of the study’s data file is shown. Moderation and moderation to neutral do not occur among the 100 arguments of this proposition.
Table 24
Figure S6: Pass rate and AUC. ( A ) Pass rate as a function of the threshold from 0.5 to 1 , over all propositions and initial stances. ( B ) Area under the pass-rate curve of (A), normalized by its maximum value of 0.5 , across different models, providing a summary measure of stance preservation performance over the range of thresholds.
Large language model reliability studies whether outputs remain correct, calibrated, robust, and controllable across prompts, contexts, and deployment conditions.
Large language models are increasingly used to simulate social media users and infer how individuals may respond to online discussions. However, it remains unclear whether these simulations reflect precise user-specific beliefs or whether they are highly sensitive to semantically independent changes in conversational contexts. In this work, we study counterfactual context revision as a framework for auditing LLM-based stance simulation. Given an original online conversation, we first infer a target user's stance toward a specific topic. We then apply controlled revision strategies to the conversational context and simulate the user's stance again under the revised context. We compare text-only revision strategies with a multimodal one that incorporates meme-based context and evaluate two main effectiveness metrics, i.e., average directional stance shift and stance transition rate. The results reveal effective and robust stance transitions in both text-only and multimodal strategies across different polarization-preference mechanisms. Our study contributes an evaluation framework for understanding the context sensitivity of LLM-based stance simulation (https://github.com/STARResearchLab/StanceShift). More broadly, it highlights both the promise and risk of using LLMs as proxies for social media users in social simulation.
Xinnong Zhang, Wanting Shan, Hanjia Lyu +2
1Fudan University · 3Shanghai Innovation Institute · University of Rochester
Large language models increasingly take part in extended conversations where they can both shape and be shaped by another party's views, yet most persuasion research measures only one side of that exchange. We introduce a controlled LLM-LLM debate framework in which two language-model roles repeatedly discuss a contested topic (climate-change attribution): one is given an assigned stance and, in some conditions, an explicit instruction to persuade, while the other receives no assigned position. Across four ordered model combinations involving GPT-6 Luna and Gemini 2.5 Flash Lite, we track both each role's explicitly reported position and a separate measure of its turn-by-turn behavior inside the conversation. Opinion dynamics differ substantially across model combinations: same-model interactions narrow the opinion gap on average, clearly for only one of the two models, while cross-model interactions widen it. Explicit reports and in-dialogue behavior capture distinct aspects of opinion dynamics, and a blanket instruction to persuade does not reliably increase influence. These findings suggest that conversational influence depends not only on assigned roles and initial positions but also on the participating models, with implications for how opinion change is measured in extended LLM interactions.
Mohammad Siahkamari, Phuoc Dat Nguyen, Stephen MacNeil
Partisan news media erode cross-partisan trust, but large language models (LLMs) offer a potential means of debiasing such content at scale. Across two pre-registered experiments, we tested whether LLM-generated debiasing of liberal news headlines could improve conservative readers' trust-relevant judgments. Study 1 found that subtle lexical debiasing (replacing emotive words with more moderate synonyms) had no effect on any outcome. Study 2 found that a more substantive reframing intervention significantly increased conservatives' perceived trustworthiness, completeness, and willingness to engage with liberal news headlines, without producing a backfire effect among a sample of liberals. In Study 1, the intervention produced robust effects among LLM-simulated silicon participants, whereas it had no impact on human readers. In Study 2, the intervention's effects among silicon participants aligned directionally with human responses but were significantly larger in magnitude for some outcomes. Moderation analyses revealed that the model's implicit theory of who responds to debiasing diverged from the psychological profile that actually predicted human responsiveness. These findings demonstrate that LLM-based debiasing can improve cross-partisan receptivity when targeting ideological framing rather than surface-level language, but that current models lack both the quantitative accuracy and qualitative psychological fidelity to evaluate their own interventions without human oversight.
Faisal Feroz, Jonas R. Kunst
Department of Experimental Psychology, University of Oxford · Uehiro Oxford Institute, University of Oxford · Psychology Programme, School of Social Sciences, Nanyang Technological University +1