cs.CLOct 3, 2026

Stance Drift: How AI-mediated Communication Distorts Our Message

Authors: Lingchong Liu, Yanfei Zhou, Jacob Bien, Y. X. Rachel Wang, Lucy Xia, Xin Tong

Organizations: Department of Information Systems, Business Statistics and Operations Management Hong Kong University of Science and Technology Hong Kong SAR · Department of Data Sciences and Operations University of Southern California Los Angeles, CA, USA · School of Mathematics and Statistics University of Sydney Sydney, Australia · Faculty of Business and Economics University of Hong Kong, Hong Kong SAR

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Large language models (LLMs) increasingly mediate human communication, from drafting emails to summarizing scientific reports, yet whether they faithfully preserve a speaker's position remains largely untested. We model AI-mediated communication as a two-step generation-extraction pipeline: one LLM produces an argument from a specified stance, and a second LLM extracts the stance from that argument. We represent the pipeline as a probabilistic state transition over five Likert-type stance categories and define the stance preservation rate (SPR) as the average probability that the extracted stance matches the initial stance. Across 112 debate propositions, none of the nine LLMs tested exceeded an SPR of 0.7 under the default configuration. Three drift patterns accounted for most of the drift: polarization, deviation from neutrality, and flipping. Among the mitigation strategies tested, including in-context learning, multiple extraction with shuffled options, assertion, and reflection, only adding medium reasoning effort to a reflection prompt for GPT-5.4 substantially improved the SPR, to 0.775, yet polarization remained the largest pattern, with 0.119 of the transition mass. An exploratory comparison with human extraction on a single proposition suggests that drift arises at both the generation and the extraction stage. These results point to a fidelity gap in AI-mediated communication, with implications for journalism, policy deliberation, scientific communication, and other domains where opinion-laden messages pass through language models.

Figures & tables

Figure 1: AI-mediated communication model. A sender generates an argument with LLM assistance from an initial stance; a receiver extracts the stance from the argument with LLM assistance.
Figure 2: Example of empirical transition matrices deviating from the ideal identity matrix. Rows are the initial stances and columns the extracted stances; the (i,j)(i,j) entry is the probability, in percent, of extracting the jj th choice given the ii th choice as the initial stance. ( A ) GPT-5.4 , default configuration, proposition “We should increase fuel tax”. ( B ) GPT-5.4 , default configuration, proposition “The use of AI should be abandoned”. ( C ) The identity matrix: perfect preservation of every stance.
Figure 3: The two-step generation and extraction framework. An initial stance on a proposition is expanded into an argument, and the stance is extracted back from the argument.
Figure 4: SPR across models under the default configuration. The solid rule marks perfect fidelity (SPR =1=1 ) and the dashed rule the uniform random baseline (SPR =0.2=0.2 ).
ModelAgree stronglyAgreeNeutralDisagreeDisagree strongly
GPT-4o-mini.933 (.021).108 (.016).717 (.026).310 (.035).784 (.032)
GPT-4.1.962 (.014).599 (.031).800 (.026).168 (.027).903 (.016)
GPT-5.4.996 (.002).147 (.022).703 (.032).136 (.023).982 (.010)
GPT-3.5-turbo.621 (.031).484 (.031).313 (.023).244 (.027).527 (.036)
gemma-3n-E4B.595 (.040).348 (.034).920 (.019).114 (.022).617 (.041)
Llama-3.3-70B.954 (.015).505 (.036).851 (.023).161 (.026).942 (.017)
Table 1: Stance preservation by initial stance for the nine models under the default configuration. ( A ) Success rates. ( B ) Proportion of propositions whose one-sided 99% CI of the success rate, with Bonferroni correction, includes 1.
ModelSPRPolarizationModerationModeration to neutralFlippingDeviation from neutrality
GPT-4o-mini.570 (.009).246 (.011).022 (.004).009 (.002).097 (.012).057 (.005)
GPT-4.1.686 (.007).244 (.005).024 (.004).001 (.000).005 (.001).040 (.005)
GPT-5.4.593 (.008).328 (.006).002 (.001).000 (.000).018 (.004).059 (.006)
GPT-3.5-turbo.438 (.008).158 (.009).087 (.006).000 (.000).180 (.014).137 (.005)
gemma-3n-E4B.519 (.013).165 (.010).041 (.006).082 (.010).178 (.015).016 (.004)
Llama-3.3-70B.683 (.011).191 (.007).014 (.003).015 (.003).069 (.009).030 (.005)
Table 2: SPR and the components of the SDR by drift pattern for the nine models under the default configuration. Mean (SE) over the 112 propositions of the per-proposition values averaged over the five initial stances; each row sums to 1 up to rounding. Per-stance values and all tested models: Tables S5 – S10 in the appendix.
Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Figure S1: Individual example illustrating stance drift in the two-step framework. Given an initial “Agree strongly” stance on the proposition “The use of AI should be abandoned” an LLM agent is first prompted, in the generation stage, to produce an argument consistent with the specified stance. In the subsequent extraction stage, a separate LLM agent—with no access to the prior interaction—is prompted to infer the stance expressed in the generated argument relative to the proposition. In this case, the model outputs a “Disagree strongly” stance, diverging from the initial input. This discrepancy indicates that a stance drift has occurred.
Figure S2: Individual example illustrating stance preservation in the two-step framework. Starting from an initial “Agree” stance on the proposition “Casinos should be banned”, the generation stage produces an argument reflecting this stance. The extraction stage subsequently identifies the stance as “Agree”, aligning with the initial input and indicating that the stance was preserved.
Figure S3: Empirical transition matrices with standard errors. The rows and columns represent the initial stances (rows) and the extracted stances (columns). The (i,j)(i,j) entry indicates the probability of returning to the jj th choice given the ii th choice as the initial stance. The color intensity reflects the transition probabilities. Each cell gives the mean transition probability with its standard error over the 100 trials, in percent. ( A ) Empirical transition matrix for the proposition “We should increase fuel tax”, derived from experiments with GPT-5.4 . The matrix reveals large deviations from the ideal identity matrix, with a clear pattern of polarization. ( B ) Empirical transition matrix for another proposition “The use of AI should be abandoned”, based on experiments with GPT-5.4 . The matrix shows substantial deviations from the ideal identity matrix, including patterns of deviation from neutrality and flipping.
Figure S4: SPR for all models tested. The solid rule marks perfect fidelity (SPR =1=1 ) and the dashed rule the uniform random baseline (SPR =0.2=0.2 ).
ModelAgree stronglyAgreeNeutralDisagreeDisagree strongly
GPT-4o-mini.933 (.021).108 (.016).717 (.026).310 (.035).784 (.032)
GPT-4.1.962 (.014).599 (.031).800 (.026).168 (.027).903 (.016)
GPT-5.4.996 (.002).147 (.022).703 (.032).136 (.023).982 (.010)
GPT-3.5-turbo.621 (.031).484 (.031).313 (.023).244 (.027).527 (.036)
gemma-3n-E4B.595 (.040).348 (.034).920 (.019).114 (.022).617 (.041)
Llama-3.3-70B.954 (.015).505 (.036).851 (.023).161 (.026).942 (.017)
Appendix
Table S1: Average success rates with respect to initial stance for all tested models.
ModelAgree stronglyAgreeNeutralDisagreeDisagree strongly
GPT-4o-mini.223 (.040).000 (.000).402 (.047).116 (.030).536 (.047)
GPT-4.1.920 (.026).286 (.043).616 (.046).054 (.021).732 (.042)
GPT-5.4.991 (.009).027 (.015).527 (.047).018 (.013).973 (.015)
GPT-3.5-turbo.000 (.000).045 (.020).018 (.013).000 (.000).080 (.026)
gemma-3n-E4B.348 (.045).098 (.028).500 (.047).009 (.009).464 (.047)
Llama-3.3-70B.920 (.026).295 (.043).714 (.043).054 (.021).902 (.028)
Appendix
Table S2: Proportion of propositions having 1 included in the 99% confidence interval (1 sided, with Bonferroni correction) of the mean diagonal probabilities, with respect to initial stance, for all tested models.
ModelAgree stronglyAgreeNeutralDisagreeDisagree strongly
GPT-4o-mini.107 (.029).000 (.000).250 (.041).071 (.024).375 (.046)
GPT-4.1.893 (.029).179 (.036).491 (.047).027 (.015).670 (.045)
GPT-5.4.973 (.015).009 (.009).402 (.047).000 (.000).955 (.020)
GPT-3.5-turbo.000 (.000).018 (.013).000 (.000).000 (.000).018 (.013)
gemma-3n-E4B.259 (.042).062 (.023).339 (.045).009 (.009).312 (.044)
Llama-3.3-70B.902 (.028).223 (.040).562 (.047).036 (.018).848 (.034)
Appendix
Table S3: Proportion of propositions having 1 included in the 99% confidence interval (1 sided) of the mean diagonal probabilities, with respect to initial stance, for all tested models.
ModelAgree stronglyAgreeNeutralDisagreeDisagree strongly
GPT-4o-mini.884 (.030).000 (.000).277 (.042).098 (.028).598 (.047)
GPT-4.1.902 (.028).188 (.037).491 (.047).036 (.018).661 (.045)
GPT-5.4.973 (.015).009 (.009).375 (.046).000 (.000).955 (.020)
GPT-3.5-turbo.241 (.041).071 (.024).009 (.009).009 (.009).250 (.041)
gemma-3n-E4B.429 (.047).071 (.024).804 (.038).009 (.009).464 (.047)
Llama-3.3-70B.893 (.029).223 (.040).536 (.047).036 (.018).848 (.034)
Appendix
Table S4: Pass rate at threshold 0.95 with respect to initial stance, for all tested models. A proposition–initial stance combination passes if its success rate exceeds 0.95; each cell is the proportion of the 112 propositions that pass.
ModelSPRPolarizationModerationModeration to neutralFlippingDeviation from neutrality
GPT-4o-mini.570 (.009).246 (.011).022 (.004).009 (.002).097 (.012).057 (.005)
GPT-4.1.686 (.007).244 (.005).024 (.004).001 (.000).005 (.001).040 (.005)
GPT-5.4.593 (.008).328 (.006).002 (.001).000 (.000).018 (.004).059 (.006)
GPT-3.5-turbo.438 (.008).158 (.009).087 (.006).000 (.000).180 (.014).137 (.005)
gemma-3n-E4B.519 (.013).165 (.010).041 (.006).082 (.010).178 (.015).016 (.004)
Llama-3.3-70B.683 (.011).191 (.007).014 (.003).015 (.003).069 (.009).030 (.005)
Appendix
Table S5: SPR and the components of the stance drift rate (SDR) by drift pattern for all tested models. As Table 2 in the main text: overall values with a uniform prior over initial stances, as fractions of the transition mass; the five pattern columns are the components of the SDR, one minus SPR, and sum to it; each row sums to 1 up to rounding; mean (SE) over the 112 propositions, with the SE computed on the per-proposition values averaged over the five initial stances.
ModelAgree stronglyAgreeNeutralDisagreeDisagree strongly
GPT-4o-mini—.589 (.040)—.642 (.037)—
GPT-4.1—.397 (.031)—.825 (.028)—
GPT-5.4—.777 (.028)—.863 (.023)—
GPT-3.5-turbo—.326 (.035)—.462 (.035)—
gemma-3n-E4B—.234 (.034)—.592 (.041)—
Llama-3.3-70B—.137 (.024)—.817 (.028)—
Appendix
Table S6: Mass of polarization with respect to initial stance, for all tested models. Fraction of the transition mass of the initial stance; mean (SE) over the 112 propositions. A dash marks an initial stance for which the pattern is empty by definition.
ModelAgree stronglyAgreeNeutralDisagreeDisagree strongly
GPT-4o-mini.000 (.000)———.108 (.021)
GPT-4.1.038 (.014)———.081 (.013)
GPT-5.4.002 (.001)———.008 (.004)
GPT-3.5-turbo.294 (.027)———.140 (.021)
gemma-3n-E4B.140 (.023)———.063 (.017)
Llama-3.3-70B.046 (.015)———.022 (.006)
Appendix
Table S7: Mass of moderation with respect to initial stance, for all tested models. Fraction of the transition mass of the initial stance; mean (SE) over the 112 propositions. A dash marks an initial stance for which the pattern is empty by definition.
ModelAgree stronglyAgreeNeutralDisagreeDisagree strongly
GPT-4o-mini.000 (.000).039 (.009)—.004 (.002).000 (.000)
GPT-4.1.000 (.000).003 (.001)—.002 (.001).000 (.000)
GPT-5.4.000 (.000).000 (.000)—.000 (.000).000 (.000)
GPT-3.5-turbo.000 (.000).000 (.000)—.001 (.000).000 (.000)
gemma-3n-E4B.027 (.009).315 (.035)—.043 (.014).023 (.010)
Llama-3.3-70B.000 (.000).073 (.013)—.001 (.001).000 (.000)
Appendix
Table S8: Mass of moderation to neutral with respect to initial stance, for all tested models. Fraction of the transition mass of the initial stance; mean (SE) over the 112 propositions. A dash marks an initial stance for which the pattern is empty by definition.
ModelAgree stronglyAgreeNeutralDisagreeDisagree strongly
GPT-4o-mini.067 (.021).264 (.034)—.045 (.016).108 (.027)
GPT-4.1.000 (.000).002 (.001)—.005 (.002).016 (.005)
GPT-5.4.002 (.001).076 (.018)—.001 (.001).011 (.009)
GPT-3.5-turbo.085 (.018).190 (.028)—.293 (.031).332 (.033)
gemma-3n-E4B.237 (.035).103 (.022)—.251 (.039).296 (.040)
Llama-3.3-70B.000 (.000).286 (.038)—.021 (.010).036 (.014)
Appendix
Table S9: Mass of flipping with respect to initial stance, for all tested models. Fraction of the transition mass of the initial stance; mean (SE) over the 112 propositions. A dash marks an initial stance for which the pattern is empty by definition.
ModelAgree stronglyAgreeNeutralDisagreeDisagree strongly
GPT-4o-mini——.283 (.026)——
GPT-4.1——.200 (.026)——
GPT-5.4——.297 (.032)——
GPT-3.5-turbo——.687 (.023)——
gemma-3n-E4B——.080 (.019)——
Llama-3.3-70B——.149 (.023)——
Appendix
Table S10: Mass of deviation from neutrality with respect to initial stance, for all tested models. Fraction of the transition mass of the initial stance; mean (SE) over the 112 propositions. A dash marks an initial stance for which the pattern is empty by definition.
Figure S5: GPT-5.4 across the four configurations, with respect to initial stance. ( A ) Success rate. ( B ) Mass of polarization. The configurations are the default configuration, reflection, third-person reflection, and third-person reflection with medium reasoning effort; the three later configurations each add one change to the previous one. Bars are means over the 112 propositions and error bars are standard errors. The values are listed in Tables S1 and S6 .
Initial StanceAgree stronglyAgreeNeutralDisagreeDisagree strongly
Generation-stage drift020501
Extraction-stage drift01913201
Appendix
Table S11: Argument counts by drift stage and initial stance for the human-extraction study. Each cell records the number of arguments (out of 20 per initial stance) attributable to each drift stage, for proposition “The use of AI should be abandoned” generated by GPT-5.4 . Generation-stage drift: the human-majority label differs from the known initial stance, indicating that the generated argument itself does not carry the intended stance signal. Extraction-stage drift: the AI-majority label differs from the human-majority label, attributing the discrepancy to the LLM extractor rather than the generated content. The two rows are not mutually exclusive.
Initial stanceGenerated argumentHuman-majority labelAI-majority labelDrift patternDrift stage
DisagreeAbandoning AI entirely would throw away major benefits in medicine, education, accessibility, scientific research, and public services. Like other powerful technologies, AI can be misused, but that argues for strong rules, transparency, and accountability—not total rejection. The United States would also harm its competitiveness and innovation capacity by stepping back while other countries continue developing it. A better approach is to limit dangerous applications, require human oversight where stakes are high, and enforce ethical standards so society can capture AI’s advantages while reducing risks.DisagreeDisagree stronglypolarizationextraction stage
AgreeAI should not be abandoned, but its use should be approached carefully. In the United States, AI already helps doctors detect diseases earlier, assists businesses with routine tasks, and supports scientific research. Rejecting it entirely would mean giving up major benefits and global competitiveness. The real issue is not whether AI should exist, but how to govern it responsibly. Strong rules on privacy, transparency, bias, and accountability can reduce harm while preserving innovation. With thoughtful oversight, AI can remain a valuable tool rather than something society must completely discard.DisagreeDisagree stronglyflippingboth stages (flipping at generation, polarization at extraction)
NeutralAI’s future depends on how it is designed, governed, and used. It can improve medical diagnosis, expand access to education, and automate dangerous tasks, yet it also raises serious concerns about bias, privacy, job disruption, and misuse. Because both benefits and harms are significant, abandoning it entirely may be premature, but embracing it without safeguards would be reckless. A balanced approach would focus on strict oversight, transparency, accountability, and limits on high-risk applications while society continues evaluating whether its advantages truly outweigh its costs.NeutralDisagreedeviation from neutralityextraction stage
Appendix
Table S12: One example argument per drift pattern from the human-extraction study. All examples come from the single proposition of the study, “The use of AI should be abandoned”, with arguments generated by GPT-5.4 and AI labels extracted by GPT-5.4 ; they illustrate the patterns on this proposition and do not estimate their frequency. Drift pattern: the pattern of the transition from the initial stance to the AI-majority label, as defined in the main text. Drift stage: generation stage if the human-majority label differs from the initial stance, extraction stage if the AI-majority label differs from the human-majority label, both stages if both differ. For each pattern the argument with the largest sum of the human-majority and AI-majority vote shares is shown, in full; four arguments share the largest sum for flipping and three for deviation from neutrality, and the first in the row order of the study’s data file is shown. Moderation and moderation to neutral do not occur among the 100 arguments of this proposition.
Table 24
Figure S6: Pass rate and AUC. ( A ) Pass rate as a function of the threshold from 0.50.5 to 11 , over all propositions and initial stances. ( B ) Area under the pass-rate curve of (A), normalized by its maximum value of 0.50.5 , across different models, providing a summary measure of stance preservation performance over the range of thresholds.
More in this topic

LLM Reliability

Large language model reliability studies whether outputs remain correct, calibrated, robust, and controllable across prompts, contexts, and deployment conditions.

See all 802 papers

Explore similar work

CardsList

  1. Revising Context, Shifting Simulated Stance: Auditing LLM-Based Stance Simulation in Online Discussions

    Jun 4, 2026Xinnong Zhang, Wanting Shan, Hanjia Lyu +2Counterfactual EvaluationLLM Auditing

    1Fudan University · 3Shanghai Innovation Institute · University of Rochester

  2. To Manipulate or Not: Opinion Dynamics in LLM Interactions

    Oct 20, 2026Mohammad Siahkamari, Phuoc Dat Nguyen, Stephen MacNeilPersuasion in Language ModelsBelief Updating in LLMs

    Temple University, Philadelphia, PA, USA

  3. Can AI Debias the News? LLM Interventions Improve Cross-Partisan Receptivity but LLMs Overestimate Their Own Effectiveness

    May 1, 2026Faisal Feroz, Jonas R. KunstLLM EvaluationHuman Behavior Simulation

    Department of Experimental Psychology, University of Oxford · Uehiro Oxford Institute, University of Oxford · Psychology Programme, School of Social Sciences, Nanyang Technological University +1