To Manipulate or Not: Opinion Dynamics in LLM Interactions

Mohammad Siahkamari; Phuoc Dat Nguyen; Stephen MacNeil

cs.AIOct 20, 2026

To Manipulate or Not: Opinion Dynamics in LLM Interactions

Authors: Mohammad Siahkamari, Phuoc Dat Nguyen, Stephen MacNeil

Organizations: Temple University, Philadelphia, PA, USA

Abstract

Large language models increasingly take part in extended conversations where they can both shape and be shaped by another party's views, yet most persuasion research measures only one side of that exchange. We introduce a controlled LLM-LLM debate framework in which two language-model roles repeatedly discuss a contested topic (climate-change attribution): one is given an assigned stance and, in some conditions, an explicit instruction to persuade, while the other receives no assigned position. Across four ordered model combinations involving GPT-6 Luna and Gemini 2.5 Flash Lite, we track both each role's explicitly reported position and a separate measure of its turn-by-turn behavior inside the conversation. Opinion dynamics differ substantially across model combinations: same-model interactions narrow the opinion gap on average, clearly for only one of the two models, while cross-model interactions widen it. Explicit reports and in-dialogue behavior capture distinct aspects of opinion dynamics, and a blanket instruction to persuade does not reliably increase influence. These findings suggest that conversational influence depends not only on assigned roles and initial positions but also on the participating models, with implications for how opinion change is measured in extended LLM interactions.

Figures & tables

Design quantity	Count
Ordered model pairings	4
Assigned Opponent positions	3
Persuasion styles	2
Statement wordings	2
Cells per pairing ( $3\times 2\times 2$ )	12
Debates per cell	10

Table 1: Sample structure. Every exact cell contains 10 independent debates.

Term	Definition
NM / M	Non-manipulative / manipulative Opponent prompt
Assigned Human-cause	Opponent prompted toward human causation
Assigned Neutral	Opponent prompted to approach the issue neutrally
Assigned Natural-cause	Opponent prompted toward natural-process causation
Self-reported position	Separate 0–6 report about the original statement
Interaction-state opinion	Cumulative state implied by in-dialogue ACCEPT/REJECT/IGNORE responses

Table 2: Experimental design and terminology.

Position	O Pre	O Post	D Pre	D Post
Human-cause	5.94	5.16	5.28	5.02
Neutral	3.86	4.58	5.34	4.89
Natural-cause	2.59	3.89	5.33	4.81

Table 3: Mean human-causation-aligned ratings by assigned Opponent position, pooled across four model pairings (

N=160

per position).

Figure 1: Self-reported topic-position trajectories for Luna Discussant–Luna Opponent. NM/M indicate non-manipulative/manipulative prompts. Panel titles give the persuasion style, the Opponent’s assigned stance (Human-cause, Neutral, Natural-cause), and which statement wording was presented (Human or Natural). Ratings are human-causation-aligned: 6 denotes strongest support for human causation and 0 strongest support for natural-process causation. Bands are 95% confidence intervals across 10 runs per cell. The other three model pairings appear in the additional model-pairing figures.

Outcome	Mean	95% CI
Discussant aligned shift	-0.41	[-0.57, -0.25]
Opponent aligned shift	0.42	[0.22, 0.61]
Discussant absolute shift	1.07	[0.94, 1.20]
Opponent absolute shift	1.52	[1.37, 1.66]
Pairwise gap change	0.01	[-0.23, 0.24]

Table 4: Primary aligned conversation-level outcomes (

N=480

D model $\to$ O model	D shift	O shift	Gap pre	Gap post	Gap change
Luna $\to$ Luna	-0.11	1.23	1.47	0.21	-1.26
Gemini $\to$ Gemini	-0.63	0.07	1.85	1.60	-0.25
Luna $\to$ Gemini	0.03	-0.79	2.20	3.03	0.83
Gemini $\to$ Luna	-0.93	1.16	1.63	2.33	0.70

Table 5: Model-pairing-specific aligned shifts and change in the absolute role gap (120 completed debates per row). D/O indicate Discussant/Opponent. Negative gap change indicates convergence.

Figure 2: In-dialogue interaction-state trajectories for Luna Discussant–Luna Opponent. NM/M denote non-manipulative/manipulative Opponent prompts. Panels distinguish topic wording and assigned stance. Lines are human-causation-aligned interaction states (6 = strongest support for human causation); bands are 95% confidence intervals across the 10 runs per cell, and shading between role means represents their gap. Additional pairings appear in the appendix.

Figure 3: Absolute Discussant–Opponent separation in self-reported topic ratings for Luna–Luna. Stages are Pre, individual rounds, and Post. Lower values indicate a smaller within-pair gap; panels cross assigned position, persuasion style, and statement wording. Bands are 95% confidence intervals across 10 runs per cell. Results for the remaining model pairings appear in the appendix.

Figure 4: Conversation-level mean change in absolute self-reported role gap, with

t

-based 95% confidence intervals (

n=120

per ordered pairing). Negative changes indicate convergence; positive changes indicate divergence. Blue points are same-model pairings and orange points are cross-model pairings. Absolute within-pair differences are invariant to reversing the statement’s rating scale.

Figure 5: Bidirectional mean pre/post self-reported position changes for each ordered model pairing, with conversation-level

t

-based 95% confidence intervals (

n=120

per pairing). Signed differences are aligned so that positive scores always indicate movement toward the human-causation position, regardless of the statement wording.

Figure 6: ACCEPT, REJECT, and IGNORE proportions by exact cell for the Discussant (top) and Opponent (bottom) in Luna–Luna debates. The remaining model-pairing versions are specified in the appendix.

Panel of Figure 7 — Figure 7: Luna–Luna: deterministic state-transition matrices for Discussant (top left) and Opponent (top right), and interaction state versus self-reported rating for Discussant (bottom left) and Opponent (bottom right), all on the human-causation-aligned scale. The dashed diagonal denotes equal ratings in the scatterplots. See the appendix for all other model pairings.

Appendix figures & tables27 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Completed debates in each of the 12 position-by-style-by-wording cells; the dashed reference indicates the target of 10 debates. Ordered pairing: GPT-6 Luna

\to

GPT-6 Luna.

Figure 9: Completed debates in each of the 12 position-by-style-by-wording cells; the dashed reference indicates the target of 10 debates. Ordered pairing: Gemini 2.5 Flash Lite

\to

Gemini 2.5 Flash Lite.

Figure 10: Completed debates in each of the 12 position-by-style-by-wording cells; the dashed reference indicates the target of 10 debates. Ordered pairing: GPT-6 Luna

\to

Gemini 2.5 Flash Lite.

Figure 11: Completed debates in each of the 12 position-by-style-by-wording cells; the dashed reference indicates the target of 10 debates. Ordered pairing: Gemini 2.5 Flash Lite

\to

GPT-6 Luna.

Figure 12: Human-causation-aligned post-minus-pre shifts for Discussant and Opponent; positive bars indicate movement toward human causation regardless of the statement wording. Ordered pairing: GPT-6 Luna

\to

GPT-6 Luna.

Figure 13: Human-causation-aligned post-minus-pre shifts for Discussant and Opponent; positive bars indicate movement toward human causation regardless of the statement wording. Ordered pairing: Gemini 2.5 Flash Lite

\to

Gemini 2.5 Flash Lite.

Figure 14: Human-causation-aligned post-minus-pre shifts for Discussant and Opponent; positive bars indicate movement toward human causation regardless of the statement wording. Ordered pairing: GPT-6 Luna

\to

Gemini 2.5 Flash Lite.

Figure 15: Human-causation-aligned post-minus-pre shifts for Discussant and Opponent; positive bars indicate movement toward human causation regardless of the statement wording. Ordered pairing: Gemini 2.5 Flash Lite

\to

GPT-6 Luna.

Figure 16: Human-causation-aligned interaction-state trajectories derived from categorical decisions (10 replicates per cell); shaded distance is the gap between role means. Ordered pairing: Gemini 2.5 Flash Lite

\to

Gemini 2.5 Flash Lite.

Figure 17: Human-causation-aligned interaction-state trajectories derived from categorical decisions (10 replicates per cell); shaded distance is the gap between role means. Ordered pairing: GPT-6 Luna

\to

Gemini 2.5 Flash Lite.

Figure 18: Human-causation-aligned interaction-state trajectories derived from categorical decisions (10 replicates per cell); shaded distance is the gap between role means. Ordered pairing: Gemini 2.5 Flash Lite

\to

GPT-6 Luna.

Figure 19: Round-level human-causation-aligned topic ratings, separated by prompt condition and statement wording (10 replicates per cell). Ordered pairing: Gemini 2.5 Flash Lite

\to

Gemini 2.5 Flash Lite.

Figure 20: Round-level human-causation-aligned topic ratings, separated by prompt condition and statement wording (10 replicates per cell). Ordered pairing: GPT-6 Luna

\to

Gemini 2.5 Flash Lite.

Figure 21: Round-level human-causation-aligned topic ratings, separated by prompt condition and statement wording (10 replicates per cell). Ordered pairing: Gemini 2.5 Flash Lite

\to

GPT-6 Luna.

Figure 22: Within-pair absolute differences in direct topic ratings over 20 rounds. Ordered pairing: Gemini 2.5 Flash Lite

\to

Gemini 2.5 Flash Lite.

Figure 23: Within-pair absolute differences in direct topic ratings over 20 rounds. Ordered pairing: GPT-6 Luna

\to

Gemini 2.5 Flash Lite.

Figure 24: Within-pair absolute differences in direct topic ratings over 20 rounds. Ordered pairing: Gemini 2.5 Flash Lite

\to

GPT-6 Luna.

Figure 25: ACCEPT/REJECT/IGNORE proportions by exact cell for Discussant (top) and Opponent (bottom). Ordered pairing: Gemini 2.5 Flash Lite

\to

Gemini 2.5 Flash Lite.

Figure 26: ACCEPT/REJECT/IGNORE proportions by exact cell for Discussant (top) and Opponent (bottom). Ordered pairing: GPT-6 Luna

\to

Gemini 2.5 Flash Lite.

Figure 27: ACCEPT/REJECT/IGNORE proportions by exact cell for Discussant (top) and Opponent (bottom). Ordered pairing: Gemini 2.5 Flash Lite

\to

GPT-6 Luna.

Panel of Figure 28 — Figure 28: Normalized interaction-state transitions for Discussant (top) and Opponent (bottom). Ordered pairing: Gemini 2.5 Flash Lite $\to$ Gemini 2.5 Flash Lite.

Panel of Figure 29 — Figure 29: Normalized interaction-state transitions for Discussant (top) and Opponent (bottom). Ordered pairing: GPT-6 Luna $\to$ Gemini 2.5 Flash Lite.

Panel of Figure 30 — Figure 30: Normalized interaction-state transitions for Discussant (top) and Opponent (bottom). Ordered pairing: Gemini 2.5 Flash Lite $\to$ GPT-6 Luna.

Figure 31: Deterministic interaction state versus direct topic rating for Discussant (top) and Opponent (bottom). Ordered pairing: Gemini 2.5 Flash Lite

\to

Gemini 2.5 Flash Lite.

Figure 32: Deterministic interaction state versus direct topic rating for Discussant (top) and Opponent (bottom). Ordered pairing: GPT-6 Luna

\to

Gemini 2.5 Flash Lite.

Figure 33: Deterministic interaction state versus direct topic rating for Discussant (top) and Opponent (bottom). Ordered pairing: Gemini 2.5 Flash Lite

\to

GPT-6 Luna.

Figure 34: Exploratory difference in mean pre/post absolute-gap change between manipulative and non-manipulative Opponent prompts within each ordered model pairing. Points and bars show difference of means and unadjusted Welch 95% confidence intervals. Negative differences indicate a more convergent (or less divergent) average gap under manipulation, not necessarily persuasion of the Discussant. All four intervals include zero; the pairing-by-manipulation interaction test is not statistically significant.

Persuasion in Language Models

Persuasion in language models studies how they influence human and AI beliefs or decisions, and how to measure and limit undue influence.

See all 79 papers

Measuring Opinion Bias and Sycophancy via LLM-based Persuasion
Apr 23, 2026Rodrigo Nogueira, Giovana Kerche Bonás, Thales Sales Almeida +7LLM Sycophancy LLM Auditing
Maritaca AI · JusBrasil
Characterizing Opinion Evolution of Networked LLMs
Jun 5, 2026Caleb Probine, Yigit Ege Bayiz, Filippos Fotiadis +3Multi-Agent LLM Systems Opinion Dynamics
The University of Texas at Austin
Peer Influence across Heterogeneous AI Models
Oct 2, 2026Frida Nøhr Laustsen, Marie Haahr Petersen, Victoria Popa +4Persuasion in Language Models Multi-Agent Communication
IT University of Copenhagen, Denmark · University of Pisa, Italy · City St. George’s, University of London, United Kingdom +1

Apr 23, 2026J/K move · Enter open · S save

Measuring Opinion Bias and Sycophancy via LLM-based Persuasion

Rodrigo Nogueira, Giovana Kerche Bonás, Thales Sales Almeida, Andrea Roque+6

Maritaca AI · JusBrasil

LLM Sycophancy LLM Auditing Language Model Bias Evaluation Persuasion in Language Models Language Model Probing

Open paper AskPDF arXiv

Large language models increasingly shape the information people consume: they are embedded in search, consulted for professional advice, deployed as agents, and used as a first stop for questions about policy, ethics, health, and politics. When such a model silently holds a position on a contested topic, that position propagates at scale into users' decisions. Eliciting a model's positions is harder than it first appears: contemporary assistants answer direct opinion questions with evasive disclaimers, and the same model may concede the opposite position once the user starts arguing one side. We propose a method, released as the open-source llm-bias-bench, for discovering the opinions an LLM actually holds on contested topics under conditions that resemble real multi-turn interaction. The method pairs two complementary free-form probes. Direct probing asks for the model's opinion across five turns of escalating pressure from a simulated user. Indirect probing never asks for an opinion and engages the model in argumentative debate, letting bias leak through how it concedes, resists, or counter-argues. Three user personas (neutral, agree, disagree) collapse into a nine-way behavioral classification that separates persona-independent positions from persona-dependent sycophancy, and an auditable LLM judge produces verdicts with textual evidence. The first instantiation ships 38 topics in Brazilian Portuguese across values, scientific consensus, philosophy, and economic policy. Applied to 13 assistants, the method surfaces findings of practical interest: argumentative debate triggers sycophancy 2-3x more than direct questioning (median 50% to 79%); models that look opinionated under direct questioning often collapse into mirroring under sustained arguments; and attacker capability matters mainly when an existing opinion must be dislodged, not when the assistant starts neutral.

Abstract

Persuasion in Language Models

Explore similar work

Measuring Opinion Bias and Sycophancy via LLM-based Persuasion

Characterizing Opinion Evolution of Networked LLMs

Peer Influence across Heterogeneous AI Models