Authors: Frida Nøhr Laustsen, Marie Haahr Petersen, Victoria Popa, Ariel Flint, Romualdo Pastor-Satorras, Andrea Baronchelli, Luca Maria Aiello
Organizations: IT University of Copenhagen, Denmark · University of Pisa, Italy · City St. George’s, University of London, United Kingdom · Universitat Politècnica de Catalunya, Spain
When two AI agents disagree, who persuades whom? As multi-agent systems increasingly combine language models of different families and sizes, the answer can determine which judgments survive interaction. Measuring persuasion as the probabilistic shift in an agent's decision after a single exchange with a dissenting peer, we test seven open-weight models across three language understanding tasks. We find that persuasion is strong: when models disagree, receivers often abandon their initial judgment after seeing a peer's answer and explanation. Surprisingly, however, neither standalone certainty nor model scale reliably predicts persuasion dynamics. Models producing almost perfectly consistent decisions in isolation can be among the most susceptible to persuasion, and small models can match larger ones as persuaders and resist their influence just as effectively. Furthermore, we show that the size of the shift depends more on the susceptibility of the listener than on the persuasiveness of the speaker. Persuasion patterns are therefore specific to each model pairing, with heterogeneity amplifying persuasion in some combinations and suppressing it in others, allowing a dissenting agent running a small model to overturn the judgments of a much larger one. These findings show that the behavior of interacting models cannot be inferred from their individual properties but must be evaluated in the combinations in which they will operate.
Figures & tables
Model
Sarcasm
Commonsense
Sentiment
+
−
+/−
+
−
+/−
+
−
+/−
gemma-27b
.916
.063
.021
.479
.495
.026
.462
.524
.014
gemma-4b
.959
.033
.008
.367
.580
.053
.424
.563
.013
llama-70b
.733
.192
.075
.430
.536
.034
.510
.475
.015
llama-8b
.855
.003
.142
.433
.132
.435
.482
.429
.089
qwen-72b
.821
.128
.051
.389
.563
.048
.456
.527
.017
Table 1. Summary of classification certainty in round 1 . Fraction of samples that, over M repetitions, are assigned always a positive label ( + ), always a negative label ( − ), or a mixed set of labels ( +/− ). Values are presented across the seven models and three datasets.
Figure 1. Influence scores Δα∣β across model pairs and datasets. For the positive influence scores (A), column-wise marginal means show how susceptible a judge is to influence from different peers, while row-wise marginal means represent the persuasiveness of a peer across different judges. For the backfiring scores (B), columns show how strongly a judge backfires in reaction to peer influence, and rows show how strongly a peer induces backfiring in judges, when a backfire event occurs. In both panels, the anti-diagonal reports the outcomes of homogeneous systems, and the off-diagonal entries those of heterogeneous systems. The largest absolute marginals in each row and column are in bold. The margin of error at 95% confidence is ≤0.01 for all combinations of judge, peer, and dataset.
Figure 2. Role of model heterogeneity in influence. Model pairs plotted as a function of their positive influence score (x-axis) and backfiring score (y-axis), colored by judge agent. Pairs with judges impervious to influence lie closer to the origin. Stars and circles denote homogeneous and heterogeneous systems, respectively. The distance between stars and circles of the same color indicates behavioral divergence between the two settings.
Figure 3. Influence scores Δα∣β across model pairs from the same family, with varying parameter size. Positive influence scores are presented in panel A, backfiring scores in panel B. The anti-diagonal reports the outcomes of homogeneous systems. The largest absolute marginals in each row and column are in bold. The margin of error at 95% confidence is ≤0.01 for all combinations of judge, peer, and dataset.
Experiment
Judge ← Peer
Δα∣β
Δα∣β−
Main
Sensitivity
Diff.
Main
Sensitivity
Diff.
Temperature
llama-70b ← qwen-72b-high
0.53
0.53
0.00
-0.27
-0.27
0.00
llama-70b ← qwen-72b-low
0.53
0.52
-0.01
-0.27
-0.29
-0.02
qwen-72b ← llama-70b-high
0.13
0.12
-0.01
-0.58
-0.57
+0.01
qwen-72b ← llama-70b-low
0.13
0.13
0.00
-0.58
-0.57
+0.01
Memoryless
llama-70b ← qwen-72b
0.53
0.53
0.00
-0.27
-0.23
+0.04
Table 2. Sensitivity analysis summary. Influence scores Δα∣β and backfiring scores Δα∣β− , comparing results across all sensitivity analyses to the corresponding main experiment. Diff. = main - sensitivity.
Figure 4. Overview of the framework . (A) In round 1, the judge and peer independently generate M outputs for the same sample i . The judge’s probabilities of generating each output label is calculated. (B) Outputs from round 1 are upsampled so that both judge and peer have M explanations for each distinct label they generated in round 1. (C) M instances of contrasting labels-explanations pairs from the judge and peer are given in input to the judge in round 2, resulting in M new labels, out of which a new label probability is calculated. The comparison between the label probability in round 1 and that in round 2 yields the Δ value. (D) The sum of Δ values for both directions of influence ( 0→1 and 1→0 ) is the final influence score.
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Model name
Provider
Repository ID
Sampling params.
gemma-3-4b
Google
unsloth/gemma-3-4b-it-unsloth-bnb-4bit
×
gemma-3-27b
Google
unsloth/gemma-3-27b-it-bnb-4bit
×
llama-3.1-8b
Meta
unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit
✓
llama-3.3-70b
Meta
unsloth/Llama-3.3-70B-Instruct-bnb-4bit
✓
qwen-2.5-7b
Alibaba Cloud
unsloth/Qwen2.5-7B-Instruct-bnb-4bit
✓
qwen-2.5-72b
Alibaba Cloud
unsloth/Qwen2.5-72B-Instruct-bnb-4bit
✓
Appendix
Table S1. Specification of models . “Sampling params” refer to whether a generation configuration-file specifying default sampling parameters is provided in the repository.
Figure S1. System prompt for all model calls.
Figure S2. Round 1 and round 2 prompts. Slightly different prompts are used for Sentiment (A) Commonsense (B), and Sarcasm (C). In Round 2, colors represent the label and explanation pairs coming from the judge’s (blue) and peer’s (red) outputs produced in round 1.
Model
Sarcasm N=137,454
Commonsense N=9,264
Sentiment N=91,964
gemma-27b
4
1
11
gemma-4b
0
8
43
llama-70b
1
12
59
llama-8b
7
44
5
qwen-72b
27
1
0
qwen-7b
21
31
18
Appendix
Table S2. Data cleaning. Number of instances discarded from baseline outputs per model and dataset. N denotes the total dataset size.
Figure S3. Conditional distribution of post-interaction certainty ( pα∣β ) given individual model certainty pα for each receiver-peer pair on the Sentiment dataset . Plots on the diagonal correspond to homogeneous systems. The leftmost column in diagonal plots is empty since samples with pα=0 in homogeneous systems cannot generate any disagreeing pairs for round 2.
Figure S4. Conditional distribution of post-interaction certainty ( pα∣β ) given individual model certainty pα for each receiver-peer pair on the Commonsense dataset . Plots on the diagonal correspond to homogeneous systems. The leftmost column in diagonal plots is empty since samples with pα=0 in homogeneous systems cannot generate any disagreeing pairs for round 2.
Figure S5. Conditional distribution of post-interaction certainty ( pα∣β ) given individual model certainty pα for each receiver-peer pair on the Sarcasm dataset . Plots on the diagonal correspond to homogeneous systems. The leftmost column in diagonal plots is empty since samples with pα=0 in homogeneous systems cannot generate any disagreeing pairs for round 2.
Figure S6. Backfiring scores for agreeing interactions across model pairs on the Sarcasm dataset . Columns shows how susceptible a receiver is to influence from the different peers, while the rows show how much influence a peer can exert on different receivers.
Figure S7. Probability of backfiring across model pairs and datasets.
Figure S8. Probability of backfiring across model pairs from the same family, with varying parameter size.
Receiver
Sarcasm
Commonsense
Sentiment
Hom. Δα∣β
Het. Δα∣β (avg.)
Diff.
Hom. Δα∣β
Het. Δα∣β (avg.)
Diff.
Hom. Δα∣β
Het. Δα∣β (avg.)
Diff.
gemma-27b
.97
.86
-.11
.76
.78
.03
.96
.87
-.09
gpt-20b
.67
.14
-.79
.57
.42
-.26
.79
.65
-.17
llama-70b
.51
.58
-.14
.54
.69
.27
.56
.71
.26
qwen-72b
.43
.39
-.09
.45
.42
-.07
.54
.46
-.15
Appendix
Table S3. Comparison of susceptibility to influence in homogeneous (Hom.) and heterogeneous (Het.) systems . For each receiver, the table reports the homogeneous positive influence score, the average positive influence score across heterogeneous interactions, and the percentage difference between the two. Positive differences indicates that the model becomes more susceptible to influence when transitioning from a homogeneous system to a heterogeneous system, while negative differences indicate that the model becomes less susceptible.
Figure S9. Role of model heterogeneity in influence . Model pairs plotted as a function of their positive influence score (x-axis) and backfiring score (y-axis), colored by peer agent. Stars and circles denote homogeneous and heterogeneous systems, respectively. Non-influential peers lie close to the origin. Those in the upper-right region are able to highly influence judges, whereas those in the lower-left region cause strong backfiring effects.
Figure S10. Prompts for sensitivity analysis. Prompt variations implementing a memoryless judge (A), no explanations given (B), and swap between judge’s and peer’s label-explanation pairs (C).
Figure S11. Effect of peer interaction on classification performance. Differences between F1 classification score in round 2 with the respective F1 score obtained by the Judge in isolation in round 1. Positive values indicate a performance improvement after judge-peer interaction.
Figure S12. Effect of peer interaction on classification performance for models of different size from the same family. Differences between F1 classification score in round 2 with the respective F1 score obtained by the Judge in isolation in round 1, across model pairs from the same family. Positive values indicate a performance improvement after judge-peer interaction.
Judge ← Peer
Match Type
Sarcasm
Commonsense
Sentiment
Δα∣β
Δα∣β−
Count
Δα∣β
Δα∣β−
Count
Δα∣β
Δα∣β−
Count
gemma-27b ← gemma-27b
0:0
–
0.00
1,000
–
-0.00
1,000
–
0.00
1,000
1:1
–
0.00
1,000
–
-0.02
1,000
–
0.00
1,000
B:B
0.97
-0.01
5,644
0.76
-0.13
482
0.96
-0.02
2,668
gemma-4b ← gemma-27b
0:0
–
-0.01
1,000
–
-0.00
1,000
–
–
–
0:1
0.99
–
1,993
0.15
–
1,587
0.65
–
5,200
Appendix
Table S4. Detailed influence scores ( Δα∣β ) and backfiring scores ( Δα∣β− ). The scores are reported together with the corresponding number of instances from which they were computed. ‘–’ indicates match types for which the score cannot be computed, as neither no positive nor negative shifts can occur given the label certainty of the judge. Recall that 0:0 and 1:1 are agreeing configurations, and the remaining match types describe disagreeing configurations.
Multi-agent LLM systems are increasingly used in practice, but their behavior is complex and hard to understand. We design a simple task to ask which model has the most influence on the final consensus when several models are mixed in one system. Experimentally, across seven model families, we find that the opinions of the other models are absorbed into the opinion of the stronger model, the one with more parameters. This happens even though the task requires no pretraining knowledge and has no correct answer. To understand this phenomenon, we interpret the agents as weighted particles moving on a simplex. Larger models carry a larger weight and therefore occupy a deeper well in the energy landscape. A large weight means strong in-context learning: a model with a large weight amplifies the majority of its context. Larger models are generally better at in-context learning, and this is what makes the larger model the influencer. We hope that this work helps to interpret multi-agent systems, whose behavior is becoming too complex for humans to understand.
Large language models increasingly take part in extended conversations where they can both shape and be shaped by another party's views, yet most persuasion research measures only one side of that exchange. We introduce a controlled LLM-LLM debate framework in which two language-model roles repeatedly discuss a contested topic (climate-change attribution): one is given an assigned stance and, in some conditions, an explicit instruction to persuade, while the other receives no assigned position. Across four ordered model combinations involving GPT-6 Luna and Gemini 2.5 Flash Lite, we track both each role's explicitly reported position and a separate measure of its turn-by-turn behavior inside the conversation. Opinion dynamics differ substantially across model combinations: same-model interactions narrow the opinion gap on average, clearly for only one of the two models, while cross-model interactions widen it. Explicit reports and in-dialogue behavior capture distinct aspects of opinion dynamics, and a blanket instruction to persuade does not reliably increase influence. These findings suggest that conversational influence depends not only on assigned roles and initial positions but also on the participating models, with implications for how opinion change is measured in extended LLM interactions.
Mohammad Siahkamari, Phuoc Dat Nguyen, Stephen MacNeil
Multi-agent LLM systems increasingly debate answers, coordinate research, simulate users, and mediate information flows, making agent-to-agent persuasion a basic but undermeasured capability. We introduce a controlled testbed for studying how goal-directed persuaders shift elicited stances in networks of LLM agents grounded in real-world ego-network topologies. Across four LLM backbones, five graphs, and 55 policy statements, we find that persuasion dynamics depend on the interaction between topology, competition, topic, and model prior. Additionally, we show that direct exposure reliably predicts next-round stance change in competing runs, and peer relays carry smaller but measurable influence, showing that agents not assigned to persuade can still transmit persuasive force. Finally, analyzing post text alone misses important movement: planned strategies are only partly realized in executed messages, action choices can diverge from message content, and persuadees rarely state the stance shifts detected by probes. These results argue for evaluating multi-agent persuasion as a trajectory- and exposure-level process, using belief probes, exposure provenance, and action logs to identify who influenced whom and whether visible language reflects underlying stance movement.