In LLM-agent opinion simulations, each agent reports the probability that a claim is true after seeing its own previous probability and the probabilities reported by its neighbors, and that number becomes its state in the next round. If the number describes the agent's position rather than the sentence it was shown, then rewriting the claim as its complement and replacing every probability p with 1 - p should return the complementary answer, so the two reports should sum to one. We test this with three small open models: Qwen3-4B, Llama-3.2-3B and Gemma-3-4B. On 2,304 agent states taken from a running network simulation, Qwen's paired reports sum to 1.18 on average, and on 48 real fact-checking claims from AVeriTeC they again exceed one. A controlled benchmark of 24 worlds, each with exactly two possible outcomes, shows the same failure in all three models. The discrepancy is two to ten times larger than the change produced by a same-meaning paraphrase, and it persists under repeated sampling and across peer group sizes and report strengths. When the reports are fed back into two populations that start from complementary states, the populations do not remain mirror images: their divergence is positive after five rounds for every model. Averaging each answer with the complement of its paired answer restores consistency by construction and lowers Brier loss averaged over the two wordings, but at the final round it beats the better of the two original wordings for only one of the three models. A schedule-by-targeting experiment on the same model gives a small, stable interaction because such sweeps keep the claim wording fixed. Consistency under rewording therefore needs its own test before simulated probabilities are read as beliefs
Figures & tables
Figure 1: Numerical agent experiments. (a) Agents exchange scalar probability reports on a fixed graph; values and network sizes are schematic. (b) Own state, ordered peer reports, and available evidence enter a fixed checkpoint, which returns one number for the next state. (c) The network study compares three schedules with one-time evidence delivery with one-time evidence delivery. Sequential agents can read already-updated neighbors; displayed identities illustrate order. (d) Targeting contrasts, factual loss, and dispersion are separate outcomes. Complement discrepancy comes from separate paired-request studies: the schedule sweep holds the claim wording fixed. The token example illustrates the controlled benchmark, not the campus claims.
Model
Quantization
Instruction in
Qwen3-4B-Inst.-2507
Q4_K_M
System
Llama-3.2-3B-Inst.
Q4_K_M
System
Gemma-3-4B-IT
QAT Q4_0
User message
Table 1: Tested models. They differ in size, quantization and prompt format, so differences between them are not attributable to architecture alone. The network, preserved-state and real-claim studies use Qwen at temperature 0.2; the controlled studies use greedy decoding except repeated sampling.
3 models; same worlds and factors; only the new second orientation is generated
1,728
Repeated sampling
3 models; 6 worlds; 80 draws per request and orientation; both evidence conditions
5,760
Table 2: Design of all studies. The first three rows are the Qwen studies and count agent updates or cases; the remaining rows are the controlled studies and count new model calls, 33,288 in total. Counts are not independent sample sizes. The first two controlled studies use all 24 worlds; the others use six.
Schedule
Δσ
Simultaneous 95% interval
Synchronous
0.0063
[−0.0095,0.0220]
Fixed sequential
0.0109
[−0.0048,0.0267]
Randomized seq.
0.0048
[−0.0109,0.0206]
Table 3: Degree-versus-random targeting contrast in final Brier loss, with crossed scenario-pair/world-seed bootstrap bands. Positive values favor degree targeting. The primary schedule interaction is reported separately in the text.
Figure 2: Population Brier loss over six rounds for the three evidence policies under each update schedule, averaged over 16 scenarios and 12 world seeds. The dotted line marks the one-time source delivery at round two; later requests carry probabilities but not the source text.
Probe / estimand
Estimate
Interval
Level
Preserved states: signed A0
0.1846
[0.1553,0.2152]
97.5%
Preserved states: Brier gain Rsrc
0.0233
[−0.0450,0.0763]
97.5%
Preserved states: absolute discrepancy
0.1899
[0.1650,0.2156]
95%
Real claims: signed discrepancy
0.1867
[0.1281,0.2449]
97.5%
Real claims: protection P
0.0250
[−0.0501,0.0994]
97.5%
Real claims: absolute discrepancy
0.3259
[0.2929,0.3622]
95%
Table 4: Paired-probe results on simulation states and real-claim requests. Preserved-state diagnostics average three recipients within each matched block. Real-claim outcomes average peer orders before nonlinear calculations, then weight claims equally. The primary 97.5% intervals adjust for two endpoints per study; the two absolute-discrepancy intervals are secondary and pointwise. These are separate estimands, not comparable standardized effect sizes.
Secondary contrast
Estimate
Pointwise 95% interval
Preserved states: source contrast in C
0.1390
[0.1240,0.1559]
Preserved states: source contrast in Pr(C>0,DB>0)
0.2947
[0.2639,0.3236]
Real claims: S+→−−S−→+
-0.1555
[−0.1940,−0.1162]
Real claims: public–private C
0.0116
[0.0075,0.0156]
Real claims: public–private DB
-0.0027
[−0.0073,0.0020]
Table 5: Attraction, factual loss, and persuasion are measured separately. Source contrasts are absent minus restored; audience contrasts are public minus private. The joint-event proportion is on a 0–1 scale. Preserved-state intervals use crossed scenario-pair/world-seed resampling; real-claim intervals resample claims within label.
Figure 3: Mean absolute complement discrepancy (filled circles) and same-polarity paraphrase sensitivity (open squares) on the 24-world benchmark, by model and evidence condition. Differences are computed within pairs before worlds and families are weighted equally. Complement discrepancy is larger in all six cases.
Figure 4: Divergence d(t) between paired populations that start from complementary states, over five synchronous rounds on six worlds. Thick lines average the two initial placements, which are shown as thin solid and dashed lines. With visible evidence, Qwen’s divergence stays roughly flat after the first round, while Llama’s and Gemma’s grow.
Figure 5: Symmetrization. Left: the same state is described in both orientations and the model answers each request in a separate call, giving q0 and q1 . Middle: q1 is mapped back to the original orientation as 1−q1 and averaged with q0 to give R . The repaired pair (R,1−R) is complementary by construction, and for this fixed pair the average Brier loss falls by δ2/4 . Right: repaired reports become the next round’s states and are compared with the unrepaired populations on factual loss and call cost. The one-step identity does not determine the outcome of this closed-loop comparison.
Figure 6: Visible-evidence Brier loss after closed-loop feedback on six worlds. Filled circles: loss averaged over the two orientations. Open triangles: the baseline’s original (left) and complementary (right) orientations separately. Symmetrization lowers the averaged loss for all three models but is not always better than the better baseline orientation.
Family
Original ( o=0 )
Complement ( o=1 )
Binary switch
on
off
Exclusive color
red
blue
Integer parity
even
odd
Threshold
below
at or above
Set membership
in set
in its complement
Event order
A before B
B before A
Table 6: The six structural families, each with four worlds. Every world defines two exhaustive, mutually exclusive outcomes at a fixed referent and time.
Figure 8: Mean absolute complement discrepancy by structural family (four worlds each) with affirmative descriptions, for each model and evidence condition. Bars at zero are measured values.
Figure 9: Change in absolute discrepancy for each world when the complementary description is an explicit negation instead of an affirmative alternative. Negative values mean the negation reduced the discrepancy. Sw: switch, Co: color, Set: membership, Par: parity, Ord: event order, Thr: threshold.
Figure 10: Discrepancy of the mean reports (circles) and mean discrepancy of independently paired draws (diamonds), from 80 draws per request on the six-world subset. Where only a diamond is visible, the two coincide.
Figure 11: Change relative to the matched baseline for each prompt control. Left: absolute complement discrepancy (open squares, evidence absent; filled circles, evidence visible). Right: visible Brier loss averaged over orientations. Evidence prose and reordering apply only with visible evidence.
Figure 12: Mean absolute complement discrepancy by peer count and symmetric report pair. Each cell averages six worlds, two priors and three opposition levels; the outlined cell is the four-peer, 0.2/0.8 setting used in the main tests.
Model
Arm
L0(5)
L1(5)
Final
Rounds
Qwen
Baseline
0.083
0.250
0.167
0.168
Reminder
0.000
0.229
0.115
0.138
Greedy average
0.083
0.250
0.167
0.168
Symmetrization
0.147
0.147
0.147
0.098
Llama
Baseline
0.071
0.345
0.208
0.196
Reminder
0.074
0.209
0.142
0.191
Table 7: Visible-evidence Brier loss after five rounds of feedback, averaged over six worlds. L0(5) and L1(5) are final losses in each orientation, Final is their mean, and Rounds averages that mean over rounds 1–5. Lower is better.
Label
Quantity
Estimate
95% interval
Supported
Signed δ
0.3488
[0.3016,0.3948]
Absolute ∣δ∣
0.3647
[0.3276,0.4042]
Protection P
0.0100
[−0.0822,0.1039]
Refuted
Signed δ
0.0246
[−0.0673,0.1163]
Absolute ∣δ∣
0.2872
[0.2307,0.3469]
Protection P
0.0400
[−0.0572,0.1323]
Table 8: Real-claim results by dataset label (24 claims each), with pointwise 95% intervals from a claim bootstrap within label.
Figure 13: Truth-directed probability on the 48 real claims as the share of peers opposing the initial position increases, averaged over claims and both wordings. Rows: initial position aligned with or opposed to the dataset label. Columns: audience. Error bars are 95% claim-bootstrap intervals; the dotted line marks probability 0.5.
LLMs are increasingly deployed as proxies for human study participants in social science experiments, yet the fidelity of this practice has rarely been tested directly. We test whether six LLMs can simulate individual human belief updates, comparing LLM outputs 1-to-1 against ground truth data from 391 UK participants on Prolific, who updated their stances on three discussion topics after reading Reddit comments. Each participant was simulated by an LLM conditioned on a persona derived from their demographic and personality trait data. We find that some LLMs (Qwen3-32B and GPT-5-Mini) can match the human post-stance distribution, but only when given participants' actual initial stances. All six models fail to simulate initial stances themselves and to produce faithful belief updates from self-generated stances. Three systematic biases emerge across all models: overrepresentation of neutral positions, more frequent but smaller belief shifts than humans, and a failure to rank comments by convincingness. Demographic and personality trait personas had no consistent effect on fidelity. LLM simulations of human belief dynamics are only reliable when grounded in realistic starting conditions, that current multi-round social media simulations rarely provide.
Sebastian Pohl, Harsh Mehta, Pranav Mambayil +4
IT:U, Interdisciplinary Transformation University Austria, Altenberger Straße 66c, 4040 Linz, Austria
Multi-agent debate is increasingly used to reach consensus among LLM agents, yet agents often yield to a unanimous majority. When an agent changes its answer, has it changed its mind or only its statement? We study this with two-hop factual questions whose intermediate entity (the bridge, e.g. the country in "the capital of the country where the Sagrada Familia is located") is never stated by anyone. Scripted peers, in the role of Asch's confederates, unanimously assert a wrong answer taken from another fact with a different bridge. At the moment the agent answers, we read the bridge from its residual stream with the Jacobian lens (J-lens) and, for comparison, the logit lens. In pre-registered tests on held-out facts with four open-weight models, agents of Qwen3.5-4B, Qwen3.6-27B and Gemma-4-E4B-it that gave in still represented their original bridge in the pre-registered layers below the output (hit@100 above a control entity: 0.85, 0.22 and 0.24), where the logit lens rarely ranked it among the top 100 tokens (0.00-0.06). These agents also represented the bridge behind the peers' answer, beyond a mention baseline. A pre-registered addendum hid the agent's earlier answer or removed it: agents that gave in still represented their original bridge in all four models (0.43, 0.29, 0.37 and 0.25 with the answer hidden), including Llama-3.1-8B-Instruct, which barely did so with its answer in view (0.03). The premise can thus be computed from the question alone while the agent states the majority's answer. Hiding the earlier answer also changed conformity: Qwen3.5-4B gave in on 89% of questions instead of 8%. In exploratory interventions, injecting the bridge's J-lens direction brought agents back to their original answer only in the two Qwen models. Stated consensus in multi-agent debate can thus overstate agreement. We also report the negative results of our pre-registered program.
LLM agents in social simulation revise their opinions implicitly, in context: how open an agent is to persuasion can neither be specified nor verified, and collective outcomes inherit the model's training prior. We introduce Bayesian Chronicle Agents (BCA), a minimal belief layer separating what an agent believes from how it speaks. Each stance is a probability, updated by one Bayesian step per utterance heard. A single prior-strength parameter κ encodes stubbornness, modeled after its role in Friedkin--Johnsen (FJ) opinion dynamics. We then sweep this parameter to yield three canonical regimes of opinion dynamics on demand (consensus, persistent disagreement, committed-minority influence), with persistent disagreement matching the FJ closed-form fixed points at R2=0.93--0.99. We further show that prescribed κ remains recoverable after the language round-trip, with perfect rank-order recovery across all four models. Explicit belief also makes simulation auditable: the layer surfaces systematic per-model stance biases that end-to-end simulation would silently absorb.
Hafsa Akbar, Daniel Platnick, Marjan Alirezaie +2
MIT Media Lab, Massachusetts Institute of Technology · Flybits Labs, Creative AI Hub · Toronto Metropolitan University