Organizations: School of Computer Science and Informatics, University of Liverpool, Liverpool, UK · Department of Computer Science, Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates
Language-model agents' beliefs can be manipulated through open channels such as social media, even when the information they receive is truthful. When agents fill gaps between fragments with self-induced relationships, they may accept those relationships as established facts and pass the resulting rumours to others. Drawing on cognitive theory of narrative understanding and film theory of montage, we introduce Generative Montage to operationalize this failure, termed narrative overfitting, by coordinating agents to arrange truthful fragments into misleading narratives. Across 1,090 rumour targets from six Twitter events, agents often accepted rumours overconfidently, with overall acceptance exceeding 65 percent in twelve of fourteen models. Acceptance remained above 50 percent after majority voting or review of the same evidence, showing how errors can reach downstream decisions. To explain this susceptibility, fragment interventions identify posts that influence belief within complete narratives. Synthetic controls then show that similar response patterns arise from valid evidence, separating belief influence from evidential support. This distinction motivates Grounded Monitor, a proof-of-concept neuro-symbolic runtime monitor that checks direct evidential support to guide agents' judgements, improving decision accuracy by 5.6 to 29.0 percentage points across original and anonymized settings. These findings expose a cognitive route to rumour amplification and support monitoring inferred relationships before agent-generated conclusions guide public decisions.
Figures & tables
Figure 1: Narrative overfitting spans models, events and downstream decisions. a , Target-acceptance rates for individual models, ordered by their overall rate. Open circles show event-specific estimates, horizontal segments span their range, and diamonds indicate the reported overall rates. Blue and orange denote proprietary and open-weight models, respectively; each estimate for a model and event combines five runs per target. b , Target acceptance accompanied by self-reported confidence of at least 0.8 in the earlier open report; values are percentages. c , Downstream target acceptance rate after majority voting or adjudication by a language-model judge given the feed and five victim agents’ reports. Dashed lines mark 50%.
Writer narrative Draft excerpt
Editor feed Post sequence and implied connection
Victim agent response Verdict, confidence and review
Charlie Hebdo h∗ : Ahmed Merabet was the first victim of the Charlie Hebdo attack
Officer Merabet stationed outside… confrontation occurred… #JeSuisAhmed trended… death confirmed in reports…
7 posts Attack starts → Officer present → Hashtag emerges → Casualty Implied connection: Presence at the attack’s onset is linked to being the first victim.
TRUE “Timeline suggests early encounter…” Confidence: 0.92 MV: Y (5/5) Judge: Y
Sydney Siege h∗ : There are 13 hostages inside the Sydney cafe
6 posts Cafe surrounded → People visible → Count emerges → 13 confirmed Implied connection: A count of people is treated as a count of hostages.
TRUE “Multiple sources indicate 13 hostages…” Confidence: 0.88 MV: Y (3/5) Judge: Y
Ferguson h∗ : Ferguson police are leading a smear campaign or character assassination of M. Brown
Police released surveillance footage… Brown’s background discussed… media focus shifted to character… community alleged bias…
8 posts Shooting → Footage release → Character focus → Bias claims Implied connection: A shift towards character coverage is interpreted as coordinated intent.
TRUE “Pattern suggests coordinated character attack…” Confidence: 0.85 MV: Y (5/5) Judge: Y
Table 1: Illustrative examples of belief manipulation by Generative Montage Framework.
Figure 2: Fragment responses and full-feed belief across models. Orange denotes attack feeds ( n=223 , spanning 37 targets) and blue denotes ordinary-report feeds ( n=464 ). a , Response-dispersion distributions (upper curves) and belief-pull summaries (lower rows) for six models, with model-specific horizontal scales. b , Each point represents a feed, positioned by its full-feed belief margin and response dispersion after averaging within-model percentile ranks across the six models. Marginal curves show the distributions of these pooled scores.
Figure 3: Payload deletion lowers target belief more than random deletion in six models. Deletions use 223 attack feeds over 37 targets. Colour identifies the evaluated model: orange shades are Qwen2.5-Instruct and blue shades are DeepSeek-R1-Distill-Qwen, with darker shades for larger models. a , Change in full-feed belief margin against change in response dispersion after payload deletion, relative to the original feed. Each point is one attack feed, each small panel is one model with its own axes, and r is the Pearson correlation within that model. b , Mean changes in nats with 95% claim-clustered bootstrap intervals, one point per model. Deletion contrasts compare payload deletion with random deletion; transplantation compares 349 same-event ordinary-report feeds before and after payload insertion. Dispersion changes and correlations are descriptive summaries of fragment responses. c , Percentage of all 223 attack feeds whose TRUE verdict was flipped by payload deletion (model colour) and by random deletion (grey).
Figure 4: Response dispersion is not specific to misleading narratives. a , Construction-control AUROCs across eight models. Evidence selection compares scorer-selected with random fragments in short templates (Fragments) and narratives (Narratives). Attack search compares complete attacks with narratives from scorer-selected (Selected control) or random fragments (Random control). Full pipeline compares attacks with short-fragment controls. AUROC above 0.5 indicates higher dispersion in the first condition, and below 0.5 in the reference condition. b , Each model’s attack AUROC (horizontal) and curation-free AUROC (vertical) relative to supported narratives. Error bars are the separately estimated 95% world-clustered bootstrap intervals. Dashed lines mark 0.5; shaded regions indicate both estimates below or above 0.5, and the dotted diagonal indicates equal AUROCs. c , Coloured ring segments show the percentage of feeds above each model’s short-fragment-control dispersion threshold, set at its 90th percentile; grey shows the remainder. Random selection denotes curation-free narratives. Extended Data Tables Extended Data and Extended Data retain all values and intervals.
Target-status selectivity within montage feeds
Diagnostic signal
Non-rumour control montage ( n=80 )
Attack montage ( n=158 )
AUROC
Mean salience
0.889
0.900
0.502
Mean grounding posts per feed
0.487
0.070
0.664
Feeds with grounding (%)
38.8
6.3
0.662
Detecting attack montage against same-event reporting
Outcome
Attack montage ( n=158 )
Chronological control ( n=158 )
Performance
Table 2: Target promotion and direct support address different diagnostic comparisons. Results are shown for ORIG. The upper block compares non-rumour control and attack montages generated by the same pipeline, with non-rumour control montages treated as the positive class for AUROC. The lower block compares attack montages with chronological control feeds from the same events (Methods); its AUROC uses salience, and the alarm rates correspond to recall and false-positive rate within this comparison.
Figure 5: GM improves acceptance decisions across models and entity settings. a , ORIG. b , ANON. Grey bars show baseline judgements and blue bars show judgements under the GM response condition. Decision accuracy counts acceptance of a non-rumour control and non-acceptance of a rumour target as correct, pooling 158 rumour targets and 80 non-rumour controls with equal weight per target. Each target was judged in five runs. Both FALSE and UNCERTAIN count as non-acceptance. The GM response condition adds an uncertain option and provides a fixed warning when the monitor raises an alarm.
Evaluated model
Charlie Hebdo
Sydney Siege
Ferguson
Ottawa Shoot.
Germanwings
Putin Missing
Overall ASR
A
C
H
A
C
H
A
C
H
A
C
H
A
C
H
A
C
H
Proprietary models
GPT-4o-mini
81.7
0.83
67.4
92.1
0.82
70.3
79.5
0.81
59.0
86.7
0.85
78.2
74.5
0.85
60.0
66.7
0.82
53.3
83.1
GPT-4o
79.4
0.85
66.9
88.5
0.83
69.1
67.0
0.83
53.0
85.5
0.85
74.5
56.4
0.89
52.7
63.3
0.75
26.7
77.4
GPT-4.1-nano
81.7
0.81
66.9
94.5
0.80
67.1
79.1
0.79
53.1
94.5
0.81
74.8
74.5
0.83
61.8
70.0
0.75
13.3
85.5
GPT-4.1-mini
79.9
0.85
68.4
80.6
0.82
50.9
68.0
0.81
42.5
77.6
0.87
67.9
63.6
0.91
63.6
20.0
0.81
16.7
72.7
Extended Data Table 1: Target acceptance and reported confidence across models and events. Results cover six events. A denotes second-stage rumour-target acceptance (ASR, %), and C denotes mean self-reported confidence in the first-stage open report on a 0–1 scale. H denotes target acceptance accompanied by first-stage confidence of at least 0.8 (HC-ASR, %). The final column gives overall ASR pooled over evaluated runs for each model; group means average these model-level rates. Background colours distinguish the evaluated model families.
Configuration
ASR (%)
HC-ASR (%)
Change in ASR (pp)
Complete construction procedure
77.0
64.9
—
Without Director-guided revision
63.5
48.0
−13.5
Without Editor sequencing
69.7
52.5
−7.3
Single construction agent
26.8
16.6
−50.2
Extended Data Table 2: Construction ablations on Charlie Hebdo. ASR measures rumour-target acceptance, and HC-ASR additionally requires confidence of at least 0.8 in the initial open report. Changes are percentage-point differences from the complete construction procedure. Removing a role also changes prompts or computation, so each comparison evaluates the resulting procedure.
Evaluated model
Prompting
ASR (%)
Qwen2.5-7B
Direct
67.8
Chain of thought
70.9 ( +3.1 )
DeepSeek-R1-7B
Direct
74.9
Chain of thought
81.7 ( +6.8 )
Extended Data Table 3: Chain-of-thought prompting in two Charlie Hebdo comparisons. Parentheses give percentage-point changes in rumour-target acceptance relative to direct prompting. Both models were evaluated on Charlie Hebdo under the indicated prompting conditions.
Extended Data Fig. 1: Association between evidence quantity and susceptibility. Rumour-target acceptance and mean confidence in the initial open report for GPT-4.1-mini on Charlie Hebdo, grouped by the number of truthful public posts. The comparison describes how acceptance varies across feed-length groups, which also differ in content.
Extended Data Fig. 2: Rumour-target acceptance after report aggregation. Downstream acceptance rates under majority voting and language-model adjudication are shown by event and evaluated model family. The five analysts were instantiated separately but interpreted the same feed.
Dispersion by refutation class
Evaluated model
Full-feed belief
Dispersion
Explicit contradiction
Later correction
Inference alone
Qwen2.5-3B
0.76 [0.71, 0.81]
0.65 [0.58, 0.72]
0.72 [0.63, 0.80]
0.64 [0.53, 0.74]
0.56 [0.46, 0.67]
Qwen2.5-7B
0.59 [0.53, 0.65]
0.71 [0.65, 0.76]
0.76 [0.70, 0.83]
0.69 [0.61, 0.76]
0.63 [0.48, 0.78]
Qwen2.5-14B
0.51 [0.43, 0.58]
0.56 [0.48, 0.64]
0.63 [0.51, 0.71]
0.53 [0.39, 0.65]
0.50 [0.29, 0.67]
DeepSeek-R1-1.5B
0.39 [0.30, 0.47]
0.75 [0.69, 0.82]
0.78 [0.67, 0.89]
0.71 [0.62, 0.81]
0.79 [0.71, 0.88]
DeepSeek-R1-7B
0.48 [0.40, 0.56]
0.61 [0.54, 0.68]
0.69 [0.62, 0.77]
0.60 [0.50, 0.71]
0.45 [0.33, 0.57]
Extended Data Table 4: Per-model separation of attack feeds from ordinary-report feeds. AUROC and 95% bootstrap intervals (3,000 resamples) compare 223 attack feeds over 37 targets with 464 same-event ordinary-report feeds. Full-feed belief margin is m(F) , and response dispersion is u(F) . Attack feeds are further grouped by how the target is refuted in the source record: explicit contradiction ( n=85 ), later correction ( n=100 ) or inference alone ( n=38 ). Pooled estimates average within-model percentile ranks for each feed before computing AUROC. Attack feeds were selected for success on a reference model’s frozen readout.
Belief
Evaluated model
Δm , deletion
Flips, payload / random (%)
Baseline TRUE (%)
Δm , transplant
Qwen2.5-3B
−2.66 [ −3.25 , −2.11 ]
19.7 / 5.8
44.4
+0.07 [ −0.13 , +0.26 ]
Qwen2.5-7B
−4.05 [ −4.96 , −3.26 ]
20.2 / 4.9
70.0
+0.23 [ +0.01 , +0.43 ]
Qwen2.5-14B
−7.71 [ −9.77 , −5.98 ]
17.0 / 5.8
54.7
+0.39 [ −0.14 , +0.93 ]
DeepSeek-R1-1.5B
−0.08 [ −0.09 , −0.06 ]
15.7 / 8.5
32.7
−0.01 [ −0.02 , +0.01 ]
DeepSeek-R1-7B
−0.69 [ −0.83 , −0.56 ]
7.2 / 1.3
91.5
−0.16 [ −0.21 , −0.11 ]
Extended Data Table 5: Payload interventions in the six-model panel. Each model identified its own payload from its panel readout and then read every feed variant, the original feed included, under seven templates. Deletion contrasts follow equation ( 10 ) over 223 attack feeds; transplantation contrasts compare 349 ordinary-report feeds before and after payload insertion. Verdict flips change m>0 in the original feed to m≤0 after deletion, with all 223 attack feeds as the denominator. Baseline TRUE is the percentage of original attack feeds with m(F)>0 ; the feeds were selected for success on a reference model outside the panel. Δm and verdict flips are the preregistered causal estimands. The dispersion contrasts and the correlation of m(F−i∗)−m(F) with u(F−i∗)−u(F) are descriptive summaries of changes in fragment responses. Intervals are 95% claim-clustered bootstrap intervals from 3,000 resamples over 37 attack-target or 251 ordinary-report-target clusters. The cross-model mean averages the six per-model estimates within each resample. Values are in nats except percentages and r .
Qwen2.5
DeepSeek-R1
Qwen3
Llama-3.1
Contrast
3B
7B
14B
1.5B
7B
14B
8B
8B
Cs vs C
0.58 [0.55, 0.60]
0.59 [0.56, 0.61]
0.61 [0.58, 0.64]
0.52 [0.51, 0.53]
0.52 [0.51, 0.54]
0.62 [0.59, 0.64]
0.55 [0.53, 0.58]
0.62 [0.59, 0.65]
Ns vs N
0.57 [0.51, 0.64]
0.56 [0.49, 0.63]
0.55 [0.48, 0.62]
0.44 [0.40, 0.49]
0.54 [0.47, 0.60]
0.56 [0.49, 0.63]
0.56 [0.49, 0.63]
0.55 [0.48, 0.62]
A vs Ns
0.44 [0.38, 0.51]
0.45 [0.38, 0.52]
0.46 [0.39, 0.54]
0.48 [0.43, 0.52]
0.48 [0.41, 0.55]
0.46 [0.39, 0.54]
0.44 [0.38, 0.52]
0.50 [0.43, 0.58]
A vs N
0.52 [0.45, 0.58]
0.51 [0.44, 0.58]
0.51 [0.43, 0.58]
0.42 [0.38, 0.47]
0.52 [0.45, 0.58]
0.52 [0.45, 0.59]
0.50 [0.42, 0.58]
0.55 [0.47, 0.61]
A vs C
0.98 [0.97, 0.99]
0.99 [0.98, 1.00]
1.00 [0.99, 1.00]
0.94 [0.92, 0.96]
0.97 [0.95, 0.98]
0.99 [0.98, 1.00]
1.00 [1.00, 1.00]
0.90 [0.86, 0.93]
Extended Data Table 6: Response dispersion across synthetic construction controls. The six-model response panel is supplemented by Qwen3-8B and Llama-3.1-8B for additional validation. C is the short-fragment control, with randomly selected relevant fragments rendered in a fixed short template. Cs uses the same template with fragments selected by a frozen target-aware scorer. N is the curation-free narrative, generated from randomly selected fragments by the full pipeline. Ns uses the same narrative pipeline with scorer-selected fragments. A is the complete attack, which searches the full evidence pool and generates a narrative. Short fragments average 46 characters, whereas narrated conditions contain a median of seven fragments of about 250 characters. The Cs versus C and Ns versus N comparisons test scorer selection; A versus Ns and A versus N compare attack search with scorer-selected and randomly selected narratives. The A versus C comparison evaluates the complete pipeline, including changes in selection, presentation and length. Values are AUROC(u) in Cohort A with 95% world-clustered bootstrap intervals (2,000 resamples), treating the first condition as positive; values below 0.5 indicate higher dispersion in the second condition.
Acceptance (%)
Above threshold (%)
Cohort
Evaluated model
Reads
Direct
Multi-step
Supported
Curation-free
Attack
A
Qwen3-8B
144
97.5
100.0
100
97
100
A
Qwen2.5-7B
144
96.3
96.8
100
95
96
A
Llama-3.1-8B
144
96.3
100.0
81
71
73
Extended Data Table 7: Acceptance and dispersion in supported narratives. These feeds narrate evidence towards a supported target h′ , with E⊨h′ . Direct targets require one world edge, whereas multi-step targets require at least two. Acceptance is the percentage of individual reads for each world and prompt template with full-feed margin m(F)>0 . AUROCs in the lower block use the first named condition as positive and include 95% world-clustered bootstrap intervals; values below 0.5 indicate higher dispersion in the second condition. The upper block also reports the percentage of feeds above the 90th percentile of short-fragment-control dispersion, a cutoff with a nominal 10% flagging rate on that reference control. The first 48 world slots in Cohort A yielded 48 feeds; evaluating each feed with three templates produced 144 reads per model.
Measure
ORIG
ANON
Target-only probes (GPT-4.1-mini)
Event recognition (%)
44.5
5.5
Correct veracity guesses (%)
22.7
1.7
Director scores
Narrative score
7.87
8.23
Sequence score
8.67
8.66
Extended Data Table 8: Entity anonymization and narrative regeneration controls. ORIG retains real entities from the 2014–2015 CoPHEME events; ANON replaces them with fictional counterparts before regenerating the feeds. Target-only probes use GPT-4.1-mini without a feed, with correct veracity guesses evaluated only for targets with resolved source annotations. Director scores assess the generated narrative and the post sequence on a ten-point scale.
Entity-surface contrasts in TRUE probability
ΔsurfaceR
ΔsurfaceANON
Evaluated model
Mean
P
Holm P
Mean
P
LOEO range
Qwen2.5-7B
−0.002
0.99
1.00
−0.012
1.00
−0.019 to +0.022
Qwen3-8B
+0.003
0.30
0.91
−0.005
0.41
−0.008 to +0.019
Llama-3.1-8B
+0.013
0.63
1.00
+0.005
0.65
+0.006 to +0.018
Extended Data Table 9: Entity-surface controls: probability contrasts and verdict rates. The upper block reports mean differences in template-averaged TRUE probability over T=158 rumour targets. Surface contrasts ΔsurfaceR and ΔsurfaceANON follow equation ( 13 ); one-sided Wilcoxon signed-rank tests assess the preregistered increases, with Holm correction for the primary R contrast. The table also gives its leave-one-event-out (LOEO) range, generation-source contrasts under both entity surfaces, and target-only TRUE probabilities under general-knowledge instructions. Surface estimates were below the effect-size threshold of 0.05, and the tested increases were not significant. The lower block gives the percentage of individual reads labelled TRUE. R and ANON denote generation over real-entity and anonymized evidence, respectively. Target-only baselines use either general-knowledge instructions or the evidence-evaluation prompt with an empty evidence block.
Relation
Operational test
Illustrative post
Graph role
grounds
The post alone makes the target necessarily true, if accurate.
“Police confirmed that several attackers coordinated the operation.”
Observation supporting h∗
abductive
The post promotes the target, but establishing it requires an additional observation or premise.
“Gunfire was reported at the War Memorial and Parliament within minutes of one another.”
Retractable assumption
undermines
The post supplies evidence against the target.
“Police concluded that a single gunman acted alone.”
Counter-observation
irrelevant
The post does not bear on the target.
“Residents were asked to remain indoors.”
No target edge
Extended Data Table 10: Operational relation labels used by GM. The illustrative target is “the Ottawa incident involved a coordinated operation by multiple attackers”. The examples illustrate the classification rule rather than quoting experimental posts. The fixed GPT-4.1-mini classifier assigns these target-relative labels under the single-post entailment criterion. Preceding posts may resolve references but do not supply additional premises; the labels enter the symbolic support representation used by the alarm rule.
Diagnostic measure
ORIG
ANON
Mean salience, non-rumour
0.889
0.866
Mean salience, rumour
0.900
0.776
Salience AUROC
0.502
0.569
Feeds with grounding, non-rumour (%)
38.8
38.8
Feeds with grounding, rumour (%)
6.3
7.0
Grounding-presence AUROC
0.662
0.659
Extended Data Table 11: Monitoring diagnostics in ORIG and ANON. The same GPT-4.1-mini classifier evaluates 158 rumour and 80 non-rumour montages in each setting. ORIG retains real entities; ANON replaces them with fictional counterparts before regenerating the feeds. Grounding denotes the presence of at least one post labelled grounds . AUROCs treat non-rumour montages as positive and use salience or binary grounding presence.
Targets ( n )
Grounding (%)
Target type
Example
Non-rumour
Rumour
Non-rumour
Rumour
AUROC
Atomic
“11 people died”; “the suspect is X”
72
120
40.3
6.7
0.670
Inferential
“the plane descended rapidly”
8
38
25.0
5.3
0.599
Extended Data Table 12: Direct-support detection by target type in the PHEME-derived Twitter sample. Results are for ORIG, using the same classifier and montage construction as Table 2 . Atomic targets specify a fact, such as a count or identity, whereas inferential targets express an interpretation of the supplied fragments. Counts refer to targets, each represented by one feed. Grounding is the percentage of feeds with at least one post labelled grounds . AUROC measures grounding-based separation within each target type, treating non-rumour controls as positive. The inferential non-rumour group contains eight targets, so its estimates are descriptive. Examples illustrate the target types.
Measure
Writer and Director
Editor and Director
First approval round
3.03±0.17
3.00±0.23
Best approval round
3.46±0.65
3.71±0.88
Director apparent-support score
8.01±0.16
8.76±0.07
Narrative length (words)
218.51±36.71
—
Sequence length (posts)
—
6.94±0.41
Extended Data Table 13: Statistics for the two construction loops. Values are mean ± s.d. across Charlie Hebdo attack instances generated with GPT-4.1-mini. Apparent-support scores are assigned by the Director during construction.
Experiment and purpose
Materials
Models and readouts
Comparison
Broad attack evaluation. Establish susceptibility across models and events.
1,090 rumour targets from six events; five analyst runs per target.
Fourteen-model panel. ASR records second-stage target acceptance; HC-ASR additionally requires first-stage confidence of at least 0.8. Mean confidence summarizes the initial open reports.
Acceptance across models and events under the common attack-evaluation protocol.
Downstream review. Test whether another judgement stage retains the initial conclusions.
The same feed and five analyst reports for each reviewed target; event-level results summarized across four model families.
Majority vote over five target-adjudication verdicts; a separately instantiated DeepSeek-R1-14B judge. Readout: downstream target acceptance.
Majority voting and language-model adjudication on shared input, as shown in Fig. 1 c.
Natural-corpus responses. Locate differences between complete-feed and isolated-fragment responses.
Attack feeds versus ordinary-report feeds; pooled and per-model comparisons, with target-refutation strata in Extended Data Table Extended Data .
Payload interventions. Test whether a fragment selected through isolated responses changes the complete-feed judgement.
Deletion: 223 attack feeds over 37 targets. Transplantation: 349 ordinary-report feeds over 251 targets.
Six-model response panel; each model selects its own payload and evaluates all feed variants under seven templates. Readouts: changes in m(F) and verdict flips, with descriptive changes in u(F) and correlations.
Payload versus random deletion; ordinary-report feeds before versus after payload insertion.
Supplementary Table 1: Experiment map: behavioural breadth and fragment influence. Each row identifies the question, sampling units, evaluated models, response measures and comparison for one analysis. Model abbreviations follow Methods; panel membership is given below and in Methods.
Experiment and purpose
Materials
Models and readouts
Comparison
Synthetic narrative controls. Distinguish narrative influence from evidential support.
Cohort A comprises 134 world slots. Supported narratives use the first 48 slots, yielding 48 feeds and 144 template reads per model.
Six-model panel plus Qwen3-8B and Llama-3.1-8B for selection contrasts. Supported narratives use the two additional models and Qwen2.5-7B. Three templates with TRUE, FALSE and INSUFFICIENT probabilities give margin, dispersion, acceptance and AUROC.
Construction contrasts, supported-narrative comparisons and dispersion-threshold exceedance in Fig. 4 ; complete values in Extended Data Tables Extended Data and Extended Data .
Entity replacement and regeneration. Examine the joint change in entity familiarity and generated material.
The focused set of 158 rumour targets and 80 non-rumour controls in ORIG and ANON.
Four-model GM panel. Readouts: target acceptance for rumour targets and non-rumour controls. GPT-4.1-mini target-only probes measure event recognition and veracity guesses.
Feeds generated from original entities versus feeds regenerated after replacement with fictional entities.
Fixed-text entity controls. Separate entity-name effects from generation-source differences.
158 rumour targets; eight conditions and three templates for each of three models.
Qwen2.5-7B, Qwen3-8B and Llama-3.1-8B. Readouts: contrasts in template-averaged TRUE probability and the percentage of individual TRUE verdicts.
Generation source crossed with real or fictional names in fixed text; target-only baselines under knowledge and evidence-evaluation prompts.
Grounded Monitor. Monitor detected single-post support and assess changes in target judgement following its assessment.
158 rumour targets and 80 non-rumour controls; a separate detection comparison pairs attack montages with 158 chronological control feeds matched for event, post count and target.
GPT-4.1-mini relation classifier; four-model GM panel. Readouts: salience and detected grounding, alarm rates, AUROC and decision accuracy.
Attack versus non-rumour control montages; attack montages versus chronological control feeds; warning plus the uncertain option versus baseline judgements in ORIG and ANON.
Supplementary Table 1: Experiment map: narrative controls, entity controls and monitoring (continued). These analyses test the specificity of fragment responses, distinguish changes in generated content from changes in entity names, and evaluate both the support monitor and changes in acceptance when agents respond to its assessment.
Event
Context
Evidence
Rumour targets
Mean cascade size
Charlie Hebdo
breaking news
1,814
265
14.8
Sydney siege
hostage incident
1,081
140
16.4
Ferguson
civil unrest
869
274
21.8
Ottawa shooting
terrorist attack
749
141
11.7
Germanwings
aviation disaster
325
144
10.0
Putin missing
political uncertainty
112
126
2.9
Supplementary Table 2: Dataset statistics for CoPHEME. The six events originate from real Twitter conversations in PHEME [ 3 ] . Evidence counts the factual source items available for narrative construction, and rumour targets are the claims used in the broad attack evaluation. Mean cascade size is the average historical engagement size of the target rumours in the source data.
LLM systems increasingly rely on groups of agents that discuss, critique, and check one another, assuming that peers will catch individual errors. We test this assumption against a minimal adversary. In a ten-agent factual discussion, one or three adversarial agents repeatedly assert a plausible false answer. After each round we probe each benign agent's private answer through a separate call whose response is hidden from its peers, and we separately label its public stance toward the claim. Across six models and three factual-QA datasets, groups remain stable without an attacker: only 0.6% of benign agents finish with the false answer. A single adversarial agent increases this rate to 44.8% and three to 60.5%. Studies of conformity typically report public agreement without private change; we find the reverse. With three adversarial agents present, benign agents hold the false answer in 51.6% of exposed turns but publicly endorse it in only 19.7%. Much of the influence therefore remains invisible in the discussion, and the false answers persist when agents are asked for their own views. Agents that retain the true answer often fail to defend it; once adopted, the false answer spreads among benign agents, and baseline knowledge does not predict robustness.
Liangjie Huang, Henry Peng Zou, Wei-Chieh Huang +3
Can one sentence manipulate an agent's belief by targeting a single implicit warrant linking evidence to a conclusion? In this paper, we introduce the one-relation attack (ORA), a novel cognitive attack on this inferential relation. ORA localizes a critical implicit warrant and performs a one-relation intervention using a single source-grounded sentence that qualifies this warrant and steers the agent toward an incorrect belief. Across eighteen models on biomedical inference and vulnerability detection, best-of-$10 attacks overturn initially correct answers at mean rates of 54.9% for proprietary and 80.0% for open-weight models. Attacks also transfer without victim feedback, and paired controls support the roles of targeting and qualification. Moreover, successful sentences can pass source-support checks and evade identification, and multi-agent debate raises team error from 50.0% to 59.5% in our balanced stress test. These findings establish implicit warrants as a cognitive attack surface for manipulating agents' beliefs.
Jinwei Hu, Youcheng Sun, Yi Dong +1
School of Computer Science and Informatics, University of Liverpool · Department of Computer Science, MBZUAI
An LLM agent shown a professional-looking market panel commits to a directional call on a provably unpredictable question far more often than one asked the bare question: across 12 frontier models, commitment rises from 6.5% to 54.0% as evidence is escalated. It commits just as readily when every number on the panel is invented: fabricating the entire display, so nothing the model can see is true except the question itself, still lifts commitment from 24.5% to 36.8%, statistically indistinguishable from the 37.6% produced by genuine market data. What unlocks confident action is not information but the authority of its packaging. The failure is narrow and locatable. Incapacity is not the answer: on matched answerable questions attached to the same panels, the same models answer essentially always, at near-perfect accuracy. Nor is it belief - stated probabilities barely move across the gradient that swings action by 48 points, and score worse than a climatological baseline. Missing judgment isn't it either: asked to classify a question's knowability before acting, models call it irreducible 90% of the time and then commit on just 0.4% of those. The act/don't-act gate is what fails, and the effect is concentrated in a few models rather than universal. Because the gate is separable, it can be trained. Supervised fine-tuning of a 3B model on 540 synthetic cases, predominantly dice, coins, jars and timers, drives commitment to 0.0% on the original cases and transfers to three unseen domains. It does not survive everything: the gate holds exactly when the response format leaves room to reason, and rigid formats that remove that room leave the model confident and wrong on questions it otherwise answers correctly. The gate is trainable and context-fragile, and deployment needs both halves of that sentence.