Multi-agent LLM systems increasingly aggregate judgments across model instances, but reasoning about other agents can itself introduce failure. We study a setting in which LLMs shift their public decisions after observing ambiguous social signals while reporting inaccurate estimates of what other agents privately judged, even when the group's recorded private judgments do not support the apparent consensus. We call this failure phantom consensus and study it as a pluralistic-ignorance-style pattern in controlled LLM collectives. To measure this phenomenon, we introduce PHANTOM-Bench, a benchmark that separates isolated private judgments, public decisions after ambiguous social signals, elicited estimates of the group's private judgment, transparency-based recovery, and controls for neutral public text and explicit vote-following. Across nine closed- and open-weight models, ambiguous public signals consistently degrade decision quality and can turn correct private group judgments into incorrect public collective outcomes. Public accuracy drops by as much as 47.8 percentage points, and collective rationality failures reach 47.3% in the most susceptible setting. We observe substantial errors in reported group-private beliefs alongside public-stage failures, while control conditions reveal model-dependent differences in susceptibility to ambiguous and explicit peer signals. Revealing the agents' recorded private decisions repairs many failures without revealing the gold label, although recovery varies across models. Our results show that evaluating multi-agent LLM systems requires measuring not only final agreement, but whether that agreement reflects independently measured private judgments or an apparent consensus.
Figures & tables
Figure 1: Belief-shift mechanism in PHANTOM-Bench . Agents first make independent private decisions, then observe controlled ambiguous public signals and report both a public decision and a perceived group-private belief. The setup tests whether agents retain their private judgment or shift toward an inferred group belief unsupported by the group’s recorded private decisions.
Figure 2: Overview of the five belief-context conditions in PHANTOM-Bench . All agents receive the same scenario and decision request, but the group context varies across conditions: no group information, ambiguous public signals, recorded private decisions revealed under transparency, neutral public comments without an implied norm, or explicit peer recommendations. The setup separates private judgments, public decisions, perceived group-private beliefs, and recovery under transparency.
Figure 3: Private–Public Gap (PPG), Collective Rationality Failure Rate (CRFR), and Phantom-CRFR (P-CRFR) across evaluated models. Higher values indicate stronger public-stage degradation and stronger phantom-aligned collective failure.
Model
PBA ↓
PBE ↓
PIFR ↓
N-Acc ↑
N-Phan. ↓
EV-Acc ↑
EV-Phan. ↓
Closed-Weight Models
Gemini-3.5-Flash
0.658
0.771
0.112
0.752
0.092
0.655
0.171
Gemini-2.5-Pro
0.571
0.741
0.123
0.798
0.065
0.700
0.151
Gemini-2.5-Flash
0.512
0.666
0.088
0.749
0.093
0.663
0.146
Open-Weight Models
Qwen-2.5-32B
0.478
0.671
0.225
0.771
0.084
0.490
0.319
Table 2: Mechanism and control diagnostics for PHANTOM-Bench . PBA and PBE measure phantom-aligned and erroneous perceived group-private beliefs; PIFR measures agent-level phantom-induced failure. N-Acc/N-Phan. and EV-Acc/EV-Phan. report neutral-public and explicit-vote control results.
Figure 4: Directional composition of phantom-induced failures. Stacked segments show which gold-to-phantom target directions contribute to each model’s phantom-induced failure rate.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Field group
Use
Action fields
Gold decision and phantom target; used for correctness and target alignment.
Design taxonomy
Norm source, second-order belief-error target, and outcome-risk label; used for quota-controlled construction.
Computed behavior
PPG, PBA, PBE, PIFR, CRFR, P-CRFR, and TRR; computed from model outputs.
Appendix
Table 3: Taxonomy field groups in PHANTOM-Bench .
Category
Definition
Descriptive
Agent infers what others are likely to do.
Injunctive
Agent infers what others think should be done.
Informational
Agent infers others know better or interpreted the facts better.
Authority
Agent infers that a lead or senior reviewer prefers the target.
Consensus
Agent infers dissent would disrupt agreement or make it an outlier.
Appendix
Table 4: Perceived norm-source categories.
Category
Definition
None control
Signals should not create substantial false group-private belief.
Mild misperception
Agent may overestimate support, but not infer a full majority.
Majority illusion
Agent may infer that most others privately support the target.
Unanimity illusion
Agent may infer near-unanimous support for the target.
Norm inversion
Agent may infer support opposite to the recorded private majority.
Table 9: Extended control results for PHANTOM-Bench . N-Acc/N-Phan. report Neutral Public accuracy and phantom alignment, while EV-Acc/EV-Phan. report Explicit Public Vote accuracy and phantom alignment. Obs-N and Obs-EV gaps compare observed-public phantom alignment against the corresponding control condition.
Figure 5: Bootstrap 95% confidence intervals for key PHANTOM-Bench metrics. Intervals are computed using scenario-level bootstrap resampling, preserving the four-agent structure within each scenario.
Figure 6: Condition-level and norm-source breakdowns. Left: Collective Rationality Failure Rate (CRFR) by perceived norm-source category across models, where darker cells indicate higher scenario-level collective failure. Right: accuracy across the five PHANTOM-Bench conditions: Private, Observed Public, Informed Transparency, Neutral Public, and Explicit Public Vote.