Multi-agent discussion gains less when dissent is withheld

Chand Sahil Mansuri; Xin Wang; Mengying Li; Bryan Acton; Rory Eckardt; Dhaval Patel; Sadamori Kojaku

physics.soc-phSep 29, 2026

Multi-agent discussion gains less when dissent is withheld

Authors: Chand Sahil Mansuri, Xin Wang, Mengying Li, Bryan Acton, Rory Eckardt, Dhaval Patel, Sadamori Kojaku

Organizations: School of Systems Science and Industrial Engineering, Binghamton University, Binghamton, NY, USA · School of Management, Binghamton University, Binghamton, NY, USA · IBM T. J. Watson Research Center, Yorktown Heights, NY, USA

Abstract

Multi-agent systems of LLMs add discussion to majority voting and are therefore expected to be more capable. However, empirical reports conflict on whether discussion improves accuracy or leads to an incorrect consensus. Here, we introduce a parsimonious model that explains when discussion improves accuracy and when it ends in an incorrect consensus, built from four behaviors repeatedly observed in LLM agents: (1) withholding dissent, (2) internalizing a stated answer, (3) reconsidering after seeing dissent, and (4) correcting toward the correct answer. The model shows that discussion can overturn an incorrect initial majority only when the withholding rate $c$ is below a critical rate $c^* = γ/(γ+ a)$ , set by the net correction rate $γ$ and the internalization rate $a$ . We estimate these rates from conversation logs with a Bayesian method and place LLM teams relative to $c^*$ . As the model predicts, the gain from discussion shrinks as withholding rises, across LLMs and on a hidden profile benchmark, HiddenBench, and MedEInst. Instructing agents not to withhold dissent increases this gain. Turning reasoning off also increases the gain, because reasoning raises the internalization rate $a$ and keeps agents from reconsidering a minority answer. These findings reconcile the conflicting reports and identify when discussion outperforms majority voting.

Figures & tables

Figure 1: (a) Each agent holds a private belief $z$ (inner circle) and states a public opinion $y$ (outer ring), and it sees the statements of the agents it observes and their majority $m_{i}$ ( $q=4$ in the schematic). (b) When the visible majority differs from its belief, an agent either withholds (probability $c$ ), states the majority, and adopts it with probability $a$ , or states its belief, reconsiders with probability $\rho d_{i}$ , and reaches the right answer with probability $r$ . (c) Share of teams ending on the correct answer against the single-agent accuracy $p$ and the withholding rate $c$ : agent-based simulation (color) and the mean-field boundary (black curve) agree. Below $p=1/2$ , discussion reaches the correct answer only while $c$ stays below $c^{*}$ ( $a=0.5$ , $\rho=1$ , $r=0.8$ , $q=3$ ).

Figure 2: (a) Hidden profile task: only the pooled private clues reveal the correct answer. (b) Withholding rate

\hat{c}

and (c) critical value

c^{*}

per LLM under instructions A (honest, blue), B (none, grey), and C (value cohesion, open). (d) Gain from discussion against the margin

\hat{c}-c^{*}

; marker shape is the LLM, striped marks have reasoning switched off, and the curve is a quadratic fit with its 95% bootstrap band. The gain falls as the margin grows. (e–g) Internalization

\hat{a}

, net correction

\hat{\gamma}

, and gain from discussion with reasoning on (filled) and off (striped), under instruction A. Intervals are 95%.

Figure 3: (a–c) Gain from discussion against the margin

\hat{c}-c^{*}

on HiddenBench (HB), MuSiQue (MuS), and MedEInst (MedE), with markers as in Fig. 2 d and a linear fit (orange) with its 95% band; settings whose LLM solved no item and the MedEInst run of qwen3.8 are not drawn (Appendix B.1 ). The gain falls as the margin grows, clearly on HB and MedE and weakly on MuS. (d) Effects of the benchmark, the LLM, and the instruction (relative to B) on the logit of the withholding rate, from one fit over the four benchmarks (HP: our hidden profile task). (e) Share of the shared evidence that supports the correct answer (black) or the agent’s own answer (orange) on the hidden profile task. (f) Evidence pooling and utilization rates on the hidden profile task under instruction A. Intervals are 95%.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Evidence pooling rate (left) and evidence utilization rate (right) on the hidden profile task, one row per model in the order of Fig. 3 e, with the three instructions on each row (A blue, B grey, C open). Intervals are 95% Wilson intervals.

Figure 5: Reasoning switched on (filled) and off (open) in the same model on the hidden profile task, one row per model, pooled over A, B and C, unlike the instruction-A panels of Fig. 2 e–g: (a) withholding rate

\hat{c}

, (b) internalization rate

\hat{a}

, (c) net correction rate

\hat{\gamma}

, and (d) gain from discussion. Bars are 95% Wilson intervals of the counts pooled over the three instructions in (a) and (b) and of the post-discussion accuracy in (d), and the range over the three instructions in (c). The reasoning-off run of gemini-3.8-flash is the lowest reasoning setting its API accepts. With reasoning on, internalization rises or stays, while net correction and the gain from discussion are lower in all six models.

Figure 6: Gain from discussion for transparent, control, and forced-majority settings, for four LLMs on the hidden profile task and HiddenBench. Points and intervals are observations, and the gray bands are the predictions registered before the experiment.

Figure 7: Rank correlations of gain with the margin and with its component rates

\hat{c}

\hat{a}

and

\hat{\gamma}

on four benchmarks, with a pooled row. Black circles and intervals show all-data estimates, and orange open squares and intervals show held-out task-split estimates, with medians and 95% intervals over 20 splits.

Figure 8: Probability that the final majority is correct against the number of agents correct at round 0, in panels for the hidden profile task, HiddenBench, MuSiQue, and MedEInst. Solid black marks are observations, and dashed gray marks are posterior-predictive model estimates from the model simulated with each setting’s rates drawn from their posterior.

Figure 9: Stacked shares of the rounds counted as withholding on the hidden profile task. Categories separate whether visible evidence supported, tied, or opposed the majority and whether the private answer stayed opposed or moved with the majority. The upper panel shows the main runs and the lower panel the reasoning-off runs.

Explore similar work

Jun 28, 2026cs.MA

Minority Sentinel: When to Overturn Majority Voting in Multi-Agent LLM Debates

Multi-Agent Debate (MAD) with Majority Voting is a dominant paradigm for improving LLM reasoning, yet its effectiveness rests on the Condorcet Jury Theorem's assumption of independent errors. Because contemporary LLMs share similar pretraining corpora, their errors are strongly correlated, causing the majority to systematically suppress correct minority opinions, a phenomenon we term Minority Truth. Through debates among three heterogeneous LLM agents on six benchmarks, we find that roughly one in four divergent cases has the minority holding the correct answer, yielding a 10-percentage-point theoretical recovery margin. We propose Minority Sentinel, a lightweight meta-classifier that extracts a multi-dimensional debate fingerprint from debate logs and trains a LightGBM model to decide when to overturn majority voting. Minority Sentinel achieves a stable Flip Precision of 81.2% with positive Net Gain across all six datasets and all 20 random seed trials, demonstrating that debate logs contain sufficient behavioral signals for a non-LLM classifier to reliably recover suppressed minorities without degrading system accuracy. The LLM-as-Judge baseline yields negative Net Gain despite higher recall, confirming that flip safety, not recovery volume, determines intervention value.

Chuan He, Zebin Chen, Zhengyi Yang +5

University of New South Wales · Euler AI · Macquarie University

Multi-Agent Debate Debate Large Language Model Decisions Majority Voting Minority Large Language Model Agents+1

Ask PDF

Apr 3, 2026cs.CL

Too Polite to Disagree: Understanding Sycophancy Propagation in Multi-Agent Systems

Large language models (LLMs) often exhibit sycophancy: agreement with user stance even when it conflicts with the model's opinion. While prior work has mostly studied this in single-agent settings, it remains underexplored in collaborative multi-agent systems. We ask whether awareness of other agents' sycophancy levels influences discussion outcomes. To investigate this, we run controlled experiments with six open-source LLMs, providing agents with peer sycophancy rankings that estimate each peer's tendency toward sycophancy. These rankings are based on scores calculated using various static (pre-discussion) and dynamic (online) strategies. We find that providing sycophancy priors reduces the influence of sycophancy-prone peers, mitigates error-cascades, and improves final discussion accuracy by an absolute 10.5%. Thus, this is a lightweight and efficient way to reduce model sycophancy during discussions and subsequently improve downstream accuracy.

Vira Kasprova, Amruta Parulekar, Abdulrahman AlRabah +5

University of Illinois Urbana-Champaign

Sycophancy Multi-Agent Large Language Model Systems

Ask PDF

Apr 29, 2026cs.MA

The Cost of Consensus: Isolated Self-Correction Prevails Over Unguided Homogeneous Multi-Agent Debate

Multi-agent debate, where teams of LLMs iteratively exchange rationales and vote on answers, is widely deployed under the assumption that peer review filters hallucinations. Yet the failure dynamics of homogeneous debate remain poorly understood, therefore we report findings from a controlled empirical study of teams of

N{=}10

homogeneous agents (Qwen2.5-7B, Llama-3.1-8B, Ministral-3-8B) across

R{=}3

debate rounds on two high-difficulty benchmarks (GSM-Hard and MMLU-Hard). We compare peer debate against isolated self-correction and a stochastic noise control that injects rationales from unrelated problems. We decompose debate failure into three model-dependent pathways: sycophantic conformity, where agents uncritically adopt majority answers (modal adoption up to 85.5%); contextual fragility, where peer rationales destabilize previously correct reasoning (vulnerability rate up to 70.0%); and consensus collapse, where plurality voting discards correct answers already present in the generation pool (oracle gap up to 32.3 percentage points). Ablations over communication density (

K \in \{2,4,9\}

) and sampling temperature (

T \in \{0.4, 0.7\}

) show that conformity reaches high levels at minimal peer exposure (

K{=}2

) and intensifies with greater initial diversity. Across all configurations, debate consumes 2.1-3.4

\times

more tokens (up to 28,631 tokens per problem) than self-correction for equal or lower accuracy. Our results indicate that, within the 7-8B parameter class, homogeneous teams without structured roles do not benefit from unguided peer exchange, and that isolated self-correction consistently offers a more favorable cost-accuracy tradeoff.

Blaž Bertalanič, Carolina Fortuna

Jožef Stefan Institute Ljubljana, Slovenia

Multi-Agent Debate Debate Iterative Self-Correction Free-Form Textual Rationales

Ask PDF