physics.soc-phNov 16, 2026

Borrowed Strength, Imported Risk: Capability Gaps and Reception Govern Heterogeneous LLM Debate

Authors: Prashanti Nilayam, Kiran Kumar Ramanna, Prashil Tumbade, Nagaraj N Bhat, Sankalp Nayak

Organizations: ServiceNow

Abstract

Heterogeneous LLM debate is often attributed to collective reasoning that creates capability beyond any member. In the matched panels we study, the aggregate gain is dominated by capability transfer: a peer supplies task competence and receivers absorb a measurable fraction. Across nine matched directions, honest benefit and first-exchange adversarial exposure open when the peer has a positive same-task advantage, and honest gain is closely described by the empirical decomposition ΔAdhon≈κΓd+ρdΔA_d^{\mathrm{hon}}\approxκΓ_d^+ρ_d, the product of that advantage and the receiver's reception. Two fixed-receiver directions provide held-out validation without refitting: reception remains stable within uncertainty as a larger peer advantage produces a larger gain. A matched, single-direction open-answer MuSiQue scope probe reaches the negative-gap side of the same gate. Qwen-7B trails the Gemma-3-12B receivers by 10.78 points; despite 62.50% reception, it produces only +1.17 points of controlled first-exchange gain. Reception cannot substitute for positive aggregate peer advantage. When an influential peer is assigned an adversarial role, the same influential peer slot removes the borrowed gain and can push the panel below homogeneous debate. Across all five full-reference settings, the best observed majority correctness does not exceed the influential member's best observed accuracy, despite unused complementary answers. Heterogeneous debate is therefore a measurable transfer decision with a measurable failure mode, not a benefit conferred by diversity alone.

Figures & tables

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Jun 18, 2026

Heterogeneous LLM Debate Under Adversarial Peers: Honest Gains, Replacement Costs, and Resilience

Heterogeneous LLM debate is motivated by the promise that diverse peers correct one another, but the same exchange that carries correction also carries adversarial influence. We measure which dominates by tracking how a heterogeneous peer changes the honest agents' revision behavior: how often they change their answer, and whether the change is corrective or harmful. We compare matched panels (homogeneous baseline, honest-mixed, and adversarial-mixed) and contaminated panels in which a malicious same-family peer is already present, spanning four model families and three reasoning benchmarks. An honest heterogeneous peer sharply lowers harmful revision, and an adversarial one reverses it. For Llama-3.1-70B defenders on MATH-hard, the honest-slot harmful-revision rate falls from 89% in the homogeneous panel to 35% with an honest peer, and an adversarial peer returns it to 90%. The conditional rate hides this damage on weak defenders, but the end-of-debate flip rate exposes it. The pattern keeps its sign across families and benchmarks while its magnitude varies with the defender-benchmark regime. We also measure the effects when an adversarial same-family peer is already present: an honest heterogeneous peer lowers both harmful revision and the rate at which initially-correct answers are lost. On the same Llama-3.1-70B setting, the added honest peer cuts the flip rate on initially-correct items from 31% under a same-family adversary to 6%. Heterogeneity is therefore not only an attack surface but, when an adversary is already present, also a defense.
Apr 29, 2026

The Cost of Consensus: Isolated Self-Correction Prevails Over Unguided Homogeneous Multi-Agent Debate

Multi-agent debate, where teams of LLMs iteratively exchange rationales and vote on answers, is widely deployed under the assumption that peer review filters hallucinations. Yet the failure dynamics of homogeneous debate remain poorly understood, therefore we report findings from a controlled empirical study of teams of N=10N{=}10 homogeneous agents (Qwen2.5-7B, Llama-3.1-8B, Ministral-3-8B) across R=3R{=}3 debate rounds on two high-difficulty benchmarks (GSM-Hard and MMLU-Hard). We compare peer debate against isolated self-correction and a stochastic noise control that injects rationales from unrelated problems. We decompose debate failure into three model-dependent pathways: sycophantic conformity, where agents uncritically adopt majority answers (modal adoption up to 85.5%); contextual fragility, where peer rationales destabilize previously correct reasoning (vulnerability rate up to 70.0%); and consensus collapse, where plurality voting discards correct answers already present in the generation pool (oracle gap up to 32.3 percentage points). Ablations over communication density (K∈{2,4,9}K \in \{2,4,9\}) and sampling temperature (T∈{0.4,0.7}T \in \{0.4, 0.7\}) show that conformity reaches high levels at minimal peer exposure (K=2K{=}2) and intensifies with greater initial diversity. Across all configurations, debate consumes 2.1-3.4×\times more tokens (up to 28,631 tokens per problem) than self-correction for equal or lower accuracy. Our results indicate that, within the 7-8B parameter class, homogeneous teams without structured roles do not benefit from unguided peer exchange, and that isolated self-correction consistently offers a more favorable cost-accuracy tradeoff.
Sep 28, 2026

Measuring Collapse and Correction in Homogeneous-Panel LLM Debate

Multi-agent large language model (LLM) debate is often evaluated by whether final answers improve, but a changed answer is not necessarily a better one: the same discussion can correct an initially wrong majority or collapse an initially correct one. Standard final-accuracy evaluations conflate these opposing transitions. We introduce an auditable protocol for homogeneous debate on multiple-choice questions (MCQs) that records each run as a transition ledger over collapse, correction, onset, and signed intervention utility. On 6,925 MMLU-Pro debates, the protocol records 253 collapses together with the corrections that offset them, and the correction side changes how interventions should be judged. Replay experiments reveal the central tradeoff: freezing every debate prevents 251 collapses but loses 804 corrections under equal weights, and a leave-one-model-out probe-gated freeze that is tuned for net utility never fires, so collapse prevention alone can recommend the wrong policy. A compact pre-debate 8-probe screen is a triage signal: its unadjusted family-level association with conditional-collapse risk is high (G=7, Spearman ρρ = 0.893, exact two-sided p = 0.0123), but initial-majority accuracy is a close comparator (ρρ = -0.821; family partial ρρ = 0.767, p = 0.0877), so we do not treat it as calibrated or capability-adjusted prediction. Round-level traces localize many collapses to the first debate round, where early disagreement can precede both collapses and corrections. Reasoning-mode models, including one model at two reasoning-effort levels, also show both collapses and corrections. We release replayable schemas, coders, audits, cost cards, and zero-API rebuild scripts so future model rows can be compared under the same denominators and signed utility ledger.