Epistemic Gain, Aleatoric Cost: Uncertainty Decomposition in Multi-Agent Debate for Math Reasoning

Dan Qiao; Binbin Chen; Fengyu Cai; Jianlong Chen; Wenhao Li; Fuxin Jiang; Zuzhi Chen; Hongyuan Zha

cs.MAMar 1, 2026

Epistemic Gain, Aleatoric Cost: Uncertainty Decomposition in Multi-Agent Debate for Math Reasoning

Authors: Dan Qiao, Binbin Chen, Fengyu Cai, Jianlong Chen, Wenhao Li, Fuxin Jiang, Zuzhi Chen, Hongyuan Zha,

Organizations: School of Data Science, The Chinese University of Hong Kong, Shenzhen, China · ByteDance, Beijing, China · Technical University of Darmstadt, Darmstadt, Germany · School of Computer Science, Tongji University, Shanghai, China ·

Abstract

Multi-Agent Debate (MAD) has shown promise in improving reasoning and reducing hallucinations, yet it remains unclear how information exchange shapes individual reasoning behavior. Empirically, MAD exhibits paradoxical phenomena, including rising accuracy with increasing token entropy and marked differences between homogeneous and heterogeneous agent combinations. In this paper, we introduce a Bayesian uncertainty analysis framework for MAD, which decomposes answer-level predictive uncertainty into epistemic uncertainty and aleatoric uncertainty, corresponding to the potential gain and cost of debate. Across multiple agent configurations, we find that effective debate depends on achieving high epistemic gain under controlled aleatoric cost. Building on this insight, we design an uncertainty-guided multi-agent reinforcement learning algorithm that encourages lower aleatoric cost and more effective epistemic information utilization. Experiments show that our approach simultaneously enhances each agent's accuracy and promotes a more productive debate process, providing an operational Bayesian perspective for understanding and improving MAD.

Figures & tables

Figure 1: Multi agent debate performance with two Qwen2.5-3B-Instruct models. As accuracy (red line) increases across debate rounds, token entropy (blue shaded area) also increases noticeably.

Panel of Figure 2 — Figure 2: Overview of uncertainty decomposition in MAD. The purple dashed area represents the system epistemic uncertainty and the gray line represents the total uncertainty. The gap between total uncertainty and Sys-EU corresponds to Sys-AU. The blue and orange lines represent the accuracy on the test datasets. Top row: Homogeneous multi-agent debate. Bottom row: Heterogeneous multi-agent debate.

Setting	Method (Round)	GSM8K		MATH500		AMC23		AIME24		AIME25		Average
Setting	Method (Round)	A0	A1	A0	A1	A0	A1	A0	A1	A0	A1	A0	A1
Homogeneous MAD	Zero-Shot ( $T=1$ )	${83.3}_{\pm 0.4}$	${83.3}_{\pm 0.4}$	${61.0}_{\pm 0.5}$	${61.0}_{\pm 0.5}$	${35.0}_{\pm 1.2}$	${45.0}_{\pm 1.1}$	${3.3}_{\pm 0.0}$	${0.0}_{\pm 0.0}$	${2.0}_{\pm 1.8}$	${2.0}_{\pm 1.8}$	${36.9}_{\pm 0.4}$	${38.3}_{\pm 0.4}$
	IPPO ( $T=1$ )	${83.4}_{\pm 0.3}$	${82.3}_{\pm 0.4}$	${65.2}_{\pm 0.5}$	${62.2}_{\pm 0.6}$	${45.0}_{\pm 1.2}$	${45.0}_{\pm 1.1}$	${3.3}_{\pm 0.4}$	${1.5}_{\pm 0.3}$	${2.7}_{\pm 0.4}$	${2.7}_{\pm 0.4}$	${39.9}_{\pm 0.4}$	${38.7}_{\pm 0.5}$
	UMAD ( $T=1$ )	${82.6}_{\pm 0.4}$	${83.7}_{\pm 0.3}$	${62.4}_{\pm 0.5}$	${63.4}_{\pm 0.4}$	${42.5}_{\pm 1.4}$	${37.5}_{\pm 1.6}$	${4.0}_{\pm 0.5}$	${2.0}_{\pm 0.4}$	${3.3}_{\pm 0.6}$	${3.3}_{\pm 0.5}$	${39.0}_{\pm 0.5}$	${38.0}_{\pm 0.6}$
	Zero-Shot ( $T=2$ )	${83.4}_{\pm 0.3}$	${84.1}_{\pm 0.4}$	${63.0}_{\pm 0.5}$	${64.6}_{\pm 0.6}$	${30.0}_{\pm 1.3}$	${40.0}_{\pm 1.2}$	${3.3}_{\pm 0.0}$	${2.0}_{\pm 1.8}$	${6.7}_{\pm 0.0}$	${2.0}_{\pm 1.8}$	${37.3}_{\pm 0.0}$	${38.5}_{\pm 0.6}$
	IPPO ( $T=2$ )	${83.4}_{\pm 0.2}$	${83.3}_{\pm 0.3}$	${65.8}_{\pm 0.4}$	${64.2}_{\pm 0.5}$	${42.5}_{\pm 1.3}$	${47.5}_{\pm 1.1}$	${4.0}_{\pm 0.6}$	${2.7}_{\pm 0.4}$	${7.3}_{\pm 0.5}$	${3.3}_{\pm 0.4}$	${40.6}_{\pm 0.4}$	${40.2}_{\pm 0.4}$

Table 1: Comparisons of individual agent accuracy across initial responses (

T=1

) and debate responses (

T=2,5

). We report pass@1 results for both homogeneous and heterogeneous MAD settings using Qwen2.5-3B-Instruct and Qwen3-4B-Instruct-2507 . Light blue + Gain and light pink - Loss indicate performance variations. Cell colors are assigned relative to the corresponding Zero-Shot baseline at the same turn (

T

). Bold values indicate the best performance among Zero-Shot , IPPO , and UMAD at

T=5

Panel of Figure 3 — Figure 3: Uncertainty dynamics (gray and purple) vs. accuracy (orange and blue). UMAD training suppresses aleatoric noise and improves both agents’ accuracies across debate turns.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Trajectory-level pairwise debate rollouts strategy.

Generation Configuration
Hyperparameter	Value
Training Temperature	$0.8$
Inference Temperature	$0.6$
Top-p	$0.95$
Max Response Length	$2,048$
Max Prompt Length	$5,120$

Appendix

Table 2: Detailed Hyperparameters for UMAD Training and Inference.

Figure 5: Homogeneous and Heterogeneous First Turn Responses Flipping Ratio.

Panel of Figure 6 — Figure 6: Overview of uncertainty decomposition in MAD. The purple dashed area represents the system epistemic uncertainty and the gray line represents the total uncertainty. The blue and orange lines represent the accuracy on the test datasets. Top two rows: Homogeneous multi-agent debate. Bottom two rows: Heterogeneous multi-agent debate.

Method	Round	GSM8K		MATH500		AMC23		AIME24		AIME25		Average
Method	Round	A0	A1	A0	A1	A0	A1	A0	A1	A0	A1	A0	A1
Full UMAD	$T=1$	$83.2{\pm}0.9$	$89.5{\pm}0.7$	$63.0{\pm}0.7$	$79.6{\pm}0.9$	$38.3{\pm}5.2$	$55.8{\pm}3.8$	$3.3{\pm}3.3$	$17.8{\pm}3.8$	$0.0{\pm}0.0$	$27.8{\pm}8.4$	$37.6{\pm}0.6$	$54.1{\pm}2.1$
Full UMAD	$T=2$	$90.0{\pm}0.7$	$90.7{\pm}0.4$	$83.3{\pm}0.3$	$82.7{\pm}1.0$	$65.8{\pm}6.3$	$62.5{\pm}2.5$	$20.0{\pm}8.8$	$28.9{\pm}1.9$	$26.7{\pm}8.8$	$35.6{\pm}5.1$	$57.2{\pm}3.6$	$60.1{\pm}1.0$
Full UMAD	$T=5$	$\mathbf{91.1{\pm}0.6}$	$91.1{\pm}0.2$	$\mathbf{87.0{\pm}0.7}$	$86.1{\pm}0.6$	$\mathbf{80.0{\pm}4.3}$	$71.7{\pm}3.8$	$\mathbf{32.2{\pm}6.9}$	$34.4{\pm}5.1$	$\mathbf{42.2{\pm}5.1}$	$\mathbf{45.6{\pm}7.7}$	$\mathbf{66.5{\pm}2.8}$	$65.8{\pm}3.1$
Intrinsic-only	$T=1$	$83.4{\pm}0.1$	$89.9{\pm}0.2$	$65.3{\pm}1.2$	$76.3{\pm}1.3$	$39.2{\pm}5.2$	$59.2{\pm}2.9$	$1.1{\pm}1.9$	$18.9{\pm}1.9$	$3.3{\pm}0.0$	$21.1{\pm}1.9$	$38.5{\pm}0.8$	$53.1{\pm}0.4$
Intrinsic-only	$T=2$	$86.9{\pm}0.6$	$91.5{\pm}0.5$	$75.8{\pm}1.1$	$83.9{\pm}0.4$	$57.5{\pm}7.5$	$72.5{\pm}2.5$	$14.4{\pm}1.9$	$32.2{\pm}3.8$	$10.0{\pm}3.3$	$35.6{\pm}5.1$	$48.9{\pm}1.7$	$63.1{\pm}2.2$

Appendix

Table 3: Three-seed mean and standard deviation for heterogeneous MAD evaluation. We report pass@1 individual agent accuracy across

T=1,2,5

. Best values are highlighted in bold.

Multi-Agent Debate

Multi-Agent Debate studies how language-model agents exchange arguments, critique answers, and combine judgments for collective reasoning.

See all 145 papers

Sep 3, 2026

Remember and Reweight: Enhancing Multi-Agent Debate with Experience Memory and Confidence Estimation

Multi-agent debate (MAD) improves the reasoning capabilities of large language models by having multiple agents iteratively refine their responses through discussion. However, MAD suffers from a critical vulnerability known as shared misconception: when a majority of agents initially converge on an incorrect answer, the debate process tends to amplify rather than correct the error. Existing methods primarily address peer skew but leave the agents' inherently biased concept priors unaddressed. To mitigate this systematic weakness, we propose R

^2

-MAD (Remember and Reweight for Multi-Agent Debate), a framework that equips agents with an experience memory accumulated from past debates. R

^2

-MAD intervenes on both failure modes through two complementary mechanisms: A debate-state-aware retrieval policy dynamically calibrates the concept prior by retrieving relevant historical evidence based on the current consensus level. Then these retrieved experiences provide a basis for estimating per-agent reliability, yielding confidence weights to modulate peer influence. Experiments on various benchmarks show that R

^2

-MAD achieves consistent improvements over existing single-agent and MAD baselines.

Xuanfa Jin, Zhijian Ma, Yongcheng Zeng +3

Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences · University College London

AI Agent Reliability Multi-Agent Debate Multi-Agent Consensus Memory-Augmented Language Model Agents Multi-Agent Deliberation

Ask PDF

Oct 12, 2026

Efficient Multi-Agent Reasoning via Confidence-Guided Adaptive Debate

Multi-agent debate has shown promise for improving the reasoning of large language models, yet recent theory suggests its benefits are highly regime-dependent. While interaction can amplify informative signals under corrective conditions, symmetric debate dynamics are neutral in expectation, often making majority voting preferable. We reconcile these views by arguing that debate is effective only when invoked at the right time and with appropriate structure. Based on this insight, we propose LASE: Leader-Adaptive Structured Engagement, a leader-centric multi-agent debate framework that selectively engages interaction only in non-neutral regimes. LASE introduces an asymmetric leader-supporter architecture that enables directed information flow and selective signal amplification, while defaulting to simple aggregation otherwise. Experiments across diverse reasoning benchmarks show that LASE achieves multi-agent-level performance with near single-agent token cost, substantially improving efficiency over static debate and voting baselines.

Seungdong Yoa, Ye Seul Sim, Suhee Yoon +5

LG AI Research, Seoul, South Korea · School of Computer Science and Engineering, Chung-Ang University, Seoul, South Korea · Division of Computer Engineering, Hankuk University of Foreign Studies, Yongin, South Korea +1

Multi-Agent Debate Multi-Agent Reasoning Efficient Language Model Reasoning Multi-Agent Communication

Ask PDF

Jun 11, 2026

ARMOR-MAD: Adaptive Routing for Heterogeneous Multi-Agent Debate in Large Language Model Reasoning

Multi-agent debate (MAD) can improve large language model reasoning, but fixed debate pipelines often waste computation and can amplify correlated errors among similar agents. We propose ARMOR-MAD, a training-free heterogeneous MAD framework that treats debate as conditional computation. ARMOR-MAD combines three components: Pre-debate Agreement Routing (PAR) decides whether independently generated Round-0 answers require debate; Early Agreement Stopping Evaluator (EASE) stops debate after convergence; and Semantic Outlier Detection (SOD) down-weights abnormal final answers during aggregation. Across MATH Level 5, GSM8K, MMLU, and MMLU-Pro, ARMOR-MAD consistently improves over fixed-round heterogeneous debate with the same model pool, reaching 65.5%, 96.5%, 90.0%, and 81.5% accuracy, respectively. The results suggest that genuine model heterogeneity and agreement-based control are both important for making MAD more accurate and efficient.

Fuqiang Niu, Bowen Zhang

School of Cyber Science and Technology, University of Science and Technology of China, Hefei, China · School of Artificial Intelligence, Shenzhen Technology University, Shenzhen, China

Multi-Agent Debate Multi-Agent Consensus LLM Routing Multi-Agent Reasoning

Ask PDF

Abstract

Multi-Agent Debate

Explore similar work

Remember and Reweight: Enhancing Multi-Agent Debate with Experience Memory and Confidence Estimation

Efficient Multi-Agent Reasoning via Confidence-Guided Adaptive Debate

ARMOR-MAD: Adaptive Routing for Heterogeneous Multi-Agent Debate in Large Language Model Reasoning