Epistemic Gain, Aleatoric Cost: Uncertainty Decomposition in Multi-Agent Debate for Math Reasoning

Dan Qiao; Binbin Chen; Fengyu Cai; Jianlong Chen; Wenhao Li; Fuxin Jiang; Zuzhi Chen; Hongyuan Zha

cs.MAMar 1, 2026

Epistemic Gain, Aleatoric Cost: Uncertainty Decomposition in Multi-Agent Debate for Math Reasoning

Authors: Dan Qiao, Binbin Chen, Fengyu Cai, Jianlong Chen, Wenhao Li, Fuxin Jiang, Zuzhi Chen, Hongyuan Zha,

Organizations: School of Data Science, The Chinese University of Hong Kong, Shenzhen, China · ByteDance, Beijing, China · Technical University of Darmstadt, Darmstadt, Germany · School of Computer Science, Tongji University, Shanghai, China ·

Abstract

Multi-Agent Debate (MAD) has shown promise in improving reasoning and reducing hallucinations, yet it remains unclear how information exchange shapes individual reasoning behavior. Empirically, MAD exhibits paradoxical phenomena, including rising accuracy with increasing token entropy and marked differences between homogeneous and heterogeneous agent combinations. In this paper, we introduce a Bayesian uncertainty analysis framework for MAD, which decomposes answer-level predictive uncertainty into epistemic uncertainty and aleatoric uncertainty, corresponding to the potential gain and cost of debate. Across multiple agent configurations, we find that effective debate depends on achieving high epistemic gain under controlled aleatoric cost. Building on this insight, we design an uncertainty-guided multi-agent reinforcement learning algorithm that encourages lower aleatoric cost and more effective epistemic information utilization. Experiments show that our approach simultaneously enhances each agent's accuracy and promotes a more productive debate process, providing an operational Bayesian perspective for understanding and improving MAD.

Figures & tables

Figure 1: Multi agent debate performance with two Qwen2.5-3B-Instruct models. As accuracy (red line) increases across debate rounds, token entropy (blue shaded area) also increases noticeably.

Panel of Figure 2 — Figure 2: Overview of uncertainty decomposition in MAD. The purple dashed area represents the system epistemic uncertainty and the gray line represents the total uncertainty. The gap between total uncertainty and Sys-EU corresponds to Sys-AU. The blue and orange lines represent the accuracy on the test datasets. Top row: Homogeneous multi-agent debate. Bottom row: Heterogeneous multi-agent debate.

Setting	Method (Round)	GSM8K		MATH500		AMC23		AIME24		AIME25		Average
Setting	Method (Round)	A0	A1	A0	A1	A0	A1	A0	A1	A0	A1	A0	A1
Homogeneous MAD	Zero-Shot ( $T=1$ )	${83.3}_{\pm 0.4}$	${83.3}_{\pm 0.4}$	${61.0}_{\pm 0.5}$	${61.0}_{\pm 0.5}$	${35.0}_{\pm 1.2}$	${45.0}_{\pm 1.1}$	${3.3}_{\pm 0.0}$	${0.0}_{\pm 0.0}$	${2.0}_{\pm 1.8}$	${2.0}_{\pm 1.8}$	${36.9}_{\pm 0.4}$	${38.3}_{\pm 0.4}$
	IPPO ( $T=1$ )	${83.4}_{\pm 0.3}$	${82.3}_{\pm 0.4}$	${65.2}_{\pm 0.5}$	${62.2}_{\pm 0.6}$	${45.0}_{\pm 1.2}$	${45.0}_{\pm 1.1}$	${3.3}_{\pm 0.4}$	${1.5}_{\pm 0.3}$	${2.7}_{\pm 0.4}$	${2.7}_{\pm 0.4}$	${39.9}_{\pm 0.4}$	${38.7}_{\pm 0.5}$
	UMAD ( $T=1$ )	${82.6}_{\pm 0.4}$	${83.7}_{\pm 0.3}$	${62.4}_{\pm 0.5}$	${63.4}_{\pm 0.4}$	${42.5}_{\pm 1.4}$	${37.5}_{\pm 1.6}$	${4.0}_{\pm 0.5}$	${2.0}_{\pm 0.4}$	${3.3}_{\pm 0.6}$	${3.3}_{\pm 0.5}$	${39.0}_{\pm 0.5}$	${38.0}_{\pm 0.6}$
	Zero-Shot ( $T=2$ )	${83.4}_{\pm 0.3}$	${84.1}_{\pm 0.4}$	${63.0}_{\pm 0.5}$	${64.6}_{\pm 0.6}$	${30.0}_{\pm 1.3}$	${40.0}_{\pm 1.2}$	${3.3}_{\pm 0.0}$	${2.0}_{\pm 1.8}$	${6.7}_{\pm 0.0}$	${2.0}_{\pm 1.8}$	${37.3}_{\pm 0.0}$	${38.5}_{\pm 0.6}$
	IPPO ( $T=2$ )	${83.4}_{\pm 0.2}$	${83.3}_{\pm 0.3}$	${65.8}_{\pm 0.4}$	${64.2}_{\pm 0.5}$	${42.5}_{\pm 1.3}$	${47.5}_{\pm 1.1}$	${4.0}_{\pm 0.6}$	${2.7}_{\pm 0.4}$	${7.3}_{\pm 0.5}$	${3.3}_{\pm 0.4}$	${40.6}_{\pm 0.4}$	${40.2}_{\pm 0.4}$

Table 1: Comparisons of individual agent accuracy across initial responses (

T=1

) and debate responses (

T=2,5

). We report pass@1 results for both homogeneous and heterogeneous MAD settings using Qwen2.5-3B-Instruct and Qwen3-4B-Instruct-2507 . Light blue + Gain and light pink - Loss indicate performance variations. Cell colors are assigned relative to the corresponding Zero-Shot baseline at the same turn (

T

). Bold values indicate the best performance among Zero-Shot , IPPO , and UMAD at

T=5

Panel of Figure 3 — Figure 3: Uncertainty dynamics (gray and purple) vs. accuracy (orange and blue). UMAD training suppresses aleatoric noise and improves both agents’ accuracies across debate turns.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Trajectory-level pairwise debate rollouts strategy.

Generation Configuration
Hyperparameter	Value
Training Temperature	$0.8$
Inference Temperature	$0.6$
Top-p	$0.95$
Max Response Length	$2,048$
Max Prompt Length	$5,120$

Appendix

Table 2: Detailed Hyperparameters for UMAD Training and Inference.

Figure 5: Homogeneous and Heterogeneous First Turn Responses Flipping Ratio.

Panel of Figure 6 — Figure 6: Overview of uncertainty decomposition in MAD. The purple dashed area represents the system epistemic uncertainty and the gray line represents the total uncertainty. The blue and orange lines represent the accuracy on the test datasets. Top two rows: Homogeneous multi-agent debate. Bottom two rows: Heterogeneous multi-agent debate.

Method	Round	GSM8K		MATH500		AMC23		AIME24		AIME25		Average
Method	Round	A0	A1	A0	A1	A0	A1	A0	A1	A0	A1	A0	A1
Full UMAD	$T=1$	$83.2{\pm}0.9$	$89.5{\pm}0.7$	$63.0{\pm}0.7$	$79.6{\pm}0.9$	$38.3{\pm}5.2$	$55.8{\pm}3.8$	$3.3{\pm}3.3$	$17.8{\pm}3.8$	$0.0{\pm}0.0$	$27.8{\pm}8.4$	$37.6{\pm}0.6$	$54.1{\pm}2.1$
Full UMAD	$T=2$	$90.0{\pm}0.7$	$90.7{\pm}0.4$	$83.3{\pm}0.3$	$82.7{\pm}1.0$	$65.8{\pm}6.3$	$62.5{\pm}2.5$	$20.0{\pm}8.8$	$28.9{\pm}1.9$	$26.7{\pm}8.8$	$35.6{\pm}5.1$	$57.2{\pm}3.6$	$60.1{\pm}1.0$
Full UMAD	$T=5$	$\mathbf{91.1{\pm}0.6}$	$91.1{\pm}0.2$	$\mathbf{87.0{\pm}0.7}$	$86.1{\pm}0.6$	$\mathbf{80.0{\pm}4.3}$	$71.7{\pm}3.8$	$\mathbf{32.2{\pm}6.9}$	$34.4{\pm}5.1$	$\mathbf{42.2{\pm}5.1}$	$\mathbf{45.6{\pm}7.7}$	$\mathbf{66.5{\pm}2.8}$	$65.8{\pm}3.1$
Intrinsic-only	$T=1$	$83.4{\pm}0.1$	$89.9{\pm}0.2$	$65.3{\pm}1.2$	$76.3{\pm}1.3$	$39.2{\pm}5.2$	$59.2{\pm}2.9$	$1.1{\pm}1.9$	$18.9{\pm}1.9$	$3.3{\pm}0.0$	$21.1{\pm}1.9$	$38.5{\pm}0.8$	$53.1{\pm}0.4$
Intrinsic-only	$T=2$	$86.9{\pm}0.6$	$91.5{\pm}0.5$	$75.8{\pm}1.1$	$83.9{\pm}0.4$	$57.5{\pm}7.5$	$72.5{\pm}2.5$	$14.4{\pm}1.9$	$32.2{\pm}3.8$	$10.0{\pm}3.3$	$35.6{\pm}5.1$	$48.9{\pm}1.7$	$63.1{\pm}2.2$

Appendix

Table 3: Three-seed mean and standard deviation for heterogeneous MAD evaluation. We report pass@1 individual agent accuracy across

T=1,2,5

. Best values are highlighted in bold.

Multi-Agent Debate

Multi-Agent Debate studies how language-model agents exchange arguments, critique answers, and combine judgments for collective reasoning.

See all 145 papers

Remember and Reweight: Enhancing Multi-Agent Debate with Experience Memory and Confidence Estimation
Sep 3, 2026Xuanfa Jin, Zhijian Ma, Yongcheng Zeng +3AI Agent Reliability Multi-Agent Debate
Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences · University College London
Efficient Multi-Agent Reasoning via Confidence-Guided Adaptive Debate
Oct 12, 2026Seungdong Yoa, Ye Seul Sim, Suhee Yoon +5Multi-Agent Debate Multi-Agent Reasoning
LG AI Research, Seoul, South Korea · School of Computer Science and Engineering, Chung-Ang University, Seoul, South Korea · Division of Computer Engineering, Hankuk University of Foreign Studies, Yongin, South Korea +1
ARMOR-MAD: Adaptive Routing for Heterogeneous Multi-Agent Debate in Large Language Model Reasoning
Jun 11, 2026Fuqiang Niu, Bowen ZhangMulti-Agent Debate Multi-Agent Consensus
School of Cyber Science and Technology, University of Science and Technology of China, Hefei, China · School of Artificial Intelligence, Shenzhen Technology University, Shenzhen, China

Sep 3, 2026J/K move · Enter open · S save

Remember and Reweight: Enhancing Multi-Agent Debate with Experience Memory and Confidence Estimation

Xuanfa Jin, Zhijian Ma, Yongcheng Zeng, Xinyu Cui+2

Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences · University College London

AI Agent Reliability Multi-Agent Debate Multi-Agent Consensus Memory-Augmented Language Model Agents Multi-Agent Deliberation

Open paper AskPDF arXiv

Multi-agent debate (MAD) improves the reasoning capabilities of large language models by having multiple agents iteratively refine their responses through discussion. However, MAD suffers from a critical vulnerability known as shared misconception: when a majority of agents initially converge on an incorrect answer, the debate process tends to amplify rather than correct the error. Existing methods primarily address peer skew but leave the agents' inherently biased concept priors unaddressed. To mitigate this systematic weakness, we propose R

^2

-MAD (Remember and Reweight for Multi-Agent Debate), a framework that equips agents with an experience memory accumulated from past debates. R

^2

-MAD intervenes on both failure modes through two complementary mechanisms: A debate-state-aware retrieval policy dynamically calibrates the concept prior by retrieving relevant historical evidence based on the current consensus level. Then these retrieved experiences provide a basis for estimating per-agent reliability, yielding confidence weights to modulate peer influence. Experiments on various benchmarks show that R

^2

-MAD achieves consistent improvements over existing single-agent and MAD baselines.

Abstract

Multi-Agent Debate

Explore similar work

Remember and Reweight: Enhancing Multi-Agent Debate with Experience Memory and Confidence Estimation

Efficient Multi-Agent Reasoning via Confidence-Guided Adaptive Debate

ARMOR-MAD: Adaptive Routing for Heterogeneous Multi-Agent Debate in Large Language Model Reasoning