Beyond Answer Sharing: TrueMAD for Targeted Peer Engagement in Multi-Agent Debate

Dezheng Bao; Sihang Du; Renhong Huang; Zhizhang Yuan; Yifei Sun; Yang Yang

cs.AIOct 13, 2026

Beyond Answer Sharing: TrueMAD for Targeted Peer Engagement in Multi-Agent Debate

Authors: Dezheng Bao, Sihang Du, Renhong Huang, Zhizhang Yuan, Yifei Sun, Yang Yang

Organizations: Zhejiang University

Abstract

Multi-agent debate (MAD) aims to improve large language model reasoning by exposing agents to alternative solutions and enabling iterative revision. However, standard broadcast-style MAD makes peer answers visible without requiring agents to engage with a particular peer claim. As a result, reliable reasoning may be suppressed by conflicting or majority responses, limiting achievable performance, while erroneous reasoning may persist and degrade final predictions without actionable feedback. We propose TrueMAD, a backbone-agnostic test-time plug-in that introduces one-to-one, free-form peer advice between answer broadcasting and self-revision. Each source agent responds to a target's current reasoning through support, rebuttal, refinement, or further questions, while the target retains control over how to revise. The same mechanism can augment standard MAD and its improved variants without parameter updates. Across AIME, GPQA-Diamond, and TruthfulQA with Qwen3-30B-A3B, TrueMAD improves agent-level and strict-majority accuracy over MAD by 3.62 and 3.52 percentage points on average, while TrueDMAD improves DMAD by 4.72 and 4.06 points, respectively. Analyses at the utterance, agent-transition, and group levels further show more peer-specific deliberation, more correction-oriented revisions, and more favorable consensus dynamics.

Figures & tables

Figure 1: Motivating example and TrueMAD overview. Left: In broadcast-only MAD, majority pressure can suppress a correct answer, while agents with distinct errors receive no targeted diagnosis. Right: TrueMAD adds one-to-one, free-form advice between broadcasting and self-revision, enabling support, rebuttal, refinement, or questions without altering the MAD pipeline.

Method	AIME		GPQA-Diamond		TruthfulQA
Method	Agent ACC	Vote ACC	Agent ACC	Vote ACC	Agent ACC	Vote ACC
Single LLM	$23.06{\scriptstyle\pm 1.27}$		$52.82{\scriptstyle\pm 0.07}$		$77.17{\scriptstyle\pm 0.19}$
MAD	$29.81{\scriptstyle\pm 2.49}$	$29.63{\scriptstyle\pm 2.57}$	$54.12{\scriptstyle\pm 1.29}$	$52.86{\scriptstyle\pm 1.27}$	$77.88{\scriptstyle\pm 0.45}$	$76.50{\scriptstyle\pm 1.32}$
promptMAD	$30.37{\scriptstyle\pm 2.39}$	$30.00{\scriptstyle\pm 2.94}$	$55.26{\scriptstyle\pm 1.52}$	$54.71{\scriptstyle\pm 1.77}$	$77.96{\scriptstyle\pm 0.29}$	$76.83{\scriptstyle\pm 0.76}$
DMAD	$30.28{\scriptstyle\pm 1.00}$	$29.63{\scriptstyle\pm 1.28}$	$56.14{\scriptstyle\pm 1.53}$	$56.06{\scriptstyle\pm 1.82}$	$77.79{\scriptstyle\pm 1.25}$	$77.33{\scriptstyle\pm 0.76}$
TrueMAD	$\mathbf{35.37}{\scriptstyle\pm 2.36}$	$\mathbf{34.81}{\scriptstyle\pm 1.70}$	$58.00{\scriptstyle\pm 0.60}$	$56.73{\scriptstyle\pm 0.29}$	$79.29{\scriptstyle\pm 0.62}$	$78.00{\scriptstyle\pm 0.87}$

Table 1: Main results on AIME, GPQA-Diamond, and TruthfulQA with Qwen3-30B-A3B . We report Agent ACC (%) and Vote ACC (%) as mean

\pm

sample standard deviation over three seeds. Bold marks the best method in each column;

\Delta

rows show the gains from adding TrueMAD .

Figure 2: Agent-level accuracy (%) across debate rounds and difficult initial conditions with Qwen3-30B-A3B . (a) Round-wise accuracy from R2 to R5 on AIME, GPQA-Diamond, and TruthfulQA. (b) R5 accuracy on questions for which at least half of the four agents are incorrect in R1.

Figure 3: Debate dynamics analysis of debate quality with Qwen3-30B-A3B . (a) LLM-judged utterance quality (1–10); lines connect MAD with TrueMAD and DMAD with TrueDMAD . (b) Agent- and group-level diagnostics. Higher is better for all metrics.

Method	AIME	GPQA-Diamond	TruthfulQA
MAD	$29.81{\scriptstyle\pm 2.49}$	$54.12{\scriptstyle\pm 1.29}$	$77.88{\scriptstyle\pm 0.45}$
ComputeMAD	$31.48{\scriptstyle\pm 1.05}$	$56.40{\scriptstyle\pm 2.63}$	$77.08{\scriptstyle\pm 1.18}$
Self-AdviceMAD	$33.33{\scriptstyle\pm 1.00}$	$54.55{\scriptstyle\pm 2.70}$	$75.21{\scriptstyle\pm 1.34}$
TrueMAD	$\mathbf{35.37}{\scriptstyle\pm 2.36}$	$\mathbf{58.00}{\scriptstyle\pm 0.60}$	$\mathbf{79.29}{\scriptstyle\pm 0.62}$

Table 2: Auxiliary-call-matched comparison with Qwen3-30B-A3B . Bold marks the best method.

Method	GPT-4.1-mini	Qwen3-32B
Single LLM	$51.57{\scriptstyle\pm 0.85}$	$18.43{\scriptstyle\pm 0.42}$
MAD	$62.87{\scriptstyle\pm 0.70}$	$23.98{\scriptstyle\pm 1.60}$
promptMAD	$63.06{\scriptstyle\pm 1.21}$	$24.91{\scriptstyle\pm 1.67}$
DMAD	$63.52{\scriptstyle\pm 0.42}$	$24.72{\scriptstyle\pm 1.82}$
TrueMAD	$67.31{\scriptstyle\pm 1.79}$	$27.69{\scriptstyle\pm 1.31}$
TrueDMAD	$\mathbf{68.61}{\scriptstyle\pm 2.20}$	$\mathbf{28.33}{\scriptstyle\pm 2.27}$

Table 3: Cross-model results on AIME. Bold marks the best method for each backbone.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Method	R2	R3	R4	R5
MAD	$27.69{\scriptstyle\pm 1.70}$	$29.35{\scriptstyle\pm 2.16}$	$30.00{\scriptstyle\pm 2.17}$	$29.81{\scriptstyle\pm 2.49}$
promptMAD	$28.52{\scriptstyle\pm 1.37}$	$29.81{\scriptstyle\pm 1.79}$	$30.09{\scriptstyle\pm 1.95}$	$30.37{\scriptstyle\pm 2.39}$
DMAD	$29.07{\scriptstyle\pm 2.23}$	$29.91{\scriptstyle\pm 1.25}$	$30.83{\scriptstyle\pm 1.55}$	$30.28{\scriptstyle\pm 1.00}$
TrueMAD	$\mathbf{30.74}{\scriptstyle\pm 1.43}$	$\mathbf{34.17}{\scriptstyle\pm 2.78}$	$\mathbf{35.00}{\scriptstyle\pm 2.93}$	$\mathbf{35.37}{\scriptstyle\pm 2.36}$
TrueDMAD	$28.98{\scriptstyle\pm 0.42}$	$31.48{\scriptstyle\pm 0.58}$	$33.52{\scriptstyle\pm 0.64}$	$34.63{\scriptstyle\pm 1.12}$

Appendix

Table 4: Round-wise agent-level accuracy (%) of Qwen3-30B-A3B on AIME. The highest mean accuracy in each round is shown in bold.

Method	R2	R3	R4	R5
MAD	$53.16{\scriptstyle\pm 1.46}$	$53.79{\scriptstyle\pm 1.16}$	$54.00{\scriptstyle\pm 0.89}$	$54.12{\scriptstyle\pm 1.29}$
promptMAD	$54.59{\scriptstyle\pm 1.56}$	$55.85{\scriptstyle\pm 1.39}$	$55.64{\scriptstyle\pm 1.51}$	$55.26{\scriptstyle\pm 1.52}$
DMAD	$54.59{\scriptstyle\pm 0.41}$	$56.40{\scriptstyle\pm 0.60}$	$55.98{\scriptstyle\pm 0.84}$	$56.14{\scriptstyle\pm 1.53}$
TrueMAD	$\mathbf{55.30}{\scriptstyle\pm 1.46}$	$56.82{\scriptstyle\pm 0.55}$	$57.70{\scriptstyle\pm 1.70}$	$58.00{\scriptstyle\pm 0.60}$
TrueDMAD	$55.01{\scriptstyle\pm 0.64}$	$\mathbf{58.50}{\scriptstyle\pm 2.22}$	$\mathbf{60.06}{\scriptstyle\pm 2.44}$	$\mathbf{62.25}{\scriptstyle\pm 1.79}$

Appendix

Table 5: Round-wise agent-level accuracy (%) of Qwen3-30B-A3B on GPQA-Diamond. The highest mean accuracy in each round is shown in bold.

Method	R2	R3	R4	R5
MAD	$77.54{\scriptstyle\pm 0.26}$	$77.79{\scriptstyle\pm 0.31}$	$77.83{\scriptstyle\pm 0.36}$	$77.88{\scriptstyle\pm 0.45}$
promptMAD	$78.00{\scriptstyle\pm 0.22}$	$77.96{\scriptstyle\pm 1.16}$	$77.96{\scriptstyle\pm 0.69}$	$77.96{\scriptstyle\pm 0.29}$
DMAD	$77.75{\scriptstyle\pm 0.45}$	$78.29{\scriptstyle\pm 0.95}$	$78.29{\scriptstyle\pm 1.91}$	$77.79{\scriptstyle\pm 1.25}$
TrueMAD	$78.62{\scriptstyle\pm 0.75}$	$79.33{\scriptstyle\pm 0.58}$	$79.46{\scriptstyle\pm 0.51}$	$79.29{\scriptstyle\pm 0.62}$
TrueDMAD	$\mathbf{79.00}{\scriptstyle\pm 1.00}$	$\mathbf{80.29}{\scriptstyle\pm 1.12}$	$\mathbf{80.71}{\scriptstyle\pm 0.19}$	$\mathbf{81.50}{\scriptstyle\pm 0.38}$

Appendix

Table 6: Round-wise agent-level accuracy (%) of Qwen3-30B-A3B on TruthfulQA. The highest mean accuracy in each round is shown in bold.

Method	Wrong maj.	Split	Correct maj.	$\geq$ Half wrong
	$k\leq 1$	$k=2$	$k\geq 3$	$k\leq 2$
MAD	$7.6$	$\mathbf{93.4}$	$98.4$	$15.0$
promptMAD	$8.1$	$92.1$	$\mathbf{100.0}$	$15.3$
DMAD	$9.1$	$86.8$	$97.4$	$15.8$
TrueMAD	$\mathbf{14.7}$	$\mathbf{93.4}$	$\mathbf{100.0}$	$\mathbf{21.4}$
TrueDMAD	$14.4$	$89.5$	$98.4$	$20.8$

Appendix

Table 7: R5 agent-level accuracy (%) of Qwen3-30B-A3B on AIME, stratified by the R1 correctness composition. The highest accuracy in each column is shown in bold.

Method	Wrong maj.	Split	Correct maj.	$\geq$ Half wrong
	$k\leq 1$	$k=2$	$k\geq 3$	$k\leq 2$
MAD	$9.8$	$54.7$	$93.7$	$20.9$
promptMAD	$10.9$	$55.0$	$95.1$	$21.8$
DMAD	$11.9$	$54.7$	$\mathbf{96.2}$	$22.5$
TrueMAD	$17.2$	$62.8$	$93.2$	$28.5$
TrueDMAD	$\mathbf{21.0}$	$\mathbf{72.8}$	$96.1$	$\mathbf{33.8}$

Appendix

Table 8: R5 agent-level accuracy (%) of Qwen3-30B-A3B on GPQA-Diamond, stratified by the R1 correctness composition. The highest accuracy in each column is shown in bold.

Method	Wrong maj.	Split	Correct maj.	$\geq$ Half wrong
	$k\leq 1$	$k=2$	$k\geq 3$	$k\leq 2$
MAD	$6.6$	$59.0$	$\mathbf{97.3}$	$15.7$
promptMAD	$11.4$	$57.0$	$96.3$	$19.4$
DMAD	$13.8$	$63.0$	$95.1$	$22.4$
TrueMAD	$15.7$	$58.0$	$96.9$	$23.1$
TrueDMAD	$\mathbf{23.3}$	$\mathbf{73.0}$	$97.0$	$\mathbf{32.0}$

Appendix

Table 9: R5 agent-level accuracy (%) of Qwen3-30B-A3B on TruthfulQA, stratified by the R1 correctness composition. The highest accuracy in each column is shown in bold.

Method	AIME			GPQA-Diamond			TruthfulQA
Method	RP	EI	NC	RP	EI	NC	RP	EI	NC
MAD	$3.16$	$4.04$	$2.76$	$3.86$	$4.83$	$3.36$	$3.84$	$4.49$	$3.11$
promptMAD	$3.41$	$4.42$	$\mathbf{3.13}$	$4.53$	$5.63$	$\mathbf{4.36}$	$4.99$	$6.12$	$\mathbf{5.17}$
DMAD	$2.88$	$4.45$	$2.81$	$4.23$	$5.96$	$3.85$	$5.13$	$6.39$	$4.38$
TrueMAD	$\mathbf{3.82}$	$\mathbf{5.03}$	$3.12$	$\mathbf{4.86}$	$6.27$	$4.23$	$\mathbf{5.42}$	$\mathbf{6.93}$	$5.12$
TrueDMAD	$3.51$	$4.88$	$3.08$	$4.68$	$\mathbf{6.30}$	$4.18$	$5.24$	$6.75$	$4.80$

Appendix

Table 10: Peer-deliberation quality scores of Qwen3-30B-A3B corresponding to Figure 3 (a). RP, EI, and NC denote Rebuttal Precision, Evidence Independence, and Novel Contribution, respectively. The highest score in each column is shown in bold.

Method	AIME			GPQA-Diamond			TruthfulQA
Method	CPL	AME	CMP 1v3	CPL	AME	CMP 1v3	CPL	AME	CMP 1v3
MAD	$0.85$	$4.6$	$25.0$	$1.28$	$3.7$	$25.4$	$4.31$	$2.1$	$14.1$
promptMAD	$0.89$	$7.2$	$33.3$	$1.42$	$4.9$	$27.7$	$3.92$	$4.6$	$34.3$
DMAD	$0.80$	$6.8$	$40.4$	$1.43$	$10.1$	$36.0$	$3.71$	$10.9$	$47.4$
TrueMAD	$\mathbf{1.96}$	$7.1$	$\mathbf{55.0}$	$1.65$	$6.6$	$32.4$	$4.81$	$8.8$	$25.0$
TrueDMAD	$1.01$	$\mathbf{10.9}$	$47.9$	$\mathbf{2.00}$	$\mathbf{12.8}$	$\mathbf{47.3}$	$\mathbf{5.49}$	$\mathbf{13.7}$	$\mathbf{54.8}$

Appendix

Table 11: Correction-preference and consensus-robustness scores of Qwen3-30B-A3B corresponding to Figure 3 (b). CPL is unitless, AME is reported in percentage points, and CMP 1v3 is reported in percent. The highest score in each column is shown in bold.

Placeholder	Runtime value
{question}	Question text; for multiple-choice datasets, this already includes the answer options.
{format_instruction}	Dataset-specific output instruction shown below.
{debater_id}	Integer identifier of the current debater, from 0 to 3 .
{answer_i}	Previous-round answer of debater i , including the current debater’s own answer.
{own_answer}	Previous-round answer of the agent producing advice.
{target_id}	Integer identifier of the debater receiving advice.

Appendix

Table 12: Runtime placeholders used in the prompt templates.

Multi-Agent Debate

Multi-Agent Debate studies how language-model agents exchange arguments, critique answers, and combine judgments for collective reasoning.

See all 140 papers

Remember and Reweight: Enhancing Multi-Agent Debate with Experience Memory and Confidence Estimation
Sep 3, 2026Xuanfa Jin, Zhijian Ma, Yongcheng Zeng +3AI Agent Reliability Multi-Agent Debate
Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences · University College London
ARMOR-MAD: Adaptive Routing for Heterogeneous Multi-Agent Debate in Large Language Model Reasoning
Jun 11, 2026Fuqiang Niu, Bowen ZhangMulti-Agent Debate Multi-Agent Consensus
School of Cyber Science and Technology, University of Science and Technology of China, Hefei, China · School of Artificial Intelligence, Shenzhen Technology University, Shenzhen, China
When and Why Does Multi-Agent Debate Fail and Does It Really Underperform?
Oct 23, 2025Yongqiang Chen, Gang Niu, James Cheng +2Multi-Agent Debate Multi-Agent Collaboration
1The Chinese University of Hong Kong · 2RIKEN Center for Advanced Intelligence Project · 3Hong Kong Baptist University +1

Sep 3, 2026J/K move · Enter open · S save

Remember and Reweight: Enhancing Multi-Agent Debate with Experience Memory and Confidence Estimation

Xuanfa Jin, Zhijian Ma, Yongcheng Zeng, Xinyu Cui+2

Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences · University College London

AI Agent Reliability Multi-Agent Debate Multi-Agent Consensus Memory-Augmented Language Model Agents Multi-Agent Deliberation

Open paper AskPDF arXiv

Multi-agent debate (MAD) improves the reasoning capabilities of large language models by having multiple agents iteratively refine their responses through discussion. However, MAD suffers from a critical vulnerability known as shared misconception: when a majority of agents initially converge on an incorrect answer, the debate process tends to amplify rather than correct the error. Existing methods primarily address peer skew but leave the agents' inherently biased concept priors unaddressed. To mitigate this systematic weakness, we propose R

^2

-MAD (Remember and Reweight for Multi-Agent Debate), a framework that equips agents with an experience memory accumulated from past debates. R

^2

-MAD intervenes on both failure modes through two complementary mechanisms: A debate-state-aware retrieval policy dynamically calibrates the concept prior by retrieving relevant historical evidence based on the current consensus level. Then these retrieved experiences provide a basis for estimating per-agent reliability, yielding confidence weights to modulate peer influence. Experiments on various benchmarks show that R

^2

-MAD achieves consistent improvements over existing single-agent and MAD baselines.

Abstract

Multi-Agent Debate

Explore similar work

Remember and Reweight: Enhancing Multi-Agent Debate with Experience Memory and Confidence Estimation

ARMOR-MAD: Adaptive Routing for Heterogeneous Multi-Agent Debate in Large Language Model Reasoning

When and Why Does Multi-Agent Debate Fail and Does It Really Underperform?