Multi-agent debate (MAD) systems leverage collective intelligence to enhance reasoning capabilities, yet existing approaches struggle to simultaneously optimize accuracy, consensus formation, and computational efficiency. Static topology methods lack adaptability to task complexity variations, while external LLM-based coordination risks introducing privileged knowledge that compromises debate neutrality. This work presents RUMAD (Reinforcement-Unifying Multi-Agent Debate), a novel framework that formulates dynamic communication topology control in MAD as a reinforcement learning (RL) problem. RUMAD employs a content-agnostic observation scheme that captures high-level debate dynamics avoiding access to raw agent reasoning content. RUMAD uses a multi-objective reward to model solution quality, cohesion and efficiency. A PPO-trained controller dynamically adjusts edge weights in the communication graph, while a dual-threshold mechanism enables fine-grained control over both agent activation and information visibility. Experimental evaluation across MMLU, GSM8K, and GPQA benchmarks demonstrates that RUMAD achieves substantial efficiency gains, reducing token costs by over 80%, while still improving reasoning accuracy compared to single LLM model and multiple MAD baselines. Notably, RUMAD trained exclusively on MMLU exhibits robust zero-shot generalization to out-of-domain (OOD) tasks, indicating that the learned communication strategies capture task-independent principles of effective multi-agent coordination. These results establish RUMAD as a efficient and robust approach for deploying multi-agent reasoning application with practical resource constraints.
Figures & tables
Figure 1. Comparison of accuracy and average token cost. Accuracy for MMLU ( ∙ ) and GSM8K ( ■ ) corresponds to the left Y-axis, while GPQA ( ▲ ) corresponds to the right Y-axis. Points closer to the top-left corner represent a superior accuracy-efficiency trade-off. The results show that RUMAD variants consistently outperform baselines by achieving higher accuracy at a lower token cost. Comparison of accuracy and token consumption. The results show that RUMAD achieves better trade-off between accuracy and efficiency.
Static Topology Methods
External-LLM Methods
RUMAD
Accuracy
∙
∙
∙
Consensus
∘
∙
∙
External Opinion Independent
∙
-
∙
Question Independent
∙
-
∙
Communication Cost Control
∘
∘
∙
Table 1. Different types of MAD methods are designed from different strategies. Mark ∙ indicates corresponding metrics are taken into design target. Mark ∘ indicates methods may have side-effect on corresponding metrics but don’t focus on them. Mark - means methods fail on corresponding aspects.
Figure 2. The process pipeline of RUMAD. In the first stage, all agents gives the initial response. In the second stage, RUMAD organizes topology in stage 2.1 and the selected agents debate with each other in stage 2.2. In the last stage, the final decision is obtained via majority voting. The process pipeline of RUMAD. In the first stage, all agents gives the initial response. In the second stage, RUMAD organizes topology in stage 2.1 and the selected agents debate with each other in stage 2.2. In the last stage, the final decision is obtained via majority voting.
Figure 3. An example debate controlled by RUMAD. An example debate controlled by RUMAD.
Task
Method (6,6)
ACC
Token Cost (k/task)
Cost Saving
MMLU
RUMAD B=12
68%
11.43
-81.74%
RUMAD B=18
68%
22.46
-64.11%
MAD
49%
62.58
N/A
S-MAD ∗
61%
33.32
-46.75%
S-MAD ∘
54%
31.70
-49.34%
GD
53%
32.17
-48.59%
Table 2. Performance of RUMAD and baselines across three evaluation datasets. Token cost is calculated as average of each topic debated. (6,6) is general mark of MAD systems indicating 6-agent, 6-round configuration. RUMAD significantly reduces token cost with comparable or improved accuracy. S-MAD with different structures are denoted as * (Star) and o (Ring). Since RUMAD is trained upon MMLU dev partition, MMLU test partition is used for evaluation. RUMAD evaluated upon GPQA-main and GSM8K is zero-shot migrated, without any finetuning with new datasets. RUMAD shows superior cross-domain generality. Note: All baseline accuracies are measured under identical 4-bit quantization and 0-shot settings for a fair comparison.
Ablation Module
MMLU
GPQA
GSM8K
ACC
TC
ACC
TC
ACC
TC
RUMAD
68.3
11.4
31.9
19.5
86.4
10.5
β distribution
65.1
18.7
29.2
28.7
84.5
17.2
w/o Rep
66.9
15.6
31.0
24.4
85.1
14.3
w/o Rt
67.2
22.2
30.1
32.6
86.0
20.2
w/o agent activation
65.4
51.9
31.7
67.5
86.1
45.0
Table 3. Ablation study of RUMAD’s key components. We report accuracy (ACC, %) and average token cost (TC, in k/task). The best result for each metric is in bold . We set B=12 for all variants.
Agents ( N )
Budget ( B )
MMLU
GSM8K
ACC
TC
ACC
TC
3
6
66.6%
2.3
85.0%
2.0
6
12
68.3%
11.4
86.4%
10.5
6
18
68.5%
22.5
88.6%
17.3
9
18
70.3%
17.6
88.5%
15.9
Table 4. Scalability analysis with varying agent numbers. We report Accuracy (ACC) and average Token Cost (TC, k/task) on MMLU and GSM8K.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Symbol
Description
Value
α1 / β1
Accuracy
1.0/2.0
α2 / β2
Consensus
0.3/0.5
α3
Progress
0.1
α4 / β3
Efficiency
0.2/0.3
α5 / β4
Improvement
1.0/0.2
α6
Sparsity (penalty)
0.1
Appendix
Table 5. Weight parameter used in RUMAD multi-objective reward functions.
Symbol
Description
Value
dh
Hidden layer dimension
128
α
Learning rate
1×10−4
γ
Discount factor
0.99
λ
GAE parameter
0.95
ε
Policy clipping threshold
0.20
c1
Value loss coefficient
0.01
Appendix
Table 6. Training hyperparameters used for the PPO-based topology controller of RUMAD.
Task
Output Format Requirements
MMLU & GPQA
Put your final choice in the form \boxed{{answer}} at the end of your response.
GSM8K
Your final answer should be a single numerical number, in the Form \boxed{{answer}}, at the end of your response.
Appendix
Table 7. Output format requirements across different tasks.
Type
Task
Prompt
System
All
Welcome to the debate! You are a seasoned debater with expertise in succinctly and persuasively expressing your viewpoints. You will be assigned to debate groups, where you will engage in discussions with fellow participants. The outcomes of each group’s deliberations will be shared among all members. It is crucial for you to leverage this information effectively in order to critically analyze the question at hand and ultimately arrive at the correct answer. Best of luck!
Starting
All
Can you answer the following question as accurately as possible? < Question > Explain your answer. < Output Format > .
Debate
All
These are the solutions from other agents: < other agent responses > . Based on the above responses with their indicated importance, can you provide an updated answer? Examine all solutions step by step. < Output Format > .
Appendix
Table 8. Prompts used in each stage of the debate.
Multi-Agent Debate (MAD) is a collaborative framework in which multiple agents iteratively refine solutions through the generation of reasoning and alternating critique cycles. Current work primarily optimizes intra-round topologies and inter-round interactions separately, limiting the adaptation of token costs to task complexity. This work introduces Heterogeneous Consensus-Progressive Reasoning for Efficient Multi-Agent Debate (HCP-MAD), leveraging consensus as a dynamic signal to facilitate progressive reasoning. The core motivation is that a majority of straightforward tasks can be effectively resolved via lightweight pair-agent debates, while complex tasks require expanded collaboration. Firstly, Heterogeneous Consensus Verification conducts rapid consensus verification using a pair of heterogeneous agents for early stopping. Next, Heterogeneous Pair-Agent Debate applies an adaptive stopping criterion to terminate mutual critique of reasoning traces. Finally, the unresolved tasks are addressed through Escalated Collective Voting by aggregating diverse perspectives from additional agents. Experiments across six benchmarks show that HCP-MAD enhances accuracy while substantially reducing token costs. Code is https://github.com/fuyu66/HCP-MAD.
Multi-agent debate (MAD) improves the reasoning capabilities of large language models by having multiple agents iteratively refine their responses through discussion. However, MAD suffers from a critical vulnerability known as shared misconception: when a majority of agents initially converge on an incorrect answer, the debate process tends to amplify rather than correct the error. Existing methods primarily address peer skew but leave the agents' inherently biased concept priors unaddressed. To mitigate this systematic weakness, we propose R2-MAD (Remember and Reweight for Multi-Agent Debate), a framework that equips agents with an experience memory accumulated from past debates. R2-MAD intervenes on both failure modes through two complementary mechanisms: A debate-state-aware retrieval policy dynamically calibrates the concept prior by retrieving relevant historical evidence based on the current consensus level. Then these retrieved experiences provide a basis for estimating per-agent reliability, yielding confidence weights to modulate peer influence. Experiments on various benchmarks show that R2-MAD achieves consistent improvements over existing single-agent and MAD baselines.
Xuanfa Jin, Zhijian Ma, Yongcheng Zeng +3
Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences · University College London
Multi-agent debate has shown promise for improving the reasoning of large language models, yet recent theory suggests its benefits are highly regime-dependent. While interaction can amplify informative signals under corrective conditions, symmetric debate dynamics are neutral in expectation, often making majority voting preferable. We reconcile these views by arguing that debate is effective only when invoked at the right time and with appropriate structure. Based on this insight, we propose LASE: Leader-Adaptive Structured Engagement, a leader-centric multi-agent debate framework that selectively engages interaction only in non-neutral regimes. LASE introduces an asymmetric leader-supporter architecture that enables directed information flow and selective signal amplification, while defaulting to simple aggregation otherwise. Experiments across diverse reasoning benchmarks show that LASE achieves multi-agent-level performance with near single-agent token cost, substantially improving efficiency over static debate and voting baselines.
Seungdong Yoa, Ye Seul Sim, Suhee Yoon +5
LG AI Research, Seoul, South Korea · School of Computer Science and Engineering, Chung-Ang University, Seoul, South Korea · Division of Computer Engineering, Hankuk University of Foreign Studies, Yongin, South Korea +1