Multi-agent debate (MAD) systems leverage collective intelligence to enhance reasoning capabilities, yet existing approaches struggle to simultaneously optimize accuracy, consensus formation, and computational efficiency. Static topology methods lack adaptability to task complexity variations, while external LLM-based coordination risks introducing privileged knowledge that compromises debate neutrality. This work presents RUMAD (Reinforcement-Unifying Multi-Agent Debate), a novel framework that formulates dynamic communication topology control in MAD as a reinforcement learning (RL) problem. RUMAD employs a content-agnostic observation scheme that captures high-level debate dynamics avoiding access to raw agent reasoning content. RUMAD uses a multi-objective reward to model solution quality, cohesion and efficiency. A PPO-trained controller dynamically adjusts edge weights in the communication graph, while a dual-threshold mechanism enables fine-grained control over both agent activation and information visibility. Experimental evaluation across MMLU, GSM8K, and GPQA benchmarks demonstrates that RUMAD achieves substantial efficiency gains, reducing token costs by over 80%, while still improving reasoning accuracy compared to single LLM model and multiple MAD baselines. Notably, RUMAD trained exclusively on MMLU exhibits robust zero-shot generalization to out-of-domain (OOD) tasks, indicating that the learned communication strategies capture task-independent principles of effective multi-agent coordination. These results establish RUMAD as a efficient and robust approach for deploying multi-agent reasoning application with practical resource constraints.
Figures & tables
Figure 1. Comparison of accuracy and average token cost. Accuracy for MMLU ( ∙ ) and GSM8K ( ■ ) corresponds to the left Y-axis, while GPQA ( ▲ ) corresponds to the right Y-axis. Points closer to the top-left corner represent a superior accuracy-efficiency trade-off. The results show that RUMAD variants consistently outperform baselines by achieving higher accuracy at a lower token cost. Comparison of accuracy and token consumption. The results show that RUMAD achieves better trade-off between accuracy and efficiency.
Static Topology Methods
External-LLM Methods
RUMAD
Accuracy
∙
∙
∙
Consensus
∘
∙
∙
External Opinion Independent
∙
-
∙
Question Independent
∙
-
∙
Communication Cost Control
∘
∘
∙
Table 1. Different types of MAD methods are designed from different strategies. Mark ∙ indicates corresponding metrics are taken into design target. Mark ∘ indicates methods may have side-effect on corresponding metrics but don’t focus on them. Mark - means methods fail on corresponding aspects.
Figure 2. The process pipeline of RUMAD. In the first stage, all agents gives the initial response. In the second stage, RUMAD organizes topology in stage 2.1 and the selected agents debate with each other in stage 2.2. In the last stage, the final decision is obtained via majority voting. The process pipeline of RUMAD. In the first stage, all agents gives the initial response. In the second stage, RUMAD organizes topology in stage 2.1 and the selected agents debate with each other in stage 2.2. In the last stage, the final decision is obtained via majority voting.
Figure 3. An example debate controlled by RUMAD. An example debate controlled by RUMAD.
Task
Method (6,6)
ACC
Token Cost (k/task)
Cost Saving
MMLU
RUMAD B=12
68%
11.43
-81.74%
RUMAD B=18
68%
22.46
-64.11%
MAD
49%
62.58
N/A
S-MAD ∗
61%
33.32
-46.75%
S-MAD ∘
54%
31.70
-49.34%
GD
53%
32.17
-48.59%
Table 2. Performance of RUMAD and baselines across three evaluation datasets. Token cost is calculated as average of each topic debated. (6,6) is general mark of MAD systems indicating 6-agent, 6-round configuration. RUMAD significantly reduces token cost with comparable or improved accuracy. S-MAD with different structures are denoted as * (Star) and o (Ring). Since RUMAD is trained upon MMLU dev partition, MMLU test partition is used for evaluation. RUMAD evaluated upon GPQA-main and GSM8K is zero-shot migrated, without any finetuning with new datasets. RUMAD shows superior cross-domain generality. Note: All baseline accuracies are measured under identical 4-bit quantization and 0-shot settings for a fair comparison.
Ablation Module
MMLU
GPQA
GSM8K
ACC
TC
ACC
TC
ACC
TC
RUMAD
68.3
11.4
31.9
19.5
86.4
10.5
β distribution
65.1
18.7
29.2
28.7
84.5
17.2
w/o Rep
66.9
15.6
31.0
24.4
85.1
14.3
w/o Rt
67.2
22.2
30.1
32.6
86.0
20.2
w/o agent activation
65.4
51.9
31.7
67.5
86.1
45.0
Table 3. Ablation study of RUMAD’s key components. We report accuracy (ACC, %) and average token cost (TC, in k/task). The best result for each metric is in bold . We set B=12 for all variants.
Agents ( N )
Budget ( B )
MMLU
GSM8K
ACC
TC
ACC
TC
3
6
66.6%
2.3
85.0%
2.0
6
12
68.3%
11.4
86.4%
10.5
6
18
68.5%
22.5
88.6%
17.3
9
18
70.3%
17.6
88.5%
15.9
Table 4. Scalability analysis with varying agent numbers. We report Accuracy (ACC) and average Token Cost (TC, k/task) on MMLU and GSM8K.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Symbol
Description
Value
α1 / β1
Accuracy
1.0/2.0
α2 / β2
Consensus
0.3/0.5
α3
Progress
0.1
α4 / β3
Efficiency
0.2/0.3
α5 / β4
Improvement
1.0/0.2
α6
Sparsity (penalty)
0.1
Appendix
Table 5. Weight parameter used in RUMAD multi-objective reward functions.
Symbol
Description
Value
dh
Hidden layer dimension
128
α
Learning rate
1×10−4
γ
Discount factor
0.99
λ
GAE parameter
0.95
ε
Policy clipping threshold
0.20
c1
Value loss coefficient
0.01
Appendix
Table 6. Training hyperparameters used for the PPO-based topology controller of RUMAD.
Task
Output Format Requirements
MMLU & GPQA
Put your final choice in the form \boxed{{answer}} at the end of your response.
GSM8K
Your final answer should be a single numerical number, in the Form \boxed{{answer}}, at the end of your response.
Appendix
Table 7. Output format requirements across different tasks.
Type
Task
Prompt
System
All
Welcome to the debate! You are a seasoned debater with expertise in succinctly and persuasively expressing your viewpoints. You will be assigned to debate groups, where you will engage in discussions with fellow participants. The outcomes of each group’s deliberations will be shared among all members. It is crucial for you to leverage this information effectively in order to critically analyze the question at hand and ultimately arrive at the correct answer. Best of luck!
Starting
All
Can you answer the following question as accurately as possible? < Question > Explain your answer. < Output Format > .
Debate
All
These are the solutions from other agents: < other agent responses > . Based on the above responses with their indicated importance, can you provide an updated answer? Examine all solutions step by step. < Output Format > .
Appendix
Table 8. Prompts used in each stage of the debate.
LG AI Research, Seoul, South Korea · School of Computer Science and Engineering, Chung-Ang University, Seoul, South Korea · Division of Computer Engineering, Hankuk University of Foreign Studies, Yongin, South Korea +1