Efficient Multi-Agent Reasoning via Confidence-Guided Adaptive Debate

Seungdong Yoa; Ye Seul Sim; Suhee Yoon; Sanghyu Yoon; Dongmin Kim; Soonyoung Lee; Bumsoo Kim; Junhyun Lee

cs.AIOct 12, 2026

Efficient Multi-Agent Reasoning via Confidence-Guided Adaptive Debate

Authors: Seungdong Yoa, Ye Seul Sim, Suhee Yoon, Sanghyu Yoon, Dongmin Kim, Soonyoung Lee, Bumsoo Kim, Junhyun Lee

Organizations: LG AI Research, Seoul, South Korea · School of Computer Science and Engineering, Chung-Ang University, Seoul, South Korea · Division of Computer Engineering, Hankuk University of Foreign Studies, Yongin, South Korea · Noah’s Farm

Abstract

Multi-agent debate has shown promise for improving the reasoning of large language models, yet recent theory suggests its benefits are highly regime-dependent. While interaction can amplify informative signals under corrective conditions, symmetric debate dynamics are neutral in expectation, often making majority voting preferable. We reconcile these views by arguing that debate is effective only when invoked at the right time and with appropriate structure. Based on this insight, we propose LASE: Leader-Adaptive Structured Engagement, a leader-centric multi-agent debate framework that selectively engages interaction only in non-neutral regimes. LASE introduces an asymmetric leader-supporter architecture that enables directed information flow and selective signal amplification, while defaulting to simple aggregation otherwise. Experiments across diverse reasoning benchmarks show that LASE achieves multi-agent-level performance with near single-agent token cost, substantially improving efficiency over static debate and voting baselines.

Figures & tables

Panel of Figure 1 — Figure 1: Performance and Efficiency Overview. (Top) Relative performance comparison (Ours=100%). Ours consistently outperforms baselines across all tasks. (Bottom) Token cost multipliers vs. Single Agent. Ours incurs minimal overhead compared to the computationally expensive MAD.

Figure 2: Overall architectures of (a) Majority Voting, (b) Multi-Agent Debate and (c) Ours. (a) aggregates independent outputs, (b) applies uniform interaction while (c) selectively invokes structured, leader-centric interaction in corrective regimes where $U=1$ in Eq. (7).

		MATH-500 (Math)				MATH-lv5 (Hard Math)				GSM-Symbolic (Robustness)
Model	Metric	Single	MV	MAD	Ours	Single	MV	MAD	Ours	Single	MV	MAD	Ours
Gemini 2.0-Flash	Accuracy ( $\uparrow$ )	80.0	86.4	74.8	87.2	55.7	56.7	56.0	57.1	88.4	91.2	88.6	93.4
	Avg. Tokens ( $\downarrow$ )	0.7k	5.2k	11.8k	1.1k	1.0k	7.4k	20.3k	1.8k	0.4k	2.6k	5.6k	0.4k
	Total Tokens ( $\downarrow$ )	0.4M	2.6M	5.9M	0.5M	1.3M	9.7M	26.9M	2.4M	0.2M	1.3M	2.8M	0.2M
GPT 4.1-mini	Accuracy ( $\uparrow$ )	73.0	73.8	73.8	74.2	79.8	84.7	86.4	86.6	94.4	94.8	93.2	94.8
	Avg. Tokens ( $\downarrow$ )	1.0k	7.3k	16.1k	1.8k	1.6k	11.7k	29.1k	2.6k	0.3k	2.4k	5.9k	0.5k

Table 1: Main Results on Reasoning and Knowledge Benchmarks. We compare ours with Single Agent, Majority Vote (MV), and Multi-Agent Debate (MAD) baselines. Ours consistently achieves multi-agent level accuracy while maintaining near single-agent token efficiency. Best results are highlighted in bold (Accuracy: across all methods; Tokens: among multi-agent frameworks).

Method	$N$	Tokens	Acc.	$\Delta$
MV	7	5.2k	86.4	-
Ours (Single)	1	1.1k	87.2	+0.8
Ours (Majority Vote)	5	5.4k	88.0	+1.6

Table 2: Compatibility with Majority Voting (MATH-500). comparison of standard MV (

N=7

) vs. our ensemble variant ( Ours-Majority Vote ,

N=5

Panel of Figure 3 — Figure 3: Calibration Analysis of Confidence Proxies. We analyze the relationship between accuracy and our two uncertainty metrics: (a) Min Logprob and (b) Entropy. The dual-axis charts show sample distribution (bars) and accuracy trend (lines). Crucially, accuracy consistently declines as uncertainty increases, empirically validating our Monotone Calibration Assumption (Assumption 3.3 ) . This confirms that instances flagged as uncertain are indeed prone to errors, establishing these metrics as effective gating signals for our adaptive framework.

Setting	Accuracy (%)	Avg. Tokens
Random Run 1	86.8	1.0k
Random Run 2	87.2	1.2k
Random Run 3	86.6	1.1k
Random Avg. (3 runs)	86.9 ${\scriptstyle(\pm 0.31)}$	1.1k
Main Reported (Ref.)	87.2	1.1k

Table 3: Robustness to Agent Permutation (MATH-500). We report three independent runs where the supporting agent order was randomly shuffled for each query. The low standard deviation (0.31) and the minimal gap (

\Delta

0.3%) from our main reported score confirm the stability of our framework.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

1. Leading Agent
System Prompt:
You are a lead problem solver. Solve the problem step by step with careful reasoning. Put your final answer in \boxed{}.
User Prompt:
Problem: {problem}
Provide your solution step by step.
IMPORTANT: Put your final numerical answer inside \boxed{}. The answer must be a plain number only - NO units, NO currency symbols, NO thousands separators, and NO percentage signs. Just the raw number.

Appendix

Table 7

2. Supporting Agent 1
System Prompt:
You are a helpful collaborator. Help the problem solver by providing hints, alternative perspectives, or clarifications.
IMPORTANT RULES:
1. Do NOT solve the problem directly - guide them to the right approach.
2. Do NOT use \boxed{} in your response - that is reserved for the lead solver.
3. Focus on explaining concepts, pointing out where they went wrong, or suggesting strategies.

Appendix

Table 8

3. Supporting Agent 2
System Prompt:
You are a senior expert consultant. A team is struggling with a difficult problem. Your role is to provide expert guidance and insights, NOT to solve the problem for them.
IMPORTANT RULES:
1. Help them understand the key concepts they’re missing.
2. Point out flaws in their reasoning and suggest the right direction.
3. Do NOT provide the final answer directly.

Appendix

Table 9

4. Supporting Agent 3
System Prompt:
You are an aggressive Devil’s Advocate. Your job is to ATTACK the solution, not to validate it.
YOUR MISSION:
1. FIND THE WEAK POINT: Every solution has a vulnerability. Find it.
2. SET LOGICAL TRAPS: Ask questions that expose hidden assumptions or flawed reasoning.
3. DEMAND RIGOROUS PROOF: ’How do you KNOW this step is valid?’ ’Prove it.’

Appendix

Table 10

Method	Accuracy	Total Tokens
Single Agent	25.7%	0.3M
Majority Voting (MV)	31.8%	2.0M
Multi-Agent Debate (MAD)	24.1%	4.4M
Ours (LASE)	32.9%	1.4M

Appendix

Table 4: Performance comparison on MATH-500 using Llama-3-8B.

$\tau_{lp}$	$\tau_{h}$	Accuracy	Avg. Tokens	Esc Rate ( $U=1$ )
0.50	0.50	74.1%	1.8k	21.4%
0.60	0.50	74.8%	1.8k	21.2%
0.80	0.50	74.2%	1.8k	22.6%
0.90	0.50	74.8%	2.6k	30.8%
0.95	0.50	75.1%	3.1k	42.8%

Appendix

Table 5: Threshold sweep analysis on MATH-500 (GPT-4.1-mini).

Method	Accuracy	Total Tokens
Single Agent	73.0%	0.5M
Mixture-of-Agents (MoA)	72.0%	3.6M
Ours (LASE)	74.2%	0.9M

Appendix

Table 6: Performance and token cost comparison with Mixture-of-Agents (MoA) on MATH-500 using GPT-4.1-mini.

Explore similar work

RUMAD: Reinforcement-Unifying Multi-Agent Debate
Feb 27, 2026Chao Wang, Han Lin, Huaze Tang +2Multi-Agent Debate Multi-Agent Coordination
Tsinghua University, Shenzhen, China · Zhejiang University, Hangzhou, China
Remember and Reweight: Enhancing Multi-Agent Debate with Experience Memory and Confidence Estimation
Sep 3, 2026Xuanfa Jin, Zhijian Ma, Yongcheng Zeng +3AI Agent Reliability Multi-Agent Debate
Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences · University College London
Group Perspective Matters: Regulating Debate Relationships Can Mitigate Blind Conformity in Multi-Agent Debate
Aug 4, 2026Hao Wu, Shoucheng Song, Chang Yao +4Multi-Agent Debate RL for LLM Agents
School of Computer Science & Technology, Beijing Jiaotong University, Beijing, China · Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, Beijing, China

Feb 27, 2026J/K move · Enter open · S save

RUMAD: Reinforcement-Unifying Multi-Agent Debate

Chao Wang, Han Lin, Huaze Tang, Huijing Lin+1

Tsinghua University, Shenzhen, China · Zhejiang University, Hangzhou, China

Multi-Agent Debate Multi-Agent Coordination Multi-Agent Communication RL for LLM Agents

Open paper AskPDF arXiv

Multi-agent debate (MAD) systems leverage collective intelligence to enhance reasoning capabilities, yet existing approaches struggle to simultaneously optimize accuracy, consensus formation, and computational efficiency. Static topology methods lack adaptability to task complexity variations, while external LLM-based coordination risks introducing privileged knowledge that compromises debate neutrality. This work presents RUMAD (Reinforcement-Unifying Multi-Agent Debate), a novel framework that formulates dynamic communication topology control in MAD as a reinforcement learning (RL) problem. RUMAD employs a content-agnostic observation scheme that captures high-level debate dynamics avoiding access to raw agent reasoning content. RUMAD uses a multi-objective reward to model solution quality, cohesion and efficiency. A PPO-trained controller dynamically adjusts edge weights in the communication graph, while a dual-threshold mechanism enables fine-grained control over both agent activation and information visibility. Experimental evaluation across MMLU, GSM8K, and GPQA benchmarks demonstrates that RUMAD achieves substantial efficiency gains, reducing token costs by over 80%, while still improving reasoning accuracy compared to single LLM model and multiple MAD baselines. Notably, RUMAD trained exclusively on MMLU exhibits robust zero-shot generalization to out-of-domain (OOD) tasks, indicating that the learned communication strategies capture task-independent principles of effective multi-agent coordination. These results establish RUMAD as a efficient and robust approach for deploying multi-agent reasoning application with practical resource constraints.

Abstract

Explore similar work

RUMAD: Reinforcement-Unifying Multi-Agent Debate

Remember and Reweight: Enhancing Multi-Agent Debate with Experience Memory and Confidence Estimation

Group Perspective Matters: Regulating Debate Relationships Can Mitigate Blind Conformity in Multi-Agent Debate