Multi-Agent Debate for Explainable Trading: Reasoning, Consensus, and Performance in Simulated Markets

Juli Huang; Alanood Alrassan; Deveen Harischandra; Theodore Wu; Veljko Skarich; Matthew Hayes

cs.MAAug 31, 2026

Multi-Agent Debate for Explainable Trading: Reasoning, Consensus, and Performance in Simulated Markets

Authors: Juli Huang, Alanood Alrassan, Deveen Harischandra, Theodore Wu, Veljko Skarich, Matthew Hayes

Organizations: Department of Engineering, Stanford University

Abstract

Large language models (LLMs) are increasingly used for financial decision-making, yet it remains unclear whether improvements in reasoning quality translate into better economic outcomes. We investigate this question using a multi-agent debate framework for portfolio allocation in historical market simulations, where specialized agents propose, critique, and revise investment decisions. Reasoning quality is evaluated across four dimensions: logical validity, evidential support, alternative consideration, and causal alignment, and compared with downstream financial performance. Across 210 controlled runs, aggregate reasoning quality shows no meaningful relationship with Sharpe ratio (r = 0.07, p = 0.29) or total return (r = 0.03, p = 0.70). Structured prompting increases measured reasoning quality from about 0.72 to 0.84 (+17.7%, Cohen's d about 2.0), but these gains do not consistently translate into higher returns. We identify sycophantic convergence as a central failure mode, where agents abandon independent positions during critique-revision cycles and converge toward similar allocations. A Jensen-Shannon divergence intervention that preserves disagreement improves Sharpe by +0.14 (p = 0.028) and Sortino by +0.25 (p = 0.026), while interventions enforcing stronger causal reasoning do not improve financial performance. Our results suggest that multi-agent debate is most valuable when it preserves independent informational signals rather than simply improving measured reasoning quality.

Figures & tables

Analysis	r	p	n
$\bar{\rho}$ vs Sharpe	$+0.07$	0.29	210
$\bar{\rho}$ vs Return	$+0.03$	0.70	210
$\Delta\rho$ vs $\Delta$ Sharpe (paired)	$-0.02$	0.87	105
$\Delta\rho$ vs $\Delta$ Return (paired)	$+0.01$	0.91	105

Table 1: Pearson correlation between reasoning quality and financial outcomes.

Metric	(A) Single Agent Basic	(B) 3 Roles Mean, Basic	(C) 3 Roles Mean, Causal	(D) 3 Roles Mean, Enriched
Excess Return % (v SPY)	$0.77\%\pm 1.03\%$	${0.65\%\pm 0.93\%}^{C}$	${1.12\%\pm 0.93\%}^{BD}$	${0.57\%\pm 0.91\%}^{C}$
Sharpe	$1.00\pm 0.30$	${0.99\pm 0.30}^{C}$	${1.07\pm 0.30}^{B}$	$1.04\pm 0.30$
Volatility	${0.22\pm 0.01}^{C\mathbf{D}}$	${0.21\pm 0.01}^{\mathbf{D}}$	${0.21\pm 0.01}^{A\mathbf{D}}$	${0.18\pm 0.01}^{\mathbf{A}\mathbf{B}\mathbf{C}}$
Max Drawdown %	${9.84\%\pm 0.97\%}^{C\mathbf{D}}$	${9.60\%\pm 0.93\%}^{C\mathbf{D}}$	${9.33\%\pm 0.90\%}^{AB\mathbf{D}}$	${8.29\%\pm 0.77\%}^{\mathbf{A}\mathbf{B}\mathbf{C}}$
Sortino	$1.76\pm 0.46$	${1.75\pm 0.47}^{C}$	${1.87\pm 0.47}^{B}$	$1.80\pm 0.47$
Num Positions	${10.77\pm 1.47}^{B}$	${11.37\pm 1.55}^{ACD}$	${10.83\pm 1.41}^{BD}$	${10.17\pm 1.21}^{BC}$

Table 2: Mean over 35 quarter+tickers scenarios

\pm

SEM. Superscripts indicate stat. sig. in paired t-test with the specified column(s). Unbolded:

p<0.15

, Bolded :

p<0.01

Configuration	N	Return %	Sharpe	Sortino	Max DD %	Excess vs SPY %
Single Agent	3	$+1.59\pm 0.99$	$0.35\pm 0.17$	$0.52\pm 0.25$	$9.73\pm 1.90$	$-1.01\pm 3.12$
Multi-Agent Mean	3	$+2.43\pm 1.23$	$0.58\pm 0.30$	$0.88\pm 0.46$	$8.61\pm 1.95$	$-0.13\pm 3.09$
Enriched Debate (3-ag.)	3	$+2.46\pm 1.74$	$0.76\pm 0.48$	$1.16\pm 0.75$	$5.33\pm 0.71$	$+0.04\pm 4.04$
Enriched Debate (4-ag.)	2	$-0.27\pm 0.29$	$-0.05\pm 0.06$	$-0.07\pm 0.09$	$6.09\pm 0.82$	$+0.14\pm 4.47$

Table 3: Financial performance across configurations (mean

\pm

SEM). Excess vs SPY is portfolio return minus S&P 500 return over the same quarter. Enriched 3-agent debate achieves the best risk-adjusted returns (highest Sharpe and Sortino, lowest max drawdown). The 4-agent 5-round debate exhibits negative returns, suggesting over-convergence degrades performance.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 1: Our simulation pipeline builds data cases from input financial sources, orchestrates multi-agent debate in a market environment, then scores the decision-making through financial performance and reasoning quality (left). LLM agents with clear roles participate in Propose-Critique-Revise debate loops. The revision tone is adjusted by PID control (center). Input sources are summarized and formatted into quarterly snapshot memos Appendix B.19 .

Metric	(A) 3 Roles Mean, Causal	(B) 3 Roles Debate, Causal	(C) 3 Roles Debate, Causal + RCA
Excess Return % (v SPY)	$1.12\%\pm 0.93\%$	$0.95\%\pm 0.96\%$	$0.80\%\pm 0.88\%$
Sharpe	$1.07\pm 0.30$	$1.01\pm 0.30$	$1.03\pm 0.30$
Volatility	${0.21\pm 0.01}^{\mathbf{B}\mathbf{C}}$	${0.20\pm 0.01}^{\mathbf{A}C}$	${0.19\pm 0.01}^{\mathbf{A}B}$
Max Drawdown %	${9.33\%\pm 0.90\%}^{B\mathbf{C}}$	${8.88\%\pm 0.83\%}^{AC}$	${8.64\%\pm 0.80\%}^{\mathbf{A}B}$
Sortino	$1.87\pm 0.47$	$1.82\pm 0.49$	$1.80\pm 0.47$
Num Positions	$10.83\pm 1.41$	$11.17\pm 1.49$	$11.14\pm 1.57$

Appendix

Table 4: Mean over 35 quarter+tickers scenarios

\pm

SEM. Superscripts indicate stat. sig. in paired t-test with the specified column(s). Unbolded:

p<0.15

, Bolded :

p<0.01

Scenario	Return %	Sharpe	Sortino	Max DD %	SPY %	Excess %
2022 Q4 Tech Sector	$-0.35$	$-0.13$	$-0.20$	$5.96$	$+4.79$	$-5.14$
2022 Q1 Inflation Shock	$+2.83$	$0.89$	$1.32$	$6.11$	$-5.16$	$+7.99$
2023 Q2 Higher-for-Longer	$+5.52$	$1.52$	$2.37$	$3.91$	$+8.27$	$-2.75$
Average	$+2.67$	$0.76$	$1.16$	$5.33$	$+2.63$	$+0.04$

Appendix

Table 5: Per-scenario results for 3-agent enriched debate (Config 3). Performance varies substantially across market regimes: the system outperforms SPY by

+7.99\%

during the 2022 Q1 inflation shock but underperforms by

-5.14\%

in the 2022 Q4 tech rally.

Round	$\bar{\rho}$	$P_{1}$ (LV)	$P_{2}$ (ES)	$P_{3}$ (AC)	$P_{4}$ (CA)
1	0.843	0.86	0.86	0.87	0.78
2	0.845	0.87	0.85	0.86	0.81
3	0.815	0.84	0.83	0.80	0.75
4	0.832	0.77	0.86	0.88	0.81
5	0.839	0.82	0.86	0.86	0.79

Appendix

Table 6: CRIT pillar scores across debate rounds (4-agent enriched debate, 2022 Q4 tech scenario).

P_{1}

=Logical Validity,

P_{2}

=Evidential Support,

P_{3}

=Alternative Consideration,

P_{4}

=Causal Alignment. The most common CRIT diagnostic is causal_overreach , particularly for the Technical agent.

Scenario	Exp	Return %	Sharpe	Sortino	Max DD %	Excess %	Status
2021 Q3 Inflation	A	$+12.40$	$3.05$	$4.65$	$4.61$	$+2.64$	OK
	B	-	-	-	-	-	Fail
2022 Q1 Inf. Shock	A	$-19.40$	$-2.56$	$-3.16$	$19.96$	$-3.05$	OK
	B	$-19.40$	$-2.56$	$-3.16$	$19.96$	$-3.05$	OK
2022 Q2 Recession	A	$-1.97$	$-0.35$	$-0.50$	$12.69$	$+3.96$	OK
	B	$-1.77$	$-0.32$	$-0.45$	$12.97$	$+4.16$	OK

Appendix

Table 7: Per-scenario LLM comparison. Exp A uses GPT-5-mini for all roles; Exp B uses Gemini-2.5-flash for Macro/Value/Judge and GPT-5-mini for Risk. Exp B failures are caused by Gemini hallucinating ticker JNJ (not in scenario universe). Where both complete, returns are comparable- e.g., 2022 Q1 produces byte-identical portfolios.

Metric	Value
Scenario	2022 Q1 Inflation Shock (10 tickers)
Return	$-16.22\%$
Sharpe	$-2.11$
Sortino	$-2.63$
Max Drawdown	$17.67\%$
SPY Return	$-16.35\%$

Appendix

Table 8: Phase-LLM experiment results (GPT-5-mini for propose/revise, Gemini-2.5-flash critique/judge). Tracked SPY closely during a severe downturn, with excess return of

+0.13\%

Metric	Mean	Best	Worst
Return (%)	-1.11	21.71	-70.10
Excess Return vs SPY (%)	1.01	11.58	-6.41
Max Drawdown (%)	7.87	1.68	16.39
Sharpe Ratio	1.06	4.38	-2.73
Trades	8.8	21	1

Appendix

Table 9: Ablation performance across 17 market scenarios.

Experiment	Return (%)	Excess vs SPY (%)	Sharpe	Final JS
Exp 1 Role-only (Macro Anthropic, Risk GPT)	-1.42	3.16	-0.54	0.7181
Exp 2 Role-only (Macro GPT, Risk Anthropic)	-2.37	2.26	-0.64	0.7312
Exp 3 Hybrid phases	-1.70	2.84	-0.53	0.7946

Appendix

Table 10: Performance comparison across LLM routing experiments.

Abl.	Config	N	Ret	Sharpe	Sortino	MaxDD	Excess vs SPY
A7	Base	35	$+1.79\pm 1.50$	$+0.50\pm 0.28$	$+0.93\pm 0.43$	$+10.03\pm 0.88$	$-1.73\pm 1.03$
A7	+JS	35	$+2.62\pm 1.61$	$\mathbf{+0.64\pm 0.29}$	$\mathbf{+1.19\pm 0.46}$	$+10.21\pm 0.95$	$-0.90\pm 1.08$
A8	Base	35	$+1.46\pm 1.21$	$+0.42\pm 0.28$	$+0.81\pm 0.44$	$+8.07\pm 0.71$	$-2.06\pm 0.87$
A8	+CRIT	35	$+1.61\pm 1.26$	$+0.46\pm 0.28$	$+0.86\pm 0.43$	$+8.27\pm 0.78$	$-1.90\pm 0.94$
A10	Base	35	$+2.61\pm 1.54$	$+0.66\pm 0.28$	$+1.18\pm 0.44$	$+9.80\pm 0.87$	$-0.91\pm 1.04$
A10	+JS	35	$+2.41\pm 1.59$	$+0.57\pm 0.27$	$+1.04\pm 0.42$	$\mathbf{+10.33\pm 0.93}$	$-1.11\pm 1.07$

Appendix

Table 11: Performance metrics across intervention ablations. Three ablations evaluate reasoning interventions in the multi-agent debate system. A7 introduces a Jensen–Shannon (JS) collapse detector in a two-agent setting ( macro , technical ), triggering when revision-stage disagreement falls below a threshold (

JS_{\text{rev}}/JS_{\text{prop}}<0.8

) and nudging agents to maintain independent reasoning. A8 applies the same JS intervention with an added risk agent (three agents total); the intervention fired frequently (97% of debates) but produced no measurable effect. A10 replaces the JS trigger with a CRIT-based reasoning intervention that fires when causal reasoning quality

(\rho_{4}<0.8)

Abl.	Metric	Diff	95% CI	t	p	d
A7	Sharpe	$+0.143$	$[+0.017,\;+0.270]$	$+2.300$	0.0277	$+0.389$
A7	Sortino	$+0.254$	$[+0.032,\;+0.476]$	$+2.328$	0.0260	$+0.393$
A7	Excess vs SPY %	$+0.828$	$[-0.009,\;+1.666]$	$+2.011$	0.0523	$+0.340$
A10	Max DD %	$+0.529$	$[-0.028,\;+1.086]$	$+1.930$	0.0620	$+0.326$

Appendix

Table 12: Paired t-test results for ablations with statistically significant or marginal effects (

p<0.10

). Green p-values indicate improvement relative to the baseline configuration, red indicates undesired outcomes.

Tier	n	$\boldsymbol{\bar{\rho}}$	Sharpe	Return (%)
Standard	56	0.833	-0.918	-3.56
Light	62	0.825	-0.940	-3.58
Intense Light	67	0.819	-0.992	-3.66
Intense	60	0.834	-0.967	-3.75

Appendix

Table 13: Prompt enrichment tiers (Standard

\rightarrow

Intense) show no improvement in reasoning or financial performance across 245 runs of Ablation 1.

Subgroup	Metric	r	p	n
Value agent $\rho$ vs Sharpe	Pearson	$-0.29$	0.001	126
Risk agent $\rho$ vs Return	Pearson	$+0.22$	0.006	152

Appendix

Table 14: Selected subgroup correlations between agent reasoning quality and performance.

Ablation	n	Debate Return	EW Return	Diff	Beats EW
Abl 1	245	$-3.64\%$	$-3.28\%$	$-0.36\%$	31%
Abl 7	70	$+2.20\%$	$+4.47\%$	$-2.26\%$	30%
Abl 8	70	$+1.54\%$	$+4.47\%$	$-2.93\%$	34%
Abl 10	70	$+2.51\%$	$+4.47\%$	$-1.96\%$	40%
Pooled	455	—	—	$\mathbf{-1.29\%}$	33%

Appendix

Table 15: Debate portfolio performance compared to an equal-weight baseline.

Pillar	Baseline	Enriched	$\Delta$	Cohen’s $d$
Evidential Support	0.51	0.85	$+0.34$	2.92
Causal Alignment	0.70	0.87	$+0.17$	2.21
Alternative Consideration	0.79	0.89	$+0.10$	1.09
Logical Validity	0.87	0.82	$-0.05$	$-0.44$

Appendix

Table 16: Improvements in CRIT reasoning pillars under structured prompts.

Figure 3: Dashboard Aggregate Stats View.

Multi-Agent Debate

Multi-Agent Debate studies how language-model agents exchange arguments, critique answers, and combine judgments for collective reasoning.

See all 144 papers

Mar 29, 2026

Toward Reliable Evaluation of LLM-Based Financial Multi-Agent Systems: Taxonomy, Coordination Primacy, and Cost Awareness

Multi-agent systems based on large language models (LLMs) for financial trading have grown rapidly since 2023, yet the field lacks a shared framework for understanding what drives performance or for evaluating claims credibly. This survey makes three contributions. First, we introduce a four-dimensional taxonomy, covering architecture pattern, coordination mechanism, memory architecture, and tool integration; applied to 12 multi-agent systems and two single-agent baselines. Second, we formulate the Coordination Primacy Hypothesis (CPH): inter-agent coordination protocol design is a primary driver of trading decision quality, often exerting greater influence than model scaling. CPH is presented as a falsifiable research hypothesis supported by tiered structural evidence rather than as an empirically validated conclusion; its definitive validation requires evaluation infrastructure that does not yet exist in the field. Third, we document five pervasive evaluation failures (look-ahead bias, survivorship bias, backtesting overfitting, transaction cost neglect, and regime-shift blindness) and show that these can reverse the sign of reported returns. Building on the CPH and the evaluation critique, we introduce the Coordination Breakeven Spread (CBS), a metric for determining whether multi-agent coordination adds genuine value net of transaction costs, and propose minimum evaluation standards as prerequisites for validating the CPH.

Phat Nguyen, Thang Pham

Georgia Institute of Technology, Atlanta GA 30332, USA · Adobe Inc., San Jose CA 95110, USA

Multi-Agent LLMs LLM Agent Evaluation Multi-Agent Coordination Algorithmic Trading Quantitative Finance

Ask PDF

May 31, 2026

FinCom: A Financial Multi-Agent Demo with Disagree-or-Commit Deliberation

Multi-agent systems powered by large language models (LLMs) are increasingly used for financial analysis and decision support. However, existing coordination schemes, especially those emphasizing consensus or debate, are vulnerable to sycophancy: agents conform to peer reasoning instead of evidence, leading to premature agreement and degraded outcomes. We introduce FinCom (Financial Committee), a governed multi-agent framework and interactive system that operationalizes the Disagree-or-Commit (DoC) protocol to embed structured dissent into financial AI committees. A central Supervisor orchestrates three ReAct-enabled specialist agents: Research, Quantitative, and Risk. Each agent is equipped with role-specific tools for retrieval, computation, and stress testing. During deliberation, agents must either explicitly critique or commit to their peers' reasoning before converging on a unified recommendation. This demonstration showcases how FinCom supports committee-style financial analysis through coordinated multi-agent interaction, including structured report generation and interactive decision support. Evaluated across the most recent financial agent benchmark, in addition to 90 internal handcrafted financial tasks using an LLM-as-a-Judge protocol, DoC improves reasoning accuracy and risk awareness significantly over a consensus-seeking baseline on both an in-house and external evaluation set. By reframing disagreement as a governance primitive rather than noise, FinCom offers a lightweight, prompt-only recipe for improving accountability, transparency, and epistemic robustness in agentic financial systems.

Chao Peter Yang, Zixiao Tan, Kaisen Yao +3

Duke University United States · ClearPath United States

LLM Sycophancy LLM Agent Orchestration Multi-Agent Debate AI Agent Governance

Ask PDF

Sep 25, 2026

The Price of Thought: Does Test-Time Reasoning Pay in LLM Trading?

While inference-time reasoning in large language models (LLMs) promises better decision making, its higher computational cost may not yield better economic outcomes. Yet reasoning controls are rarely evaluated as economic interventions, where changes in model outputs must translate into better portfolios after trading costs. We conduct a controlled study of representative LLMs from the DeepSeek, GPT, and Gemini families. We vary reasoning effort while holding information available at each formation date, prompts, output formats, and portfolio construction fixed. Our evaluation covers a full year of U.S. equities under three input conditions: numerical, identifiable news, and masked news. It includes more than 800,000 asset predictions and repeated model generations. Across all three model families, additional reasoning does not produce a reliable improvement in net portfolio returns. For DeepSeek, where we examine the full progression from no reasoning to maximum reasoning, performance is nonmonotonic. Repeated generations also produce unstable treatment effects and portfolio selections, even when overall scores remain similar. These findings show that additional reasoning can change financial decisions without reliably improving their economic value, motivating validation for each task before deployment.

Jiayi Chen, Guiling Wang

Department of Computer Science New Jersey Institute of Technology Newark, New Jersey, USA

LLM Reasoning Efficient Language Model Reasoning LLM Reliability

Ask PDF

Abstract

Multi-Agent Debate

Explore similar work

Toward Reliable Evaluation of LLM-Based Financial Multi-Agent Systems: Taxonomy, Coordination Primacy, and Cost Awareness

FinCom: A Financial Multi-Agent Demo with Disagree-or-Commit Deliberation

The Price of Thought: Does Test-Time Reasoning Pay in LLM Trading?