Multi-Agent Debate for Explainable Trading: Reasoning, Consensus, and Performance in Simulated Markets

Juli Huang; Alanood Alrassan; Deveen Harischandra; Theodore Wu; Veljko Skarich; Matthew Hayes

cs.MAAug 31, 2026

Multi-Agent Debate for Explainable Trading: Reasoning, Consensus, and Performance in Simulated Markets

Authors: Juli Huang, Alanood Alrassan, Deveen Harischandra, Theodore Wu, Veljko Skarich, Matthew Hayes

Organizations: Department of Engineering, Stanford University

Abstract

Large language models (LLMs) are increasingly used for financial decision-making, yet it remains unclear whether improvements in reasoning quality translate into better economic outcomes. We investigate this question using a multi-agent debate framework for portfolio allocation in historical market simulations, where specialized agents propose, critique, and revise investment decisions. Reasoning quality is evaluated across four dimensions: logical validity, evidential support, alternative consideration, and causal alignment, and compared with downstream financial performance. Across 210 controlled runs, aggregate reasoning quality shows no meaningful relationship with Sharpe ratio (r = 0.07, p = 0.29) or total return (r = 0.03, p = 0.70). Structured prompting increases measured reasoning quality from about 0.72 to 0.84 (+17.7%, Cohen's d about 2.0), but these gains do not consistently translate into higher returns. We identify sycophantic convergence as a central failure mode, where agents abandon independent positions during critique-revision cycles and converge toward similar allocations. A Jensen-Shannon divergence intervention that preserves disagreement improves Sharpe by +0.14 (p = 0.028) and Sortino by +0.25 (p = 0.026), while interventions enforcing stronger causal reasoning do not improve financial performance. Our results suggest that multi-agent debate is most valuable when it preserves independent informational signals rather than simply improving measured reasoning quality.

Figures & tables

Analysis	r	p	n
$\bar{\rho}$ vs Sharpe	$+0.07$	0.29	210
$\bar{\rho}$ vs Return	$+0.03$	0.70	210
$\Delta\rho$ vs $\Delta$ Sharpe (paired)	$-0.02$	0.87	105
$\Delta\rho$ vs $\Delta$ Return (paired)	$+0.01$	0.91	105

Table 1: Pearson correlation between reasoning quality and financial outcomes.

Metric	(A) Single Agent Basic	(B) 3 Roles Mean, Basic	(C) 3 Roles Mean, Causal	(D) 3 Roles Mean, Enriched
Excess Return % (v SPY)	$0.77\%\pm 1.03\%$	${0.65\%\pm 0.93\%}^{C}$	${1.12\%\pm 0.93\%}^{BD}$	${0.57\%\pm 0.91\%}^{C}$
Sharpe	$1.00\pm 0.30$	${0.99\pm 0.30}^{C}$	${1.07\pm 0.30}^{B}$	$1.04\pm 0.30$
Volatility	${0.22\pm 0.01}^{C\mathbf{D}}$	${0.21\pm 0.01}^{\mathbf{D}}$	${0.21\pm 0.01}^{A\mathbf{D}}$	${0.18\pm 0.01}^{\mathbf{A}\mathbf{B}\mathbf{C}}$
Max Drawdown %	${9.84\%\pm 0.97\%}^{C\mathbf{D}}$	${9.60\%\pm 0.93\%}^{C\mathbf{D}}$	${9.33\%\pm 0.90\%}^{AB\mathbf{D}}$	${8.29\%\pm 0.77\%}^{\mathbf{A}\mathbf{B}\mathbf{C}}$
Sortino	$1.76\pm 0.46$	${1.75\pm 0.47}^{C}$	${1.87\pm 0.47}^{B}$	$1.80\pm 0.47$
Num Positions	${10.77\pm 1.47}^{B}$	${11.37\pm 1.55}^{ACD}$	${10.83\pm 1.41}^{BD}$	${10.17\pm 1.21}^{BC}$

Table 2: Mean over 35 quarter+tickers scenarios

\pm

SEM. Superscripts indicate stat. sig. in paired t-test with the specified column(s). Unbolded:

p<0.15

, Bolded :

p<0.01

Configuration	N	Return %	Sharpe	Sortino	Max DD %	Excess vs SPY %
Single Agent	3	$+1.59\pm 0.99$	$0.35\pm 0.17$	$0.52\pm 0.25$	$9.73\pm 1.90$	$-1.01\pm 3.12$
Multi-Agent Mean	3	$+2.43\pm 1.23$	$0.58\pm 0.30$	$0.88\pm 0.46$	$8.61\pm 1.95$	$-0.13\pm 3.09$
Enriched Debate (3-ag.)	3	$+2.46\pm 1.74$	$0.76\pm 0.48$	$1.16\pm 0.75$	$5.33\pm 0.71$	$+0.04\pm 4.04$
Enriched Debate (4-ag.)	2	$-0.27\pm 0.29$	$-0.05\pm 0.06$	$-0.07\pm 0.09$	$6.09\pm 0.82$	$+0.14\pm 4.47$

Table 3: Financial performance across configurations (mean

\pm

SEM). Excess vs SPY is portfolio return minus S&P 500 return over the same quarter. Enriched 3-agent debate achieves the best risk-adjusted returns (highest Sharpe and Sortino, lowest max drawdown). The 4-agent 5-round debate exhibits negative returns, suggesting over-convergence degrades performance.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 1: Our simulation pipeline builds data cases from input financial sources, orchestrates multi-agent debate in a market environment, then scores the decision-making through financial performance and reasoning quality (left). LLM agents with clear roles participate in Propose-Critique-Revise debate loops. The revision tone is adjusted by PID control (center). Input sources are summarized and formatted into quarterly snapshot memos Appendix B.19 .

Metric	(A) 3 Roles Mean, Causal	(B) 3 Roles Debate, Causal	(C) 3 Roles Debate, Causal + RCA
Excess Return % (v SPY)	$1.12\%\pm 0.93\%$	$0.95\%\pm 0.96\%$	$0.80\%\pm 0.88\%$
Sharpe	$1.07\pm 0.30$	$1.01\pm 0.30$	$1.03\pm 0.30$
Volatility	${0.21\pm 0.01}^{\mathbf{B}\mathbf{C}}$	${0.20\pm 0.01}^{\mathbf{A}C}$	${0.19\pm 0.01}^{\mathbf{A}B}$
Max Drawdown %	${9.33\%\pm 0.90\%}^{B\mathbf{C}}$	${8.88\%\pm 0.83\%}^{AC}$	${8.64\%\pm 0.80\%}^{\mathbf{A}B}$
Sortino	$1.87\pm 0.47$	$1.82\pm 0.49$	$1.80\pm 0.47$
Num Positions	$10.83\pm 1.41$	$11.17\pm 1.49$	$11.14\pm 1.57$

Appendix

Table 4: Mean over 35 quarter+tickers scenarios

\pm

SEM. Superscripts indicate stat. sig. in paired t-test with the specified column(s). Unbolded:

p<0.15

, Bolded :

p<0.01

Scenario	Return %	Sharpe	Sortino	Max DD %	SPY %	Excess %
2022 Q4 Tech Sector	$-0.35$	$-0.13$	$-0.20$	$5.96$	$+4.79$	$-5.14$
2022 Q1 Inflation Shock	$+2.83$	$0.89$	$1.32$	$6.11$	$-5.16$	$+7.99$
2023 Q2 Higher-for-Longer	$+5.52$	$1.52$	$2.37$	$3.91$	$+8.27$	$-2.75$
Average	$+2.67$	$0.76$	$1.16$	$5.33$	$+2.63$	$+0.04$

Appendix

Table 5: Per-scenario results for 3-agent enriched debate (Config 3). Performance varies substantially across market regimes: the system outperforms SPY by

+7.99\%

during the 2022 Q1 inflation shock but underperforms by

-5.14\%

in the 2022 Q4 tech rally.

Round	$\bar{\rho}$	$P_{1}$ (LV)	$P_{2}$ (ES)	$P_{3}$ (AC)	$P_{4}$ (CA)
1	0.843	0.86	0.86	0.87	0.78
2	0.845	0.87	0.85	0.86	0.81
3	0.815	0.84	0.83	0.80	0.75
4	0.832	0.77	0.86	0.88	0.81
5	0.839	0.82	0.86	0.86	0.79

Appendix

Table 6: CRIT pillar scores across debate rounds (4-agent enriched debate, 2022 Q4 tech scenario).

P_{1}

=Logical Validity,

P_{2}

=Evidential Support,

P_{3}

=Alternative Consideration,

P_{4}

=Causal Alignment. The most common CRIT diagnostic is causal_overreach , particularly for the Technical agent.

Scenario	Exp	Return %	Sharpe	Sortino	Max DD %	Excess %	Status
2021 Q3 Inflation	A	$+12.40$	$3.05$	$4.65$	$4.61$	$+2.64$	OK
	B	-	-	-	-	-	Fail
2022 Q1 Inf. Shock	A	$-19.40$	$-2.56$	$-3.16$	$19.96$	$-3.05$	OK
	B	$-19.40$	$-2.56$	$-3.16$	$19.96$	$-3.05$	OK
2022 Q2 Recession	A	$-1.97$	$-0.35$	$-0.50$	$12.69$	$+3.96$	OK
	B	$-1.77$	$-0.32$	$-0.45$	$12.97$	$+4.16$	OK

Appendix

Table 7: Per-scenario LLM comparison. Exp A uses GPT-5-mini for all roles; Exp B uses Gemini-2.5-flash for Macro/Value/Judge and GPT-5-mini for Risk. Exp B failures are caused by Gemini hallucinating ticker JNJ (not in scenario universe). Where both complete, returns are comparable- e.g., 2022 Q1 produces byte-identical portfolios.

Metric	Value
Scenario	2022 Q1 Inflation Shock (10 tickers)
Return	$-16.22\%$
Sharpe	$-2.11$
Sortino	$-2.63$
Max Drawdown	$17.67\%$
SPY Return	$-16.35\%$

Appendix

Table 8: Phase-LLM experiment results (GPT-5-mini for propose/revise, Gemini-2.5-flash critique/judge). Tracked SPY closely during a severe downturn, with excess return of

+0.13\%

Metric	Mean	Best	Worst
Return (%)	-1.11	21.71	-70.10
Excess Return vs SPY (%)	1.01	11.58	-6.41
Max Drawdown (%)	7.87	1.68	16.39
Sharpe Ratio	1.06	4.38	-2.73
Trades	8.8	21	1

Appendix

Table 9: Ablation performance across 17 market scenarios.

Experiment	Return (%)	Excess vs SPY (%)	Sharpe	Final JS
Exp 1 Role-only (Macro Anthropic, Risk GPT)	-1.42	3.16	-0.54	0.7181
Exp 2 Role-only (Macro GPT, Risk Anthropic)	-2.37	2.26	-0.64	0.7312
Exp 3 Hybrid phases	-1.70	2.84	-0.53	0.7946

Appendix

Table 10: Performance comparison across LLM routing experiments.

Abl.	Config	N	Ret	Sharpe	Sortino	MaxDD	Excess vs SPY
A7	Base	35	$+1.79\pm 1.50$	$+0.50\pm 0.28$	$+0.93\pm 0.43$	$+10.03\pm 0.88$	$-1.73\pm 1.03$
A7	+JS	35	$+2.62\pm 1.61$	$\mathbf{+0.64\pm 0.29}$	$\mathbf{+1.19\pm 0.46}$	$+10.21\pm 0.95$	$-0.90\pm 1.08$
A8	Base	35	$+1.46\pm 1.21$	$+0.42\pm 0.28$	$+0.81\pm 0.44$	$+8.07\pm 0.71$	$-2.06\pm 0.87$
A8	+CRIT	35	$+1.61\pm 1.26$	$+0.46\pm 0.28$	$+0.86\pm 0.43$	$+8.27\pm 0.78$	$-1.90\pm 0.94$
A10	Base	35	$+2.61\pm 1.54$	$+0.66\pm 0.28$	$+1.18\pm 0.44$	$+9.80\pm 0.87$	$-0.91\pm 1.04$
A10	+JS	35	$+2.41\pm 1.59$	$+0.57\pm 0.27$	$+1.04\pm 0.42$	$\mathbf{+10.33\pm 0.93}$	$-1.11\pm 1.07$

Appendix

Table 11: Performance metrics across intervention ablations. Three ablations evaluate reasoning interventions in the multi-agent debate system. A7 introduces a Jensen–Shannon (JS) collapse detector in a two-agent setting ( macro , technical ), triggering when revision-stage disagreement falls below a threshold (

JS_{\text{rev}}/JS_{\text{prop}}<0.8

) and nudging agents to maintain independent reasoning. A8 applies the same JS intervention with an added risk agent (three agents total); the intervention fired frequently (97% of debates) but produced no measurable effect. A10 replaces the JS trigger with a CRIT-based reasoning intervention that fires when causal reasoning quality

(\rho_{4}<0.8)

Abl.	Metric	Diff	95% CI	t	p	d
A7	Sharpe	$+0.143$	$[+0.017,\;+0.270]$	$+2.300$	0.0277	$+0.389$
A7	Sortino	$+0.254$	$[+0.032,\;+0.476]$	$+2.328$	0.0260	$+0.393$
A7	Excess vs SPY %	$+0.828$	$[-0.009,\;+1.666]$	$+2.011$	0.0523	$+0.340$
A10	Max DD %	$+0.529$	$[-0.028,\;+1.086]$	$+1.930$	0.0620	$+0.326$

Appendix

Table 12: Paired t-test results for ablations with statistically significant or marginal effects (

p<0.10

). Green p-values indicate improvement relative to the baseline configuration, red indicates undesired outcomes.

Tier	n	$\boldsymbol{\bar{\rho}}$	Sharpe	Return (%)
Standard	56	0.833	-0.918	-3.56
Light	62	0.825	-0.940	-3.58
Intense Light	67	0.819	-0.992	-3.66
Intense	60	0.834	-0.967	-3.75

Appendix

Table 13: Prompt enrichment tiers (Standard

\rightarrow

Intense) show no improvement in reasoning or financial performance across 245 runs of Ablation 1.

Subgroup	Metric	r	p	n
Value agent $\rho$ vs Sharpe	Pearson	$-0.29$	0.001	126
Risk agent $\rho$ vs Return	Pearson	$+0.22$	0.006	152

Appendix

Table 14: Selected subgroup correlations between agent reasoning quality and performance.

Ablation	n	Debate Return	EW Return	Diff	Beats EW
Abl 1	245	$-3.64\%$	$-3.28\%$	$-0.36\%$	31%
Abl 7	70	$+2.20\%$	$+4.47\%$	$-2.26\%$	30%
Abl 8	70	$+1.54\%$	$+4.47\%$	$-2.93\%$	34%
Abl 10	70	$+2.51\%$	$+4.47\%$	$-1.96\%$	40%
Pooled	455	—	—	$\mathbf{-1.29\%}$	33%

Appendix

Table 15: Debate portfolio performance compared to an equal-weight baseline.

Pillar	Baseline	Enriched	$\Delta$	Cohen’s $d$
Evidential Support	0.51	0.85	$+0.34$	2.92
Causal Alignment	0.70	0.87	$+0.17$	2.21
Alternative Consideration	0.79	0.89	$+0.10$	1.09
Logical Validity	0.87	0.82	$-0.05$	$-0.44$

Appendix

Table 16: Improvements in CRIT reasoning pillars under structured prompts.

Figure 3: Dashboard Aggregate Stats View.

Multi-Agent Debate

Multi-Agent Debate studies how language-model agents exchange arguments, critique answers, and combine judgments for collective reasoning.

See all 144 papers

Toward Reliable Evaluation of LLM-Based Financial Multi-Agent Systems: Taxonomy, Coordination Primacy, and Cost Awareness
Mar 29, 2026Phat Nguyen, Thang PhamMulti-Agent LLMs LLM Agent Evaluation
Georgia Institute of Technology, Atlanta GA 30332, USA · Adobe Inc., San Jose CA 95110, USA
FinCom: A Financial Multi-Agent Demo with Disagree-or-Commit Deliberation
May 31, 2026Chao Peter Yang, Zixiao Tan, Kaisen Yao +3LLM Sycophancy LLM Agent Orchestration
Duke University United States · ClearPath United States
The Price of Thought: Does Test-Time Reasoning Pay in LLM Trading?
Sep 25, 2026Jiayi Chen, Guiling WangLLM Reasoning Efficient Language Model Reasoning
Department of Computer Science New Jersey Institute of Technology Newark, New Jersey, USA

Mar 29, 2026J/K move · Enter open · S save

Toward Reliable Evaluation of LLM-Based Financial Multi-Agent Systems: Taxonomy, Coordination Primacy, and Cost Awareness

Phat Nguyen, Thang Pham

Georgia Institute of Technology, Atlanta GA 30332, USA · Adobe Inc., San Jose CA 95110, USA

Multi-Agent LLMs LLM Agent Evaluation Multi-Agent Coordination Algorithmic Trading Quantitative Finance

Open paper AskPDF arXiv

Multi-agent systems based on large language models (LLMs) for financial trading have grown rapidly since 2023, yet the field lacks a shared framework for understanding what drives performance or for evaluating claims credibly. This survey makes three contributions. First, we introduce a four-dimensional taxonomy, covering architecture pattern, coordination mechanism, memory architecture, and tool integration; applied to 12 multi-agent systems and two single-agent baselines. Second, we formulate the Coordination Primacy Hypothesis (CPH): inter-agent coordination protocol design is a primary driver of trading decision quality, often exerting greater influence than model scaling. CPH is presented as a falsifiable research hypothesis supported by tiered structural evidence rather than as an empirically validated conclusion; its definitive validation requires evaluation infrastructure that does not yet exist in the field. Third, we document five pervasive evaluation failures (look-ahead bias, survivorship bias, backtesting overfitting, transaction cost neglect, and regime-shift blindness) and show that these can reverse the sign of reported returns. Building on the CPH and the evaluation critique, we introduce the Coordination Breakeven Spread (CBS), a metric for determining whether multi-agent coordination adds genuine value net of transaction costs, and propose minimum evaluation standards as prerequisites for validating the CPH.

Abstract

Multi-Agent Debate

Explore similar work

Toward Reliable Evaluation of LLM-Based Financial Multi-Agent Systems: Taxonomy, Coordination Primacy, and Cost Awareness

FinCom: A Financial Multi-Agent Demo with Disagree-or-Commit Deliberation

The Price of Thought: Does Test-Time Reasoning Pay in LLM Trading?