Quant-Arena: Benchmarking LLM Agents for Quantitative Trading Research

Houde Qian; Zheng Jiang; Wenyi Li; Chenxi Xu; Xinjie Lin; Yuzhao Peng; Ruoyu Han; Youjie Zheng

cs.CLOct 21, 2026

Quant-Arena: Benchmarking LLM Agents for Quantitative Trading Research

Authors: Houde Qian, Zheng Jiang, Wenyi Li, Chenxi Xu, Xinjie Lin, Yuzhao Peng, Ruoyu Han, Youjie Zheng,

Organizations: Navers Lab, Einsia.AI

Abstract

Quantitative research is an iterative loop: a researcher proposes a market hypothesis, implements it as an executable signal, tests it on historical data, and decides what to keep. Existing benchmarks score formulas, code, or individual trading decisions, leaving open whether large language models (LLMs) can complete this loop and produce strategies that work beyond the research period. We introduce Quant-Arena, a benchmark and public leaderboard for end-to-end quantitative research on real A-share market data. A model constructs features and stock-scoring code, receives feedback only on 2022--2023, and freezes one strategy for assessment on the sealed year 2024. A cash-account evaluator applies trading constraints and costs, including market impact estimated from a simulated order book. Scores are dominated by realized excess return and complemented by signal quality and the diversity of features used by the strategy. We evaluate nine frontier models, among which GPT-6 Astra scores highest; yet no model beats the equal-weight market in 2024. Quant-Arena is open to submissions from any model or agent harness.

Figures & tables

Figure 2: A model constructs features and stock-scoring code from raw market observations, revises candidates using research-period feedback, and freezes one strategy for assessment on the sealed period under common portfolio and execution rules.

Property	Specification
Stock universe	Selected universe of 3,042 stocks
Inputs	Daily market data and raw Level-2 quotes, orders, and trades
Research period	2022–2023
Out-of-sample period	Sealed year 2024

Table 1: Market data and temporal split used by the benchmark.

Panel of Table 2 — Table 2: Iterative results for every model and supported reasoning effort, Models are ordered by their best total score, and efforts increase within each model. Shaded rows mark each model’s highest-scoring effort, the configuration used by the diagnostics in Sections 3.3 – 3.6 . Scores are on a 0–100 scale. Financial and signal metrics refer to the sealed 2024 window; the gross equal-weight market ARR is 17.1%. MDD is a negative return. Bold marks the best value in each column.

Figure 3: Mean next-day return by daily score decile for the nine models’ best iterative configurations in 2024, in basis points (bp). H–L is the highest-minus-lowest decile spread, and the last entry in each header is the number of days with a signal.

Figure 4: Cash-account NAV of the iterative Astra run at xhigh effort in 2024, against the gross equal-weight market. Shaded bands mark days on which Astra holds stocks.

Model	Efforts	$S$			Window $\Delta$		ARR (%)
		Iter.	One-shot	$\Delta$	$\Delta S_{\mathrm{in}}$	$\Delta S_{\mathrm{out}}$	Iter.	One-shot	$\Delta$
GPT-6 Astra	xhigh / high	51.29	35.22	16.07	15.67	16.24	10.7	$-4.8$	15.5
GPT-6 Sol	xhigh / low	51.14	32.15	18.99	17.21	19.74	8.6	$-7.6$	16.2
GPT-5.6 Terra	max / none	47.58	42.14	5.44	3.81	6.14	8.4	9.5	$-1.1$
Claude Opus 5.5	medium / medium	44.28	42.61	1.67	9.59	$-1.73$	4.4	5.6	$-1.2$
Gemini 3.8 Flash	medium / medium	43.04	35.91	7.13	17.96	2.48	$-2.9$	1.6	$-4.5$

Table 3: Paired iterative and one-shot results, each at the supported effort with the highest total score within its setting (efforts listed as iterative / one-shot).

\Delta

is iterative minus one-shot.

S_{\mathrm{in}}

and

S_{\mathrm{out}}

are the research-period and sealed-period scores, and ARR differences are in percentage points. The gross equal-weight market ARR is 17.1%.

Figure 5: Research-period progress of the nine iterative runs. (a) Best-so-far excess ARR gain over the first valid candidate; means use the runs available at each index. (b) First, best, and committed candidates on 2022–2023.

Figure 6: Holding-value-weighted style exposure of the nine models’ best iterative configurations in 2024, in cross-sectional standard deviations relative to the market. Astra’s estimates use its 61 invested days.

Figure 7: Cash-account NAV in 2024 for the nine models’ best iterative configurations and Kepler-1, against the gross equal-weight market.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Field	Specification
Market and universe	Selected A-shares; static universe of 3,042 stocks; missing and delisted stocks retained
Input profile	Raw daily and Level-2 market data through a date-scoped data interface; Python strategies
Windows	In-sample research 2022–2023 with authorized warmup history; sealed out-of-sample 2024; participants may define training and validation splits within the research period
Budget	At most 5 research evaluations
Feedback (iterative track)	Aggregate return, signal, and coverage statistics on the research window, plus execution errors
Portfolio	Top-75 equal-weight long portfolio, boundary ties retained; Top-200/400 diagnostics

Appendix

Table 4: Public specification of the selected A-share suite.

Panel of Table 5 — Table 5: One-shot results for every model and supported reasoning effort, Layout and shading follow Table 2 ; shaded rows mark each model’s highest-scoring one-shot effort. The one-shot setting receives no performance feedback.

Figure 8: Pairwise mean daily Spearman signal correlations (a) and Top-75 Jaccard overlaps (b) among the nine models’ best iterative configurations in 2024 (36 pairs). Portfolio overlap measures the selected score sets, rather than identical realized trades under execution constraints.

Financial Language Model Evaluation

See all 29 papers

QuantEval: A Benchmark for Financial Quantitative Tasks in Large Language Models
Jan 13, 2026Zhaolu Kang, Junhao Gong, Wenqing Hu +13Financial Language Model Evaluation LLM Evaluation
Peking University · Tsinghua University · Fudan University +6
Strat-LLM: Stratified Strategy Alignment for LLM-based Stock Trading with Real-time Multi-Source Signals
May 7, 2026Wenliang Huang, Zengyi YuLLM Evaluation LLM Alignment
School of Economics Zhejiang University of Technology Hangzhou, China · Faculty of Education East China Normal University Shanghai, China
From Knowing to Doing: A Memory-Controlled Benchmark for LLM Trading Agents on Stock Markets
May 27, 2026Taojie Zhu, Wentao Zhao, Rui Sun +7Benchmark Contamination LLM Agent Evaluation
1Tsinghua University, 2Stepfun · 2Stepfun · 4Shanghai Jiao Tong University +3

Jan 13, 2026J/K move · Enter open · S save

QuantEval: A Benchmark for Financial Quantitative Tasks in Large Language Models

Zhaolu Kang, Junhao Gong, Wenqing Hu, Yingjie He+12

Peking University · Tsinghua University · Fudan University+6

Financial Language Model Evaluation LLM Evaluation Numerical Reasoning in Language Models Quantitative Finance LLM Code Generation

Open paper AskPDF arXiv

Large Language Models (LLMs) have shown strong capabilities in finance-oriented question answering, yet their evaluation on professional quantitative workflows remains fragmented and often stops at factual or document-grounded QA. We introduce QuantEval, a benchmark for evaluating whether LLMs can complete the compute--implement--validate loop required in quantitative finance. QuantEval contains 1,575 expert-validated instances across three complementary dimensions: (i) knowledge-based QA, (ii) quantitative mathematical reasoning, and (iii) quantitative strategy coding. The coding task is evaluated with a deterministic CTA-style backtesting framework that executes model-generated strategies and compares risk--return metrics against expert references, moving beyond syntax or template matching. We evaluate 13 open-source and proprietary LLMs, report prompt and reliability analyses, and observe substantial gaps relative to human experts, especially in multi-step reasoning and framework-constrained coding. We further conduct supervised fine-tuning and reinforcement learning experiments on domain-aligned data, showing that training can improve quantitative reasoning while also exposing trade-offs in factual recall. QuantEval releases the dataset, evaluation scripts, and full backtesting configuration to support reproducible research on LLMs for quantitative finance.

Abstract

Financial Language Model Evaluation

Explore similar work

QuantEval: A Benchmark for Financial Quantitative Tasks in Large Language Models

Strat-LLM: Stratified Strategy Alignment for LLM-based Stock Trading with Real-time Multi-Source Signals

From Knowing to Doing: A Memory-Controlled Benchmark for LLM Trading Agents on Stock Markets