What LLM Trading Agents Actually Do in Production: A Six-Month, Population-Scale Record from Two Fleets

T. J. Barton; Chris Constantakis; Patti Hauseman; Annie Mous; Alaska Hoffman; Brian Bergeron; Hunter Goodreau

cs.AISep 4, 2026

What LLM Trading Agents Actually Do in Production: A Six-Month, Population-Scale Record from Two Fleets

Authors: T. J. Barton, Chris Constantakis, Patti Hauseman, Annie Mous, Alaska Hoffman, Brian Bergeron, Hunter Goodreau

Organizations: DX Research Group (DXRG)

Abstract

We present a continuous, population-scale measurement record of autonomous language-model trading agents operating in production across two systems with one design lineage: DX Terminal Pro (3,505 user-funded vaults trading real ETH in Base memecoin markets for 21 days, February to March 2026) and the DXAP live alpha fleet (500 to 599 user-created agents all-history, 91 to 117 concurrently active, trading Hyperliquid perpetuals, June to August 2026). The record spans roughly six months, 7.5M single-model invocations with about 300K onchain actions, and a further 231,638 multi-tool turns producing 14,596 fills. Four findings carry the paper. First, the operating layer determines behavior more than anything written in strategy text: a risk slider explains leverage (+0.425 per level), agent fixed effects absorb 60% of variance, and a leaderboard render boundary causally routes selection (regression discontinuity 1.75x at the top-3 cut). Second, sizing is volatility-blind: median leverage is 5.0x in every volatility sextile, and one posture-slider cell (11% of the book) holds 62% of liquidations. Third, agents capture almost none of the upside they reach: 43.2% of positions saw at least +300 bps of favorable excursion within 24h, yet 49.3% of those closed with a negative trade return; a mechanical bracket recovers +39.0 bps per position. Fourth, neither fleet shows a directional edge. The DXAP fleet is not profitable and trails a matched Hyperliquid retail benchmark (41% vs. 50% roundtrip win rate). A paired-replay league of frontier models on 416 captured production scenarios finds decision quality statistically indistinguishable at this horizon, while choice stability differs sharply across model families. Every headline survives day-clustered inference, permutation nulls, and a common-fee restatement; the paper closes with a 17-rule methodology canon bought with our own retractions.

Figures & tables

	DX Terminal Pro	DXAP live alpha
Window	Feb 26 to Mar 18, 2026 (21 days)	Jun 8 to Aug 15, 2026 (69 days and running)
Venue	12 memecoin tokens, Uniswap V4 pools on Base	$\sim$ 99–100 perpetuals on Hyperliquid (incl. HIP-3 synthetics)
Capital	Real ETH: 4,984 ETH deposited, 4,589 withdrawn (92%)	Mostly paper at $10,000 start with live prices; small real-capital book (5,035 real-money fills, 6 of 19 users, notionals$ 14–$2,000)
Population	3,505 funded vaults (3,454 active)	500–599 agents all-history; 91–117 concurrently active
Model	Qwen3-235B-A22B-Thinking-2507 via SGLang, temp 0.6	OpenRouter mix; qwen3.7-plus dominates (86/91 active on Jul 22)
Cadence	$\sim$ 5 min/agent; exactly one tool call (buy/sell/observe)	15m/30m/1h/4h schedules + agent-created triggers; 10-tool manifest, mandatory finalize_turn

Table 1: The two production systems. DX Terminal Pro figures are the canonical run facts of [ 1 ] and its lab record; DXAP figures are from the live ClickHouse analytics store as of Aug 15, 2026.

Figure 1: One continuous record across two production systems. The DX Terminal Pro window (shaded; Feb 26 to Mar 18) is drawn from the canonical run facts; DXAP daily active agents (solid, left axis) and cumulative finalized turns (dashed, right axis) are computed from the ClickHouse analytics_agent_turns_finalized table, Jun 8 to Aug 15, 2026.

Figure 2: DXAP agents vs. a matched Hyperliquid retail sample, trailing window ending Aug 15, 2026. Left: daily win rate. Center: long share of entries (count-based). Right: 4-day open-to-close roundtrips (ours

n=353

, retail

n=16{,}009

) and share of week-active accounts with positive net ROI. Source: hl4d_summary.json , daily pipeline of Aug 15, 14:00 UTC.

Metric (window)	DXAP agents	HL retail
Roundtrip win rate (4d)	41% ( $n=353$ )	50% ( $n=16{,}009$ )
Median hold (4d)	2.05 h	2.18 h
Median roundtrip return (4d)	$-0.094\%$	$+0.002\%$
Accounts net-positive (week)	15% ( $n=117$ )	53% ( $n=4{,}828$ )
Long share of entries (count)	$\sim$ 78%	$\sim$ 60%
Effective leverage, median	0.096 $\times$	0.022 $\times$

Table 2: Fleet vs. retail benchmark, trailing windows ending Aug 15, 2026. Retail sample: 1,961 leaderboard traders in the

1K–

100K equity band.

Figure 3: Selection by gainers-leaderboard rank at entry (Jun 16 to Jul 24; 2,339 entry observations across gainers ranks 1–15, plotted through rank 12). Terracotta bars are rendered ranks; the dashed line is the render boundary. The registered RDD estimate at the 3

|

4 cut is

1.75\times

[1.49, 2.06]; the raw rank-3/rank-4 count ratio is

1.81\times

(290/160).

Figure 4: Volatility-blind sizing across 6,400 closed DXAP positions. Left: market volatility per sextile vs. median chosen leverage (flat at

5.0\times

). Right: median realized return per sextile (bars) and liquidation rate (dashed).

Sextile	$n$	Vol (bps/hr)	Med. lev	Med. notional	Liq. rate	Med. realized
S1 calmest	1,067	30.8	5.0 $\times$	$3,500	0.7%	$-10.6$
S2	1,067	49.2	5.0 $\times$	$4,000	2.2%	$-34.3$
S3	1,066	70.3	5.0 $\times$	$5,000	4.6%	$-44.5$
S4	1,067	93.9	5.0 $\times$	$5,140	3.9%	$-49.0$
S5	1,066	121.8	5.0 $\times$	$5,405	3.6%	$-54.9$
S6 wildest	1,067	175.2	5.0 $\times$	$5,000	4.3%	$-98.2$

Table 3: Closed positions by entry-time volatility sextile (Jun 8 to Jul 24). Realized return is median net bps; bracketed-entry quality is flat throughout, so the degradation is sizing, not selection.

Figure 5: Liquidations by strategy posture $\times$ tradeFrequency slider (6,400 closed positions, 205 liquidations). The boxed cell (momentum $\times$ frequency 5) holds 128 of 205 (62%). Cell counts are from the style-lab position master; the odds ratio is the day-stratified Mantel–Haenszel estimate.

Figure 6: H2 anchor ladder:

\Delta

PR-AUC over the frozen concrete baseline when pooling at successive points in the turn card. Error bars are 90% day-clustered intervals. The entire increment sits at the order boundary.

Figure 7: The capture gap: maximum favorable excursion within 24h vs. realized return for 6,400 closed positions (hexbin, log counts; clipped at the 0.5/99.5 percentiles for display). The dashed line marks $+300$ bps MFE.

Figure 8: The June 2026 model decision league. Left: winsorized replay regret against the achievable envelope (bps, lower is better) for three frontier models over 416 captured production scenarios across 7 days at temperature 0.6; whiskers are day

\to

8h-block hierarchical bootstrap intervals, and the shaded band is the range every interval covers. Right: choice stability in the forced-action arm (flat removed; 280 cells), the fraction of cells in which the model’s choice flips across temperature-0.6 repeats; selection quality spans rank percentile 0.486–0.508 (0.5 is chance). Replay measures behavior under a frozen context and says nothing about live P&L.

Algorithmic Trading

Algorithmic trading uses computational models to generate signals, allocate portfolios, and execute orders across financial markets.

See all 144 papers

Apr 28, 2026

Operating-Layer Controls for Onchain Language-Model Agents Under Real Capital

We study reliability in autonomous language-model agents that translate user mandates into validated tool actions under real capital. The setting is DX Terminal Pro, a 21-day deployment in which 3,505 user-funded agents traded real ETH in a bounded onchain market. Users configured vaults through structured controls and natural-language strategies, but only agents could choose normal buy/sell trades. The system produced 7.5M agent invocations, roughly 300K onchain actions, about $20M in volume, more than 5,000 ETH deployed, roughly 70B inference tokens, and 99.9% settlement success for policy-valid submitted transactions. Long-running agents accumulated thousands of sequential decisions, including 6,000+ prompt-state-action cycles for continuously active agents, yielding a large-scale trace from user mandate to rendered prompt, reasoning, validation, portfolio state, and settlement. Reliability did not come from the base model alone; it emerged from the operating layer around the model: prompt compilation, typed controls, policy validation, execution guards, memory design, and trace-level observability. Pre-launch testing exposed failures that text-only benchmarks rarely measure, including fabricated trading rules, fee paralysis, numeric anchoring, cadence trading, and misread tokenomics. Targeted harness changes reduced fabricated sell rules from 57% to 3%, reduced fee-led observations from 32.5% to below 10%, and increased capital deployment from 42.9% to 78.0% in an affected test population. We show that capital-managing agents should be evaluated across the full path from user mandate to prompt, validated action, and settlement.

T. J. Barton, Chris Constantakis, Patti Hauseman +4

DX Research Group (DXRG)

AI Agent Reliability LLM Agent Evaluation AI Agent Monitoring Long-Horizon LLM Agents LLM Agent Reliability

Ask PDF

May 19, 2026

Agentic Trading: When LLM Agents Meet Financial Markets

A growing body of work explores how Large Language Models (LLMs) can be embedded in trading systems as agents that perceive market information, retrieve context, reason about decisions, emit tradable actions, and adapt under market feedback. This paper reframes LLM-based trading agents as expert-system decision pipelines and presents an audit-oriented evidence map of 77 included studies in a protocol-coded snapshot screened through 2026-03-09. A primary empirical subset (n=19) satisfies the minimum boundary of Action Output plus Closed-Loop Evaluation; the remaining 58 included studies are retained as background and design context. The central empirical finding is protocol incomparability: within the primary subset, only 2/19 studies report extractable time-consistent split protocols, 1/19 reports an explicit transaction-cost model, 1/19 documents universe or survivorship handling, 11/19 report execution timing or semantics, 15/19 are coded as R0, and no study reaches R3 reproducibility. We therefore use Architecture-Capability-Adaptation as a working analytical lens rather than a validated taxonomy, and we foreground the evidence ledger, reproducibility audit, and reporting checklist as the main contributions. The resulting survey shows that architectural experimentation is expanding rapidly, while comparable evaluation protocols, execution semantics, and reproducible artifacts remain the field's immediate bottlenecks.

Yihan Xia, Panpan You, Taotao Wang +4

College of Electronic and Information Engineering, Shenzhen University, Shenzhen, China

LLM Agent Evaluation Finance Algorithmic Trading

Ask PDF

Aug 1, 2025

ContestTrade: A Multi-Agent Trading System Based on Internal Contest Mechanism

In financial trading, large language model (LLM)-based agents demonstrate significant potential, but their decisions can be sensitive to noisy and non-stationary market information. We propose ContestTrade, a multi-agent trading system with an internal competitive mechanism inspired by institutional investment workflows. The system consists of two specialized teams: (1) a Data Team that processes and condenses massive market data into diversified textual factors optimized for constrained LLM context windows, and (2) a Research Team that produces parallelized multipath trading decisions via tool-augmented deep research. The core design is a "Quantify-Predict-Allocate" contest mechanism within each team: agent outputs are scored only after market outcomes become observable, future utility is predicted from historical scores, and resources are allocated to agents with positive predicted utility. In a post-2024 A-share backtest, ContestTrade achieves higher backtested return and risk-adjusted performance than the evaluated baselines. We further describe the temporal protocol, implementation choices, and limitations to clarify the scope of these results.

Rui Sun, Li Zhao, Zuoyou Jiang +5

1Stepfun · 2FinStep

Multi-Agent LLM Systems LLM Agent Orchestration Multi-Agent Systems Algorithmic Trading

Ask PDF

Abstract

Algorithmic Trading

Explore similar work

Operating-Layer Controls for Onchain Language-Model Agents Under Real Capital

Agentic Trading: When LLM Agents Meet Financial Markets

ContestTrade: A Multi-Agent Trading System Based on Internal Contest Mechanism