cs.AINov 17, 2026

The Only Edge Was Memorized in Training: LLM Trading Agents Around Earnings and Bad News

Authors: Jackson Wang

Abstract

Large language model (LLM) trading agents are tested on news, but news reaches stock prices within minutes, and a benchmark that replays headlines at their publication time finds no alpha in trading on them. Money can then remain in two places, before news whose date is known, such as earnings announcements, and after bad news, if the market's fall went too far. However, studies of LLMs and earnings read documents published with or after the announcement, and studies of reversal sort monthly portfolios, so neither tells whether an agent can trade these moments after costs. In this paper, we test LLM agents at both moments on the same look-ahead-free benchmark, before 598 earnings announcements and after 409 bad-news drops of large US stocks. Every input is cut at the decision close, staged agents are compared with single calls, simple rules and a logistic regression, and every decision is made with and without the stock's name. Across five models from four vendors, no agent beat the simple bets. Before earnings, the agents mostly bought and earned what buying before every announcement earned, and after bad news, a Gemini-based agent called almost every fall an overreaction while no agent picked better dips than buying them all. The only gains came from what the models had memorized in training. Given only the ticker and the dates, a GPT-based model called the direction of 59.2% of the announcements before its training cutoff and 53.3% after it. Agent builders should therefore compare an event agent with the simple bet and test it blind after the training cutoff.

Figures & tables

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. From Knowing to Doing: A Memory-Controlled Benchmark for LLM Trading Agents on Stock Markets

    May 27, 2026Taojie Zhu, Wentao Zhao, Rui Sun +7Benchmark ContaminationLLM Agent Evaluation

  2. Inside the Trading Loop: Markets, Models, and Moves with LLM Agents

    Nov 8, 2026Renee Jia, Di Mu, Yao Yu +1portfolio managementAlgorithmic Trading