Do LLMs Understand Limit Order Book Dynamics?

Junxiao Chen; Paul Glasserman

cs.AIAug 24, 2026

Do LLMs Understand Limit Order Book Dynamics?

Authors: Junxiao Chen, Paul Glasserman

Organizations: Columbia Business School

Abstract

A large language model (LLM) trained on synthetic limit order book (LOB) data achieves near perfect scores in generating valid sequences of LOB events. However, the LLM's implicit world model fails to learn the state of the LOB. This deficiency leads to biased estimates and spurious predictability in using the LLM to forecast future LOB events. Our analysis uses novel tests of an LLM's world model, extending prior work from deterministic settings to the stochastic dynamics needed for the LOB.

Figures & tables

Figure 1: Representation of a limit order book

Model	Valid	SP	Extra	Compress.
	Traversal	Rate	Steps	Score
SP	99.4%	99.4% (100%)	0.0 (0.0)	61.1%
RW-S	98.5%	90.6% (10.4%)	0.1 (16.3)	42.0%
RW-L	95.5%	83.7% (4.2%)	0.3 (43.8)	48.0%
Baseline	0.7%	0.7% (4.2%)	0.0 (43.8)	99.9%

Table 1: Diagnostic results

Step	Event	Resulting State
0	Start	$(2,4,4,1,0)$
1	ask_p3_2	$(3,4,4,1,2)$
2	bid_p2_1	$(3,4,4,1,1)$
3	ask_p2_1	$(3,4,4,2,1)$
4	w_ask_p1_2	$(3,4,2,2,1)$
5	ask_p3_1	$(3,4,2,2,2)$

Table 2: Prefix used in kernel-level TV

Action Token	Next State	True Prob.	LLM’s Prob.
w_ask_p1_2	$(3,4,0,2,2)$	0.050	0.245
w_ask_p1_1	$(3,4,1,2,2)$	0.050	0.241
ask_p2_2	$(3,4,2,4,2)$	0.050	0.190
ask_p1_1	$(3,4,3,2,2)$	0.050	0.132
ask_p1_2	$(3,4,4,2,2)$	0.050	0.106

Table 3: Next-event probabilities in kernel-level TV

Figure 2: Illustration of history- and kernel-level TV

Model	TV Type	$H=1$	$H=2$	$H=3$
SP	Kernel-level	0.46	0.72	0.84
SP	History-level	0.33	0.51	0.59
RW-S	Kernel-level	0.11	0.19	0.28
RW-S	History-level	0.07	0.14	0.23
RW-L	Kernel-level	0.08	0.15	0.21
RW-L	History-level	0.06	0.11	0.16

Table 4: Multi-step forecast TV distances

Event Token	Next State	$Q_{\theta,1}(a\mid h_{1})$	$Q_{\theta,1}(a\mid h_{2})$
bid_p2_2	$(1,2,5,2,2)$	0.278	0.002
ask_p3_2	$(1,4,5,0,0)$	0.056	0.313
m_ask_2	$(1,4,5,0,0)$	0.047	0.233
w_bid_p3_2	$(1,4,5,0,0)$	0.028	0.199
ask_p2_2	$(2,3,5,2,2)$	0.053	0.208
ask_p2_1	$(2,3,5,1,2)$	0.128	0.002

Table 5: Next-event probabilities in history-level TV

LOB Setting	# of States	Model	Valid Traversal	SP Rate	Compress. Score	Kernel-level TV			History-level TV
LOB Setting	# of States	Model	Valid Traversal	SP Rate	Compress. Score	$H=1$	$H=2$	$H=3$	$H=1$	$H=2$	$H=3$
$(P,V_{1},V_{2})$ $=(3,12,3)$	8281	SP	99.4%	99.4% (100%)	47.0%	0.64	0.86	0.93	0.34	0.45	0.57
		RW-S	97.8%	88.6% (11.5%)	29.0%	0.14	0.22	0.35	0.10	0.13	0.26
		RW-L	94.1%	81.2% (4.0%)	34.0%	0.13	0.19	0.32	0.08	0.14	0.18
		Baseline	0.3%	0.3% (4.0%)	99.8%	0.00	0.02	0.01	0.01	0.00	0.01
$(P,V_{1},V_{2})$ $=(8,2,2)$	41553	SP	97.1%	96.8% (100%)	53.6%	0.61	0.79	0.92	0.39	0.56	0.62

Table 6: Diagnostic and multi-step forecast results for larger LOB settings

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Prefix	Start State	Event Sequence
$h_{1}$	$(2,3,2,2,1)$	ask_p1_2 , w_bid_p3_1 , ask_p2_2 , ask_p1_1 , bid_p3_2 , bid_p2_1 , m_ask_1 , bid_p3_1 , w_bid_p3_2 , ask_p2_1 , ask_p3_2 , bid_p1_1 , bid_p3_2 , m_ask_1 , m_bid_2 , m_bid_1 , m_bid_2 , bid_p1_1 , ask_p1_2 , bid_p3_2
$h_{2}$	$(1,2,3,1,2)$	bid_p3_2 , bid_p2_1 , bid_p2_2 , m_bid_2 , ask_p1_1 , w_bid_p3_1 , w_bid_p3_1 , ask_p1_2 , w_bid_p3_2 , ask_p3_1 , m_ask_2 , bid_p3_1 , ask_p3_1 , w_ask_p1_1 , ask_p2_2 , ask_p1_1 , m_ask_1 , ask_p1_1 , m_bid_2 , bid_p3_2

Appendix

Table 7: Two prefixes in history-level TV

Limit Order Book Modeling

See all 20 papers

LOBERT: Generative AI Foundation Model for Limit Order Book Messages
Nov 16, 2025Eljas Linna, Kestutis Baltakys, Alexandros Iosifidis +1Limit Order Book Forecasting Quantitative Finance
Data Science Research Centre Tampere University
Probabilistic Limit Order Book Forecasting with Conditional Diffusion Models
Oct 18, 2026Junyu Chen, Franklin Liu, Moshu Xu +1Limit Order Book Forecasting Limit Order Book Modeling
Department of Statistics and Data Science, Tsinghua University, Beijing, China
Understanding the Inner Workings of a State-Space Foundation Model for Limit Order Books
Oct 20, 2026Valentin Mohl, Alexandre Bismuth, Aramis Marti-Shahandeh +6Limit Order Book Modeling State Representation Learning
British Open-ended Learning and Discovery Lab (BOLD), Department of Computer Science, University of Oxford, Oxford, United Kingdom · Harvard University, Cambridge, MA, United States · British Open-ended Learning and Discovery Lab (BOLD), Department of Engineering Science, University of Oxford, Oxford, United Kingdom +3

Nov 16, 2025J/K move · Enter open · S save

LOBERT: Generative AI Foundation Model for Limit Order Book Messages

Eljas Linna, Kestutis Baltakys, Alexandros Iosifidis, Juho Kanniainen

Data Science Research Centre Tampere University

Limit Order Book Forecasting Quantitative Finance Time Series Forecasting Limit Order Book Modeling Next-Event Prediction Time-Series Foundation Models

Open paper AskPDF arXiv

Modeling the dynamics of financial Limit Order Books (LOB) at the message level is challenging due to irregular event timing, rapid regime shifts, and the reactions of high-frequency traders to visible order flow. Previous LOB models require cumbersome data representations and lack adaptability outside their original tasks, leading us to introduce LOBERT, a general-purpose encoder-only foundation model for LOB data suitable for downstream fine-tuning. LOBERT adapts the original BERT architecture for LOB data by using a novel tokenization scheme that treats complete multi-dimensional messages as single tokens while retaining continuous representations of price, volume, and time. With these methods, LOBERT achieves leading performance in tasks such as predicting mid-price movements and next messages, while reducing the required context length compared to previous methods.

Abstract

Limit Order Book Modeling

Explore similar work

LOBERT: Generative AI Foundation Model for Limit Order Book Messages

Probabilistic Limit Order Book Forecasting with Conditional Diffusion Models

Understanding the Inner Workings of a State-Space Foundation Model for Limit Order Books