cs.CLNov 17, 2026

Can LLMs Understand What They Predict? Evaluating Latent Causal Structure Inference in Financial Markets

Authors: Yixiang Zheng, Yuechen Jiang, Zhiwei Liu, Jimin Huang, Yuyan Wang, Sophia Ananiadou

Organizations: National Centre for Text Mining, The University of Manchester · The Fin AI · ELLIS Manchester

Abstract

Large language models (LLMs) are increasingly evaluated through their ability to predict future outcomes, implicitly assuming that predictive success reflects an understanding of the underlying system. However, accurate predictions can often be achieved without recovering the latent mechanisms that generate observable outcomes. To examine this assumption, we introduce LUCID, a benchmark that evaluates whether models can infer latent market structures from event descriptions and historical financial observations. Models generate event-induced causal graphs representing inferred market structure and are evaluated through a reconstruction-based framework. Our evaluation of 6 open-weight and proprietary LLMs shows that adding their generated graphs generally reduces reconstruction quality compared with using historical observations alone, although a few metrics improve. These findings suggest that inferring informative, event-specific market structures from observable evidence remains challenging for the evaluated LLMs.

Figures & tables

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Are the Financial Reasoning from LLMs Credible? A Real World Test over Long-Horizon Statements

    Jul 22, 2026Xinke Tong, Xuanming Zhang, Tianyi Tang +10Numerical Reasoning in Language ModelsLLM Evaluation

  2. PolyBench: Benchmarking LLM Forecasting and Trading Capabilities on Live Prediction Market Data

    Apr 3, 2026Pu Cheng, Juncheng Liu, Yunshen LongForecasting BenchmarksLLM Evaluation

  3. Hindcast: Replaying Prediction Markets to Evaluate LLM Forecasters

    Jul 15, 2026Xiao Ye, Jacob Dineen, Evan Zhu +3LLM EvaluationForecasting Benchmarks