cs.AINov 7, 2026

Approximate Recall, Approximate Forecasts: Recall as a Diagnostic for LLM Forecasting Errors

Authors: Prashanth Bhaskara, Shubhaankar Gupta, Seojoon Yeon

Organizations: University of Chicago.

Abstract

Large Language Models (LLMs) exhibit approximate recall: they retain coarse associations about past events while losing fidelity on specific details. Past research on LLMs has quantified different models' forecasting ability in real-world events using prediction markets, but has not studied how losing said fidelity impacts their forecasting abilities. For example: while an LLM correctly recalls that the answer to: ''Did Anna Sawai win the Best Drama Actress in the 76th Emmy Awards?'' is ''Yes,'' it incorrectly states the event date as September 22 rather than September 15. In this paper, we introduce a novel metric called the Recall Fidelity Score to study how imprecision in internalized knowledge propagates to forecasting capability on future events. On resolved historical markets, we find that higher local recall fidelity around semantically similar past events predicts lower forecasting error on resolved markets, and the same signal predicts smaller distance from open markets prices.

Figures & tables

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Hindcast: Replaying Prediction Markets to Evaluate LLM Forecasters

    Jul 15, 2026Xiao Ye, Jacob Dineen, Evan Zhu +3LLM EvaluationForecasting Benchmarks

  2. TimeSeek: Temporal Reliability of Agentic Forecasters

    Apr 5, 2026Hamza Mostafa, Om Shastri, Dennis LeeLLM Agent ReliabilityForecasting Benchmarks

  3. ForecastGym: A Live, Controlled, and Replayable Benchmark for Forecasting Agents

    Nov 17, 2026Yicheng Tao, Yuante Li, Zipei FanForecasting BenchmarksProbabilistic Forecasting