Large Language Models (LLMs) exhibit approximate recall: they retain coarse associations about past events while losing fidelity on specific details. Past research on LLMs has quantified different models' forecasting ability in real-world events using prediction markets, but has not studied how losing said fidelity impacts their forecasting abilities. For example: while an LLM correctly recalls that the answer to: ''Did Anna Sawai win the Best Drama Actress in the 76th Emmy Awards?'' is ''Yes,'' it incorrectly states the event date as September 22 rather than September 15. In this paper, we introduce a novel metric called the Recall Fidelity Score to study how imprecision in internalized knowledge propagates to forecasting capability on future events. On resolved historical markets, we find that higher local recall fidelity around semantically similar past events predicts lower forecasting error on resolved markets, and the same signal predicts smaller distance from open markets prices.
Figures & tables
Specification
Intercept
βRm,d
R2
M1: Rm,d
0.226∗∗∗
−0.204∗∗∗
0.398
M2: +Dentropy
0.286∗∗∗
−0.222∗∗∗
0.429
M3: +Model FE
0.285∗∗∗
−0.223∗∗∗
0.691
Table 1: Domain-level regression of recall Brier scores on Recall Fidelity Score Rm,d across N=36 (model, domain) cells. Entropy defined in A.8
Figure 1: Recall Fidelity Scores (null = 0) and Log Loss for all models grouped by domain. Elections and Economics domains seem to produce strongest recall across models. Exact figures reported in A.9
Figure 2: Confident-wrong recall produces high Brier scores despite reduced confidence, while correct and partial recall yield near-zero Brier at high confidence. Confident-wrong cells are separated from all other states by an order of magnitude in Brier.
Figure 3: Mean forecast log-loss (from unresolved markets) versus mean Recall Fidelity color coded by model across nine domains.
Figure 4: Local recall error-type decomposition. Coefficients and 95% confidence intervals come from a market-level OLS regression of forecast Brier loss on the shares of the ten nearest historical recall markets falling into each recall category, with model and forecast-market fixed effects and standard errors clustered by condition ID.
Figure 5: Regression coefficients for local historical recall across open-market Brier-proxy specifications. Coefficient on Rm,i,10local with 95% confidence intervals using standard errors clustered by condition ID shown. Higher local recall significantly predicts lower squared forecast-market disagreement and lower absolute forecast-market distance.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Open-market proxy regression coefficients with 95% confidence intervals by condition ID. Local historical recall is negatively associated with squared and absolute forecast-market disagreement in market fixed-effect specifications, while the domain-level specification shows that market entropy is strongly positively associated with forecast-market disagreement.
Panel
Outcome
Main regressor
Controls / FE
β^ (SE)
p
N / clusters
Historical local
Bm,i
Rm,i,10local
Model + market FE
−0.0652(0.0231)
0.0048
4,897/1,264
Historical weighted
Bm,i
Rm,i,10local,w
Model + market FE
−0.0710(0.0225)
0.0016
4,897/1,264
Domain M1
Bˉm,d
Rm,d
None
−0.2038(0.0430)
<0.001
36 / –
Domain M2
Bˉm,d
Rm,d
Dd
−0.2216(0.0445)
<0.001
36 / –
Domain M3
Bˉm,d
Rm,d
Model FE + Dd
−0.2227(0.0361)
<0.001
36 / –
Appendix
Table 2: Historical resolved-market and aggregate domain-level recall regressions. The reported coefficient is always the coefficient on the recall-fidelity regressor. In Domain M2, γ^D=−0.1185 ( SE=0.0891 , p=0.193 ); in Domain M3, γ^D=−0.1192 ( SE=0.0691 , p=0.0948 ).
Panel
Outcome
Term
Controls / FE
β^ (SE)
p
N / clusters
Open market FE
Bm,j
Rm,j,10local
Model + market FE
−0.0284(0.0108)
0.0084
5,724/1,897
Open market FE
Am,j
Rm,j,10local
Model + market FE
−0.0448(0.0152)
0.0032
5,724/1,897
Open market FE
Lm,j
Rm,j,10local
Model + market FE
−0.0606(0.0696)
0.384
5,724/1,897
Open domain FE
Bm,j
Rm,j,10local
Model + domain FE + H(πj)
0.0071(0.0154)
0.642
5,724/1,897
Open domain FE
Bm,j
H(πj)
Model + domain FE
0.0721(0.0137)
<0.001
5,724/1,897
Open domain FE
Lm,j
Rm,j,10local
Model + domain FE + H(πj)
0.0140(0.0544)
0.797
5,724/1,897
Appendix
Table 3: Open-market proxy regressions. Market fixed effects compare models on the same unresolved market. Domain fixed effects with entropy controls test whether local recall explains broader domain-level variation after accounting for market-price uncertainty.
Figure 7: Murphy Breakdown of recall Brier for each model. A lower reliability score is more favorable while a higher resolution score is more favorable.
Model
n
Brier
Uncertainty
Reliability
Resolution
Null Rate
Claude Opus 4.7
1150
0.1017
0.2110
0.0102
0.0031
0.3174
Claude Sonnet 4.6
1150
0.1406
0.2110
0.0239
0.0052
0.3252
Gemini 3.1 Pro
1150
0.0877
0.2110
0.0080
0.0042
0.2591
GPT-5.5
0 993
0.0816
0.2132
0.0075
0.0022
0.2115
Appendix
Table 4: Murphy decomposition components and null prediction rates by model. Lower values are better for Brier, Reliability, and Null Rate, while higher values are better for Resolution.