Language models are now used to read central-bank text for economic forecasting, but most were trained on text written after the documents they read. A model reading a 2021 FOMC statement may already know that inflation rose in 2022, so part of its apparent skill can be memory of the outcome, a temporal form of test-set contamination. We measure this memory on FOMC statements and minutes from 2000 to 2026 by comparing modern open models with chronologically consistent models, which are trained only on text from before a fixed date. Asked about past GDP outcomes given only a date, models that cannot know the answer still reach an AUC of up to 0.64 from priors on dates and eras, so chance is the wrong baseline. Against these memory-free models on the same questions, most modern models recall which years had strong growth. Membership-inference scores sort documents by era and document type rather than by whether a model saw them, as exact counts in an open pretraining corpus show. In real-time nowcasts, every lookahead premium we estimate on US GDP is smaller than the forecast comparison can detect at macroeconomic sample sizes, and none on GDP or inflation survives correction for multiple testing. Lookahead audits should therefore measure memory directly against memory-free models on the same items.
Figures & tables
Figure 1: The pipeline at one forecast origin τ . A temporal gate routes each FOMC document to the ChronoGPT checkpoint whose training ended the year before it; ungated readers see every document. An output gate turns next-token label scores into a five-dimensional policy signal and abstains when three wordings of a question disagree in sign. The same readers answer date-only memory probes. At each origin the nowcaster takes one vintage of the Philadelphia Fed Real-Time Data Set (RTDSM), the eleven indicators as published by that date. It fills the unpublished months, the ragged edge, with a second-order autoregression on each indicator’s monthly changes (AR(2) fill) and reduces the target’s block of indicators to three principal components (block PCA). A ridge regression of the target on these factors and any text signal, the ridge bridge, gives the nowcast y^τ . Every nowcast is scored against the first release, the first published estimate of the quarter.
Figure 2: The study on one calendar. ChronoGPT checkpoints end on 31 December of each year from 1999 to 2024; a document is read by the checkpoint that ends the year before it. The modern readers were released in 2024, so their training text covers most of the test window. Four fixed ChronoGPT checkpoints, and three DatedGPT checkpoints on the probes, form the ungated ladder. Specification selection uses only origins before the test window.
Figure 3: Lookahead premium of each leaky reader over the gated reader, all quarters (dot, 95% quarter-block interval) and without 2021 (bar).
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: The ChronoGPT ladder: each checkpoint on outcomes it could have seen (filled) and could not (open), GDP probe AUC (left) and GDP nowcast premium over the gated reader with 95% quarter-block intervals (right). Memory predicts filled above open; no difference is resolvable.
Figure 5: (a) First-release GDP growth and three nowcasts at the first origin of each quarter. (b) Running sum of the gated reader’s squared error minus the ungated ChronoGPT-2024 reader’s: the gap rises before 2020, falls in 2020 and rises in 2021.
Figure 6: Share of document-dimension readings set to zero because the three wordings of the question disagreed in sign. Against its unguarded signal, the gated reader’s guarded signal changes GDP accuracy by −2.3 %; with unguarded signals throughout, the GDP premiums run from −3.0 % to +1.3 % instead of −0.9 % to +3.4 % (Table 3 ).
Data contamination in language models examines how training exposure to evaluation data affects measured capability, benchmark validity, and model behavior.
Department of Electronics and Communication Engineering, Jaypee Institute of Information Technology, Noida, India · Department of Computer Science and Engineering, Indian Institute of Technology Patna, India