PALM: Point-in-Time Adaptation for Financial Language Models

Seunghan Lee; Jun Seo; Jaehoon Lee; Junhyeok Kang; Sangjun Han; Sungdong Yoo; Minjae Kim; Tae Yoon Lim

cs.LGSep 2, 2026

PALM: Point-in-Time Adaptation for Financial Language Models

Authors: Seunghan Lee, Jun Seo, Jaehoon Lee, Junhyeok Kang, Sangjun Han, Sungdong Yoo, Minjae Kim, Tae Yoon Lim, +4 more

Organizations: LG AI Research

Abstract

Language models used in financial backtests suffer from look-ahead bias, as a model trained on text published after the study period has already observed the outcomes it is asked to predict. To handle this issue, point-in-time (PIT) language models are pretrained on chronologically filtered corpora and released as one checkpoint per calendar year, each with a documented cutoff. However, each additional year costs a full pretraining run, and whether that run is necessary has never been tested. In this paper, we show that the annual pretraining run is not necessary. We instead compare each checkpoint against the newer one that replaced it, and find that the newer checkpoint scores no better on the same evaluation window. Motivated by this observation, we propose PALM (Point-in-time Adaptation for financial Language Models), a simple yet effective alternative to annual pretraining that fits a low-rank adapter on text published before the decision date without modifying any pretrained weight. We further find that a small adapter is enough to add a new period to the knowledge an old checkpoint already encodes, and that this outperforms continued pretraining. We validate PALM on a decade of financial news and on various families of PIT models, whose cutoffs span two decades and whose sizes range from 1.3 to 4.2B. Code is available at: https://github.com/seunghan96/palm.

Figures & tables

Figure 1. Naive LLM vs. PIT LLM vs. PALM (Ours). A naive LLM has read the evaluation period and is ineligible. A PIT suite avoids this with one pretraining run per year, whereas PALM needs one model and one small adapter per year. A timeline from 2017 to 2029 with three rows. The naive model reads text past the decision date at 2026 and its overrun is hatched as look-ahead bias. The point-in-time suite is four bars, one per calendar year, each stopping before the decision date. Our row is one bar of the same pretrained text followed by four small adapter blocks, one per year.

Line of work	Point-in-Time	Finance	Adaptation
Financial LLMs ( Ke et al., 2019 ; Lopez-Lira and Tang, 2026 ; Lee et al., 2026 )		✓
PIT language models ( He et al., 2025a ; Kelly et al., 2026 ; Yan et al., 2026 )	✓	✓
Continued pretraining ( Lazaridou et al., 2021 ; Dhingra et al., 2022 ; Luu et al., 2022 )			✓
Adapters ( Houlsby et al., 2019 ; Hu et al., 2022 )			✓
PALM (Ours)	✓	✓	✓

Table 1. Positioning of PALM. Point-in-Time means that no text published after the decision date was read.

Figure 2. Key concepts and notation. A timeline from 2017 to 2029. Four horizontal bars, bracketed on the left as one point-in-time suite, each covering the text one vintage has read up to its own cutoff. Three bars stop before the decision date at 2026 and are marked eligible, and one runs past it into the shaded evaluation window and is marked ineligible. A dotted line marks the cutoff of the topmost vintage and a double arrow from it to the decision date is labelled staleness.

Figure 3. Overall framework of PALM. (a) Signal Extraction : A dated article is turned into a signal by contrasting upward against downward continuations in a single forward pass, and the signals are ranked within a date. The blue square marks the position whose next-token distribution is read. (b) Point-in-Time Adaptation : Every pretrained weight is frozen and a low-rank adapter is fitted on the text published between the checkpoint cutoff and the decision date. Two boxed panels. The left panel takes a dated news article, ends the prompt where a direction word would follow, reads the log-odds between upward and downward continuations out of one forward pass, and ranks the assets of that date. The right panel freezes the pretrained weights and trains a rank-r adapter on the text published between the cutoff and the decision date, with an arrow showing that the adapter reads only that interval.

Component	Setting
[1] Data
Corpus	FNSPID financial news ( Dong et al., 2024 )
Universe	1,099 US tickers
Panel span	2014 to 2023
Target	Next-session return
[2] Models

Table 2. Experimental setup: Data, Models, Evaluation.

Suite	Scored		Eligible		Excluded cutoffs
Suite	$n$	Cutoffs	$n$	Cutoffs	Excluded cutoffs
ChronoGPT	26	1999–2024	21	1999–2019	2020 to latest
ChronoGPT-Instruct	21	2004–2024	16	2004–2019
DatedGPT	12	2013–2024	7	2013–2019
Total	59	—	44	—	15

Table 3. Model checkpoints. Every suite releases one vintage per calendar year without gaps, and the counts differ only through where each suite starts and where it stops.

Suite	Slope of IC per year	$p$ -value
ChronoGPT	+0.00004	0.40
ChronoGPT-Instruct	+0.00011	0.12
DatedGPT	+0.00034	0.55
Pooled	+0.00007	0.14

Table 4. Effect of staleness. No suite carries the negative slope the annual pretraining run is meant to prevent, meaning that a further year leaves the signal where it found it .

Backbone		IC	Return	Sharpe	Wins
ChronoGPT	$-$	7.50	8.13	1.42	20/21 (95%)
ChronoGPT	$+$ PALM	9.18	9.33	1.60	20/21 (95%)
ChronoGPT-Instruct	$-$	9.28	8.90	1.52	14/16 (88%)
ChronoGPT-Instruct	$+$ PALM	9.87	8.94	1.50	14/16 (88%)
DatedGPT	$-$	5.51	2.98	0.55	6/7 (86%)
DatedGPT	$+$ PALM	7.02	3.62	0.67	6/7 (86%)

Table 5. Effectiveness of PALM across backbones. The same adapter raises the IC on every suite, showing that the gain does not depend on which model PALM is attached to .

Figure 4. Effect of staleness and of PALM. (a) Information coefficient against staleness on the fixed evaluation window, which leaves the checkpoint as the only quantity varying between points. Points are demeaned within suite, matching the specification of Table 4 . (b) The same checkpoints before and after PALM, paired on the days both systems scored. Two panels. The left one scatters the information coefficient against staleness for every eligible checkpoint of three suites, with a fitted line that does not slope downward. The right one connects each checkpoint to itself before and after the adapter, and almost every connector rises.

Figure 5. Effect of the adapter capacity. The gain in the IC is positive at every adapter rank

r

, and a small rank already reaches it, showing that a few directions carry the update . The gain in the information coefficient plotted against adapter rank on a log scale from one to thirty-two, with a bootstrap interval at each rank. The curve is positive throughout, rises to a peak at small rank, and falls towards zero at the largest rank.

Backbone: PIT-4B	IC	Return	Sharpe
w/o update	4.38	4.08	0.72
w/ continued pretraining	4.48	4.30	0.77
w/ PALM	5.76	4.97	0.89

Table 6. Continued pretraining against PALM, on PIT-4B ( Kelly et al., 2026 ) , whose vintages come from one run. Six more years of the run leave the score where they found it while the adapter moves it, showing that compute is not the missing ingredient .

Age of base	IC		Return		Sharpe
Age of base	$-$	$+$ PALM	$-$	$+$ PALM	$-$	$+$ PALM
0 to 3 years	6.58	7.71	6.69	6.70	1.17	1.13
4 to 6 years	7.72	9.29	7.31	7.60	1.26	1.29
7 to 10 years	7.97	9.23	8.57	9.78	1.47	1.66
11 years or more	8.47	9.63	7.70	8.68	1.34	1.51

Table 7. How often a suite must be retrained. The adapted score does not fall as the base ages, and the oldest group adapted outscores the newest group unadapted, showing that no retraining horizon appears within two decades of cutoffs . The daily difference in information coefficient between the adapted and unadapted checkpoints, accumulated over the evaluation window. The curve climbs steadily from the start of the window to its end rather than jumping in one stretch.

The update raises the score
Ticker	Headline	Shift	Return
GCC	Commodity ETFs rally on the Ukraine war	$+2.01$	$+1.31\%$
VYM	The most popular equity income ETFs	$+1.81$	$-0.25\%$
TWLO	Three tech stocks in the sweet spot	$+1.73$	$+1.23\%$
The update lowers the score
NIO	Why Tesla, Nio and Nikola stocks fell	$-2.23$	$-0.57\%$

Table 8. Articles whose score moves most under PALM, for a 2015 checkpoint. Most of them concern a firm or an event the vintage had never encountered, showing that the update lands on the material the vintage could not have seen . A horizontal bar chart of the words whose score moves most under the adapter. Terms are listed on the vertical axis and the mean shift in standard deviations on the horizontal one, with roughly equal numbers moving up and down.

(a) What the gain needs
What was changed	IC		Return		Sharpe
What was changed	Gain	Wins	Gain	Wins	Gain	Wins
Words scrambled, rank 2	+0.48	59%	-0.24	52%	-0.05	43%
Words scrambled, rank 8	-0.19	43%	-2.39	23%	-0.39	23%
(b) What it could reach
Text read past the date	+1.05	84%	+0.83	57%	+0.12	57%

Table 9. What the gain needs, and what it could reach. The gain needs the eligible text, and nothing beyond it .

Budget	IC		Return		Sharpe
Budget	Gain	Wins	Gain	Wins	Gain	Wins
200 steps	+0.17	55%	+0.42	55%	+0.05	52%
600 steps	+1.26	91%	+0.69	61%	+0.10	61%
1200 steps	+1.29	84%	+0.69	55%	+0.08	52%
Rate $2\times 10^{-5}$	+0.14	59%	+0.23	66%	+0.02	61%

Table 10. Sensitivity analysis 1 - Training budget. The gain appears only once the adapter has been trained enough to fit the corpus, showing that the update has to be trained and not merely attached to the model .

Decision date	IC		Return		Sharpe
Decision date	Gain	Wins	Gain	Wins	Gain	Wins
2019	+1.09	87%	+1.46	74%	+0.18	71%
2021	+1.26	91%	+0.69	61%	+0.10	61%
2022	+0.47	66%	+0.31	47%	+0.03	45%

Table 11. Sensitivity analysis 2 - Decision date. Moving the date by two years in either direction leaves the gain in place, showing that the result does not depend on our window .

Projection	IC		Return		Sharpe
Projection	Gain	Wins	Gain	Wins	Gain	Wins
Attention	+0.63	74%	+0.01	49%	-0.05	47%
FFN	+0.82	82%	+0.18	57%	+0.04	57%
Attention + FFN	+1.26	91%	+0.69	61%	+0.10	61%

Table 12. Sensitivity analysis 3 - Adapted projections. Neither half of the block reproduces the gain on its own, showing that the update is not carried by one part of the layer (FFN: feed-forward projections).

% of text	IC		Return		Sharpe
% of text	Gain	Wins	Gain	Wins	Gain	Wins
10%	+1.02	80%	+0.75	66%	+0.09	59%
25%	+1.01	82%	+0.42	59%	+0.05	59%
50%	+1.11	86%	+0.53	57%	+0.07	45%
100%	+1.26	91%	+0.69	61%	+0.10	61%

Table 13. Sensitivity analysis 4 - Share of eligible text. A tenth of the corpus carries four fifths of what all of it delivers, showing that the gain sits in the text nearest the decision date .

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

The panel
Property	Value
Source	FNSPID financial news ( Dong et al., 2024 )
Universe	1,099 US tickers
Span	2 January 2014 to 27 December 2023
Observations	252,000 article-days over 2,514 trading days
Articles per day	99 at the median, 100 on average

Appendix

Table 14. Details of FNSPID. Fewer than a third of the observations sit in the evaluation window, and every eligible vintage is scored on all of them, showing that the comparison rests on the same articles whichever checkpoint is being read .

Suite	Hugging Face identifier	Width	Vintages
Suite	Hugging Face identifier	Width	Released	Eligible
ChronoGPT	manelalab/chrono-gpt-v1-*	1536	26 (1999–2024)	21 (1999–2019)
ChronoGPT-Instruct	manelalab/chrono-gpt-instruct-v1-*	1536	21 (2004–2024)	16 (2004–2019)
DatedGPT	datedgpt/datedgpt-*-base	2048	12 (2013–2024)	7 (2013–2019)
PIT-4B	Diamegs/PIT-4B-*	4096	Monthly (2013–2024)	7 used (2013–2019)

Appendix

Table 15. The four suites. The three suites we pair release one vintage per calendar year without gaps and their eligible counts differ only through where each suite starts, while PIT-4B publishes monthly checkpoints from which we take one per calendar year, showing that the paired pool is set by release history rather than by any choice of ours .

The adapter
Setting	Value
Rank and scale	$r=2$ , $\alpha=2r$
Placement	Attention and feed-forward projections of every block
Initialization	$A$ random
	$B=0$
	Every pretrained weight frozen

Appendix

Table 16. The settings that hold throughout. Everything except the rank, the steps and the rate is fixed once and never tuned per suite, showing that the gain is not the product of a search over configurations .

Suite	IC		Return		Sharpe
Suite	Slope	$p$ -value	Slope	$p$ -value	Slope	$p$ -value
ChronoGPT	+0.00004	0.40	-0.0024	$<$ 0.01	-0.038	$<$ 0.01
ChronoGPT-Instruct	+0.00011	0.12	+0.0002	0.83	+0.003	0.82
DatedGPT	+0.00034	0.55	+0.0003	0.96	+0.014	0.90
Pooled	+0.00007	0.14	-0.0016	0.01	-0.024	0.01

Appendix

Table 17. Effect of staleness on all three measures. The IC slope is positive in every suite while the portfolio slopes fall, and the only significant portfolio slopes belong to the one suite that supplies half the pool, showing that the disagreement between the two kinds of score rests on a single release history .

Regressand	Slope on staleness	$p$ -value
$-$	$+0.000066$	0.14
$+$ PALM	$+0.000038$	0.35
Gain	$-0.000029$	0.33

Appendix

Table 18. The age of the base checkpoint, as regressions rather than as groups. Neither the adapted score nor the gain over the unadapted checkpoint declines with staleness, showing that the grouping of Table 6 survives a continuous test .

ChronoGPT			ChronoGPT-Instruct			DatedGPT
Cutoff	$-$	$+$ PALM	Cutoff	$-$	$+$ PALM	Cutoff	$-$	$+$ PALM
1999	10.61	10.58	2004	8.00	8.46	2013	5.74	6.85
2000	8.73	9.97	2005	9.02	10.23	2014	5.95	4.75
2001	6.80	8.37	2006	13.01	11.90	2015	10.48	11.03
2002	6.90	9.50	2007	9.52	9.28	2016	2.86	7.96
2003	5.40	7.21	2008	9.69	10.36	2017	3.86	4.00

Appendix

Table 19. Every eligible checkpoint, without and with PALM. The update raises the IC on 40 of the 44, and the four it does not are spread across all three suites, showing that the average is not carried by a handful of checkpoints .

	IC		Return		Sharpe
Rank	Gain	Wins	Gain	Wins	Gain	Wins
1	+0.74	77%	+0.22	61%	+0.02	59%
2	+1.26	91%	+0.69	61%	+0.10	61%
3	+1.37	91%	+0.96	68%	+0.15	66%
4	+1.32	95%	+0.95	59%	+0.14	55%
6	+1.26	84%	+0.65	59%	+0.09	57%

Appendix

Table 20. Effect of the adapter capacity, on all three reported metrics. All three peak at rank three and fall away from it, and the portfolio measures turn negative beyond rank twelve while the IC stays positive, showing that capacity past a few directions costs the portfolio measures more than it buys them .

Backbone	Width	$r{=}1$	$r{=}2$	$r{=}4$	$r{=}8$	$r{=}16$
ChronoGPT	1536	+0.85	+1.68	+1.83	+1.72	+0.93
ChronoGPT-Instruct	1536	+0.39	+0.59	+0.47	-0.15	-0.52
DatedGPT	2048	+1.19	+1.50	+1.72	+0.76	+1.81
PIT-4B	4096	+0.67	-0.05	+0.07	+0.37	+1.16

Appendix

Table 21. Adapter capacity against backbone width. The best rank is four or less on the narrow suites (ChronoGPT, ChronoGPT-Instruct) and sixteen on the wide ones (DatedGPT, PIT-4B), and the rank we report is therefore too small for the widest suite rather than ineffective, showing that the capacity of the update should be read against the width of the backbone .

Wording	$\mathcal{W}^{+}$ / $\mathcal{W}^{-}$	IC		Gain		$p$
Wording	$\mathcal{W}^{+}$ / $\mathcal{W}^{-}$	$-$	$+$ PALM	IC	Wins	$p$
	rise, increase, go up, gain
Reported	fall, decrease, go down, drop	6.65	8.32	$+1.67$	18/19 (95%)	$<$ 0.001
	climb, advance, strengthen, rally
Alternative A	slide, retreat, weaken, tumble	5.23	8.27	$+3.04$	17/19 (89%)	0.0003
	up, higher, better, stronger

Appendix

Table 22. The same comparison read through three disjoint sets of direction words. The unadapted score moves with the wording while the adapted score does not, showing that the gain belongs to the update and not to the words that read it out .

Ticker	$f_{\theta}$	Return	Headline
ABT	$+1.53$	$-2.22\%$	Abbott rewards shareholders as shares peak
INCY	$+1.42$	$+1.15\%$	Incyte reports a survival benefit
VALE	$+1.36$	$-0.73\%$	Nucor acquires a majority stake
SPWR	$-1.22$	$-10.78\%$	California to reform solar policy
PCG	$-1.27$	$-0.90\%$	California to reform solar policy
AAL	$-1.27$	$-1.23\%$	Why airline stocks are falling

Appendix

Table 23. One scored date, 13 December 2021, for the ChronoGPT vintage cut at the end of 2019. Signals are compared within the date, and the individual outcomes disagree with the ordering as often as they agree, showing that the signal lives in the average over many dates rather than in any one of them .

	IC		Return		Sharpe
Schedule	Gain	Wins	Gain	Wins	Gain	Wins
One pass, whole window	+1.26	91%	+0.69	61%	+0.10	61%
One pass per year	+0.80	80%	+0.34	64%	+0.04	57%

Appendix

Table 24. Adaptation schedule. Reading the eligible interval once beats walking it year by year, showing that the update does not need to follow the order the text was published in .

	IC		Return		Sharpe
Date	Gain	Wins	Gain	Wins	Gain	Wins
2017	+1.33	97%	+2.01	97%	+0.28	91%
2018	+1.70	94%	+2.13	89%	+0.30	89%
2019	+1.09	87%	+1.46	74%	+0.18	71%
2020	+1.41	85%	+1.26	63%	+0.17	61%
2021	+1.26	91%	+0.69	61%	+0.10	61%

Appendix

Table 25. Decision date. The IC gain is positive at every date, showing that the reading of the IC does not depend on where the window sits .

	IC		Return		Sharpe
Share	Gain	Wins	Gain	Wins	Gain	Wins
5%	+0.97	78%	+1.07	73%	+0.15	66%
10%	+1.02	80%	+0.75	66%	+0.09	59%
25%	+1.01	82%	+0.42	59%	+0.05	59%
50%	+1.11	86%	+0.53	57%	+0.07	45%
75%	+1.05	84%	+0.44	61%	+0.06	55%

Appendix

Table 26. Share of the eligible text. A small slice nearest the decision date carries most of the gain, showing that what the update needs is the text closest to the date rather than the volume of it .

Number of steps, at a rate of $10^{-4}$
	IC		Return		Sharpe
Budget	Gain	Wins	Gain	Wins	Gain	Wins
100 steps	-0.00	58%	+0.05	51%	-0.00	47%
200 steps	+0.17	55%	+0.42	55%	+0.05	52%
300 steps	+0.37	73%	+0.30	59%	+0.03	57%
600 steps	+1.26	91%	+0.69	61%	+0.10	61%

Appendix

Table 27. Training budget. The gain appears only once the adapter has been trained past a floor and fades once it is trained well beyond it, showing that the update has to be trained rather than merely attached .

	IC		Return		Sharpe
Tokens	Gain	Wins	Gain	Wins	Gain	Wins
64 tokens	+1.30	91%	+0.65	59%	+0.07	57%
128 tokens	+1.33	93%	+0.68	59%	+0.09	59%
256 tokens	+1.26	91%	+0.69	61%	+0.10	61%
512 tokens	+1.18	89%	+0.59	61%	+0.09	64%
1024 tokens	+1.19	91%	+0.70	61%	+0.10	61%

Appendix

Table 28. Tokens read per article. Sixty-four tokens deliver what a thousand do, showing that the update is carried by the opening of an article rather than by its length .

	IC		Return		Sharpe
Seed	Gain	Wins	Gain	Wins	Gain	Wins
Seed 0	+1.26	91%	+0.69	61%	+0.10	61%
Seed 1	+1.11	91%	+0.57	59%	+0.08	59%
Seed 2	+1.24	91%	+0.86	68%	+0.12	55%
Seed 3	+1.08	82%	+0.56	57%	+0.07	50%
Seed 4	+1.35	85%	+0.73	64%	+0.10	62%

Appendix

Table 29. Random seed. Seven seeds place the IC gain between

+1.08

and

+1.35

, showing that the result is not an artifact of the one initialization we happened to report .

Explore similar work

Scaling Point-in-Time Language Models
Apr 24, 2026Bryan Kelly, Semyon Malamud, Johannes Schwab +1Decoder-Only Language Models Temporal Data Leakage
Yale School of Management · AQR Capital Management · NBER +3
Summoning the Oracle to Slay It: Mitigating Look-Ahead Bias in Financial Backtesting with Large Language Models
May 23, 2026Weixian Waylon Li, Mengyu Wang, Tiejun MaLanguage Model Decoding Quantitative Finance
University of Edinburgh
A Fast and Effective Solution to the Problem of Look-ahead Bias in LLMs
Dec 7, 2025Humzah Merchant, Bradford LevyLLM-Based Time Series Forecasting LLM Unlearning
University of Chicago Chicago, IL 60637

Apr 24, 2026, cs.CLJ/K move · Enter open · S save

Scaling Point-in-Time Language Models

Bryan Kelly, Semyon Malamud, Johannes Schwab, Teng Andrea Xu

Yale School of Management · AQR Capital Management · NBER+3

Decoder-Only Language Models Temporal Data Leakage LLM Training

Open paper AskPDF arXiv

Large language models trained on unrestricted internet corpora inevitably embed information from the future, introducing lookahead bias that compromises the validity of backtests and causal inference in finance and the social sciences. Point-in-time language models--trained exclusively on text available up to each calendar date--eliminate this leakage by construction, but existing efforts typically produce models that lag substantially behind their unconstrained counterparts. We show that this performance gap can be substantially narrowed through scale. Training decoder-only transformers with up to 4 billion parameters on 1 trillion chronologically filtered tokens from FineWeb, we construct a sequence of monthly model checkpoints spanning 2013-2024. Across a range of common-sense reasoning and language understanding benchmarks, our models approach the performance of leading open-weight models of comparable size (e.g., Gemma-3-4B and LLaMA-7B) trained on temporally unrestricted data, although a performance gap remains on several tasks. Instruction fine-tuning via LoRA further improves downstream usability. We release the complete pipeline--including dataset construction, training infrastructure, and evaluation code--to enable reproducible point-in-time language modeling and to support research applications that require strict temporal validity.

Abstract

Explore similar work

Scaling Point-in-Time Language Models

Summoning the Oracle to Slay It: Mitigating Look-Ahead Bias in Financial Backtesting with Large Language Models

A Fast and Effective Solution to the Problem of Look-ahead Bias in LLMs