Rethinking the Role of LLMs in Time Series Forecasting

Xin Qiu; Junlong Tong; Yirong Sun; Yunpu Ma; Wei Zhang; Xiaoyu Shen

cs.CLFeb 16, 2026

Rethinking the Role of LLMs in Time Series Forecasting

Authors: Xin Qiu, Junlong Tong, Yirong Sun, Yunpu Ma, Wei Zhang, Xiaoyu Shen

Organizations: Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative, Institute of Digital Twin, Eastern Institute of Technology, Ningbo · Zhejiang University · LMU Munich

Abstract

Large language models (LLMs) have been introduced to time series forecasting (TSF) to incorporate contextual knowledge beyond numerical signals. However, existing studies question whether LLMs provide genuine benefits, often reporting comparable performance without LLMs. We show that such conclusions stem from limited evaluation settings and do not hold at scale. We conduct a large-scale study of LLM-based TSF (LLM4TSF) across 8 billion observations, 17 forecasting scenarios, 4 horizons, multiple alignment strategies, and both in-domain and out-of-domain settings. Our results demonstrate that LLM4TS indeed improves forecasting performance, with especially large gains in cross-domain generalization. Pre-alignment outperforming post-alignment in over 90% of tasks. Both pretrained knowledge and model architecture of LLMs contribute and play complementary roles: pretraining is critical under distribution shifts, while architecture excels at modeling complex temporal dynamics. Moreover, under large-scale mixed distributions, a fully intact LLM becomes indispensable, as confirmed by token-level routing analysis and prompt-based improvements. Overall, Our findings overturn prior negative assessments, establish clear conditions under which LLMs are not only useful, and provide practical guidance for effective model design. We release our code at https://github.com/EIT-NLP/LLM4TSF.

Figures & tables

Figure 1: Two mainstream alignment strategies for LLM4TSF.

Figure 2: Comparison of LLM4TSF with pre-alignment and post-alignment strategies under single-dataset learning.

Figure 3: Comparison of LLM4TSF performance with pre- and post-alignment under single- and cross-dataset paradigm. Negative and Positive values indicate MAE decreases and increases under cross-dataset learning compared to single-dataset learning.

Types	Zero-Shot Test					5% Few-Shot Test
Models	Pre-align	Post-align	Chronos	UniTS	Moirai	UniTime	TimeLLM
Wind	1.015	0.963	1.422	1.358	1.236	1.358	1.321
Solar	0.228	0.274	0.434	0.871	0.936	0.218	0.577
AQShunyi	0.612	0.688	0.808	0.890	0.668	0.905	0.859
CzenLan	0.274	0.288	0.298	0.738	0.660	0.401	0.319
ZafNoo	0.547	0.583	0.550	0.668	0.543	0.803	0.594

Table 1: Out-of-domain test performance (MSE, averaged over horizons

\{96,192,336,720\}

). Bold and underlined indicate the best and second-best results for each dataset.

Figure 4: Original architecture and two ablation variants.

Models	LLM4TSF (Pre-align)						LLM4TSF (Post-align)
	w/ Pre-training		w/o Pre-training		w/o LLM		w/ Pre-training		w/o Pre-training		w/o LLM
	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE
In-Domain Test
ETTh1	0.445	0.447	0.460	0.471	0.432	0.438	0.436	0.431	0.428	0.420	0.435	0.442
ETTh2	0.388	0.348	0.414	0.377	0.407	0.363	0.396	0.357	0.429	0.403	0.439	0.424
ETTm1	0.375	0.353	0.393	0.382	0.363	0.328	0.384	0.354	0.373	0.340	0.378	0.351

Table 2: Ablation results of LLM4TSF (Pre-align) and LLM4TSF (Post-align) , averaged over horizons

\{96,192,336,720\}

. For each dataset, the best MAE is highlighted in red , and the best MSE is highlighted in blue , respectively.

Figure 5: Statistical properties of different datasets. Datasets highlighted in red indicate cases where the w/ pre-training setting underperforms the corresponding ablation baselines in test.

Figure 6: As shifting and transition increase, TSF becomes more challenging with higher MAE, while the advantage of LLMs becomes evident. Moreover, even w/o pre-training, a trainable LLM backbone outperforms non-LLM under high-transition regimes.

Figure 7: Relationship between the token ratio passing through the LLM (averaged over horizons

\{96,192,336,720\}

) and shifting across different datasets. The ”- - -” & ”- - -” indicate the average ratios of each dataset, and the ”- - -” denotes the reference value of 0.5 .

Figure 8: Routing analysis illustration. For each TS token, a trained router decides whether to route it to the LLM or skip. The LLM backbone is either w/ pre-training or w/o pre-training.

Figure 9: (a) With a pre-trained LLM backbone, tokens of passing LLM are concentrated in high shifting regions. (b) With pre-training, tokens of passing LLM are concentrated in high transition.

	LLM4TSF (Pre-align)			LLM4TSF (Post-align)
	w/ Pre- training	w/o Pre- training	Change	w/ Pre- training	w/o Pre- training	Change
In-Domain Test
ETTh1	17%	24%	+7%	31%	57%	+26%
ETTh2	64%	43%	-21%	71%	62%	-9%
ETTm1	19%	14%	-5%	24%	66%	+42%
ETTm2	73%	28%	-45%	76%	68%	-8%

Table 3: Impact of pretrained LLM parameters on the token passing ratio through the LLM, averaged over horizons

96,192,336,720

Figure 10: LLM4TSF(Post-align) performance and token passing ratios through the LLM under three training strategies: full-parameter fine-tuning, LoRA fine-tuning, and training PE+LN.

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 11: Proportional distribution of training datasets across ten real-world domains.

Dataset	Variables	Timestamps	Dataset	Variables	Timestamps
ETTh1	7	14,400	Wind	7	48,673
ETTh2	7	14,400	Solar	137	52,560
ETTm1	7	57,600	AQShunyi	11	35,064
ETTm2	7	57,600	CzenLan	11	19,934
Weather	21	52,696	ZafNoo	11	19,225
Traffic	862	17,544	NASDAQ	5	1,244

Appendix

Table 4: Summary of datasets used in experiments. The table reports the number of variables and timestamps for each time series dataset.

Figure 12: Analysis of the 17 in-domain and out-of-domain datasets from Shifting, Stationarity, Transition, Seasonality, and Trend.

Input: Time series $\mathbf{X}\in\mathbb{R}^{T\times 1}$
Output: Shifting $\delta\in(0,1)$
1:	Normalize $\mathbf{X}$ using z-score normalization to obtain $\mathbf{Z}\in\mathbb{R}^{T\times 1}$ .
2:	Compute $Z_{\min}=\min(\mathbf{Z})$ and $Z_{\max}=\max(\mathbf{Z})$ .
3:	Construct $m$ uniformly spaced value levels:
	$\ell_{i}=Z_{\min}+\frac{i-1}{m-1}(Z_{\max}-Z_{\min}),\quad i=1,\ldots,m.$
4:	for each level $\ell_{i}$ do

Appendix

Table 5: Algorithm: Shifting Computation

Input: Time series $\mathbf{X}=\langle x_{1},x_{2},\ldots,x_{T}\rangle\in\mathbb{R}^{T\times 1}$
Output: Stationarity indicator $\gamma\in\{0,1\}$ of $\mathbf{X}$
1:	Compute the Augmented Dickey–Fuller (ADF) statistic $s\leftarrow\mathrm{ADF}(\mathbf{X})$ .
2:	return $\gamma=\begin{cases}1,&\text{if }s\leq 0.05,\\ 0,&\text{otherwise}.\end{cases}$

Appendix

Table 6: Algorithm 2: Stationarity Computation

Input: Time series $\mathbf{X}\in\mathbb{R}^{T\times 1}$
Output: Transition $\Delta\in\left(0,\tfrac{1}{3}\right)$
1:	Estimate the characteristic lag $\tau$ as the first zero-crossing point of the autocorrelation function of $\mathbf{X}$ .
2:	Downsample $\mathbf{X}$ with stride $\tau$ to obtain a reduced sequence $\mathbf{Y}\in\mathbb{R}^{T^{\prime}\times 1}$ .
3:	Obtain the rank ordering of $\mathbf{Y}$ by computing the permutation index $\mathbf{r}=\mathrm{argsort}(\mathbf{Y}).$
4:	Discretize $\mathbf{Y}$ into a symbolic sequence $\mathbf{Z}\in\{0,1,2\}^{T^{\prime}}$ via $Z_{j}=\left\lfloor\frac{3\,r_{j}}{T^{\prime}}\right\rfloor,\quad j=1,\ldots,T^{\prime}.$
5:	Initialize a transition count matrix $\mathbf{M}\in\mathbb{R}^{3\times 3}$ with zeros.

Appendix

Table 7: Algorithm: Transition Computation

Input: Time series $\mathbf{X}=\{x_{1},x_{2},\ldots,x_{T}\}\in\mathbb{R}^{T\times 1}$
Output: Seasonality value $\zeta\in(0,1)$
1:	Decompose $\mathbf{X}$ into seasonal, trend, and residual components using STL: $\mathbf{X}=\mathbf{S}+\mathbf{T}+\mathbf{R}.$
2:	Compute the variance of the residual component $\mathbf{R}$ .
3:	Compute the variance of the combined seasonal and residual component $\mathbf{S}+\mathbf{R}$ .
4:	return $\zeta=\max\!\left(0,\;1-\frac{\mathrm{var}(\mathbf{R})}{\mathrm{var}(\mathbf{S}+\mathbf{R})}\right).$

Appendix

Table 8: Algorithm: Seasonality Computation

Input: Time series $\mathbf{X}=\{x_{1},x_{2},\ldots,x_{T}\}\in\mathbb{R}^{T\times 1}$
Output: Trend value $\beta\in(0,1)$
1:	Decompose $\mathbf{X}$ into seasonal, trend, and residual components using STL: $\mathbf{X}=\mathbf{S}+\mathbf{T}+\mathbf{R}.$
2:	Compute the variance of the residual component $\mathbf{R}$ .
3:	Compute the variance of the combined non-seasonal component $\mathbf{T}+\mathbf{R}$ .
4:	return $\beta=\max\!\left(0,\;1-\frac{\mathrm{var}(\mathbf{R})}{\mathrm{var}(\mathbf{T}+\mathbf{R})}\right).$

Appendix

Table 9: Algorithm: Trend Computation

Figure 13: Comparison of LLM4TSF performance with pre- and post-alignment under single- and cross-dataset paradigm. Negative and Positive values indicate MSE decreases and increases under cross-dataset learning compared to single-dataset learning.

Figure 14: Effect of training data on cross-dataset learning performance. Both MAE and MSE decrease as the ratio of data increases.

Dataset	Horizon	Pre-Single	Pre-Cross	Post-Single	Post-Cross
ETTh1	96	0.407	0.414	0.421	0.398
	192	0.435	0.437	0.432	0.420
	336	0.452	0.448	0.462	0.446
	720	0.461	0.479	0.499	0.481
ETTh2	96	0.368	0.333	0.374	0.352
	192	0.399	0.374	0.416	0.379

Appendix

Table 10: MAE comparison under different settings.

Dataset	Horizon	Pre-Single	Pre-Cross	Post-Single	Post-Cross
ETTh1	96	0.404	0.412	0.420	0.377
	192	0.435	0.439	0.455	0.416
	336	0.472	0.455	0.488	0.444
	720	0.485	0.481	0.566	0.488
ETTh2	96	0.303	0.279	0.307	0.283
	192	0.361	0.347	0.375	0.354

Appendix

Table 11: MSE comparison under different settings.

Figure 15: Effect of model truncation on forecasting performance.

	In-Domain Test				Out-of-Domain Test
	w/ Prompt		w/o Prompt		w/ Prompt		w/o Prompt
LLM	MSE	Ratio(%)	MSE	Ratio(%)	MSE	Ratio(%)	MSE	Ratio(%)
LLM4TSF(Pre-align)
GPT-2	0.471	46.7	0.488 $\uparrow$	41.5 $\downarrow$	0.524	72.6	0.673 $\uparrow$	37.9 $\downarrow$
Qwen-3 0.6B	0.485	48.2	0.495 $\uparrow$	42.2 $\downarrow$	0.541	70.9	0.726 $\uparrow$	40.6 $\downarrow$
Qwen-3 1.7B	0.477	45.9	0.502 $\uparrow$	42.9 $\downarrow$	0.529	74.6	0.679 $\uparrow$	35.5 $\downarrow$

Appendix

Table 12: Comparison of average MSE and token ratios across different LLM backbones under w/ prompt or w/o prompt.

Dataset	Horizon	Chronos	UniTS	MOIRAI	UniTime	TimeLLM
Wind	96	0.696	0.755	0.640	0.685	0.664
	192	0.767	0.823	0.722	0.786	0.767
	336	0.848	0.881	0.809	0.878	0.862
	720	0.934	0.945	0.866	0.953	0.945
Solar	96	0.327	0.611	0.477	0.281	0.313
	192	0.339	0.655	0.526	0.270	0.377

Appendix

Table 13: MAE of baseline out-of-domain test performance.

Dataset	Horizon	Chronos	UniTS	MOIRAI	UniTime	TimeLLM
Wind	96	1.250	1.038	0.963	1.022	0.981
	192	1.357	1.224	1.199	1.241	1.201
	336	1.428	1.516	1.268	1.482	1.444
	720	1.653	1.653	1.513	1.688	1.658
Solar	96	0.418	0.779	0.858	0.216	0.402
	192	0.403	0.822	0.913	0.209	0.518

Appendix

Table 14: MSE of baseline out-of-domain test performance.

Figure 16: Impact of stationarity, seasonality, and trend on forecasting performance

Figure 17: When using LLMs w/ pre-training, the distribution of samples passed the LLM is insensitive to variations in stationarity, seasonality, and trend, exhibiting a relatively uniform pattern.

Figure 18: Under the LLM w/ pre-training setting, the pass distribution remains uniform with respect to these properties; however, due to the randomly initialized LLM under the pre-alignment scheme, the proportion of skipped samples is higher than that of passed samples.

Figure 33

Figure 19: Illustration of synthetic TS generated under different property control levels, where higher levels indicate stronger effects.

Require: Dataset $D$ , routing module parameters $\theta$ , temperature $\tau$
Output: Differentiable token-level routing decisions $\mathbf{y}$
1:	Initialize routing module parameters $\theta$
2:	for each training iteration do
3:	Sample a token of TS segments $X\sim D$
4:	for each token $t$ in $X$ do
5:	Compute routing logits $\mathbf{z}_{t}=[z_{t,1},z_{t,2}]$ using the routing module

Appendix

Table 15: Algorithm: Token-Level Routing

Hyperparameter	Value
Framework	HuggingFace Transformers
Distributed Training	DeepSpeed, Accelerate
GPU	$4\times$ NVIDIA A100 40GB / H100 80GB
Random Seed	Fixed 2026
Optimizer	AdamW
Learning Rate	$1\times 10^{-4}$

Appendix

Table 16: Training Configuration

Routing Parameter	Value
Number of Routing Paths	2
Routing Module Output	Logits $\mathbf{z}=[z_{1},z_{2}]$
Gumbel Noise Distribution	Gumbel $(0,1)$
Temperature $\tau$ (init)	1.0
Temperature $\tau$ (final)	0.1
Discrete Sampling	Gumbel-Max for $\mathbf{y}_{\text{hard}}$

Appendix

Table 17: Routing Analysis Configuration

LLM-Based Time Series Forecasting

LLM-based time series forecasting uses large language models to predict future values from numerical histories and contextual data.

See all 100 papers

Markovian Linguistic-Temporal Bridge: Unlocking the Potential of LLMs for Time Series Forecasting
May 19, 2025Siming Sun, Kai Zhang, Xuejun Jiang +2Time Series Forecasting LLM-Based Time Series Forecasting
Zhejiang University
FM-LLM: A frequency-enhanced mixture-of-experts framework for adapting LLMs to time series forecasting
Aug 12, 2026Rentao Gu, Yihang Ding, Junjie Li +5Zero-Shot Time Series Forecasting Time Series Forecasting
Beijing University of Posts and Telecommunications, Beijing, 100876, China · China Telecom Research Institute, Beijing, 100032, China
LLM Pretraining Shapes a Generalizable Manifold: Insights into Cross-Modal Transfer to Time Series
May 19, 2026Alexis Roger, Prateek Humane, Zhenghan Tai +4Cross-Modal Learning Language Model Pretraining
Université de Montréal Mila - Quebec AI Institute

May 19, 2025J/K move · Enter open · S save

Markovian Linguistic-Temporal Bridge: Unlocking the Potential of LLMs for Time Series Forecasting

Siming Sun, Kai Zhang, Xuejun Jiang, Wenchao Meng+1

Zhejiang University

Time Series Forecasting LLM-Based Time Series Forecasting

Open paper AskPDF arXiv

Adapting pretrained Large Language Models (LLMs) for time series forecasting primarily relies on token-level linguistic-temporal alignment, leading to the stacking of logically disjointed tokens as input. While empirically effective, these methods overlook a fundamental capability of LLMs: modeling linguistic logic and structure, rather than merely processing token features. To address this limitation, we propose the Markovian-Guided Structure-Aware Alignment (MGSAA). Our core contribution is a framework that transcends pointwise feature matching to achieve global structural isomorphism between the linguistic and temporal domains. Specifically, MGSAA distills latent evolutionary patterns of language within LLMs into a Markovian state transition graph, which is transferred as a structural prior to the time series domain. Under this prior, time series patches are decoded into latent states and then aligned via state-constrained cross-attention. Ultimately, MGSAA generates a token sequence topologically isomorphic to the LLM's inherent mental structure, reactivating its reasoning capabilities for forecasting. Comprehensive evaluations across multiple benchmarks demonstrate that MGSAA achieves state-of-the-art performance, providing an innovative solution for cross-modal alignment in LLM for time series forecasting. Code is available at https://github.com/sunzju/MGSAA.

Abstract

LLM-Based Time Series Forecasting

Explore similar work

Markovian Linguistic-Temporal Bridge: Unlocking the Potential of LLMs for Time Series Forecasting

FM-LLM: A frequency-enhanced mixture-of-experts framework for adapting LLMs to time series forecasting

LLM Pretraining Shapes a Generalizable Manifold: Insights into Cross-Modal Transfer to Time Series