cs.CLFeb 16, 2026

Rethinking the Role of LLMs in Time Series Forecasting

Authors: Xin Qiu, Junlong Tong, Yirong Sun, Yunpu Ma, Wei Zhang, Xiaoyu Shen

Organizations: Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative, Institute of Digital Twin, Eastern Institute of Technology, Ningbo · Zhejiang University · LMU Munich

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Large language models (LLMs) have been introduced to time series forecasting (TSF) to incorporate contextual knowledge beyond numerical signals. However, existing studies question whether LLMs provide genuine benefits, often reporting comparable performance without LLMs. We show that such conclusions stem from limited evaluation settings and do not hold at scale. We conduct a large-scale study of LLM-based TSF (LLM4TSF) across 8 billion observations, 17 forecasting scenarios, 4 horizons, multiple alignment strategies, and both in-domain and out-of-domain settings. Our results demonstrate that LLM4TS indeed improves forecasting performance, with especially large gains in cross-domain generalization. Pre-alignment outperforming post-alignment in over 90% of tasks. Both pretrained knowledge and model architecture of LLMs contribute and play complementary roles: pretraining is critical under distribution shifts, while architecture excels at modeling complex temporal dynamics. Moreover, under large-scale mixed distributions, a fully intact LLM becomes indispensable, as confirmed by token-level routing analysis and prompt-based improvements. Overall, Our findings overturn prior negative assessments, establish clear conditions under which LLMs are not only useful, and provide practical guidance for effective model design. We release our code at https://github.com/EIT-NLP/LLM4TSF.

Figures & tables

Figure 1: Two mainstream alignment strategies for LLM4TSF.
Figure 2: Comparison of LLM4TSF with pre-alignment and post-alignment strategies under single-dataset learning.
Figure 3: Comparison of LLM4TSF performance with pre- and post-alignment under single- and cross-dataset paradigm. Negative and Positive values indicate MAE decreases and increases under cross-dataset learning compared to single-dataset learning.
TypesZero-Shot Test5% Few-Shot Test
ModelsPre-alignPost-alignChronosUniTSMoiraiUniTimeTimeLLM
Wind1.0150.9631.4221.3581.2361.3581.321
Solar0.2280.2740.4340.8710.9360.2180.577
AQShunyi0.6120.6880.8080.8900.6680.9050.859
CzenLan0.2740.2880.2980.7380.6600.4010.319
ZafNoo0.5470.5830.5500.6680.5430.8030.594
Table 1: Out-of-domain test performance (MSE, averaged over horizons {96,192,336,720}\{96,192,336,720\} ). Bold and underlined indicate the best and second-best results for each dataset.
Figure 4: Original architecture and two ablation variants.
ModelsLLM4TSF (Pre-align)LLM4TSF (Post-align)
w/ Pre-trainingw/o Pre-trainingw/o LLMw/ Pre-trainingw/o Pre-trainingw/o LLM
MAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSE
In-Domain Test
ETTh10.4450.4470.4600.4710.4320.4380.4360.4310.4280.4200.4350.442
ETTh20.3880.3480.4140.3770.4070.3630.3960.3570.4290.4030.4390.424
ETTm10.3750.3530.3930.3820.3630.3280.3840.3540.3730.3400.3780.351
Table 2: Ablation results of LLM4TSF (Pre-align) and LLM4TSF (Post-align) , averaged over horizons {96,192,336,720}\{96,192,336,720\} . For each dataset, the best MAE is highlighted in red , and the best MSE is highlighted in blue , respectively.
Figure 5: Statistical properties of different datasets. Datasets highlighted in red indicate cases where the w/ pre-training setting underperforms the corresponding ablation baselines in test.
Figure 6: As shifting and transition increase, TSF becomes more challenging with higher MAE, while the advantage of LLMs becomes evident. Moreover, even w/o pre-training, a trainable LLM backbone outperforms non-LLM under high-transition regimes.
Figure 7: Relationship between the token ratio passing through the LLM (averaged over horizons {96,192,336,720}\{96,192,336,720\} ) and shifting across different datasets. The ”- - -” & ”- - -” indicate the average ratios of each dataset, and the ”- - -” denotes the reference value of 0.5 .
Figure 8: Routing analysis illustration. For each TS token, a trained router decides whether to route it to the LLM or skip. The LLM backbone is either w/ pre-training or w/o pre-training.
Figure 9: (a) With a pre-trained LLM backbone, tokens of passing LLM are concentrated in high shifting regions. (b) With pre-training, tokens of passing LLM are concentrated in high transition.
 LLM4TSF (Pre-align)LLM4TSF (Post-align)
w/ Pre- trainingw/o Pre- trainingChangew/ Pre- trainingw/o Pre- trainingChange
In-Domain Test
ETTh117%24%+7%31%57%+26%
ETTh264%43%-21%71%62%-9%
ETTm119%14%-5%24%66%+42%
ETTm273%28%-45%76%68%-8%
Table 3: Impact of pretrained LLM parameters on the token passing ratio through the LLM, averaged over horizons 96,192,336,72096,192,336,720 .
Figure 10: LLM4TSF(Post-align) performance and token passing ratios through the LLM under three training strategies: full-parameter fine-tuning, LoRA fine-tuning, and training PE+LN.
Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 11: Proportional distribution of training datasets across ten real-world domains.
DatasetVariablesTimestampsDatasetVariablesTimestamps
ETTh1714,400Wind748,673
ETTh2714,400Solar13752,560
ETTm1757,600AQShunyi1135,064
ETTm2757,600CzenLan1119,934
Weather2152,696ZafNoo1119,225
Traffic86217,544NASDAQ51,244
Appendix
Table 4: Summary of datasets used in experiments. The table reports the number of variables and timestamps for each time series dataset.
Figure 12: Analysis of the 17 in-domain and out-of-domain datasets from Shifting, Stationarity, Transition, Seasonality, and Trend.
Input: Time series X∈RT×1\mathbf{X}\in\mathbb{R}^{T\times 1}
Output: Shifting δ∈(0,1)\delta\in(0,1)
1:Normalize X\mathbf{X} using z-score normalization to obtain Z∈RT×1\mathbf{Z}\in\mathbb{R}^{T\times 1} .
2:Compute Zmin⁡=min⁡(Z)Z_{\min}=\min(\mathbf{Z}) and Zmax⁡=max⁡(Z)Z_{\max}=\max(\mathbf{Z}) .
3:Construct mm uniformly spaced value levels:
 ℓi=Zmin⁡+i−1m−1(Zmax⁡−Zmin⁡),i=1,…,m.\ell_{i}=Z_{\min}+\frac{i-1}{m-1}(Z_{\max}-Z_{\min}),\quad i=1,\ldots,m.
4:for each level ℓi\ell_{i} do
Appendix
Table 5: Algorithm: Shifting Computation
Input: Time series X=⟨x1,x2,…,xT⟩∈RT×1\mathbf{X}=\langle x_{1},x_{2},\ldots,x_{T}\rangle\in\mathbb{R}^{T\times 1}
Output: Stationarity indicator γ∈{0,1}\gamma\in\{0,1\} of X\mathbf{X}
1:Compute the Augmented Dickey–Fuller (ADF) statistic s←ADF(X)s\leftarrow\mathrm{ADF}(\mathbf{X}) .
2:return γ={1,if s≤0.05,0,otherwise.\gamma=\begin{cases}1,&\text{if }s\leq 0.05,\\ 0,&\text{otherwise}.\end{cases}
Appendix
Table 6: Algorithm 2: Stationarity Computation
Input: Time series X∈RT×1\mathbf{X}\in\mathbb{R}^{T\times 1}
Output: Transition Δ∈(0,13)\Delta\in\left(0,\tfrac{1}{3}\right)
1:Estimate the characteristic lag τ\tau as the first zero-crossing point of the autocorrelation function of X\mathbf{X} .
2:Downsample X\mathbf{X} with stride τ\tau to obtain a reduced sequence Y∈RT′×1\mathbf{Y}\in\mathbb{R}^{T^{\prime}\times 1} .
3:Obtain the rank ordering of Y\mathbf{Y} by computing the permutation index r=argsort(Y).\mathbf{r}=\mathrm{argsort}(\mathbf{Y}).
4:Discretize Y\mathbf{Y} into a symbolic sequence Z∈{0,1,2}T′\mathbf{Z}\in\{0,1,2\}^{T^{\prime}} via Zj=⌊3 rjT′⌋,j=1,…,T′.Z_{j}=\left\lfloor\frac{3\,r_{j}}{T^{\prime}}\right\rfloor,\quad j=1,\ldots,T^{\prime}.
5:Initialize a transition count matrix M∈R3×3\mathbf{M}\in\mathbb{R}^{3\times 3} with zeros.
Appendix
Table 7: Algorithm: Transition Computation
Input: Time series X={x1,x2,…,xT}∈RT×1\mathbf{X}=\{x_{1},x_{2},\ldots,x_{T}\}\in\mathbb{R}^{T\times 1}
Output: Seasonality value ζ∈(0,1)\zeta\in(0,1)
1:Decompose X\mathbf{X} into seasonal, trend, and residual components using STL: X=S+T+R.\mathbf{X}=\mathbf{S}+\mathbf{T}+\mathbf{R}.
2:Compute the variance of the residual component R\mathbf{R} .
3:Compute the variance of the combined seasonal and residual component S+R\mathbf{S}+\mathbf{R} .
4:return ζ=max⁡ ⁣(0,  1−var(R)var(S+R)).\zeta=\max\!\left(0,\;1-\frac{\mathrm{var}(\mathbf{R})}{\mathrm{var}(\mathbf{S}+\mathbf{R})}\right).
Appendix
Table 8: Algorithm: Seasonality Computation
Input: Time series X={x1,x2,…,xT}∈RT×1\mathbf{X}=\{x_{1},x_{2},\ldots,x_{T}\}\in\mathbb{R}^{T\times 1}
Output: Trend value β∈(0,1)\beta\in(0,1)
1:Decompose X\mathbf{X} into seasonal, trend, and residual components using STL: X=S+T+R.\mathbf{X}=\mathbf{S}+\mathbf{T}+\mathbf{R}.
2:Compute the variance of the residual component R\mathbf{R} .
3:Compute the variance of the combined non-seasonal component T+R\mathbf{T}+\mathbf{R} .
4:return β=max⁡ ⁣(0,  1−var(R)var(T+R)).\beta=\max\!\left(0,\;1-\frac{\mathrm{var}(\mathbf{R})}{\mathrm{var}(\mathbf{T}+\mathbf{R})}\right).
Appendix
Table 9: Algorithm: Trend Computation
Figure 13: Comparison of LLM4TSF performance with pre- and post-alignment under single- and cross-dataset paradigm. Negative and Positive values indicate MSE decreases and increases under cross-dataset learning compared to single-dataset learning.
Figure 14: Effect of training data on cross-dataset learning performance. Both MAE and MSE decrease as the ratio of data increases.
DatasetHorizonPre-SinglePre-CrossPost-SinglePost-Cross
ETTh1960.4070.4140.4210.398
 1920.4350.4370.4320.420
 3360.4520.4480.4620.446
 7200.4610.4790.4990.481
ETTh2960.3680.3330.3740.352
 1920.3990.3740.4160.379
Appendix
Table 10: MAE comparison under different settings.
DatasetHorizonPre-SinglePre-CrossPost-SinglePost-Cross
ETTh1960.4040.4120.4200.377
 1920.4350.4390.4550.416
 3360.4720.4550.4880.444
 7200.4850.4810.5660.488
ETTh2960.3030.2790.3070.283
 1920.3610.3470.3750.354
Appendix
Table 11: MSE comparison under different settings.
Figure 15: Effect of model truncation on forecasting performance.
 In-Domain TestOut-of-Domain Test
w/ Promptw/o Promptw/ Promptw/o Prompt
LLMMSERatio(%)MSERatio(%)MSERatio(%)MSERatio(%)
LLM4TSF(Pre-align)
GPT-20.47146.70.488 ↑\uparrow41.5 ↓\downarrow0.52472.60.673 ↑\uparrow37.9 ↓\downarrow
Qwen-3 0.6B0.48548.20.495 ↑\uparrow42.2 ↓\downarrow0.54170.90.726 ↑\uparrow40.6 ↓\downarrow
Qwen-3 1.7B0.47745.90.502 ↑\uparrow42.9 ↓\downarrow0.52974.60.679 ↑\uparrow35.5 ↓\downarrow
Appendix
Table 12: Comparison of average MSE and token ratios across different LLM backbones under w/ prompt or w/o prompt.
DatasetHorizonChronosUniTSMOIRAIUniTimeTimeLLM
Wind960.6960.7550.6400.6850.664
 1920.7670.8230.7220.7860.767
 3360.8480.8810.8090.8780.862
 7200.9340.9450.8660.9530.945
Solar960.3270.6110.4770.2810.313
 1920.3390.6550.5260.2700.377
Appendix
Table 13: MAE of baseline out-of-domain test performance.
DatasetHorizonChronosUniTSMOIRAIUniTimeTimeLLM
Wind961.2501.0380.9631.0220.981
 1921.3571.2241.1991.2411.201
 3361.4281.5161.2681.4821.444
 7201.6531.6531.5131.6881.658
Solar960.4180.7790.8580.2160.402
 1920.4030.8220.9130.2090.518
Appendix
Table 14: MSE of baseline out-of-domain test performance.
Figure 16: Impact of stationarity, seasonality, and trend on forecasting performance
Figure 17: When using LLMs w/ pre-training, the distribution of samples passed the LLM is insensitive to variations in stationarity, seasonality, and trend, exhibiting a relatively uniform pattern.
Figure 18: Under the LLM w/ pre-training setting, the pass distribution remains uniform with respect to these properties; however, due to the randomly initialized LLM under the pre-alignment scheme, the proportion of skipped samples is higher than that of passed samples.
Figure 33
Figure 19: Illustration of synthetic TS generated under different property control levels, where higher levels indicate stronger effects.
Require: Dataset DD , routing module parameters θ\theta , temperature τ\tau
Output: Differentiable token-level routing decisions y\mathbf{y}
1:Initialize routing module parameters θ\theta
2:for each training iteration do
3:Sample a token of TS segments X∼DX\sim D
4:for each token tt in XX do
5:Compute routing logits zt=[zt,1,zt,2]\mathbf{z}_{t}=[z_{t,1},z_{t,2}] using the routing module
Appendix
Table 15: Algorithm: Token-Level Routing
HyperparameterValue
FrameworkHuggingFace Transformers
Distributed TrainingDeepSpeed, Accelerate
GPU4×4\times NVIDIA A100 40GB / H100 80GB
Random SeedFixed 2026
OptimizerAdamW
Learning Rate1×10−41\times 10^{-4}
Appendix
Table 16: Training Configuration
Routing ParameterValue
Number of Routing Paths2
Routing Module OutputLogits z=[z1,z2]\mathbf{z}=[z_{1},z_{2}]
Gumbel Noise DistributionGumbel (0,1)(0,1)
Temperature τ\tau (init)1.0
Temperature τ\tau (final)0.1
Discrete SamplingGumbel-Max for yhard\mathbf{y}_{\text{hard}}
Appendix
Table 17: Routing Analysis Configuration
More in this topic

LLM-Based Time Series Forecasting

LLM-based time series forecasting uses large language models to predict future values from numerical histories and contextual data.

See all 100 papers

Explore similar work

CardsList
May 19, 2025

Markovian Linguistic-Temporal Bridge: Unlocking the Potential of LLMs for Time Series Forecasting

Adapting pretrained Large Language Models (LLMs) for time series forecasting primarily relies on token-level linguistic-temporal alignment, leading to the stacking of logically disjointed tokens as input. While empirically effective, these methods overlook a fundamental capability of LLMs: modeling linguistic logic and structure, rather than merely processing token features. To address this limitation, we propose the Markovian-Guided Structure-Aware Alignment (MGSAA). Our core contribution is a framework that transcends pointwise feature matching to achieve global structural isomorphism between the linguistic and temporal domains. Specifically, MGSAA distills latent evolutionary patterns of language within LLMs into a Markovian state transition graph, which is transferred as a structural prior to the time series domain. Under this prior, time series patches are decoded into latent states and then aligned via state-constrained cross-attention. Ultimately, MGSAA generates a token sequence topologically isomorphic to the LLM's inherent mental structure, reactivating its reasoning capabilities for forecasting. Comprehensive evaluations across multiple benchmarks demonstrate that MGSAA achieves state-of-the-art performance, providing an innovative solution for cross-modal alignment in LLM for time series forecasting. Code is available at https://github.com/sunzju/MGSAA.

Siming Sun, Kai Zhang, Xuejun Jiang +2

Zhejiang University

Time Series ForecastingLLM-Based Time Series Forecasting
AskPDF
Aug 12, 2026

FM-LLM: A frequency-enhanced mixture-of-experts framework for adapting LLMs to time series forecasting

Recent advances in Large Language Models (LLMs) have spurred cross-modal solutions for time-series forecasting. However, existing methods rely heavily on textual prompts for modality alignment-introducing nontrivial computational overhead and failing to leverage the rich spectral dynamics inherent in time-series data. To enable prompt-free, frequency-aware adaptation of frozen LLMs, we propose FM-LLM (Frequency-Enhanced Mixture-of-Experts for adapting LLMs to Time Series Forecasting), an autoregressive framework grounded in constrained asymmetric coupling. A Fourier Analysis Network (FAN)-based spectral token aligner injects structured harmonic representations directly into the frozen LLM with numerical compatibility. An asymmetric Mixture-of-Experts (MoE) decoder enforces role separation: shared experts with lightweight FAN layers reconstruct the global periodic backbone, while routed experts-restricted to standard FFNs-specialize in modeling non-periodic residual dynamics. A time-frequency hybrid loss function jointly optimizes temporal accuracy and spectral consistency, mitigating error accumulation during long-horizon autoregressive rollouts. Evaluated across eleven public benchmarks, FM-LLM achieves state-of-the-art performance on 59 out of 78 evaluation metrics. Compared to the strongest autoregressive LLM-based baseline, it delivers average improvements of 5.3% in MSE and 5.6% in MAE, with maximum gains reaching 8.0% for MSE and 8.4% for MAE. FM-LLM also demonstrates robust transferability, maintaining superior performance in 10% few-shot and zero-shot forecasting scenarios.

Rentao Gu, Yihang Ding, Junjie Li +5

Beijing University of Posts and Telecommunications, Beijing, 100876, China · China Telecom Research Institute, Beijing, 100032, China

Zero-Shot Time Series ForecastingTime Series ForecastingLLM-Based Time Series ForecastingMixture-of-Experts ModelsAutoregressive Modeling
AskPDF
May 19, 2026

LLM Pretraining Shapes a Generalizable Manifold: Insights into Cross-Modal Transfer to Time Series

Can language-pretrained transformers become effective time-series forecasters, and why? In this paper, we show that cross-modal transfer arises because language pretraining preconditions time series training with a reusable manifold. A linear probe on frozen LLM states decodes realistic time-series trajectories without paired supervision, and retrieval in this projected space yields competitive forecasts, showing that structure and dynamics exist before finetuning. Pretrained initialization also improves optimization, producing coherent gradients and a highly anisotropic loss landscape unlike random initialization. Finetuning then acts as low-dimensional alignment, reusing existing directions rather than learning temporal primitives from scratch, as evidenced by low-rank updates, subspace alignment, and shared features for periodicity, trend, and repetition. Together, these results support a geometric account of LLM-to-time-series transfer: language pretraining builds the manifold, and finetuning projects numerical dynamics onto task-relevant directions.

Alexis Roger, Prateek Humane, Zhenghan Tai +4

Université de Montréal Mila - Quebec AI Institute

Cross-Modal LearningLanguage Model PretrainingRepresentation GeometryLLM-Based Time Series ForecastingTransfer LearningTime Series Representation Learning
AskPDF