Markovian Linguistic-Temporal Bridge: Unlocking the Potential of LLMs for Time Series Forecasting

Siming Sun; Kai Zhang; Xuejun Jiang; Wenchao Meng; Qinmin Yang

cs.AIMay 19, 2025

Markovian Linguistic-Temporal Bridge: Unlocking the Potential of LLMs for Time Series Forecasting

Authors: Siming Sun, Kai Zhang, Xuejun Jiang, Wenchao Meng, Qinmin Yang

Organizations: Zhejiang University

Abstract

Adapting pretrained Large Language Models (LLMs) for time series forecasting primarily relies on token-level linguistic-temporal alignment, leading to the stacking of logically disjointed tokens as input. While empirically effective, these methods overlook a fundamental capability of LLMs: modeling linguistic logic and structure, rather than merely processing token features. To address this limitation, we propose the Markovian-Guided Structure-Aware Alignment (MGSAA). Our core contribution is a framework that transcends pointwise feature matching to achieve global structural isomorphism between the linguistic and temporal domains. Specifically, MGSAA distills latent evolutionary patterns of language within LLMs into a Markovian state transition graph, which is transferred as a structural prior to the time series domain. Under this prior, time series patches are decoded into latent states and then aligned via state-constrained cross-attention. Ultimately, MGSAA generates a token sequence topologically isomorphic to the LLM's inherent mental structure, reactivating its reasoning capabilities for forecasting. Comprehensive evaluations across multiple benchmarks demonstrate that MGSAA achieves state-of-the-art performance, providing an innovative solution for cross-modal alignment in LLM for time series forecasting. Code is available at https://github.com/sunzju/MGSAA.

Figures & tables

Figure 1: An overview of MGSAA. The pipeline begins with Language Structure Distillation in LLM , which is executed independently and outputs structure prior for downstream modules. Based on the prior, Structure-Constrained State Decoding is performed first to assign latent states to patches, followed by State-Conditioned Semantic Alignment to execute alignment via the latent states, finally, the LLM for Forecasting module produces prediction.

Figure 2: The workflow of Markovian structure constrained latent state decoding

Models	MGSAA		FSCA		TimeLLM		GPT4TS		PatchTST		iTransformer		FEDformer		Stationary		Autoformer		TimesNet		DLinear
Models	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTm1	0.378	0.383	0.400	0.402	0.392	0.403	0.390	0.398	0.388	0.402	0.407	0.411	0.446	0.455	0.529	0.478	0.614	0.526	0.410	0.418	0.404	0.408
ETTm2	0.280	0.320	0.283	0.326	0.291	0.334	0.284	0.329	0.290	0.333	0.291	0.334	0.303	0.349	0.517	0.438	0.332	0.368	0.295	0.332	0.355	0.401
ETTh1	0.433	0.427	0.438	0.441	0.454	0.447	0.448	0.437	0.448	0.447	0.464	0.455	0.439	0.457	0.629	0.560	0.501	0.490	0.460	0.455	0.460	0.456
ETTh2	0.370	0.392	0.380	0.409	0.387	0.408	0.382	0.408	0.384	0.413	0.383	0.407	0.442	0.453	0.544	0.498	0.458	0.462	0.407	0.421	0.564	0.519
Weather	0.253	0.271	0.260	0.280	0.273	0.290	0.264	0.284	0.258	0.280	0.260	0.281	0.313	0.364	0.288	0.309	0.376	0.406	0.259	0.285	0.266	0.318

Table 1: Long-term forecasting results. The input sequence length

T

is set to 96 for all baselines. All the results are averaged from 4 different prediction lengths

H\in\{96,192,336,720\}

. Bold : best, underlined : second best.

Models		MGSAA	FSCA	TimeLLM	GPT4TS	PatchTST	iTransformer	FEDformer	Stationary	Autoformer	TimesNet	DLinear	N-HiTS	N-BEATS
Average	SMAPE	11.707	11.828	12.494	12.365	12.059	12.142	13.160	12.780	12.909	12.880	13.639	12.035	12.250
	MASE	1.567	1.580	1.731	1.767	1.623	1.631	1.775	1.756	1.771	1.836	2.095	1.625	1.698
	OWA	0.841	0.850	0.913	0.918	0.869	0.874	0.949	0.930	0.939	0.955	1.051	0.869	0.896

Table 2: Short-term forecasting results on M4 dataset. The prediction length is set to

[6,48]

, and the input lengths are configured to be double the forecasting horizons. Bold : best, underlined : second best.

Models	MGSAA		FSCA		TimeLLM		GPT4TS		PatchTST		iTransformer		FEDformer		Stationary		TimesNet		DLinear
Models	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTm1	0.505	0.445	0.602	0.503	0.565	0.491	0.607	0.498	0.619	0.496	0.580	0.497	0.696	0.570	1.070	0.680	0.674	0.535	0.568	0.500
ETTm2	0.297	0.331	0.302	0.339	0.303	0.341	0.303	0.337	0.302	0.342	0.306	0.344	0.358	0.394	0.428	0.414	0.322	0.355	0.329	0.383
ETTh1	0.598	0.520	0.695	0.553	0.616	0.530	0.688	0.554	0.623	0.532	0.748	0.587	0.749	0.608	0.845	0.631	0.864	0.625	0.647	0.552
ETTh2	0.423	0.422	0.403	0.416	0.497	0.465	0.574	0.495	0.490	0.459	0.443	0.443	0.553	0.525	0.772	0.588	0.487	0.468	0.446	0.461

Table 3: Few-shot forecasting results on 10% training data of ETT datasets. All the results are averaged from 4 different prediction lengths

H\in\{96,192,336,720\}

Figure 3: Markovian state transition heatmap. Left: language domain; Right: time series domain.

Figure 4: Latent state distributions over a single time series sample. The x-axis denotes the sequence of temporal patches, and the y-axis corresponds to latent state categories.

Ablation Setting	ETTh1		Traffic
Ablation Setting	MSE	MAE	MSE	MAE
MGSAA(Full Model)	0.433	0.427	0.439	0.268
A. w/o Alignment	0.464	0.447	0.461	0.285
B. w/o Structural Prior	0.448	0.439	0.454	0.278
C. w/o Residual	0.445	0.433	0.448	0.273

Table 4: Ablation study on the effectiveness of structure-aware semantic alignment on ETTm1 and Traffic datasets.

Panel of Figure 5 — Figure 5: Hyperparameter sensitivity results on the number of latent states and top- $k$ tokens.

Models	MGSAA	Time-LLM	FSCA
Total Params (M)	87.20	148.89	89.28
Trainable Params (M)	4.06	52.80	8.17
Training Time / Step (s)	0.014	0.058	0.070
Inference Time / Batch (s)	0.004	0.024	0.026

Table 5: Computational cost comparison of LLM-based models.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Time Steps	Frequency	Domain
M4-Yearly	23,000	Yearly	Demographic
M4-Quarterly	24,000	Quarterly	Finance
M4-Monthly	48,000	Monthly	Industry
M4-Weekly	359	Weekly	Macro
M4-Daily	4,227	Daily	Micro
M4-Hourly	414	Hourly	Other

Appendix

Table 6: Statistics of the M4 dataset.

Figure 6: The result of LLM latent variable clustering.

Models	Metrics	ETT1			ETT2
Models	Metrics	96	192	336	96	192	336
Time-LLM(8)	MSE	0.383	0.440	0.480	0.316	0.397	0.430
Time-LLM(8)	MAE	0.408	0.452	0.492	0.360	0.407	0.437
MGSAA(8)	MSE	0.373	0.425	0.467	0.295	0.370	0.410
MGSAA(8)	MAE	0.391	0.423	0.442	0.339	0.386	0.427

Appendix

Table 7: Performance comparison on ETT1 and ETT2 datasets using the LLaMA-2(7B) backbone.

Dataset	Horizon	MGSAA		FSCA		TimeLLM		GPT4TS		PatchTST		iTransformer		FEDformer		Stationary		Autoformer		TimesNet		DLinear
Dataset	Horizon	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTm1	96	0.313	0.341	0.331	0.368	0.331	0.370	0.330	0.365	0.324	0.365	0.341	0.376	0.366	0.411	0.428	0.428	0.558	0.499	0.334	0.375	0.346	0.374
	192	0.359	0.370	0.389	0.384	0.370	0.389	0.368	0.383	0.372	0.392	0.381	0.395	0.434	0.446	0.498	0.457	0.606	0.511	0.407	0.413	0.382	0.391
	336	0.388	0.391	0.407	0.406	0.411	0.413	0.401	0.404	0.398	0.409	0.419	0.419	0.476	0.472	0.584	0.498	0.714	0.568	0.415	0.422	0.415	0.415
	720	0.454	0.431	0.472	0.449	0.458	0.441	0.461	0.439	0.457	0.444	0.486	0.456	0.507	0.490	0.608	0.529	0.578	0.527	0.486	0.461	0.473	0.451
	Avg	0.378	0.383	0.400	0.402	0.392	0.403	0.390	0.398	0.388	0.402	0.407	0.411	0.446	0.455	0.529	0.478	0.614	0.526	0.410	0.418	0.404	0.408

Appendix

Table 8: Full results for long-term forecasting with different prediction lengths

H\in\{96,192,336,720\}

. The input sequence length for all baseline methods is set to 96. Avg. denotes the average over all prediction lengths. Bold : best, underlined : second best.

Models		MGSAA	FSCA	TimeLLM	GPT4TS	PatchTST	iTransformer	FEDformer	Stationary	Autoformer	TimesNet	DLinear	N-HiTS	N-BEATS
Yearly	SMAPE	13.186	13.288	13.750	14.822	13.477	13.652	14.021	14.727	13.974	15.378	16.965	13.422	13.487
	MASE	2.960	2.974	3.055	3.618	3.019	3.095	3.036	3.078	3.134	3.554	4.283	3.056	3.036
	OWA	0.776	0.781	0.805	0.909	0.792	0.807	0.811	0.807	0.822	0.918	1.058	0.795	0.795
Quarterly	SMAPE	10.029	10.037	10.671	10.411	10.380	10.353	11.100	10.958	11.338	10.465	12.145	10.185	10.564
	MASE	1.169	1.174	1.276	1.232	1.233	1.209	1.350	1.325	1.365	1.227	1.520	1.180	1.252
	OWA	0.882	0.884	0.950	0.922	0.921	0.911	0.996	0.981	1.012	0.923	1.106	0.893	0.936

Appendix

Table 9: Full results for short-term forecasting on the M4 dataset. The input and prediction lengths are set to

[12,96]

and

[6,48]

, respectively. The reported average represents the weighted average results across different datasets. Bold : best, underlined : second best.

Dataset	Horizon	MGSAA		FSCA		TimeLLM		GPT4TS		PatchTST		iTransformer		FEDformer		Stationary		TimesNet		DLinear
Dataset	Horizon	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTm1	96	0.457	0.415	0.573	0.494	0.571	0.494	0.613	0.494	0.654	0.509	0.571	0.487	0.602	0.523	1.032	0.649	0.585	0.504	0.552	0.488
	192	0.476	0.430	0.574	0.487	0.509	0.457	0.596	0.489	0.617	0.481	0.557	0.480	0.641	0.545	1.025	0.656	0.609	0.516	0.546	0.487
	336	0.512	0.452	0.618	0.509	0.576	0.498	0.595	0.499	0.575	0.488	0.582	0.503	0.769	0.606	1.040	0.682	0.736	0.573	0.567	0.501
	720	0.577	0.485	0.644	0.521	0.606	0.517	0.623	0.511	0.631	0.507	0.612	0.518	0.772	0.605	1.182	0.734	0.768	0.548	0.606	0.523
	Avg	0.505	0.445	0.602	0.503	0.565	0.491	0.607	0.498	0.619	0.496	0.580	0.497	0.696	0.570	1.070	0.680	0.674	0.535	0.568	0.500

Appendix

Table 10: Full results for few-term forecasting on 10% training data of ETT datasets with different prediction lengths

H\in\{96,192,336,720\}

. The input sequence length for all baseline methods is set to 96. Avg. denotes the average over all prediction lengths. Bold : best, underlined : second best.

Dataset	Horizon	MGSAA		FSCA		TimeLLM		GPT4TS		PatchTST		iTransformer		FEDformer		Stationary		TimesNet		DLinear
Dataset	Horizon	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
$\text{h1}\rightarrow\text{h2}$	96	0.308	0.346	0.284	0.335	0.300	0.343	0.301	0.345	0.302	0.346	0.301	0.348	0.367	0.410	0.487	0.474	0.333	0.373	0.313	0.370
	192	0.379	0.391	0.363	0.386	0.386	0.397	0.382	0.392	0.380	0.396	0.380	0.395	0.446	0.456	0.551	0.503	0.409	0.412	0.413	0.433
	336	0.418	0.424	0.373	0.405	0.429	0.435	0.420	0.427	0.423	0.431	0.421	0.430	0.482	0.490	0.723	0.579	0.478	0.461	0.490	0.485
	720	0.420	0.445	0.406	0.428	0.424	0.440	0.424	0.439	0.428	0.446	0.433	0.447	0.526	0.527	0.656	0.566	0.475	0.469	0.646	0.576
	Avg	0.381	0.401	0.356	0.388	0.385	0.404	0.382	0.401	0.383	0.405	0.384	0.405	0.455	0.471	0.604	0.530	0.424	0.429	0.465	0.466

Appendix

Table 11: Full results for zero-shot learning on ETT datasets with different prediction lengths

H\in\{96,192,336,720\}

. ‘h1’, ‘h2’, ‘m1’, and ‘m2’ denote ETTh1, ETTh2, ETTm1, and ETTm2.

\text{A}\rightarrow\text{B}

indicates training on dataset A and testing on dataset B. The input sequence length for all baseline methods is set to 96. Bold : best, underlined : second best.

Time Series Forecasting

Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.

See all 858 papers

Aug 12, 2026

FM-LLM: A frequency-enhanced mixture-of-experts framework for adapting LLMs to time series forecasting

Recent advances in Large Language Models (LLMs) have spurred cross-modal solutions for time-series forecasting. However, existing methods rely heavily on textual prompts for modality alignment-introducing nontrivial computational overhead and failing to leverage the rich spectral dynamics inherent in time-series data. To enable prompt-free, frequency-aware adaptation of frozen LLMs, we propose FM-LLM (Frequency-Enhanced Mixture-of-Experts for adapting LLMs to Time Series Forecasting), an autoregressive framework grounded in constrained asymmetric coupling. A Fourier Analysis Network (FAN)-based spectral token aligner injects structured harmonic representations directly into the frozen LLM with numerical compatibility. An asymmetric Mixture-of-Experts (MoE) decoder enforces role separation: shared experts with lightweight FAN layers reconstruct the global periodic backbone, while routed experts-restricted to standard FFNs-specialize in modeling non-periodic residual dynamics. A time-frequency hybrid loss function jointly optimizes temporal accuracy and spectral consistency, mitigating error accumulation during long-horizon autoregressive rollouts. Evaluated across eleven public benchmarks, FM-LLM achieves state-of-the-art performance on 59 out of 78 evaluation metrics. Compared to the strongest autoregressive LLM-based baseline, it delivers average improvements of 5.3% in MSE and 5.6% in MAE, with maximum gains reaching 8.0% for MSE and 8.4% for MAE. FM-LLM also demonstrates robust transferability, maintaining superior performance in 10% few-shot and zero-shot forecasting scenarios.

Rentao Gu, Yihang Ding, Junjie Li +5

Beijing University of Posts and Telecommunications, Beijing, 100876, China · China Telecom Research Institute, Beijing, 100032, China

Zero-Shot Time Series Forecasting Time Series Forecasting LLM-Based Time Series Forecasting Mixture-of-Experts Models Autoregressive Modeling

Ask PDF

Feb 16, 2026

Rethinking the Role of LLMs in Time Series Forecasting

Large language models (LLMs) have been introduced to time series forecasting (TSF) to incorporate contextual knowledge beyond numerical signals. However, existing studies question whether LLMs provide genuine benefits, often reporting comparable performance without LLMs. We show that such conclusions stem from limited evaluation settings and do not hold at scale. We conduct a large-scale study of LLM-based TSF (LLM4TSF) across 8 billion observations, 17 forecasting scenarios, 4 horizons, multiple alignment strategies, and both in-domain and out-of-domain settings. Our results demonstrate that LLM4TS indeed improves forecasting performance, with especially large gains in cross-domain generalization. Pre-alignment outperforming post-alignment in over 90% of tasks. Both pretrained knowledge and model architecture of LLMs contribute and play complementary roles: pretraining is critical under distribution shifts, while architecture excels at modeling complex temporal dynamics. Moreover, under large-scale mixed distributions, a fully intact LLM becomes indispensable, as confirmed by token-level routing analysis and prompt-based improvements. Overall, Our findings overturn prior negative assessments, establish clear conditions under which LLMs are not only useful, and provide practical guidance for effective model design. We release our code at https://github.com/EIT-NLP/LLM4TSF.

Xin Qiu, Junlong Tong, Yirong Sun +3

Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative, Institute of Digital Twin, Eastern Institute of Technology, Ningbo · Zhejiang University · LMU Munich

LLM-Based Time Series Forecasting Time Series Forecasting

Ask PDF

Oct 5, 2026

FreSia: Frequency-Semantic Instantiation and Alignment for Multivariate Time Series Analysis

Large Language Models (LLMs) have shown strong potential in multivariate time series forecasting and anomaly detection. Existing studies predominantly inject temporal information into LLMs via direct numerical tokenization or heuristic textual descriptions. However, LLMs still face difficulty in perceiving the underlying structural patterns of numerical time series, particularly the seasonal and trend components obscured by discrete numerical tokens. To bridge this gap, we propose FreSia, a frequency-aware framework that establishes an effective alignment between the semantic space of LLMs and the frequency space of time series. Specifically, FGPrompt, a Frequency-Guided Prompt mechanism within FreSia, distills the frequency-domain structures of time series and projects them into prompts tailored to the semantic space of LLMs. Furthermore, we introduce a Global-driven Context Learning (GCL) component, which uses a global CLS-driven probe to generate global context to bridge the time-frequency domain gap and fuse the multi-modal information. Experiments on eight forecasting benchmarks show that FreSia achieves average improvements of 13.48% and 8.06% in MSE and MAE, respectively.

Yubo Wang, Hui He, Hezhe Qiao +2

School of Computer Science and Technology, Beijing Institute of Technology, No.5 Zhongguancun South Street, Beijing, 100081, China · School of Computing and Information Systems, Singapore Management University, 80 Stamford Road, 178902, Singapore

Multivariate Time Series Forecasting Frequency-Domain Time Series Modeling Time Series Forecasting LLM-Based Time Series Forecasting

Ask PDF

Abstract

Time Series Forecasting

Explore similar work

FM-LLM: A frequency-enhanced mixture-of-experts framework for adapting LLMs to time series forecasting

Rethinking the Role of LLMs in Time Series Forecasting

FreSia: Frequency-Semantic Instantiation and Alignment for Multivariate Time Series Analysis