Markovian Linguistic-Temporal Bridge: Unlocking the Potential of LLMs for Time Series Forecasting

Siming Sun; Kai Zhang; Xuejun Jiang; Wenchao Meng; Qinmin Yang

cs.AIMay 19, 2025

Markovian Linguistic-Temporal Bridge: Unlocking the Potential of LLMs for Time Series Forecasting

Authors: Siming Sun, Kai Zhang, Xuejun Jiang, Wenchao Meng, Qinmin Yang

Organizations: Zhejiang University

Abstract

Adapting pretrained Large Language Models (LLMs) for time series forecasting primarily relies on token-level linguistic-temporal alignment, leading to the stacking of logically disjointed tokens as input. While empirically effective, these methods overlook a fundamental capability of LLMs: modeling linguistic logic and structure, rather than merely processing token features. To address this limitation, we propose the Markovian-Guided Structure-Aware Alignment (MGSAA). Our core contribution is a framework that transcends pointwise feature matching to achieve global structural isomorphism between the linguistic and temporal domains. Specifically, MGSAA distills latent evolutionary patterns of language within LLMs into a Markovian state transition graph, which is transferred as a structural prior to the time series domain. Under this prior, time series patches are decoded into latent states and then aligned via state-constrained cross-attention. Ultimately, MGSAA generates a token sequence topologically isomorphic to the LLM's inherent mental structure, reactivating its reasoning capabilities for forecasting. Comprehensive evaluations across multiple benchmarks demonstrate that MGSAA achieves state-of-the-art performance, providing an innovative solution for cross-modal alignment in LLM for time series forecasting. Code is available at https://github.com/sunzju/MGSAA.

Figures & tables

Figure 1: An overview of MGSAA. The pipeline begins with Language Structure Distillation in LLM , which is executed independently and outputs structure prior for downstream modules. Based on the prior, Structure-Constrained State Decoding is performed first to assign latent states to patches, followed by State-Conditioned Semantic Alignment to execute alignment via the latent states, finally, the LLM for Forecasting module produces prediction.

Figure 2: The workflow of Markovian structure constrained latent state decoding

Models	MGSAA		FSCA		TimeLLM		GPT4TS		PatchTST		iTransformer		FEDformer		Stationary		Autoformer		TimesNet		DLinear
Models	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTm1	0.378	0.383	0.400	0.402	0.392	0.403	0.390	0.398	0.388	0.402	0.407	0.411	0.446	0.455	0.529	0.478	0.614	0.526	0.410	0.418	0.404	0.408
ETTm2	0.280	0.320	0.283	0.326	0.291	0.334	0.284	0.329	0.290	0.333	0.291	0.334	0.303	0.349	0.517	0.438	0.332	0.368	0.295	0.332	0.355	0.401
ETTh1	0.433	0.427	0.438	0.441	0.454	0.447	0.448	0.437	0.448	0.447	0.464	0.455	0.439	0.457	0.629	0.560	0.501	0.490	0.460	0.455	0.460	0.456
ETTh2	0.370	0.392	0.380	0.409	0.387	0.408	0.382	0.408	0.384	0.413	0.383	0.407	0.442	0.453	0.544	0.498	0.458	0.462	0.407	0.421	0.564	0.519
Weather	0.253	0.271	0.260	0.280	0.273	0.290	0.264	0.284	0.258	0.280	0.260	0.281	0.313	0.364	0.288	0.309	0.376	0.406	0.259	0.285	0.266	0.318

Table 1: Long-term forecasting results. The input sequence length

T

is set to 96 for all baselines. All the results are averaged from 4 different prediction lengths

H\in\{96,192,336,720\}

. Bold : best, underlined : second best.

Models		MGSAA	FSCA	TimeLLM	GPT4TS	PatchTST	iTransformer	FEDformer	Stationary	Autoformer	TimesNet	DLinear	N-HiTS	N-BEATS
Average	SMAPE	11.707	11.828	12.494	12.365	12.059	12.142	13.160	12.780	12.909	12.880	13.639	12.035	12.250
	MASE	1.567	1.580	1.731	1.767	1.623	1.631	1.775	1.756	1.771	1.836	2.095	1.625	1.698
	OWA	0.841	0.850	0.913	0.918	0.869	0.874	0.949	0.930	0.939	0.955	1.051	0.869	0.896

Table 2: Short-term forecasting results on M4 dataset. The prediction length is set to

[6,48]

, and the input lengths are configured to be double the forecasting horizons. Bold : best, underlined : second best.

Models	MGSAA		FSCA		TimeLLM		GPT4TS		PatchTST		iTransformer		FEDformer		Stationary		TimesNet		DLinear
Models	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTm1	0.505	0.445	0.602	0.503	0.565	0.491	0.607	0.498	0.619	0.496	0.580	0.497	0.696	0.570	1.070	0.680	0.674	0.535	0.568	0.500
ETTm2	0.297	0.331	0.302	0.339	0.303	0.341	0.303	0.337	0.302	0.342	0.306	0.344	0.358	0.394	0.428	0.414	0.322	0.355	0.329	0.383
ETTh1	0.598	0.520	0.695	0.553	0.616	0.530	0.688	0.554	0.623	0.532	0.748	0.587	0.749	0.608	0.845	0.631	0.864	0.625	0.647	0.552
ETTh2	0.423	0.422	0.403	0.416	0.497	0.465	0.574	0.495	0.490	0.459	0.443	0.443	0.553	0.525	0.772	0.588	0.487	0.468	0.446	0.461

Table 3: Few-shot forecasting results on 10% training data of ETT datasets. All the results are averaged from 4 different prediction lengths

H\in\{96,192,336,720\}

Figure 3: Markovian state transition heatmap. Left: language domain; Right: time series domain.

Figure 4: Latent state distributions over a single time series sample. The x-axis denotes the sequence of temporal patches, and the y-axis corresponds to latent state categories.

Ablation Setting	ETTh1		Traffic
Ablation Setting	MSE	MAE	MSE	MAE
MGSAA(Full Model)	0.433	0.427	0.439	0.268
A. w/o Alignment	0.464	0.447	0.461	0.285
B. w/o Structural Prior	0.448	0.439	0.454	0.278
C. w/o Residual	0.445	0.433	0.448	0.273

Table 4: Ablation study on the effectiveness of structure-aware semantic alignment on ETTm1 and Traffic datasets.

Panel of Figure 5 — Figure 5: Hyperparameter sensitivity results on the number of latent states and top- $k$ tokens.

Models	MGSAA	Time-LLM	FSCA
Total Params (M)	87.20	148.89	89.28
Trainable Params (M)	4.06	52.80	8.17
Training Time / Step (s)	0.014	0.058	0.070
Inference Time / Batch (s)	0.004	0.024	0.026

Table 5: Computational cost comparison of LLM-based models.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Time Steps	Frequency	Domain
M4-Yearly	23,000	Yearly	Demographic
M4-Quarterly	24,000	Quarterly	Finance
M4-Monthly	48,000	Monthly	Industry
M4-Weekly	359	Weekly	Macro
M4-Daily	4,227	Daily	Micro
M4-Hourly	414	Hourly	Other

Appendix

Table 6: Statistics of the M4 dataset.

Figure 6: The result of LLM latent variable clustering.

Models	Metrics	ETT1			ETT2
Models	Metrics	96	192	336	96	192	336
Time-LLM(8)	MSE	0.383	0.440	0.480	0.316	0.397	0.430
Time-LLM(8)	MAE	0.408	0.452	0.492	0.360	0.407	0.437
MGSAA(8)	MSE	0.373	0.425	0.467	0.295	0.370	0.410
MGSAA(8)	MAE	0.391	0.423	0.442	0.339	0.386	0.427

Appendix

Table 7: Performance comparison on ETT1 and ETT2 datasets using the LLaMA-2(7B) backbone.

Dataset	Horizon	MGSAA		FSCA		TimeLLM		GPT4TS		PatchTST		iTransformer		FEDformer		Stationary		Autoformer		TimesNet		DLinear
Dataset	Horizon	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTm1	96	0.313	0.341	0.331	0.368	0.331	0.370	0.330	0.365	0.324	0.365	0.341	0.376	0.366	0.411	0.428	0.428	0.558	0.499	0.334	0.375	0.346	0.374
	192	0.359	0.370	0.389	0.384	0.370	0.389	0.368	0.383	0.372	0.392	0.381	0.395	0.434	0.446	0.498	0.457	0.606	0.511	0.407	0.413	0.382	0.391
	336	0.388	0.391	0.407	0.406	0.411	0.413	0.401	0.404	0.398	0.409	0.419	0.419	0.476	0.472	0.584	0.498	0.714	0.568	0.415	0.422	0.415	0.415
	720	0.454	0.431	0.472	0.449	0.458	0.441	0.461	0.439	0.457	0.444	0.486	0.456	0.507	0.490	0.608	0.529	0.578	0.527	0.486	0.461	0.473	0.451
	Avg	0.378	0.383	0.400	0.402	0.392	0.403	0.390	0.398	0.388	0.402	0.407	0.411	0.446	0.455	0.529	0.478	0.614	0.526	0.410	0.418	0.404	0.408

Appendix

Table 8: Full results for long-term forecasting with different prediction lengths

H\in\{96,192,336,720\}

. The input sequence length for all baseline methods is set to 96. Avg. denotes the average over all prediction lengths. Bold : best, underlined : second best.

Models		MGSAA	FSCA	TimeLLM	GPT4TS	PatchTST	iTransformer	FEDformer	Stationary	Autoformer	TimesNet	DLinear	N-HiTS	N-BEATS
Yearly	SMAPE	13.186	13.288	13.750	14.822	13.477	13.652	14.021	14.727	13.974	15.378	16.965	13.422	13.487
	MASE	2.960	2.974	3.055	3.618	3.019	3.095	3.036	3.078	3.134	3.554	4.283	3.056	3.036
	OWA	0.776	0.781	0.805	0.909	0.792	0.807	0.811	0.807	0.822	0.918	1.058	0.795	0.795
Quarterly	SMAPE	10.029	10.037	10.671	10.411	10.380	10.353	11.100	10.958	11.338	10.465	12.145	10.185	10.564
	MASE	1.169	1.174	1.276	1.232	1.233	1.209	1.350	1.325	1.365	1.227	1.520	1.180	1.252
	OWA	0.882	0.884	0.950	0.922	0.921	0.911	0.996	0.981	1.012	0.923	1.106	0.893	0.936

Appendix

Table 9: Full results for short-term forecasting on the M4 dataset. The input and prediction lengths are set to

[12,96]

and

[6,48]

, respectively. The reported average represents the weighted average results across different datasets. Bold : best, underlined : second best.

Dataset	Horizon	MGSAA		FSCA		TimeLLM		GPT4TS		PatchTST		iTransformer		FEDformer		Stationary		TimesNet		DLinear
Dataset	Horizon	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTm1	96	0.457	0.415	0.573	0.494	0.571	0.494	0.613	0.494	0.654	0.509	0.571	0.487	0.602	0.523	1.032	0.649	0.585	0.504	0.552	0.488
	192	0.476	0.430	0.574	0.487	0.509	0.457	0.596	0.489	0.617	0.481	0.557	0.480	0.641	0.545	1.025	0.656	0.609	0.516	0.546	0.487
	336	0.512	0.452	0.618	0.509	0.576	0.498	0.595	0.499	0.575	0.488	0.582	0.503	0.769	0.606	1.040	0.682	0.736	0.573	0.567	0.501
	720	0.577	0.485	0.644	0.521	0.606	0.517	0.623	0.511	0.631	0.507	0.612	0.518	0.772	0.605	1.182	0.734	0.768	0.548	0.606	0.523
	Avg	0.505	0.445	0.602	0.503	0.565	0.491	0.607	0.498	0.619	0.496	0.580	0.497	0.696	0.570	1.070	0.680	0.674	0.535	0.568	0.500

Appendix

Table 10: Full results for few-term forecasting on 10% training data of ETT datasets with different prediction lengths

H\in\{96,192,336,720\}

. The input sequence length for all baseline methods is set to 96. Avg. denotes the average over all prediction lengths. Bold : best, underlined : second best.

Dataset	Horizon	MGSAA		FSCA		TimeLLM		GPT4TS		PatchTST		iTransformer		FEDformer		Stationary		TimesNet		DLinear
Dataset	Horizon	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
$\text{h1}\rightarrow\text{h2}$	96	0.308	0.346	0.284	0.335	0.300	0.343	0.301	0.345	0.302	0.346	0.301	0.348	0.367	0.410	0.487	0.474	0.333	0.373	0.313	0.370
	192	0.379	0.391	0.363	0.386	0.386	0.397	0.382	0.392	0.380	0.396	0.380	0.395	0.446	0.456	0.551	0.503	0.409	0.412	0.413	0.433
	336	0.418	0.424	0.373	0.405	0.429	0.435	0.420	0.427	0.423	0.431	0.421	0.430	0.482	0.490	0.723	0.579	0.478	0.461	0.490	0.485
	720	0.420	0.445	0.406	0.428	0.424	0.440	0.424	0.439	0.428	0.446	0.433	0.447	0.526	0.527	0.656	0.566	0.475	0.469	0.646	0.576
	Avg	0.381	0.401	0.356	0.388	0.385	0.404	0.382	0.401	0.383	0.405	0.384	0.405	0.455	0.471	0.604	0.530	0.424	0.429	0.465	0.466

Appendix

Table 11: Full results for zero-shot learning on ETT datasets with different prediction lengths

H\in\{96,192,336,720\}

. ‘h1’, ‘h2’, ‘m1’, and ‘m2’ denote ETTh1, ETTh2, ETTm1, and ETTm2.

\text{A}\rightarrow\text{B}

indicates training on dataset A and testing on dataset B. The input sequence length for all baseline methods is set to 96. Bold : best, underlined : second best.

Time Series Forecasting

Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.

See all 858 papers

FM-LLM: A frequency-enhanced mixture-of-experts framework for adapting LLMs to time series forecasting
Aug 12, 2026Rentao Gu, Yihang Ding, Junjie Li +5Zero-Shot Time Series Forecasting Time Series Forecasting
Beijing University of Posts and Telecommunications, Beijing, 100876, China · China Telecom Research Institute, Beijing, 100032, China
Rethinking the Role of LLMs in Time Series Forecasting
Feb 16, 2026Xin Qiu, Junlong Tong, Yirong Sun +3LLM-Based Time Series Forecasting Time Series Forecasting
Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative, Institute of Digital Twin, Eastern Institute of Technology, Ningbo · Zhejiang University · LMU Munich
FreSia: Frequency-Semantic Instantiation and Alignment for Multivariate Time Series Analysis
Oct 5, 2026Yubo Wang, Hui He, Hezhe Qiao +2Multivariate Time Series Forecasting Frequency-Domain Time Series Modeling
School of Computer Science and Technology, Beijing Institute of Technology, No.5 Zhongguancun South Street, Beijing, 100081, China · School of Computing and Information Systems, Singapore Management University, 80 Stamford Road, 178902, Singapore

Aug 12, 2026J/K move · Enter open · S save

FM-LLM: A frequency-enhanced mixture-of-experts framework for adapting LLMs to time series forecasting

Rentao Gu, Yihang Ding, Junjie Li, Yi Ding+4

Beijing University of Posts and Telecommunications, Beijing, 100876, China · China Telecom Research Institute, Beijing, 100032, China

Zero-Shot Time Series Forecasting Time Series Forecasting LLM-Based Time Series Forecasting Mixture-of-Experts Models Autoregressive Modeling

Open paper AskPDF arXiv

Recent advances in Large Language Models (LLMs) have spurred cross-modal solutions for time-series forecasting. However, existing methods rely heavily on textual prompts for modality alignment-introducing nontrivial computational overhead and failing to leverage the rich spectral dynamics inherent in time-series data. To enable prompt-free, frequency-aware adaptation of frozen LLMs, we propose FM-LLM (Frequency-Enhanced Mixture-of-Experts for adapting LLMs to Time Series Forecasting), an autoregressive framework grounded in constrained asymmetric coupling. A Fourier Analysis Network (FAN)-based spectral token aligner injects structured harmonic representations directly into the frozen LLM with numerical compatibility. An asymmetric Mixture-of-Experts (MoE) decoder enforces role separation: shared experts with lightweight FAN layers reconstruct the global periodic backbone, while routed experts-restricted to standard FFNs-specialize in modeling non-periodic residual dynamics. A time-frequency hybrid loss function jointly optimizes temporal accuracy and spectral consistency, mitigating error accumulation during long-horizon autoregressive rollouts. Evaluated across eleven public benchmarks, FM-LLM achieves state-of-the-art performance on 59 out of 78 evaluation metrics. Compared to the strongest autoregressive LLM-based baseline, it delivers average improvements of 5.3% in MSE and 5.6% in MAE, with maximum gains reaching 8.0% for MSE and 8.4% for MAE. FM-LLM also demonstrates robust transferability, maintaining superior performance in 10% few-shot and zero-shot forecasting scenarios.

Abstract

Time Series Forecasting

Explore similar work

FM-LLM: A frequency-enhanced mixture-of-experts framework for adapting LLMs to time series forecasting

Rethinking the Role of LLMs in Time Series Forecasting

FreSia: Frequency-Semantic Instantiation and Alignment for Multivariate Time Series Analysis