One-for-All: A Lightweight Stabilized and Parameter-Efficient Pre-trained LLM for Time Series Forecasting

Prasanjit Dey; Soumyabrata Dev; Bianca Schoen-Phelan

cs.LGMar 31, 2026

One-for-All: A Lightweight Stabilized and Parameter-Efficient Pre-trained LLM for Time Series Forecasting

Authors: Prasanjit Dey, Soumyabrata Dev, Bianca Schoen-Phelan

Organizations: ADAPT SFI Research Centre, School of Computer Science, Technological University Dublin, Ireland · ADAPT SFI Research Centre, School of Computer Science, University College Dublin

Abstract

We address the challenge of adapting pre-trained Large Language Models (LLMs) for multivariate time-series analysis, where their deployment is often hindered by prohibitive computational and memory demands. Our solution, One-for-All, introduces Gaussian Rank-Stabilized Low-Rank Adapters (rsLoRA) to enable parameter-efficient fine-tuning of frozen LLMs. While inspired by LoRA, rsLoRA introduces a mathematically grounded rank-stabilization mechanism that enables provable gradient stability at low ranks a novel contribution absent in prior PEFT methods. Our framework injects trainable rank decomposition matrices (rank 16) into positional embeddings and output layers, while keeping self-attention weights fixed. This design reduces trainable parameters by 6.8 $\times$ (vs. TimesNet), 21 $\times$ (vs. GPT4TS), and 11.8 $\times$ (vs. TIME-LLM), while achieving a 168-1,776 $\times$ smaller memory footprint (2.2MiB vs. 340MiB-4.18GiB in SOTA models). Rigorous evaluation across six time-series tasks demonstrates that One-for-All achieves state-of-the-art efficiency-accuracy trade-offs: 5.5 $\times$ higher parameter efficiency (MSE=5.50) than TimesNet and 21 $\times$ better than GPT4TS, while matching their forecasting accuracy (MSE=0.33). The framework's stability is validated through consistent performance across diverse horizons (96-720 steps) and datasets (ETT, Weather, M3, M4), with 98.3% fewer parameters than conventional transformers. These advances enable deployment on edge devices for healthcare, finance, and environmental monitoring without compromising performance.

Figures & tables

Fig. 1: One-for-All Framework: A parameter-efficient LLM unifying long-term, few-shot, zero-shot, short-term forecasting, classification, and anomaly detection. By integrating Gaussian rank-stabilized LoRA (rsLoRA) into positional embeddings and output layers while freezing the pre-trained LLM weights, we minimize trainable parameters without compromising stability.

Fig. 2: Comparison of model efficiency across different forecast horizons. (a) Trainable parameters (in millions, log scale) and (b) Model size (in MiB, log scale) for various time-series forecasting approaches. The One-for-All model (highlighted in red) demonstrates consistently low parameter counts and memory footprint across all horizons, while other methods (dashed lines) show varying computational requirements. Notably, large pretrained models (e.g., TIME-LLM and GPT4TS) exhibit significantly higher resource demands, particularly at longer horizons (336 and 720). The Avg column represents the average across all horizons, further emphasizing the efficiency advantages of the proposed approach.

Model	Parameters (M)	Memory (MiB)	Eff.*MSE/MAE	ETTh1	ETTh2	ETTm1	ETTm2	Weather	Avg
				MSE/MAE	MSE/MAE	MSE/MAE	MSE/MAE	MSE/MAE	MSE/MAE
One-for-All	0.55	2.2	5.50/5.05	0.43/0.43	0.36/0.39	0.36/0.38	0.26/0.33	0.23/0.27	0.33/0.36
GPT4TS	11.7	371.0	0.26/0.24	0.43/0.43	0.35/0.40	0.35/0.38	0.27/0.33	0.24/0.27	0.33/0.36
TIME-LLM	6.46	3907	0.50/0.44	0.41/0.42	0.34/0.38	0.33/0.37	0.25/0.32	0.23/0.26	0.31/0.35
TEST	2.00	701	1.56/1.39	0.42/0.43	0.34/0.38	0.35/0.38	0.26/0.32	0.23/0.27	0.32/0.36
TEMPO	5.00	710	0.54/0.51	0.43/0.43	0.36/0.40	0.50/0.46	0.28/0.33	0.28/0.32	0.37/0.39

TABLE I: Comparative analysis of efficiency-accuracy trade-offs in long-term time-series forecasting. One-for-All achieves state-of-the-art efficiency with minimal resource overhead: At 5.5

\times

higher parameter efficiency (Eff.*MSE=5.50 vs. 0.05–4.41 in SOTA) and 2.2MiB memory (170–1800

\times

smaller than pretrained models), while matching top accuracy (Avg MSE=0.33) across all benchmarks.

Fig. 3: Trade-offs Between Model Accuracy, Efficiency, and Scalability for Long-Term Forecasting. The average MSE (y-axis) measures prediction accuracy (lower is better), while the number of trainable parameters (x-axis) reflects model efficiency (leftward is better). The bubble sizes represent memory usage (smaller is better), highlighting scalability constraints.

Model	Parameters (M)	Memory (MiB)	Eff.*MSE/MAE	ETTh1	ETTh2	ETTm1	ETTm2	Weather	Avg
				MSE/MAE	MSE/MAE	MSE/MAE	MSE/MAE	MSE/MAE	MSE/MAE
One-for-All	0.55	2.2	4.37/4.48	0.65/0.55	0.43/0.44	0.47/0.44	0.29/0.33	0.24/0.27	0.416/0.406
GPT4TS	11.7	371.0	0.21/0.21	0.59/0.53	0.40/0.42	0.47/0.44	0.30/0.34	0.24/0.27	0.400/0.400
TIME-LLM	6.46	3907	0.43/0.40	0.56/0.52	0.37/0.40	0.41/0.43	0.28/0.33	0.24/0.28	0.372/0.392
TEST	2.00	701	1.24/1.26	0.59/0.53	0.40/0.43	0.46/0.45	0.32/0.31	0.25/0.27	0.404/0.398
TimesNet	0.63	2.9	3.02/3.46	0.87/0.63	0.48/0.47	0.68/0.54	0.32/0.35	0.28/0.30	0.526/0.458

TABLE II: Comparative analysis of efficiency-accuracy trade-offs in few-shot time-series forecasting with 10% data. One-for-All achieves Pareto-optimal efficiency with minimal resource overhead: At 4.4

\times

higher parameter efficiency (Eff.*MSE=4.37 vs. 0.04–3.02 in SOTA) and only 0.55M parameters (14–180

\times

smaller than pretrained models), while matching top accuracy (Avg MSE=0.42) across all benchmarks.

Model	Parameters (M)	Memory (MiB)	Eff.*MSE/MAE	ETTh1	ETTh2	ETTm1	ETTm2	Weather	Avg
				MSE/MAE	MSE/MAE	MSE/MAE	MSE/MAE	MSE/MAE	MSE/MAE
One-for-All	0.55	2.2	3.92/4.17	0.71/0.57	0.52/0.49	0.52/0.47	0.30/0.35	0.27/0.30	0.464/0.436
GPT4TS	11.7	371.0	0.20/0.20	0.68/0.56	0.40/0.43	0.47/0.45	0.31/0.35	0.27/0.30	0.426/0.418
TIME-LLM	6.46	3907	0.39/0.38	0.63/0.55	0.39/0.42	0.43/0.44	0.28/0.32	0.26/0.31	0.398/0.408
TimesNet	0.63	2.9	2.89/3.39	0.92/0.64	0.46/0.45	0.72/0.56	0.35/0.37	0.30/0.32	0.550/0.468
EFDformer	16.8	87.8	0.12/0.13	0.66/0.56	0.44/0.45	0.73/0.59	0.40/0.41	0.31/0.35	0.508/0.472

TABLE III: Comparative analysis of efficiency-accuracy trade-offs in few-shot time-series forecasting with 5% data. One-for-All achieves Pareto-optimal efficiency: With 3.9

\times

higher parameter efficiency (Eff.*MSE=3.92 vs. 0.04–2.89 in SOTA) and only 0.55M parameters (10–150

\times

smaller than pretrained models), while maintaining competitive accuracy (Avg MSE=0.46).

Fig. 4: Trade-offs Between Model Accuracy, Efficiency, and Scalability for few-shot Forecasting with 10% data. The average MSE (y-axis) measures prediction accuracy (lower is better), while the number of trainable parameters (x-axis) reflects model efficiency (leftward is better). The bubble sizes represent memory usage (smaller is better), highlighting scalability constraints.

Transfer	Frequency	Method
Transfer	Frequency	One-for-All	GPT4TS	TimeNet	FEDformer	TStationary	ETSformer	LightTS	Autoformer	Informer	Reformer
M3 $\rightarrow$ M4	Yearly	13.53	16.42	18.75	16.00	17.05	20.56	15.63	16.18	19.70	16.03
	Quarterly	11.00	10.13	12.26	9.48	12.56	11.65	9.40	13.92	13.00	9.76
	Monthly	15.28	14.10	14.01	15.12	16.82	16.97	24.60	16.91	15.91	14.80
	Avg	13.27	13.55	15.01	13.53	15.48	16.39	16.54	15.67	16.20	13.53
M4 $\rightarrow$ M3	Yearly	19.23	13.74	15.65	13.88	14.98	27.84	13.78	14.55	18.54	15.65

TABLE IV: Zero-shot forecasting performance (sMAPE) showing cross-domain transfer learning results between M3 and M4 datasets. The One-for-All framework achieves 3 best performances (underlined) and superior average accuracy (13.27) in M3

\rightarrow

M4 transfer compared to GPT4TS (13.55), while maintaining competitive results in the reverse direction.

Method	SMAPE					MASE					OWS
	Yearly	Quarterly	Monthly	Others	Avg	Yearly	Quarterly	Monthly	Others	Avg	Yearly	Quarterly	Monthly	Others	Avg
One-for-All	14.29	10.51	13.11	5.37	12.37	3.34	1.21	0.96	3.66	1.70	0.85	0.92	0.90	1.14	0.90
GPT4TS	14.85	10.37	12.87	5.29	12.35	3.61	1.22	0.95	3.62	1.76	0.91	0.91	0.89	1.12	0.91
TimeNet	13.47	10.06	12.70	5.02	11.89	3.05	1.17	0.94	3.34	1.60	0.79	0.88	0.88	1.05	0.85
FEDformer	13.63	10.66	14.25	4.86	12.78	3.07	1.25	1.12	3.25	1.71	0.80	0.94	1.02	1.02	0.91
ETSformer	16.73	12.63	14.65	5.72	14.20	4.25	1.73	1.22	4.13	2.18	1.04	1.20	1.08	1.25	1.09

TABLE V: Short-term forecasting performance on M4 benchmark dataset comparing One-for-All against state-of-the-art methods across three key metrics (SMAPE, MASE, OWS) and four frequency domains. The One-for-All framework achieves competitive average performance (SMAPE: 12.37, MASE: 1.70, OWS: 0.90), ranking second overall while maintaining consistent results across all frequency types.

Fig. 5: Accuracy (%) of One-for-All (red) versus baseline models (blue) across six time-series datasets. The proposed model performs robustly, matching or exceeding specialized approaches in most tasks, particularly on Japanese Vowels (98%) and SCP1 (93%). Blue colors denote comparative baselines.

Methods	SMD			MSL			SMAP			SWaT			PSM			Avg
	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1
One-for-All	87.60	80.30	83.80	82.30	81.40	81.90	89.80	53.60	67.10	92.00	92.40	92.20	98.60	95.50	97.10	90.06	80.64	84.42
GPT4TS	88.89	84.98	86.89	82.00	82.91	82.45	90.60	60.95	72.88	92.20	96.34	94.23	98.62	95.68	97.13	90.46	84.17	86.72
TimeNet	87.91	81.54	84.61	89.54	75.36	81.84	90.14	56.40	69.39	90.75	95.40	93.02	98.51	96.20	97.34	91.37	80.98	85.24
FEDformer	87.95	82.39	85.08	77.14	80.07	78.57	90.47	58.10	70.76	90.17	96.42	93.19	97.31	97.16	97.23	88.61	82.83	84.97
TStationary	88.33	81.21	84.62	68.55	89.14	77.50	89.37	59.02	71.09	68.03	96.75	79.88	97.82	96.76	97.29	82.42	84.58	82.08

TABLE VI: Anomaly detection performance comparison (Precision/P, Recall/R, F1-score) across five benchmark datasets and average results. The One-for-All framework achieves competitive performance (Avg F1: 84.42) while maintaining consistency across datasets, particularly excelling on industrial datasets SWaT (F1: 92.20) and PSM (F1: 97.10).

Fig. 6: Performance-resource trade-off analysis of One-for-All framework across different rank configurations. (a) Prediction error (MSE) for both long-term and few-shot forecasting tasks, showing performance saturation beyond Rank 16. (b) Logarithmic scaling of model parameters with increasing rank, demonstrating sub-linear growth. The red vertical line marks the optimal operating point (Rank 16) where prediction accuracy stabilizes while maintaining minimal resource requirements. Shaded regions indicate the recommended deployment range (Ranks 16-64) for different application constraints.

	Resource Budget		Long-term Forecasting				Few-shot Forecasting (10%)
Rank	Param. (M)	Mem. (MiB)	Param.	Mem.	MSE	MAE	Param.	Mem.	MSE	MAE
Rank 2	✓	✓	0.068	0.277	0.41	0.42	0.068	0.277	0.49	0.45
Rank 4	✓	✓	0.137	0.055	0.37	0.40	0.137	0.055	0.45	0.42
Rank 8	✓	✓	0.275	1.100	0.33	0.37	0.275	1.100	0.42	0.42
Rank 16	✓	✓	0.550	2.200	0.33	0.36	0.550	2.200	0.42	0.41
Rank 32	✓	✓	1.100	4.425	0.33	0.36	1.100	4.425	0.42	0.41

TABLE VII: Comprehensive analysis of One-for-All framework’s performance and resource efficiency across different rank configurations (2–1024). The table compares parameter counts (millions), memory requirements (MiB), and prediction accuracy (MSE/MAE) for both long-term and few-shot (10% training data) forecasting tasks. Gray highlighting indicates the optimal efficiency frontier (Ranks 16–64) where performance plateaus while maintaining minimal resource overhead.

Panel of Figure 14 — Fig. 7: Few-shot forecasting results for five datasets, showcasing the GPT-2-based models: One-for-All and GPT4TS, with a prediction horizon of 96. The green line represents the ground truth time-series data, while the red line shows the predicted time-series data.

Panel of Figure 15 — Fig. 8: Long-term forecasting results for the ETTm1 dataset showcasing the One-for-All framework with different ranks: (a) Rank 2, (b) Rank 4, (c) Rank 8, (d) Rank 16, (e) Rank 32, (f) Rank 64, (g) Rank 128, (h) Rank 256, (i) Rank 512, and (j) Rank 1024, under the prediction horizon of 96. The green line represents the ground truth time-series data, while the red line depicts the predicted time-series data.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Dimension	Length	Prediction Length	Frequency	Domain
ETTh	7	17420	{96, 192, 336, 720}	1 hour	Temperature
ETTm	7	69680	{96, 192, 336, 720}	15 min	Temperature
Weather	22	52696	{96, 192, 336, 720}	10 min	Weather

Appendix

Table 1: Dataset specifications for few-shot and long-term forecasting. Each dataset varies in temporal resolution, series length, and prediction horizons. The few-shot setting uses only 10% of the training data, while long-term experiments evaluate model performance over increasing forecast horizons.

Dataset	Frequency	Length	Prediction Length	Mapping
				M4	M3
M3	Yearly	645	6	Yearly	–
M3	Quarterly	756	8	Quarterly	–
M3	Monthly	1428	18	Monthly	–
M4	Yearly	23000	6	–	Yearly
M4	Quarterly	24000	8	–	Quarterly

Appendix

Table 2: Summary of the M3 and M4 datasets used for zero-shot forecasting. The table includes frequency, series length, and prediction horizons. The mapping columns indicate the corresponding source-target frequency pairs used during M3

\rightarrow

M4 transfer, where models trained on M3 are evaluated on M4 without fine-tuning.

	Dataset	Hyperparameters					Training Process
		GPT layer	Patch Size	Input length	Label length	Heads	Learning rate	Loss	Batch Size	Epochs
Few/Long	ETTh1	6	16	336	168	4	$10^{-3}$	MSE	256	10
	ETTh2	6	16	336	168	4	$10^{-3}$	MSE	256	10
	ETTm1	6	16	512	48	4	$10^{-3}$	MSE	256	10
	ETTm2	6	16	512	48	4	$10^{-3}$	MSE	256	10
	Weather	6	16	512	48	4	$10^{-3}$	MSE	512	10

Appendix

Table 3: Experiment configuration for the One-for-All Framework: long-term forecasting, few-shot forecasting, and zero-shot forecasting of time-series data.

	Method	One-for-All		GPT4TS		TIME-LLM		TEST		TEMPO		TimeNet		FEDformer		TStationary		ETSformer		Autoformer		Informer		Reformer
		MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTh1	96	0.38	0.40	0.38	0.40	0.36	0.39	0.37	0.40	0.40	0.41	0.38	0.40	0.38	0.42	0.51	0.49	0.49	0.48	0.45	0.46	0.86	0.71	0.84	0.73
	192	0.42	0.42	0.42	0.42	0.40	0.42	0.42	0.42	0.43	0.42	0.44	0.43	0.42	0.45	0.53	0.50	0.54	0.50	0.50	0.48	1.00	0.79	0.92	0.77
	336	0.44	0.43	0.44	0.43	0.43	0.43	0.42	0.44	0.44	0.43	0.49	0.47	0.46	0.46	0.59	0.53	0.57	0.52	0.52	0.50	1.11	0.81	1.20	0.83
	720	0.47	0.47	0.48	0.46	0.44	0.46	0.45	0.47	0.44	0.45	0.52	0.50	0.51	0.51	0.64	0.62	0.56	0.53	0.51	0.51	1.18	0.86	1.26	0.89
	Avg	0.43	0.43	0.43	0.43	0.41	0.42	0.42	0.43	0.43	0.43	0.46	0.45	0.44	0.46	0.57	0.54	0.54	0.51	0.50	0.49	1.04	0.79	1.06	0.81

Appendix

Table 4: Comprehensive long-term forecasting results across multiple deep learning models and transformer variants on six benchmark datasets (ETTh1, ETTh2, ETTm1, ETTm2, and Weather) for four prediction lengths: 96, 192, 336, and 720. Performance is reported using Mean Squared Error (MSE) and Mean Absolute Error (MAE). The “Avg” row represents the average performance across all horizons for each dataset.

	Method	One-for-All		GPT4TS		TIME-LLM		TEST		TimeNet		FEDformer		TStationary		ETSformer		Autoformer		Informer		Reformer
		MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTh1	96	0.51	0.48	0.46	0.46	0.45	0.46	0.46	0.46	0.86	0.63	0.51	0.50	0.92	0.64	1.11	0.81	0.61	0.55	1.18	0.79	1.18	0.79
	192	0.62	0.52	0.57	0.52	0.49	0.48	0.57	0.52	0.80	0.59	0.62	0.55	0.91	0.63	1.15	0.82	0.72	0.60	1.20	0.81	1.29	0.85
	336	0.75	0.59	0.61	0.53	0.59	0.54	0.61	0.53	0.94	0.65	0.69	0.57	0.94	0.64	1.18	0.83	0.75	0.62	1.20	0.81	1.29	0.85
	720	0.72	0.59	0.72	0.59	0.70	0.61	0.72	0.60	0.88	0.64	0.73	0.61	0.89	0.64	1.27	0.87	0.72	0.62	1.22	0.82	1.22	0.84
	Avg	0.65	0.55	0.59	0.53	0.56	0.52	0.59	0.53	0.87	0.63	0.64	0.56	0.92	0.64	1.18	0.83	0.70	0.60	1.20	0.81	1.25	0.83

Appendix

Table 5: Few-shot forecasting results using 10% of the training data across four standard prediction lengths

N\in\{96,192,336,720\}

for six benchmark datasets (ETTh1, ETTh2, ETTm1, ETTm2, and Weather). This table highlights the robustness and generalisability of various models under constrained training data regimes.

	Method	One-for-All		GPT4TS		TIME-LLM		TimeNet		FEDformer		TStationary		ETSformer		Autoformer		Informer		Reformer
		MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTh1	96	0.63	0.54	0.54	0.51	0.48	0.47	0.89	0.62	0.59	0.53	0.95	0.65	1.17	0.83	0.68	0.57	1.22	0.81	1.20	0.79
	192	0.75	0.58	0.75	0.58	0.63	0.54	0.94	0.66	0.65	0.56	0.94	0.64	1.22	0.85	0.72	0.60	1.25	0.83	1.27	0.85
	336	0.75	0.59	0.75	0.59	0.77	0.63	0.94	0.65	0.73	0.59	0.93	0.64	1.18	0.83	0.76	0.62	1.20	0.81	1.25	0.86
	720	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–
	Avg	0.71	0.57	0.68	0.56	0.63	0.55	0.92	0.64	0.66	0.56	0.94	0.64	1.19	0.84	0.72	0.60	1.22	0.82	1.24	0.83

Appendix

Table 6: Few-shot forecasting results using only 5% of the training data across the same four prediction lengths

N\in\{96,192,336,720\}

. This setting simulates extreme data-scarce conditions. The results offer insights into the sample efficiency and stability of each model when operating under minimal supervision. “–” indicates that training data is not available.

	Variant	Long-term Forecasting with Different Rank
		Rank 2		Rank 4		Rank 8		Rank 16		Rank 32		Rank 64		Rank 128		Rank 256		Rank 512		Rank 1024
		MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTh1	96	0.55	0.51	0.48	0.47	0.40	0.42	0.38	0.40	0.38	0.40	0.38	0.40	0.38	0.40	0.38	0.40	0.38	0.40	0.39	0.40
	192	0.59	0.52	0.51	0.49	0.43	0.44	0.42	0.42	0.42	0.42	0.42	0.42	0.42	0.43	0.42	0.42	0.42	0.43	0.42	0.43
	336	0.60	0.53	0.53	0.50	0.45	0.45	0.44	0.43	0.45	0.45	0.45	0.45	0.45	0.44	0.45	0.45	0.45	0.45	0.45	0.45
	720	0.58	0.54	0.53	0.52	0.47	0.48	0.47	0.47	0.47	0.48	0.46	0.47	0.46	0.47	0.47	0.48	0.46	0.47	0.48	0.48

Appendix

Table 7: Performance evaluation of the One-for-All model with varying low-rank configurations for long-term forecasting. The table presents the MSE and MAE across four forecasting horizons (

N\in\{96,192,336,720\}

) on five benchmark datasets (ETTh1, ETTh2, ETTm1, ETTm2, and Weather). Each column group corresponds to a specific rank used in the model, enabling a comprehensive comparison of prediction accuracy across different rank settings.

	Variant	Few-shot Forecasting with Different Rank
		Rank 2		Rank 4		Rank 8		Rank 16		Rank 32		Rank 64		Rank 128		Rank 256		Rank 512		Rank 1024
		MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTh1	96	0.72	0.56	0.73	0.56	0.52	0.49	0.51	0.48	0.49	0.47	0.49	0.47	0.52	0.49	0.53	0.49	0.53	0.50	0.53	0.49
	192	0.75	0.58	0.75	0.58	0.64	0.54	0.62	0.52	0.63	0.54	0.64	0.54	0.64	0.55	0.66	0.55	0.68	0.56	0.66	0.55
	336	0.74	0.59	0.75	0.59	0.75	0.59	0.75	0.59	0.74	0.59	0.75	0.59	0.75	0.59	0.76	0.59	0.75	0.59	0.75	0.59
	720	0.74	0.61	0.74	0.61	0.74	0.61	0.72	0.59	0.74	0.61	0.74	0.61	0.75	0.61	0.75	0.61	0.74	0.61	0.74	0.61

Appendix

Table 8: Comprehensive evaluation of the One-for-All model under Few-Shot Forecasting using 10% of training data. The table presents the model’s performance across four prediction horizons (

N\in\{96,192,336,720\}

) on five datasets (ETTh1, ETTh2, ETTm1, ETTm2, Weather), with varying low-rank configurations. Each cell reports the MSE and MAE metrics. Bold values indicate the best performance for each horizon and dataset across all ranks. The results highlight the impact of rank selection on model efficiency and accuracy under constrained data availability.

	Variant	Few-shot Forecasting with Different Rank
		Rank 2		Rank 4		Rank 8		Rank 16		Rank 32		Rank 64		Rank 128		Rank 256		Rank 512		Rank 1024
		MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTh1	96	0.74	0.57	0.63	0.53	0.64	0.54	0.63	0.54	0.63	0.55	0.63	0.55	0.61	0.53	0.62	0.54	0.69	0.55	0.63	0.53
	192	0.75	0.59	0.75	0.58	0.75	0.58	0.75	0.58	0.75	0.58	0.75	0.59	0.75	0.58	0.75	0.58	0.75	0.58	0.75	0.58
	336	0.75	0.60	0.75	0.60	0.75	0.59	0.75	0.59	0.75	0.59	0.75	0.59	0.76	0.60	0.75	0.60	0.75	0.60	0.75	0.59
	720	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–

Appendix

Table 9: Comprehensive evaluation of the One-for-All model under Few-Shot Forecasting using 5% of training data. The table presents the model’s performance across four prediction horizons (

N\in\{96,192,336,720\}

	Variant	Rank 2	Rank 4	Rank 8	Rank 16	Rank 32	Rank 64	Rank 128	Rank 256	Rank 512	Rank 1024
	Metric	sMAPE	sMAPE	sMAPE	sMAPE	sMAPE	sMAPE	sMAPE	sMAPE	sMAPE	sMAPE
M3 $\rightarrow$ M4	Yearly	13.80	13.67	13.60	13.53	13.54	13.51	13.41	13.69	13.57	13.73
	Quarterly	11.31	11.03	10.86	11.00	10.92	11.09	10.89	11.12	10.91	11.45
	Monthly	15.52	15.28	15.16	15.28	15.39	14.90	15.07	14.94	14.85	14.95
M4 $\rightarrow$ M3	Yearly	22.77	22.25	20.75	19.23	19.11	18.01	18.16	17.87	17.61	17.44
	Quarterly	11.92	11.83	11.57	11.77	11.66	11.01	11.66	11.28	11.47	11.20

Appendix

Table 10: Zero-shot forecasting performance of the One-for-All model across varying LoRA ranks on the M3 and M4 datasets. The table reports sMAPE scores for different frequency categories (Yearly, Quarterly, Monthly) in both M3

\rightarrow

M4 and M4

\rightarrow

M3 transfer settings. The highlighted values correspond to Rank 16, used as the baseline configuration. Lower sMAPE indicates better performance.

Method	SMAPE					MASE					OWS
	Yearly	Quarterly	Monthly	Others	Avg	Yearly	Quarterly	Monthly	Others	Avg	Yearly	Quarterly	Monthly	Others	Avg
Rank 2	15.28	10.48	12.96	5.29	12.51	3.71	1.23	0.96	3.59	1.79	0.93	0.92	0.90	1.12	0.93
Rank 4	14.29	10.51	13.11	5.37	12.37	3.34	1.21	0.96	3.66	1.70	0.85	0.92	0.90	1.14	0.90
Rank 8	15.21	10.50	13.02	5.34	12.53	3.76	1.24	0.96	3.60	1.80	0.93	0.92	0.90	1.13	0.93
Rank 16	14.79	10.36	12.96	5.34	12.38	3.59	1.22	0.96	3.57	1.76	0.90	0.91	0.90	1.21	0.91
Rank 32	15.05	10.49	13.36	5.45	12.66	3.68	1.23	0.99	3.63	1.80	0.92	0.92	0.92	1.14	0.93

Appendix

Table 11: Short-term forecasting performance on the M4 dataset across various ranks, evaluated using three performance metrics: SMAPE, MASE, and OWS. The results are presented for different data frequencies (Yearly, Quarterly, Monthly, and Others) with the average performance shown in the “Avg” column. The table summarizes the impact of different rank configurations on model performance for each metric. Lower values indicate better forecasting accuracy.

Dateset	Rank 2	Rank 4	Rank 8	Rank 16	Rank 32	Rank 64	Rank 128	Rank 256	Rank 512	Rank 1024
Ethanol	28%	30%	32%	32%	30%	29%	30%	31%	30%	32%
Face Detection	56%	62%	65%	66%	66%	66%	66%	66%	68%	67%
Heartbeat	72%	75%	76%	75%	75%	76%	76%	76%	76%	75%
Japanese Vowels	46%	81%	95%	97%	97%	97%	98%	97%	98%	97%
SCP1	83%	89%	91%	91%	91%	92%	91%	92%	91%	93%
SCP2	50%	55%	60%	57%	53%	57%	55%	58%	57%	55%

Appendix

Table 12: Detailed accuracy analysis of the One-for-All model with varying ranks (Rank 2 to Rank 1024) for classification tasks across multiple datasets. The accuracy for each dataset is reported as a percentage, showing the model’s performance at different rank configurations. The table includes results for datasets such as Ethanol, Face Detection, Heartbeat, Japanese Vowels, SCP1, and SCP2.

Dateset	Rank 2	Rank 4	Rank 8	Rank 16	Rank 32	Rank 64	Rank 128	Rank 256	Rank 512	Rank 1024
MSL	80%	83%	82%	82%	82%	82%	82%	82%	82%	82%
PSM	96%	96%	97%	97%	97%	97%	97%	97%	94%	94%
SMAP	66%	67%	67%	67%	67%	67%	67%	67%	67%	67%
SMD	82%	82%	83%	84%	84%	84%	84%	84%	84%	84%
SWAT	88%	92%	92%	92%	92%	92%	92%	92%	92%	92%
Avg	82%	84%	84%	84%	84%	84%	84%	84%	84%	84%

Appendix

Table 13: Detailed F1-score (as %) analysis of the One-for-All model with varying ranks (Rank 2 to Rank 1024) for anomaly detection tasks across multiple datasets. The F1-score, representing the harmonic mean of precision and recall, is reported for datasets such as MSL, PSM, SMAP, SMD, and SWAT. The “Avg” row provides the average F1-score across all datasets for each rank configuration. The table highlights the model’s performance in anomaly detection at different ranks.

Variant	Trainable parameters (M) for Few-shot Forecasting and Long-term Forecasting
	96			192			336			720			Avg
	Tran.*	%All*	Mem.*	Tran.	%All	Mem.	Tran.	%All	Mem.	Tran.	%All	Mem.	Tran.	%All	Mem.
Rank 2	0.068	0.080%	0.275	0.068	0.077%	0.276	0.067	0.074%	0.277	0.069	0.066%	0.280	0.068	0.074%	0.277
Rank 4	0.136	0.160%	0.548	0.136	0.155%	0.550	0.137	0.148%	0.552	0.139	0.132%	0.558	0.137	0.149%	0.552
Rank 8	0.273	0.320%	1.100	0.273	0.309%	1.100	0.275	0.295%	1.100	0.278	0.263%	1.100	0.275	0.297%	1.100
Rank 16	0.546	0.638%	2.200	0.547	0.617%	2.200	0.550	0.589%	2.200	0.556	0.526%	2.200	0.550	0.593%	2.200

Appendix

Table 14: Efficiency analysis of the One-for-All model with varying ranks (Rank 2 to Rank 1024) for few-shot forecasting and long-term forecasting tasks using the ETTh1 and ETTh2 datasets. The table presents the trainable parameters (in millions), the percentage of total model parameters, and the memory required to save the trained model (in MiB) for each rank across both forecasting tasks. The results highlight the impact of increasing model rank on parameter count, memory usage, and the overall efficiency of the model, showcasing the trade-offs between model size and performance.

Variant	Trainable parameters (M) for Few-shot Forecasting and Long-term Forecasting
	96			192			336			720			Avg
	Tran.*	%All*	Mem.*	Tran.	%All	Mem.	Tran.	%All	Mem.	Tran.	%All	Mem.	Tran.	%All	Mem.
Rank 2	0.054	0.064%	0.220	0.054	0.062%	0.220	0.054	0.060%	0.222	0.055	0.055%	0.225	0.054	0.060%	0.222
Rank 4	0.108	0.128%	0.437	0.109	0.125%	0.439	0.109	0.121%	0.441	0.111	0.111%	0.447	0.109	0.121%	0.441
Rank 8	0.217	0.257%	0.873	0.218	0.251%	0.876	0.219	0.242%	0.881	0.222	0.221%	0.893	0.219	0.243%	0.881
Rank 16	0.435	0.513%	1.700	0.437	0.501%	1.800	0.439	0.483%	1.800	0.445	0.442%	1.800	0.439	0.485%	1.775

Appendix

Table 15: Efficiency analysis of the One-for-All model with varying ranks (Rank 2 to Rank 1024) for few-shot forecasting and long-term forecasting tasks using the ETTm1 and ETTm2 datasets. The table presents the trainable parameters (in millions), the percentage of total model parameters, and the memory required to save the trained model (in MiB) for each rank across both forecasting tasks. The results highlight the impact of increasing model rank on parameter count, memory usage, and the overall efficiency of the model, showcasing the trade-offs between model size and performance.

Variant	Trainable parameters (M) for Few-shot Forecasting and Long-term Forecasting
	96			192			336			720			Avg
	Tran.*	%All*	Mem.*	Tran.	%All	Mem.	Tran.	%All	Mem.	Tran.	%All	Mem.	Tran.	%All	Mem.
Rank 2	0.102	0.117%	0.410	0.102	0.111%	0.411	0.102	0.104%	0.412	0.103	0.087%	0.415	0.102	0.105%	0.412
Rank 4	0.204	0.235%	0.818	0.204	0.223%	0.820	0.205	0.207%	0.822	0.206	0.175%	0.828	0.205	0.210%	0.822
Rank 8	0.408	0.469%	1.600	0.409	0.445%	1.600	0.410	0.415%	1.600	0.413	0.351%	1.700	0.410	0.420%	1.625
Rank 16	0.816	0.933%	3.300	0.818	0.887%	3.300	0.820	0.826%	3.300	0.826	0.699%	3.300	0.820	0.836%	3.300

Appendix

Table 16: Efficiency analysis of the One-for-All model with varying ranks (Rank 2 to Rank 1024) for few-shot forecasting and long-term forecasting tasks using the Weather datasets. The table presents the trainable parameters (in millions), the percentage of total model parameters, and the memory required to save the trained model (in MiB) for each rank across both forecasting tasks. The results highlight the impact of increasing model rank on parameter count, memory usage, and the overall efficiency of the model, showcasing the trade-offs between model size and performance.

Models	Trainable parameters (M) and checkpoint size (MiB) for few-shot forecasting and long-term forecasting
	96			192			336			720			Avg
	Tran.*	%All*	Mem.*	Tran.	%All	Mem.	Tran.	%All	Mem.	Tran.	%All	Mem.	Tran.	%All	Mem.
GPT4TS	3.916	4.60%	340.1	7.012	7.95%	352.5	11.65	12.5%	371.1	24.04	22.8%	420.6	11.65	11.9%	371.0
TIME-LLM (LLaMA-7B)	6.390	0.20%	3678	6.420	0.20%	3812	6.480	0.2%	3960	6.550	0.2%	4176	6.460	0.2%	3907
TEST (GPT-2 Medium)	–	–	–	–	–	–	–	—	–	–	–	–	$\sim$ 1-2	$\sim$ 0.3–0.6%	$\sim$ 701
TEMPO (GPT-2 Medium)	–	–	–	–	–	–	–	–	–	–	–	–	$\sim$ 3-5	$\sim$ 0.9%-1.4%	$\sim$ 710

Appendix

Table 17: Efficiency analysis of state-of-the-art models in few-shot and long-term forecasting tasks on the ETTh1 and ETTh2 datasets. The table provides the trainable parameters (in millions), the percentage of total parameters, and the memory required for saving the trained model (in MiB) across various forecasting tasks. The models compared include GPT4TS, TIME-LLM (LLaMA-7B), TEST (GPT-2 Medium), TEMPO (GPT-2 Medium), TimesNet, EFDformer, Stationary, ETSformer, Autoformer, Informer, and Reformer, with performance metrics shown for different ranks of forecasting. This analysis helps in understanding the trade-offs between model size, efficiency, and forecasting accuracy.

LLM-Based Time Series Forecasting

LLM-based time series forecasting uses large language models to predict future values from numerical histories and contextual data.

See all 100 papers

LLM Pretraining Shapes a Generalizable Manifold: Insights into Cross-Modal Transfer to Time Series
May 19, 2026Alexis Roger, Prateek Humane, Zhenghan Tai +4Cross-Modal Learning Language Model Pretraining
Université de Montréal Mila - Quebec AI Institute
Parameter-Efficient Adaptation of Pretrained Language Models for Time-Series Forecasting
Sep 14, 2026Tamanna Kumavat, Georg Brunner, Kyriakos FlourisTime Series Forecasting LLM-Based Time Series Forecasting
University of Zurich, Switzerland · Computer Vision Laboratory, ETH Zurich, Switzerland · MRC Biostatistics Unit, University of Cambridge, United Kingdom
FM-LLM: A frequency-enhanced mixture-of-experts framework for adapting LLMs to time series forecasting
Aug 12, 2026Rentao Gu, Yihang Ding, Junjie Li +5Zero-Shot Time Series Forecasting Time Series Forecasting
Beijing University of Posts and Telecommunications, Beijing, 100876, China · China Telecom Research Institute, Beijing, 100032, China

May 19, 2026J/K move · Enter open · S save

LLM Pretraining Shapes a Generalizable Manifold: Insights into Cross-Modal Transfer to Time Series

Alexis Roger, Prateek Humane, Zhenghan Tai, Gwen Legate+3

Université de Montréal Mila - Quebec AI Institute

Cross-Modal Learning Language Model Pretraining Representation Geometry LLM-Based Time Series Forecasting Transfer Learning Time Series Representation Learning

Open paper AskPDF arXiv

Can language-pretrained transformers become effective time-series forecasters, and why? In this paper, we show that cross-modal transfer arises because language pretraining preconditions time series training with a reusable manifold. A linear probe on frozen LLM states decodes realistic time-series trajectories without paired supervision, and retrieval in this projected space yields competitive forecasts, showing that structure and dynamics exist before finetuning. Pretrained initialization also improves optimization, producing coherent gradients and a highly anisotropic loss landscape unlike random initialization. Finetuning then acts as low-dimensional alignment, reusing existing directions rather than learning temporal primitives from scratch, as evidenced by low-rank updates, subspace alignment, and shared features for periodicity, trend, and repetition. Together, these results support a geometric account of LLM-to-time-series transfer: language pretraining builds the manifold, and finetuning projects numerical dynamics onto task-relevant directions.

Abstract

LLM-Based Time Series Forecasting

Explore similar work

LLM Pretraining Shapes a Generalizable Manifold: Insights into Cross-Modal Transfer to Time Series

Parameter-Efficient Adaptation of Pretrained Language Models for Time-Series Forecasting

FM-LLM: A frequency-enhanced mixture-of-experts framework for adapting LLMs to time series forecasting