Organizations: ADAPT SFI Research Centre, School of Computer Science, Technological University Dublin, Ireland · ADAPT SFI Research Centre, School of Computer Science, University College Dublin
We address the challenge of adapting pre-trained Large Language Models (LLMs) for multivariate time-series analysis, where their deployment is often hindered by prohibitive computational and memory demands. Our solution, One-for-All, introduces Gaussian Rank-Stabilized Low-Rank Adapters (rsLoRA) to enable parameter-efficient fine-tuning of frozen LLMs. While inspired by LoRA, rsLoRA introduces a mathematically grounded rank-stabilization mechanism that enables provable gradient stability at low ranks a novel contribution absent in prior PEFT methods. Our framework injects trainable rank decomposition matrices (rank 16) into positional embeddings and output layers, while keeping self-attention weights fixed. This design reduces trainable parameters by 6.8× (vs. TimesNet), 21× (vs. GPT4TS), and 11.8× (vs. TIME-LLM), while achieving a 168-1,776× smaller memory footprint (2.2MiB vs. 340MiB-4.18GiB in SOTA models). Rigorous evaluation across six time-series tasks demonstrates that One-for-All achieves state-of-the-art efficiency-accuracy trade-offs: 5.5× higher parameter efficiency (MSE=5.50) than TimesNet and 21× better than GPT4TS, while matching their forecasting accuracy (MSE=0.33). The framework's stability is validated through consistent performance across diverse horizons (96-720 steps) and datasets (ETT, Weather, M3, M4), with 98.3% fewer parameters than conventional transformers. These advances enable deployment on edge devices for healthcare, finance, and environmental monitoring without compromising performance.
Figures & tables
Fig. 1: One-for-All Framework: A parameter-efficient LLM unifying long-term, few-shot, zero-shot, short-term forecasting, classification, and anomaly detection. By integrating Gaussian rank-stabilized LoRA (rsLoRA) into positional embeddings and output layers while freezing the pre-trained LLM weights, we minimize trainable parameters without compromising stability.
Fig. 2: Comparison of model efficiency across different forecast horizons. (a) Trainable parameters (in millions, log scale) and (b) Model size (in MiB, log scale) for various time-series forecasting approaches. The One-for-All model (highlighted in red) demonstrates consistently low parameter counts and memory footprint across all horizons, while other methods (dashed lines) show varying computational requirements. Notably, large pretrained models (e.g., TIME-LLM and GPT4TS) exhibit significantly higher resource demands, particularly at longer horizons (336 and 720). The Avg column represents the average across all horizons, further emphasizing the efficiency advantages of the proposed approach.
Model
Parameters (M)
Memory (MiB)
Eff.*MSE/MAE
ETTh1
ETTh2
ETTm1
ETTm2
Weather
Avg
MSE/MAE
MSE/MAE
MSE/MAE
MSE/MAE
MSE/MAE
MSE/MAE
One-for-All
0.55
2.2
5.50/5.05
0.43/0.43
0.36/0.39
0.36/0.38
0.26/0.33
0.23/0.27
0.33/0.36
GPT4TS
11.7
371.0
0.26/0.24
0.43/0.43
0.35/0.40
0.35/0.38
0.27/0.33
0.24/0.27
0.33/0.36
TIME-LLM
6.46
3907
0.50/0.44
0.41/0.42
0.34/0.38
0.33/0.37
0.25/0.32
0.23/0.26
0.31/0.35
TEST
2.00
701
1.56/1.39
0.42/0.43
0.34/0.38
0.35/0.38
0.26/0.32
0.23/0.27
0.32/0.36
TEMPO
5.00
710
0.54/0.51
0.43/0.43
0.36/0.40
0.50/0.46
0.28/0.33
0.28/0.32
0.37/0.39
TABLE I: Comparative analysis of efficiency-accuracy trade-offs in long-term time-series forecasting. One-for-All achieves state-of-the-art efficiency with minimal resource overhead: At 5.5 × higher parameter efficiency (Eff.*MSE=5.50 vs. 0.05–4.41 in SOTA) and 2.2MiB memory (170–1800 × smaller than pretrained models), while matching top accuracy (Avg MSE=0.33) across all benchmarks.
Fig. 3: Trade-offs Between Model Accuracy, Efficiency, and Scalability for Long-Term Forecasting. The average MSE (y-axis) measures prediction accuracy (lower is better), while the number of trainable parameters (x-axis) reflects model efficiency (leftward is better). The bubble sizes represent memory usage (smaller is better), highlighting scalability constraints.
Model
Parameters (M)
Memory (MiB)
Eff.*MSE/MAE
ETTh1
ETTh2
ETTm1
ETTm2
Weather
Avg
MSE/MAE
MSE/MAE
MSE/MAE
MSE/MAE
MSE/MAE
MSE/MAE
One-for-All
0.55
2.2
4.37/4.48
0.65/0.55
0.43/0.44
0.47/0.44
0.29/0.33
0.24/0.27
0.416/0.406
GPT4TS
11.7
371.0
0.21/0.21
0.59/0.53
0.40/0.42
0.47/0.44
0.30/0.34
0.24/0.27
0.400/0.400
TIME-LLM
6.46
3907
0.43/0.40
0.56/0.52
0.37/0.40
0.41/0.43
0.28/0.33
0.24/0.28
0.372/0.392
TEST
2.00
701
1.24/1.26
0.59/0.53
0.40/0.43
0.46/0.45
0.32/0.31
0.25/0.27
0.404/0.398
TimesNet
0.63
2.9
3.02/3.46
0.87/0.63
0.48/0.47
0.68/0.54
0.32/0.35
0.28/0.30
0.526/0.458
TABLE II: Comparative analysis of efficiency-accuracy trade-offs in few-shot time-series forecasting with 10% data. One-for-All achieves Pareto-optimal efficiency with minimal resource overhead: At 4.4 × higher parameter efficiency (Eff.*MSE=4.37 vs. 0.04–3.02 in SOTA) and only 0.55M parameters (14–180 × smaller than pretrained models), while matching top accuracy (Avg MSE=0.42) across all benchmarks.
Model
Parameters (M)
Memory (MiB)
Eff.*MSE/MAE
ETTh1
ETTh2
ETTm1
ETTm2
Weather
Avg
MSE/MAE
MSE/MAE
MSE/MAE
MSE/MAE
MSE/MAE
MSE/MAE
One-for-All
0.55
2.2
3.92/4.17
0.71/0.57
0.52/0.49
0.52/0.47
0.30/0.35
0.27/0.30
0.464/0.436
GPT4TS
11.7
371.0
0.20/0.20
0.68/0.56
0.40/0.43
0.47/0.45
0.31/0.35
0.27/0.30
0.426/0.418
TIME-LLM
6.46
3907
0.39/0.38
0.63/0.55
0.39/0.42
0.43/0.44
0.28/0.32
0.26/0.31
0.398/0.408
TimesNet
0.63
2.9
2.89/3.39
0.92/0.64
0.46/0.45
0.72/0.56
0.35/0.37
0.30/0.32
0.550/0.468
EFDformer
16.8
87.8
0.12/0.13
0.66/0.56
0.44/0.45
0.73/0.59
0.40/0.41
0.31/0.35
0.508/0.472
TABLE III: Comparative analysis of efficiency-accuracy trade-offs in few-shot time-series forecasting with 5% data. One-for-All achieves Pareto-optimal efficiency: With 3.9 × higher parameter efficiency (Eff.*MSE=3.92 vs. 0.04–2.89 in SOTA) and only 0.55M parameters (10–150 × smaller than pretrained models), while maintaining competitive accuracy (Avg MSE=0.46).
Fig. 4: Trade-offs Between Model Accuracy, Efficiency, and Scalability for few-shot Forecasting with 10% data. The average MSE (y-axis) measures prediction accuracy (lower is better), while the number of trainable parameters (x-axis) reflects model efficiency (leftward is better). The bubble sizes represent memory usage (smaller is better), highlighting scalability constraints.
Transfer
Frequency
Method
One-for-All
GPT4TS
TimeNet
FEDformer
TStationary
ETSformer
LightTS
Autoformer
Informer
Reformer
M3 → M4
Yearly
13.53
16.42
18.75
16.00
17.05
20.56
15.63
16.18
19.70
16.03
Quarterly
11.00
10.13
12.26
9.48
12.56
11.65
9.40
13.92
13.00
9.76
Monthly
15.28
14.10
14.01
15.12
16.82
16.97
24.60
16.91
15.91
14.80
Avg
13.27
13.55
15.01
13.53
15.48
16.39
16.54
15.67
16.20
13.53
M4 → M3
Yearly
19.23
13.74
15.65
13.88
14.98
27.84
13.78
14.55
18.54
15.65
TABLE IV: Zero-shot forecasting performance (sMAPE) showing cross-domain transfer learning results between M3 and M4 datasets. The One-for-All framework achieves 3 best performances (underlined) and superior average accuracy (13.27) in M3 → M4 transfer compared to GPT4TS (13.55), while maintaining competitive results in the reverse direction.
Method
SMAPE
MASE
OWS
Yearly
Quarterly
Monthly
Others
Avg
Yearly
Quarterly
Monthly
Others
Avg
Yearly
Quarterly
Monthly
Others
Avg
One-for-All
14.29
10.51
13.11
5.37
12.37
3.34
1.21
0.96
3.66
1.70
0.85
0.92
0.90
1.14
0.90
GPT4TS
14.85
10.37
12.87
5.29
12.35
3.61
1.22
0.95
3.62
1.76
0.91
0.91
0.89
1.12
0.91
TimeNet
13.47
10.06
12.70
5.02
11.89
3.05
1.17
0.94
3.34
1.60
0.79
0.88
0.88
1.05
0.85
FEDformer
13.63
10.66
14.25
4.86
12.78
3.07
1.25
1.12
3.25
1.71
0.80
0.94
1.02
1.02
0.91
ETSformer
16.73
12.63
14.65
5.72
14.20
4.25
1.73
1.22
4.13
2.18
1.04
1.20
1.08
1.25
1.09
TABLE V: Short-term forecasting performance on M4 benchmark dataset comparing One-for-All against state-of-the-art methods across three key metrics (SMAPE, MASE, OWS) and four frequency domains. The One-for-All framework achieves competitive average performance (SMAPE: 12.37, MASE: 1.70, OWS: 0.90), ranking second overall while maintaining consistent results across all frequency types.
Fig. 5: Accuracy (%) of One-for-All (red) versus baseline models (blue) across six time-series datasets. The proposed model performs robustly, matching or exceeding specialized approaches in most tasks, particularly on Japanese Vowels (98%) and SCP1 (93%). Blue colors denote comparative baselines.
Methods
SMD
MSL
SMAP
SWaT
PSM
Avg
P
R
F1
P
R
F1
P
R
F1
P
R
F1
P
R
F1
P
R
F1
One-for-All
87.60
80.30
83.80
82.30
81.40
81.90
89.80
53.60
67.10
92.00
92.40
92.20
98.60
95.50
97.10
90.06
80.64
84.42
GPT4TS
88.89
84.98
86.89
82.00
82.91
82.45
90.60
60.95
72.88
92.20
96.34
94.23
98.62
95.68
97.13
90.46
84.17
86.72
TimeNet
87.91
81.54
84.61
89.54
75.36
81.84
90.14
56.40
69.39
90.75
95.40
93.02
98.51
96.20
97.34
91.37
80.98
85.24
FEDformer
87.95
82.39
85.08
77.14
80.07
78.57
90.47
58.10
70.76
90.17
96.42
93.19
97.31
97.16
97.23
88.61
82.83
84.97
TStationary
88.33
81.21
84.62
68.55
89.14
77.50
89.37
59.02
71.09
68.03
96.75
79.88
97.82
96.76
97.29
82.42
84.58
82.08
TABLE VI: Anomaly detection performance comparison (Precision/P, Recall/R, F1-score) across five benchmark datasets and average results. The One-for-All framework achieves competitive performance (Avg F1: 84.42) while maintaining consistency across datasets, particularly excelling on industrial datasets SWaT (F1: 92.20) and PSM (F1: 97.10).
Fig. 6: Performance-resource trade-off analysis of One-for-All framework across different rank configurations. (a) Prediction error (MSE) for both long-term and few-shot forecasting tasks, showing performance saturation beyond Rank 16. (b) Logarithmic scaling of model parameters with increasing rank, demonstrating sub-linear growth. The red vertical line marks the optimal operating point (Rank 16) where prediction accuracy stabilizes while maintaining minimal resource requirements. Shaded regions indicate the recommended deployment range (Ranks 16-64) for different application constraints.
Resource Budget
Long-term Forecasting
Few-shot Forecasting (10%)
Rank
Param. (M)
Mem. (MiB)
Param.
Mem.
MSE
MAE
Param.
Mem.
MSE
MAE
Rank 2
✓
✓
0.068
0.277
0.41
0.42
0.068
0.277
0.49
0.45
Rank 4
✓
✓
0.137
0.055
0.37
0.40
0.137
0.055
0.45
0.42
Rank 8
✓
✓
0.275
1.100
0.33
0.37
0.275
1.100
0.42
0.42
Rank 16
✓
✓
0.550
2.200
0.33
0.36
0.550
2.200
0.42
0.41
Rank 32
✓
✓
1.100
4.425
0.33
0.36
1.100
4.425
0.42
0.41
TABLE VII: Comprehensive analysis of One-for-All framework’s performance and resource efficiency across different rank configurations (2–1024). The table compares parameter counts (millions), memory requirements (MiB), and prediction accuracy (MSE/MAE) for both long-term and few-shot (10% training data) forecasting tasks. Gray highlighting indicates the optimal efficiency frontier (Ranks 16–64) where performance plateaus while maintaining minimal resource overhead.
Fig. 7: Few-shot forecasting results for five datasets, showcasing the GPT-2-based models: One-for-All and GPT4TS, with a prediction horizon of 96. The green line represents the ground truth time-series data, while the red line shows the predicted time-series data.
Fig. 8: Long-term forecasting results for the ETTm1 dataset showcasing the One-for-All framework with different ranks: (a) Rank 2, (b) Rank 4, (c) Rank 8, (d) Rank 16, (e) Rank 32, (f) Rank 64, (g) Rank 128, (h) Rank 256, (i) Rank 512, and (j) Rank 1024, under the prediction horizon of 96. The green line represents the ground truth time-series data, while the red line depicts the predicted time-series data.
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Dimension
Length
Prediction Length
Frequency
Domain
ETTh
7
17420
{96, 192, 336, 720}
1 hour
Temperature
ETTm
7
69680
{96, 192, 336, 720}
15 min
Temperature
Weather
22
52696
{96, 192, 336, 720}
10 min
Weather
Appendix
Table 1: Dataset specifications for few-shot and long-term forecasting. Each dataset varies in temporal resolution, series length, and prediction horizons. The few-shot setting uses only 10% of the training data, while long-term experiments evaluate model performance over increasing forecast horizons.
Dataset
Frequency
Length
Prediction Length
Mapping
M4
M3
M3
Yearly
645
6
Yearly
–
M3
Quarterly
756
8
Quarterly
–
M3
Monthly
1428
18
Monthly
–
M4
Yearly
23000
6
–
Yearly
M4
Quarterly
24000
8
–
Quarterly
Appendix
Table 2: Summary of the M3 and M4 datasets used for zero-shot forecasting. The table includes frequency, series length, and prediction horizons. The mapping columns indicate the corresponding source-target frequency pairs used during M3 → M4 transfer, where models trained on M3 are evaluated on M4 without fine-tuning.
Dataset
Hyperparameters
Training Process
GPT layer
Patch Size
Input length
Label length
Heads
Learning rate
Loss
Batch Size
Epochs
Few/Long
ETTh1
6
16
336
168
4
10−3
MSE
256
10
ETTh2
6
16
336
168
4
10−3
MSE
256
10
ETTm1
6
16
512
48
4
10−3
MSE
256
10
ETTm2
6
16
512
48
4
10−3
MSE
256
10
Weather
6
16
512
48
4
10−3
MSE
512
10
Appendix
Table 3: Experiment configuration for the One-for-All Framework: long-term forecasting, few-shot forecasting, and zero-shot forecasting of time-series data.
Method
One-for-All
GPT4TS
TIME-LLM
TEST
TEMPO
TimeNet
FEDformer
TStationary
ETSformer
Autoformer
Informer
Reformer
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
96
0.38
0.40
0.38
0.40
0.36
0.39
0.37
0.40
0.40
0.41
0.38
0.40
0.38
0.42
0.51
0.49
0.49
0.48
0.45
0.46
0.86
0.71
0.84
0.73
192
0.42
0.42
0.42
0.42
0.40
0.42
0.42
0.42
0.43
0.42
0.44
0.43
0.42
0.45
0.53
0.50
0.54
0.50
0.50
0.48
1.00
0.79
0.92
0.77
336
0.44
0.43
0.44
0.43
0.43
0.43
0.42
0.44
0.44
0.43
0.49
0.47
0.46
0.46
0.59
0.53
0.57
0.52
0.52
0.50
1.11
0.81
1.20
0.83
720
0.47
0.47
0.48
0.46
0.44
0.46
0.45
0.47
0.44
0.45
0.52
0.50
0.51
0.51
0.64
0.62
0.56
0.53
0.51
0.51
1.18
0.86
1.26
0.89
Avg
0.43
0.43
0.43
0.43
0.41
0.42
0.42
0.43
0.43
0.43
0.46
0.45
0.44
0.46
0.57
0.54
0.54
0.51
0.50
0.49
1.04
0.79
1.06
0.81
Appendix
Table 4: Comprehensive long-term forecasting results across multiple deep learning models and transformer variants on six benchmark datasets (ETTh1, ETTh2, ETTm1, ETTm2, and Weather) for four prediction lengths: 96, 192, 336, and 720. Performance is reported using Mean Squared Error (MSE) and Mean Absolute Error (MAE). The “Avg” row represents the average performance across all horizons for each dataset.
Method
One-for-All
GPT4TS
TIME-LLM
TEST
TimeNet
FEDformer
TStationary
ETSformer
Autoformer
Informer
Reformer
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
96
0.51
0.48
0.46
0.46
0.45
0.46
0.46
0.46
0.86
0.63
0.51
0.50
0.92
0.64
1.11
0.81
0.61
0.55
1.18
0.79
1.18
0.79
192
0.62
0.52
0.57
0.52
0.49
0.48
0.57
0.52
0.80
0.59
0.62
0.55
0.91
0.63
1.15
0.82
0.72
0.60
1.20
0.81
1.29
0.85
336
0.75
0.59
0.61
0.53
0.59
0.54
0.61
0.53
0.94
0.65
0.69
0.57
0.94
0.64
1.18
0.83
0.75
0.62
1.20
0.81
1.29
0.85
720
0.72
0.59
0.72
0.59
0.70
0.61
0.72
0.60
0.88
0.64
0.73
0.61
0.89
0.64
1.27
0.87
0.72
0.62
1.22
0.82
1.22
0.84
Avg
0.65
0.55
0.59
0.53
0.56
0.52
0.59
0.53
0.87
0.63
0.64
0.56
0.92
0.64
1.18
0.83
0.70
0.60
1.20
0.81
1.25
0.83
Appendix
Table 5: Few-shot forecasting results using 10% of the training data across four standard prediction lengths N∈{96,192,336,720} for six benchmark datasets (ETTh1, ETTh2, ETTm1, ETTm2, and Weather). This table highlights the robustness and generalisability of various models under constrained training data regimes.
Method
One-for-All
GPT4TS
TIME-LLM
TimeNet
FEDformer
TStationary
ETSformer
Autoformer
Informer
Reformer
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
96
0.63
0.54
0.54
0.51
0.48
0.47
0.89
0.62
0.59
0.53
0.95
0.65
1.17
0.83
0.68
0.57
1.22
0.81
1.20
0.79
192
0.75
0.58
0.75
0.58
0.63
0.54
0.94
0.66
0.65
0.56
0.94
0.64
1.22
0.85
0.72
0.60
1.25
0.83
1.27
0.85
336
0.75
0.59
0.75
0.59
0.77
0.63
0.94
0.65
0.73
0.59
0.93
0.64
1.18
0.83
0.76
0.62
1.20
0.81
1.25
0.86
720
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
Avg
0.71
0.57
0.68
0.56
0.63
0.55
0.92
0.64
0.66
0.56
0.94
0.64
1.19
0.84
0.72
0.60
1.22
0.82
1.24
0.83
Appendix
Table 6: Few-shot forecasting results using only 5% of the training data across the same four prediction lengths N∈{96,192,336,720} . This setting simulates extreme data-scarce conditions. The results offer insights into the sample efficiency and stability of each model when operating under minimal supervision. “–” indicates that training data is not available.
Variant
Long-term Forecasting with Different Rank
Rank 2
Rank 4
Rank 8
Rank 16
Rank 32
Rank 64
Rank 128
Rank 256
Rank 512
Rank 1024
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
96
0.55
0.51
0.48
0.47
0.40
0.42
0.38
0.40
0.38
0.40
0.38
0.40
0.38
0.40
0.38
0.40
0.38
0.40
0.39
0.40
192
0.59
0.52
0.51
0.49
0.43
0.44
0.42
0.42
0.42
0.42
0.42
0.42
0.42
0.43
0.42
0.42
0.42
0.43
0.42
0.43
336
0.60
0.53
0.53
0.50
0.45
0.45
0.44
0.43
0.45
0.45
0.45
0.45
0.45
0.44
0.45
0.45
0.45
0.45
0.45
0.45
720
0.58
0.54
0.53
0.52
0.47
0.48
0.47
0.47
0.47
0.48
0.46
0.47
0.46
0.47
0.47
0.48
0.46
0.47
0.48
0.48
Appendix
Table 7: Performance evaluation of the One-for-All model with varying low-rank configurations for long-term forecasting. The table presents the MSE and MAE across four forecasting horizons ( N∈{96,192,336,720} ) on five benchmark datasets (ETTh1, ETTh2, ETTm1, ETTm2, and Weather). Each column group corresponds to a specific rank used in the model, enabling a comprehensive comparison of prediction accuracy across different rank settings.
Variant
Few-shot Forecasting with Different Rank
Rank 2
Rank 4
Rank 8
Rank 16
Rank 32
Rank 64
Rank 128
Rank 256
Rank 512
Rank 1024
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
96
0.72
0.56
0.73
0.56
0.52
0.49
0.51
0.48
0.49
0.47
0.49
0.47
0.52
0.49
0.53
0.49
0.53
0.50
0.53
0.49
192
0.75
0.58
0.75
0.58
0.64
0.54
0.62
0.52
0.63
0.54
0.64
0.54
0.64
0.55
0.66
0.55
0.68
0.56
0.66
0.55
336
0.74
0.59
0.75
0.59
0.75
0.59
0.75
0.59
0.74
0.59
0.75
0.59
0.75
0.59
0.76
0.59
0.75
0.59
0.75
0.59
720
0.74
0.61
0.74
0.61
0.74
0.61
0.72
0.59
0.74
0.61
0.74
0.61
0.75
0.61
0.75
0.61
0.74
0.61
0.74
0.61
Appendix
Table 8: Comprehensive evaluation of the One-for-All model under Few-Shot Forecasting using 10% of training data. The table presents the model’s performance across four prediction horizons ( N∈{96,192,336,720} ) on five datasets (ETTh1, ETTh2, ETTm1, ETTm2, Weather), with varying low-rank configurations. Each cell reports the MSE and MAE metrics. Bold values indicate the best performance for each horizon and dataset across all ranks. The results highlight the impact of rank selection on model efficiency and accuracy under constrained data availability.
Variant
Few-shot Forecasting with Different Rank
Rank 2
Rank 4
Rank 8
Rank 16
Rank 32
Rank 64
Rank 128
Rank 256
Rank 512
Rank 1024
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
96
0.74
0.57
0.63
0.53
0.64
0.54
0.63
0.54
0.63
0.55
0.63
0.55
0.61
0.53
0.62
0.54
0.69
0.55
0.63
0.53
192
0.75
0.59
0.75
0.58
0.75
0.58
0.75
0.58
0.75
0.58
0.75
0.59
0.75
0.58
0.75
0.58
0.75
0.58
0.75
0.58
336
0.75
0.60
0.75
0.60
0.75
0.59
0.75
0.59
0.75
0.59
0.75
0.59
0.76
0.60
0.75
0.60
0.75
0.60
0.75
0.59
720
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
Appendix
Table 9: Comprehensive evaluation of the One-for-All model under Few-Shot Forecasting using 5% of training data. The table presents the model’s performance across four prediction horizons ( N∈{96,192,336,720} ) on five datasets (ETTh1, ETTh2, ETTm1, ETTm2, Weather), with varying low-rank configurations. Each cell reports the MSE and MAE metrics. Bold values indicate the best performance for each horizon and dataset across all ranks. The results highlight the impact of rank selection on model efficiency and accuracy under constrained data availability. “–” indicates that training data is not available.
Variant
Rank 2
Rank 4
Rank 8
Rank 16
Rank 32
Rank 64
Rank 128
Rank 256
Rank 512
Rank 1024
Metric
sMAPE
sMAPE
sMAPE
sMAPE
sMAPE
sMAPE
sMAPE
sMAPE
sMAPE
sMAPE
M3 → M4
Yearly
13.80
13.67
13.60
13.53
13.54
13.51
13.41
13.69
13.57
13.73
Quarterly
11.31
11.03
10.86
11.00
10.92
11.09
10.89
11.12
10.91
11.45
Monthly
15.52
15.28
15.16
15.28
15.39
14.90
15.07
14.94
14.85
14.95
M4 → M3
Yearly
22.77
22.25
20.75
19.23
19.11
18.01
18.16
17.87
17.61
17.44
Quarterly
11.92
11.83
11.57
11.77
11.66
11.01
11.66
11.28
11.47
11.20
Appendix
Table 10: Zero-shot forecasting performance of the One-for-All model across varying LoRA ranks on the M3 and M4 datasets. The table reports sMAPE scores for different frequency categories (Yearly, Quarterly, Monthly) in both M3 → M4 and M4 → M3 transfer settings. The highlighted values correspond to Rank 16, used as the baseline configuration. Lower sMAPE indicates better performance.
Method
SMAPE
MASE
OWS
Yearly
Quarterly
Monthly
Others
Avg
Yearly
Quarterly
Monthly
Others
Avg
Yearly
Quarterly
Monthly
Others
Avg
Rank 2
15.28
10.48
12.96
5.29
12.51
3.71
1.23
0.96
3.59
1.79
0.93
0.92
0.90
1.12
0.93
Rank 4
14.29
10.51
13.11
5.37
12.37
3.34
1.21
0.96
3.66
1.70
0.85
0.92
0.90
1.14
0.90
Rank 8
15.21
10.50
13.02
5.34
12.53
3.76
1.24
0.96
3.60
1.80
0.93
0.92
0.90
1.13
0.93
Rank 16
14.79
10.36
12.96
5.34
12.38
3.59
1.22
0.96
3.57
1.76
0.90
0.91
0.90
1.21
0.91
Rank 32
15.05
10.49
13.36
5.45
12.66
3.68
1.23
0.99
3.63
1.80
0.92
0.92
0.92
1.14
0.93
Appendix
Table 11: Short-term forecasting performance on the M4 dataset across various ranks, evaluated using three performance metrics: SMAPE, MASE, and OWS. The results are presented for different data frequencies (Yearly, Quarterly, Monthly, and Others) with the average performance shown in the “Avg” column. The table summarizes the impact of different rank configurations on model performance for each metric. Lower values indicate better forecasting accuracy.
Dateset
Rank 2
Rank 4
Rank 8
Rank 16
Rank 32
Rank 64
Rank 128
Rank 256
Rank 512
Rank 1024
Ethanol
28%
30%
32%
32%
30%
29%
30%
31%
30%
32%
Face Detection
56%
62%
65%
66%
66%
66%
66%
66%
68%
67%
Heartbeat
72%
75%
76%
75%
75%
76%
76%
76%
76%
75%
Japanese Vowels
46%
81%
95%
97%
97%
97%
98%
97%
98%
97%
SCP1
83%
89%
91%
91%
91%
92%
91%
92%
91%
93%
SCP2
50%
55%
60%
57%
53%
57%
55%
58%
57%
55%
Appendix
Table 12: Detailed accuracy analysis of the One-for-All model with varying ranks (Rank 2 to Rank 1024) for classification tasks across multiple datasets. The accuracy for each dataset is reported as a percentage, showing the model’s performance at different rank configurations. The table includes results for datasets such as Ethanol, Face Detection, Heartbeat, Japanese Vowels, SCP1, and SCP2.
Dateset
Rank 2
Rank 4
Rank 8
Rank 16
Rank 32
Rank 64
Rank 128
Rank 256
Rank 512
Rank 1024
MSL
80%
83%
82%
82%
82%
82%
82%
82%
82%
82%
PSM
96%
96%
97%
97%
97%
97%
97%
97%
94%
94%
SMAP
66%
67%
67%
67%
67%
67%
67%
67%
67%
67%
SMD
82%
82%
83%
84%
84%
84%
84%
84%
84%
84%
SWAT
88%
92%
92%
92%
92%
92%
92%
92%
92%
92%
Avg
82%
84%
84%
84%
84%
84%
84%
84%
84%
84%
Appendix
Table 13: Detailed F1-score (as %) analysis of the One-for-All model with varying ranks (Rank 2 to Rank 1024) for anomaly detection tasks across multiple datasets. The F1-score, representing the harmonic mean of precision and recall, is reported for datasets such as MSL, PSM, SMAP, SMD, and SWAT. The “Avg” row provides the average F1-score across all datasets for each rank configuration. The table highlights the model’s performance in anomaly detection at different ranks.
Variant
Trainable parameters (M) for Few-shot Forecasting and Long-term Forecasting
96
192
336
720
Avg
Tran.*
%All*
Mem.*
Tran.
%All
Mem.
Tran.
%All
Mem.
Tran.
%All
Mem.
Tran.
%All
Mem.
Rank 2
0.068
0.080%
0.275
0.068
0.077%
0.276
0.067
0.074%
0.277
0.069
0.066%
0.280
0.068
0.074%
0.277
Rank 4
0.136
0.160%
0.548
0.136
0.155%
0.550
0.137
0.148%
0.552
0.139
0.132%
0.558
0.137
0.149%
0.552
Rank 8
0.273
0.320%
1.100
0.273
0.309%
1.100
0.275
0.295%
1.100
0.278
0.263%
1.100
0.275
0.297%
1.100
Rank 16
0.546
0.638%
2.200
0.547
0.617%
2.200
0.550
0.589%
2.200
0.556
0.526%
2.200
0.550
0.593%
2.200
Appendix
Table 14: Efficiency analysis of the One-for-All model with varying ranks (Rank 2 to Rank 1024) for few-shot forecasting and long-term forecasting tasks using the ETTh1 and ETTh2 datasets. The table presents the trainable parameters (in millions), the percentage of total model parameters, and the memory required to save the trained model (in MiB) for each rank across both forecasting tasks. The results highlight the impact of increasing model rank on parameter count, memory usage, and the overall efficiency of the model, showcasing the trade-offs between model size and performance.
Variant
Trainable parameters (M) for Few-shot Forecasting and Long-term Forecasting
96
192
336
720
Avg
Tran.*
%All*
Mem.*
Tran.
%All
Mem.
Tran.
%All
Mem.
Tran.
%All
Mem.
Tran.
%All
Mem.
Rank 2
0.054
0.064%
0.220
0.054
0.062%
0.220
0.054
0.060%
0.222
0.055
0.055%
0.225
0.054
0.060%
0.222
Rank 4
0.108
0.128%
0.437
0.109
0.125%
0.439
0.109
0.121%
0.441
0.111
0.111%
0.447
0.109
0.121%
0.441
Rank 8
0.217
0.257%
0.873
0.218
0.251%
0.876
0.219
0.242%
0.881
0.222
0.221%
0.893
0.219
0.243%
0.881
Rank 16
0.435
0.513%
1.700
0.437
0.501%
1.800
0.439
0.483%
1.800
0.445
0.442%
1.800
0.439
0.485%
1.775
Appendix
Table 15: Efficiency analysis of the One-for-All model with varying ranks (Rank 2 to Rank 1024) for few-shot forecasting and long-term forecasting tasks using the ETTm1 and ETTm2 datasets. The table presents the trainable parameters (in millions), the percentage of total model parameters, and the memory required to save the trained model (in MiB) for each rank across both forecasting tasks. The results highlight the impact of increasing model rank on parameter count, memory usage, and the overall efficiency of the model, showcasing the trade-offs between model size and performance.
Variant
Trainable parameters (M) for Few-shot Forecasting and Long-term Forecasting
96
192
336
720
Avg
Tran.*
%All*
Mem.*
Tran.
%All
Mem.
Tran.
%All
Mem.
Tran.
%All
Mem.
Tran.
%All
Mem.
Rank 2
0.102
0.117%
0.410
0.102
0.111%
0.411
0.102
0.104%
0.412
0.103
0.087%
0.415
0.102
0.105%
0.412
Rank 4
0.204
0.235%
0.818
0.204
0.223%
0.820
0.205
0.207%
0.822
0.206
0.175%
0.828
0.205
0.210%
0.822
Rank 8
0.408
0.469%
1.600
0.409
0.445%
1.600
0.410
0.415%
1.600
0.413
0.351%
1.700
0.410
0.420%
1.625
Rank 16
0.816
0.933%
3.300
0.818
0.887%
3.300
0.820
0.826%
3.300
0.826
0.699%
3.300
0.820
0.836%
3.300
Appendix
Table 16: Efficiency analysis of the One-for-All model with varying ranks (Rank 2 to Rank 1024) for few-shot forecasting and long-term forecasting tasks using the Weather datasets. The table presents the trainable parameters (in millions), the percentage of total model parameters, and the memory required to save the trained model (in MiB) for each rank across both forecasting tasks. The results highlight the impact of increasing model rank on parameter count, memory usage, and the overall efficiency of the model, showcasing the trade-offs between model size and performance.
Models
Trainable parameters (M) and checkpoint size (MiB) for few-shot forecasting and long-term forecasting
96
192
336
720
Avg
Tran.*
%All*
Mem.*
Tran.
%All
Mem.
Tran.
%All
Mem.
Tran.
%All
Mem.
Tran.
%All
Mem.
GPT4TS
3.916
4.60%
340.1
7.012
7.95%
352.5
11.65
12.5%
371.1
24.04
22.8%
420.6
11.65
11.9%
371.0
TIME-LLM (LLaMA-7B)
6.390
0.20%
3678
6.420
0.20%
3812
6.480
0.2%
3960
6.550
0.2%
4176
6.460
0.2%
3907
TEST (GPT-2 Medium)
–
–
–
–
–
–
–
—
–
–
–
–
∼ 1-2
∼ 0.3–0.6%
∼ 701
TEMPO (GPT-2 Medium)
–
–
–
–
–
–
–
–
–
–
–
–
∼ 3-5
∼ 0.9%-1.4%
∼ 710
Appendix
Table 17: Efficiency analysis of state-of-the-art models in few-shot and long-term forecasting tasks on the ETTh1 and ETTh2 datasets. The table provides the trainable parameters (in millions), the percentage of total parameters, and the memory required for saving the trained model (in MiB) across various forecasting tasks. The models compared include GPT4TS, TIME-LLM (LLaMA-7B), TEST (GPT-2 Medium), TEMPO (GPT-2 Medium), TimesNet, EFDformer, Stationary, ETSformer, Autoformer, Informer, and Reformer, with performance metrics shown for different ranks of forecasting. This analysis helps in understanding the trade-offs between model size, efficiency, and forecasting accuracy.
Can language-pretrained transformers become effective time-series forecasters, and why? In this paper, we show that cross-modal transfer arises because language pretraining preconditions time series training with a reusable manifold. A linear probe on frozen LLM states decodes realistic time-series trajectories without paired supervision, and retrieval in this projected space yields competitive forecasts, showing that structure and dynamics exist before finetuning. Pretrained initialization also improves optimization, producing coherent gradients and a highly anisotropic loss landscape unlike random initialization. Finetuning then acts as low-dimensional alignment, reusing existing directions rather than learning temporal primitives from scratch, as evidenced by low-rank updates, subspace alignment, and shared features for periodicity, trend, and repetition. Together, these results support a geometric account of LLM-to-time-series transfer: language pretraining builds the manifold, and finetuning projects numerical dynamics onto task-relevant directions.
We study the adaptation of pretrained language models to univariate time-series forecasting through a parameter-efficient transfer learning framework, with the goal of understanding which design choices drive effective cross-modal transfer. While language models operate on discrete textual tokens, time series consist of continuous numerical observations with temporal dependencies. To bridge this modality gap, we project fixed-length time-series patches directly into the embedding space of a pretrained GPT-2 backbone, bypassing textual tokenization and treating the Transformer as a generic sequence encoder. Through controlled ablation studies on seven benchmark datasets spanning energy, weather, traffic, and finance, we analyze the effects of (i)~representation strategy (continuous embeddings versus textual serialisation), (ii)~adaptation regime (frozen backbone versus partial or full fine-tuning), (iii)~architectural components such as adapters, pooling strategies, and prediction heads, and (iv)~input context length. Continuous patch-based embeddings consistently outperform textual prompting and randomly initialised backbones. The adapted pipeline attains MASE within the range of specialised forecasting architectures while updating less than 1% of total model parameters. Results further indicate that freezing the pretrained backbone and training lightweight projection and adapter modules provides a favourable accuracy--efficiency trade-off with stable behaviour across varying context lengths.
Tamanna Kumavat, Georg Brunner, Kyriakos Flouris
University of Zurich, Switzerland · Computer Vision Laboratory, ETH Zurich, Switzerland · MRC Biostatistics Unit, University of Cambridge, United Kingdom
Recent advances in Large Language Models (LLMs) have spurred cross-modal solutions for time-series forecasting. However, existing methods rely heavily on textual prompts for modality alignment-introducing nontrivial computational overhead and failing to leverage the rich spectral dynamics inherent in time-series data. To enable prompt-free, frequency-aware adaptation of frozen LLMs, we propose FM-LLM (Frequency-Enhanced Mixture-of-Experts for adapting LLMs to Time Series Forecasting), an autoregressive framework grounded in constrained asymmetric coupling. A Fourier Analysis Network (FAN)-based spectral token aligner injects structured harmonic representations directly into the frozen LLM with numerical compatibility. An asymmetric Mixture-of-Experts (MoE) decoder enforces role separation: shared experts with lightweight FAN layers reconstruct the global periodic backbone, while routed experts-restricted to standard FFNs-specialize in modeling non-periodic residual dynamics. A time-frequency hybrid loss function jointly optimizes temporal accuracy and spectral consistency, mitigating error accumulation during long-horizon autoregressive rollouts. Evaluated across eleven public benchmarks, FM-LLM achieves state-of-the-art performance on 59 out of 78 evaluation metrics. Compared to the strongest autoregressive LLM-based baseline, it delivers average improvements of 5.3% in MSE and 5.6% in MAE, with maximum gains reaching 8.0% for MSE and 8.4% for MAE. FM-LLM also demonstrates robust transferability, maintaining superior performance in 10% few-shot and zero-shot forecasting scenarios.
Rentao Gu, Yihang Ding, Junjie Li +5
Beijing University of Posts and Telecommunications, Beijing, 100876, China · China Telecom Research Institute, Beijing, 100032, China