Transformer-based models have gained increasing attention in time series research, driving interest in Large Language Models (LLMs) and foundation models for time series analysis. As the field moves toward multi-modality, Large Vision Models (LVMs) are emerging as a promising direction. In the past, the effectiveness of Transformer and LLMs in time series has been debated. When it comes to LVMs, a similar question arises: are LVMs truely useful for time series analysis? To address it, we design and conduct the first principled study involving 4 LVMs, 8 imaging methods, 18 datasets and 26 baselines across both high-level (classification) and low-level (forecasting) tasks, with extensive ablation analysis. Our findings indicate LVMs are indeed useful for time series classification but face challenges in forecasting. Although effective, the contemporary best LVM forecasters are limited to specific types of LVMs and imaging methods, exhibit a bias toward forecasting periods, and have limited ability to utilize long look-back windows. We hope our findings could serve as a cornerstone for future research on LVM- and multimodal-based solutions to different time series tasks.
Figures & tables
Figure 1: An overview of (a) different imaging methods, (b) LVM-based time series classification, (c) LVM with linear head for forecasting, (d) LVM encoder-decoder for forecasting. In (a), MVH encodes MTS, others encode UTS. (b)(c) apply to all LVMs (ViT, Swin, MAE, SimMIM) in this study. (d) applies to MAE and SimMIM with UVH/MVH images. Table 1 summarizes their applicability.
Task
Imaging
ViT
Swin
MAE
SimMIM
Classification
All
(b)
(b)
(b)
(b)
Forecasting
UVH,MVH
(c)
(c)
(d)
(d)
Forecasting
Other
(c)
(c)
(c)
(c)
Table 1: LVM framework summary. (b)(c)(d) indicates the frameworks in Fig. 1 .
Figure 2: Model comparison in TSC. The results are averaged over 10 UEA datasets. See Table 9 in Appendix B.1 for full results.
Method
MAE
ViT
Time-LLM
GPT4TS
CALF
Dlinear
PatchTST
TimesNet
FEDformer
Autoformer
Metrics
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
0.409
0.419
0.445
0.449
0.418
0.432
0.418
0.421
0.432
0.431
0.423
0.437
0.413
0.431
0.458
0.450
0.440
0.460
0.496
0.487
ETTh2
0.357
0.390
0.389
0.411
0.361
0.396
0.354
0.389
0.351
0.384
0.431
0.447
0.330
0.379
0.414
0.427
0.437
0.449
0.450
0.459
ETTm1
0.345
0.374
0.409
0.422
0.356
0.377
0.363
0.378
0.396
0.391
0.357
0.379
0.351
0.381
0.400
0.406
0.448
0.452
0.588
0.517
ETTm2
0.268
0.327
0.300
0.337
0.261
0.316
0.254
0.311
0.283
0.323
0.267
0.334
0.255
0.315
0.291
0.333
0.305
0.349
0.327
0.371
Weather
0.225
0.258
0.234
0.273
0.244
0.270
0.227
0.255
0.251
0.274
0.249
0.300
0.226
0.264
0.259
0.287
0.309
0.360
0.338
0.382
Table 2: Model comparison in TSF. The results are averaged over different prediction lengths. See Table 11 in Appendix B.2 for full results. Red and Blue numbers are the the best and second best results. # Wins is the number of times the method performed best.
Figure 3: Comparison of 4 LVMs on TSC (accuracy) and TSF (MSE). ↑ ( ↓ ) indicates a higher (lower) value is better. Two taxonomies of the LVMs: (1) supervised (ViT, Swin) vs. self-supervised (MAE, SimMIM), (2) using global attention (ViT, MAE) vs. window-based attention (Swin, SimMIM).
Figure 4: Average rank of different imaging methods in (a) TSC task, and (b) TSF task. Lower rank is better.
Task
TSC Task (accuracy (%) ↑ )
TSF Task (MSE ↓ )
Dataset
UWave.
Spoken.
Handwrit.
FaceDetect.
ETTh1
ETTm1
Illiness
Weather
RQ3
(a) All parameters
88.4
98.5
36.4
67.4
0.558
0.399
1.781
0.273
(b) All but CLS & Mask
87.5
98.2
35.2
66.3
0.530
0.408
1.783
0.275
(c) MLP & norm
88.7
98.4
35.5
67.1
0.532
0.396
1.737
0.264
(d) Norm
81.6
98.0
28.5
65.2
0.409
0.345
1.837
0.225
(e) Zero-shot
84.0
98.5
27.8
66.7
0.452
0.420
2.037
0.308
Table 3: Ablation analysis of LVMs. For classification, higher accuracy indicates better performance. For forecasting, lower MSE is preferred. Full results are in Appendices B.5 and B.6 .
Figure 5: Key components in LVMs’ Transformer block.
Task
Classification
Forecasting
Dataset
UWave.
Spoken.
Handwrit.
FaceDetect.
ETTh1
ETTm1
Illiness
Weather
Sf-All
w/o-LVM
78.2%
49.7%
81.7%
19.3%
76.2%
98.4%
116.4%
24.1%
LVM2Attn
86.4%
50.6%
89.9%
22.4%
79.7%
117.1%
109.1%
24.4%
LVM
80.7%
84.7%
91.5%
29.2%
83.8%
118.4%
162.8%
44.5%
Sf-Half
w/o-LVM
6.6%
12.4%
74.6%
10.8%
14.4%
28.3%
41.6%
2.4%
LVM2Attn
8.7%
11.6%
83.6%
11.3%
19.5%
44.8%
69.3%
2.4%
Table 4: Performance drop of the compared models under different temporal perturbations. Red color marks the largest drop for each perturbation strategy. Full results are in Appendix B.7 .
Method
LVM
1st Baseline (task specific)
2nd Baseline (task specific)
Task
Dataset
# Param (M)
Train (min)
Inference(ms)
# Param (M)
Train (min)
Inference(ms)
# Param (M)
Time (min)
Inference(ms)
TSC
UWave.
89.43
2.83
11.52
82.23
1.19
57.61
2.42
0.39
1.69
Handwrit.
97.59
5.18
23.72
83.62
1.33
50.51
2.47
0.51
0.78
TSF
ETTh1
111.91
9.99
4.32
3.75
0.52
0.18
85.02
10.46
0.50
Weather
111.91
207.83
1.50
6.90
16.97
0.10
86.64
94.10
0.35
Table 5: Computational costs of LVMs and two best baselines in TSC (GPT4TS, TimesNet) and TSF (PatchTST, GPT4TS). The forecasting costs are measured with prediction length 96.
Figure 6: Inference time vs. performance of compared methods on TSC (accuracy) using UWaveGesture, SpokenArabicDigits, and TSF (MSE) using ETTh1, Weather. Full results are in Appendix B.8 .
Figure 12
Figure 9: An illustration of UVH.
Figure 10: TSF performance (MSE) of MAE with varying look-back window (or context) lengths.
Appendix figures & tables26 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Variates
Series Length
Dataset Size
Classes
EthanolConcentration
3
1751
(261, 263, 263)
4
FaceDetection
144
62
(5890, 3524, 3524)
2
Handwriting
3
152
(150, 850, 850)
26
Heartbeat
61
405
(204, 205,205)
2
Japanese Vowels
12
29
(270, 370, 370)
9
PEMS-SF
963
144
(267, 173, 173)
7
Appendix
Table 6: Statistics of the datasets for TSC. “Dataset Size” is organized in (Train, Validation, Test).
Dataset
# Variates
Series Length
Dataset Size
Frequency
ETTh1
7
17420
(8545, 2881, 2881)
Hourly
ETTh2
7
17420
(8545, 2881, 2881)
Hourly
ETTm1
7
69680
(34465, 11521, 11521)
15 mins
ETTm2
7
69680
(34465, 11521, 11521)
15 mins
Weather
321
52696
(36792, 5271, 10540)
10 mins
Illness
7
966
(617, 74, 170)
Weekly
Appendix
Table 7: Statistics of the datasets for TSF. “Dataset Size” is organized in (Train, Validation, Test).
ETTh1, ETTh2
ETTm1, ETTm2
Weather
Illness
Traffic
Electricity
Top 3 Period
{24, 576, 384}
{96, 576, 384}
{144, 72, 576}
{52, 26, 17}
{24, 12, 168}
{24, 164, 82}
Segment Length L
24
96
144
52
24
24
Appendix
Table 8: Top-3 potential periods by FFT and segment lengths for UVH on 8 TSF datasets.
Dataset
MAE
ViT
XGBoost
Rocket
LSTNet
LSSL
TCN
Trans.
Re.
In.
Pyra.
Auto.
Station.
FED.
ETS.
Flow.
Dlinear
LightTS
TimesNet
GPT4TS
EthanolConcentration
41.4
49.4
43.7
45.2
39.9
31.1
28.9
32.7
31.9
31.6
30.8
31.6
32.7
31.2
28.1
33.8
32.6
29.7
35.7
34.2
FaceDetection
65.4
67.4
63.3
64.7
65.7
66.7
52.8
67.3
68.6
67.0
65.7
68.4
68.0
66.0
66.3
67.6
68.0
67.5
68.6
69.2
Handwriting
39.5
36.4
15.8
58.8
25.8
24.6
53.3
32.0
27.4
32.8
29.4
36.7
31.6
28.0
32.5
33.8
27.0
26.1
32.1
32.7
Heartbeat
86.8
74.6
73.2
75.6
77.1
72.7
75.6
76.1
77.1
80.5
75.6
74.6
73.7
73.7
71.2
77.6
75.1
75.1
78.0
77.2
Japanese Vowels
95.4
98.3
86.5
96.2
98.1
98.4
98.9
98.7
97.8
98.9
98.4
96.2
99.2
98.4
95.9
98.9
96.2
96.2
98.4
98.6
PEMS-SF
84.4
84.2
98.3
75.1
86.7
86.1
68.8
82.1
82.7
81.5
83.2
82.7
87.3
80.9
86.0
83.8
75.1
88.4
89.6
87.9
Appendix
Table 9: Accuracy (%) of the compared methods in TSC on 10 benchmark datasets. Red numbers are the the best results. # Wins is the number of times the method performs the best.
Dataset
MAE
ViT
EthanolConcentration
41.4 ± 0.5
49.4 ± 0.9
FaceDetection
65.4 ± 1.2
67.4 ± 1.5
Handwriting
39.5 ± 1.5
36.4 ± 1.3
Heartbeat
86.8 ± 2.1
74.6 ± 0.6
Japanese Vowels
95.4 ± 0.3
98.3 ± 0.3
PEMS-SF
84.4 ± 0.4
84.2 ± 0.5
Appendix
Table 10: Standard deviation of LVMs on TSC datasets.
Method
MAE
ViT
Time-LLM
GPT4TS
CALF
Dlinear
PatchTST
TimesNet
FEDformer
Autoformer
Metrics
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
96
0.356
0.383
0.398
0.401
0.376
0.402
0.370
0.389
0.370
0.393
0.375
0.399
0.370
0.399
0.384
0.402
0.376
0.419
0.449
0.459
192
0.395
0.406
0.439
0.445
0.407
0.421
0.412
0.413
0.429
0.426
0.405
0.416
0.413
0.421
0.436
0.429
0.420
0.448
0.500
0.482
336
0.417
0.424
0.462
0.458
0.430
0.438
0.448
0.431
0.451
0.440
0.439
0.443
0.422
0.436
0.491
0.469
0.459
0.465
0.521
0.496
720
0.467
0.463
0.479
0.491
0.457
0.468
0.441
0.449
0.476
0.466
0.472
0.490
0.447
0.466
0.521
0.500
0.506
0.507
0.514
0.512
ETTh2
96
0.297
0.341
0.302
0.355
0.286
0.346
0.280
0.335
0.284
0.336
0.289
0.353
0.274
0.336
0.340
0.374
0.358
0.397
0.346
0.388
Appendix
Table 11: MSE and MAE evaluation of the compared methods in TSF on benchmark datasets. Red ( Blue ) numbers are the best (second best) results on each prediction length per dataset. # Wins is the number of times the method performs the best.
Method
MAE
ViT
Metrics
MSE
MAE
MSE
MAE
ETTh1
96
0.356 ± 0.001
0.383 ± 0.005
0.398 ± 0.011
0.401 ± 0.012
192
0.395 ± 0.001
0.406 ± 0.001
0.439 ± 0.005
0.445 ± 0.003
336
0.417 ± 0.001
0.424 ± 0.001
0.462 ± 0.004
0.458 ± 0.004
720
0.467 ± 0.012
0.463 ± 0.010
0.479 ± 0.011
0.491 ± 0.008
ETTh2
96
0.297 ± 0.000
0.341 ± 0.004
0.302 ± 0.001
0.355 ± 0.000
Appendix
Table 12: Standard deviation of LVMs on TSF datasets.
Dataset
MAE
SimMiM
ViT
Swin
UWaveGestureLibrary
85.0
83.1
88.4
78.9
SpokenArabicDigits
98.5
88.2
98.5
87.3
Handwriting
39.5
29.8
36.4
33.1
FaceDetection
65.4
57.8
67.4
50.3
Average
72.1
64.7
72.6
62.4
Appendix
Table 13: Accuracy (%) comparison between self-supervised LVMs and supervised LVMs on TSC benchmark datasets. Red numbers indicate the best performance for each dataset.
Dataset
Self-Supervised
Supervised
Model
MAE
SimMIM
ViT
Swin
Metrics
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
96
0.356
0.383
0.362
0.383
0.398
0.401
0.407
0.429
192
0.395
0.406
0.407
0.412
0.439
0.445
0.442
0.458
336
0.417
0.424
0.422
0.417
0.462
0.458
0.467
0.481
720
0.467
0.463
0.462
0.455
0.479
0.491
0.470
0.497
Appendix
Table 14: MSE and MAE Comparison between self-supervised LVMs and supervised LVMs on TSF datasets. Red numbers indicate the best performance for each prediction length per dataset.
Dataset
GAF
MVH
RP
STFT
Wave.
Filter.
UVH
Lineplot
EthanolConcentration
49.4
30.7
43.7
31.9
27.3
28.1
28.5
25.2
FaceDetection
67.4
68.3
65.5
61.1
63.9
64.7
67.7
50.3
Handwriting
36.4
30.8
45.1
28.2
34.0
22.3
25.8
15.9
Heartbeat
74.6
77.5
71.7
74.7
72.6
73.1
78.0
53.7
Japanese Vowels
98.3
97.8
87.8
94.8
94.9
97.0
96.4
65.7
PEMS-SF
84.2
87.2
80.1
68.5
84.7
71.2
88.1
73.4
Appendix
Table 15: Accuracy (%) comparison of 8 imaging methods on TSC benchmark datasets. Red numbers indicate the best performance for each dataset.
Imaging Method
GAF
GAF †
MVH
RP
STFT
Wave.
Filter.
UVH
Lineplot
Dataset
Metrics
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
96
0.986
0.783
1.224
0.850
0.484
0.471
0.969
0.771
0.534
0.533
0.621
0.582
0.820
0.684
0.356
0.383
0.902
0.751
192
1.004
0.797
1.227
0.854
0.575
0.517
0.971
0.775
0.621
0.587
0.650
0.600
0.864
0.707
0.395
0.406
1.204
0.894
336
1.038
0.820
1.214
0.857
0.623
0.546
0.989
0.788
0.602
0.573
0.681
0.616
0.827
0.693
0.417
0.424
1.223
0.901
720
1.008
0.812
1.190
0.863
0.737
0.612
1.062
0.825
0.669
0.621
0.699
0.633
0.858
0.720
0.467
0.463
1.150
0.852
Average
1.009
0.803
1.214
0.856
0.605
0.537
0.998
0.790
0.607
0.579
0.663
0.608
0.842
0.701
0.409
0.419
1.120
0.850
Appendix
Table 16: MSE and MAE comparison of 8 imaging methods on TSF benchmark datasets. Red numbers indicate the best performance for each dataset. GAF represents applying GAF with the framework in Fig. 1 (c). GAF † represents applying GAF with the framework in Fig. 1 (d).
Dataset
(a)
(b)
(c)
(d)
(e)
(f)
UWaveGestureLibrary
88.4
87.5
88.7
81.6
84.0
73.4
SpokenArabicDigits
98.5
98.2
98.4
98.0
98.5
97.0
Handwriting
36.4
35.2
35.5
28.5
27.8
24.3
FaceDetection
67.4
66.3
67.1
65.2
66.7
65.0
Appendix
Table 17: Accuracy (%) comparison of different fine-tuning strategies for on TSC benchmark datasets. Red numbers indicate the best performance for each dataset.
Fine-tuning Strategy
(a)
(b)
(c)
(d)
(e)
(f)
Dataset
Metrics
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
96
0.512
0.448
0.481
0.435
0.477
0.418
0.356
0.383
0.426
0.397
0.412
0.431
192
0.511
0.453
0.520
0.455
0.526
0.456
0.395
0.406
0.448
0.417
0.462
0.462
336
0.610
0.512
0.537
0.484
0.584
0.497
0.417
0.424
0.478
0.439
0.489
0.479
720
0.598
0.523
0.581
0.526
0.539
0.493
0.467
0.463
0.454
0.453
0.536
0.514
Average
0.558
0.484
0.530
0.475
0.532
0.466
0.409
0.419
0.452
0.427
0.475
0.472
Appendix
Table 18: MSE and MAE comparison of different fine-tuning strategies on TSF benchmark datasets. Red numbers indicate the best performance for each dataset.
Figure 11: Illustration of LVM’s ablation models. (a) is the model w/o-LVM , which replaces the Transformer blocks in LVMs with a linear layer. (b) is the model LVM2Attn , which replaces the Transformer blocks in LVMs with a single mult-head attention layer.
Dataset
LVMs
w/o-LVM
LVM2Attn
UWaveGestureLibrary
73.4
78.6
80.1
SpokenArabicDigits
97.0
96.4
96.5
Handwriting
24.3
22.4
20.7
FaceDetection
65.0
64.1
66.2
Appendix
Table 19: Accuracy (%) comparison between LVM architecture and ablation models on TSC benchmark datasets. Red numbers indicate the best performance for each dataset.
Model
LVMs
w/o-LVM
LVM2Attn
Dataset
Metrics
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
96
0.412
0.431
0.392
0.410
0.391
0.417
192
0.462
0.462
0.418
0.426
0.414
0.435
336
0.489
0.479
0.441
0.443
0.438
0.452
720
0.536
0.514
0.441
0.465
0.469
0.485
Average
0.475
0.472
0.423
0.436
0.428
0.447
Appendix
Table 20: MSE and MAE comparison between LVM architecture and ablation models on TSF benchmark datasets. Red numbers indicate the best performance for each dataset.
Figure 12: Accuracy vs. inference time of the compared methods on TSC benchmark datasets. Green marker stands for LVM, Red marker stands for GPT4TS and Blue marker stands for TimesNet.
Figure 13: MSE vs. inference time of the compared methods on TSF benchmark datasets. Green marker stands for LVM, Red marker stands for PatchTST and Blue marker stands for GPT4TS.
Figure 14: MAE vs. inference time of the compared methods on TSF benchmark datasets. Green marker stands for LVM, Red marker stands for PatchTST and Blue marker stands for GPT4TS.
Model
MAE
SimMIM
Pre-trained
Enc w/o Dec
Dec w/o Enc
Pre-trained
Enc w/o Dec
Dec w/o Enc
Dataset
Metrics
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
96
0.356
0.383
0.420
0.423
0.396
0.401
0.362
0.383
0.466
0.426
0.412
0.418
192
0.395
0.406
0.445
0.446
0.399
0.414
0.407
0.412
0.496
0.455
0.457
0.446
336
0.417
0.424
0.489
0.484
0.441
0.433
0.422
0.417
0.499
0.474
0.581
0.520
720
0.467
0.463
0.582
0.543
0.426
0.451
0.462
0.455
0.505
0.481
0.564
0.526
Appendix
Table 23: MSE and MAE comparison of self-supervised LVMs with either the pre-trained encoder ( Dec w/o Enc ) or decoder ( Enc w/o Dec ) excluded on TSF benchmark datasets.
Figure 15: Forecasting performance (MAE) of an LVM w.r.t. varying segment length used in UVH imaging. n ( green ) estimates the difficulty of forecasting.
Look-back Window
48
96
192
336
720
1152
1728
2304
Dataset
Metrics
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
96
0.376
0.395
0.373
0.390
0.364
0.383
0.356
0.383
0.347
0.375
0.347
0.376
0.344
0.376
0.373
0.402
192
0.440
0.431
0.424
0.418
0.411
0.412
0.395
0.406
0.385
0.405
0.384
0.402
0.391
0.408
0.399
0.417
336
0.474
0.450
0.471
0.445
0.456
0.437
0.417
0.424
0.408
0.418
0.410
0.418
0.395
0.413
0.408
0.423
720
0.485
0.477
0.482
0.471
0.469
0.465
0.467
0.463
0.468
0.460
0.432
0.440
0.425
0.442
0.424
0.442
Average
0.444
0.438
0.438
0.431
0.425
0.424
0.409
0.419
0.402
0.415
0.393
0.409
0.389
0.410
0.401
0.421
Appendix
Table 24: The MSE and MAE performance of LVMs across different look-back window lengths on TSF benchmark datasets.
Figure 16: Examples of GAF images on the first channel of multivariate time series with 152 time steps randomly drawn from five classes in the Handwriting dataset.
Figure 17: Examples of GAF images on the first channel of multivariate time series with 336 time steps randomly drawn from five classes in the UWaveGestureLibrary dataset.
Figure 18: Visualization of LVM’s inductive bias during TSF on a random example from the ETTh1 dataset (period is 24 time steps). From (a) to (d), the segment length vary within {24, 32, 36, 48}.
Figure 19: Visualization of LVM’s inductive bias during TSF on a random example from the Traffic dataset (period is 24 time steps). From (a) to (d), the segment length vary within {24, 32, 36, 48}.
Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.
Adapting vision models for time series analysis is compelling, yet all existing approaches are falling short of dedicated time series foundation models (TSFMs) in classification. In this work, we propose Time Vision Transformer (TiViT), the first framework that successfully unlocks the representational power of frozen Vision Transformers (ViTs) pretrained on large-scale image datasets for time series classification. TiViT achieves state-of-the-art performance without any finetuning by utilizing the hidden representations of OpenCLIP models. We explore the structure of TiViT representations and find that intermediate ViT layers with high intrinsic dimension are the most effective for time series classification. Furthermore, we assess the alignment between TiViT and TSFM representation spaces and identify a strong complementarity, with additional performance gains achieved through feature concatenation. Finally, we unfreeze the ViT backbone of TiViT for continual pretraining and contrastive alignment with TSFMs on time series, enhancing the performance of lightweight TiViT variants. Our findings reveal a new direction for the domain and task adaptation of vision foundation models. Code is available at https://github.com/ExplainableML/TiViT.
Simon Roschmann, Quentin Bouniot, Vasilii Feofanov +2
Helmholtz Munich · Technical University of Munich · Munich Center for Machine Learning +2
Large language models (LLMs) have been introduced to time series forecasting (TSF) to incorporate contextual knowledge beyond numerical signals. However, existing studies question whether LLMs provide genuine benefits, often reporting comparable performance without LLMs. We show that such conclusions stem from limited evaluation settings and do not hold at scale. We conduct a large-scale study of LLM-based TSF (LLM4TSF) across 8 billion observations, 17 forecasting scenarios, 4 horizons, multiple alignment strategies, and both in-domain and out-of-domain settings. Our results demonstrate that LLM4TS indeed improves forecasting performance, with especially large gains in cross-domain generalization. Pre-alignment outperforming post-alignment in over 90% of tasks. Both pretrained knowledge and model architecture of LLMs contribute and play complementary roles: pretraining is critical under distribution shifts, while architecture excels at modeling complex temporal dynamics. Moreover, under large-scale mixed distributions, a fully intact LLM becomes indispensable, as confirmed by token-level routing analysis and prompt-based improvements. Overall, Our findings overturn prior negative assessments, establish clear conditions under which LLMs are not only useful, and provide practical guidance for effective model design. We release our code at https://github.com/EIT-NLP/LLM4TSF.
Xin Qiu, Junlong Tong, Yirong Sun +3
Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative, Institute of Digital Twin, Eastern Institute of Technology, Ningbo · Zhejiang University · LMU Munich
High-quality time series forecasting is pivotal for real-world decision-making. However, traditional point-wise metrics often fail to reveal complex temporal patterns and align poorly with human intuitive preferences. While the ''LLM-as-a-Judge'' paradigm has revolutionized text evaluation by providing flexible, human-aligned judgment, its application to time series remains largely unexplored. In this paper, we leverage Vision-Language Models (VLMs) as judges for time series forecasting, harnessing their ability to comprehend time series plots grounded in textual information. Specifically, we propose a novel framework integrating micro- and macro-level judgments informed by contextual information to evaluate time series forecasting. To this end, we introduce TimeVista, a comprehensive VLM-as-a-Judge benchmark comprising 5563 time series samples paired with detailed evaluation rubrics. Extensive meta-evaluations demonstrate that VLMs are highly reliable judges, achieving significantly higher consistency with human preferences than conventional metrics. Building upon our benchmark, we comprehensively assess recent Time Series Foundation Models (TSFMs) under the VLM-as-a-Judge paradigm. Our results demonstrate that VLMs serve as robust and interpretable judges, providing a comprehensive, human-aligned standard for evaluating time series models.
Zhi Chen, Yuxuan Wang, Jialong Wu +5
School of Software, BNRist, Tsinghua University, Beijing 100084, China