The recent surge of interest in Time Series Foundation Models has rapidly advanced the field. Yet, heterogeneous training setups across studies make it difficult to determine whether performance improvements came from architectural innovations or effective data engineering. In this work, we investigate whether a standard patch Transformer architecture, paired with a straightforward training protocol, can match the zero-shot forecasting performance of state-of-the-art TSFMs. We conduct a comprehensive ablation study covering model scaling, data composition, and training techniques to examine the ingredients behind this performance. Our findings show that pretraining data composition, contiguous patch masking, and context length have a substantial effect on forecasting accuracy, and that the generic architecture itself can deliver high performance with good scalability. By strictly controlling these experimental variables, we provide comprehensive empirical results on model scaling across multiple dimensions. Furthermore, we establish that the effectiveness of this training protocol generalizes robustly across benchmarks and alternative architectures. We release our pretrained models and document the detailed training protocol and findings to establish a transparent, reproducible baseline for future research.
Figures & tables
Figure 1
Figure 3: Forecasting performance on the GIFT-Eval benchmark. The four metrics (MASE, MASE Rank, CRPS, CRPS Rank) are aggregated over the 97 test cases. Lower values indicate better performance. The methods are sorted based on CRPS Rank. Ours (Extended) is trained on a corpus that overlaps 23 of the 97 test cases and is therefore not a zero-shot result (Section 3.1 ).
Figure 3Table 4Table 5
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
Hyperparameter
Value
Rationale / Source
Architecture
Context Length ( T )
8192
A large context window for general usage. ( Ansari et al., 2025 )
Patch Size ( L )
16
Non-overlapping (stride =L ), fixed across datasets and frequencies, following PatchTST ( Nie et al., 2023 ) ; see Appendix F.1.3 .
Number of Layers ( Nlayer )
20
Modified from ViT-Large.
Embedding Dimension ( d )
1024
From ViT-Large.
Attention Head Dimension
64
Standard choice for Transformers.
Appendix
Table 6: Detailed Hyperparameter Configuration. The values are selected based on standard practices in TSFMs and Transformers.
Subset of Chronos , LOTSA , Synthetic , Finance, IoT.
YingLong ( Wang et al., 2025 )
300M
Encoder
8192
Subset of Chronos , KernelSynth .
Moirai-2 ( Woo et al., 2024 )
14M
Encoder
512
GIFT-Eval-Pretrain , Training split of GIFT-Eval , TSMixup , KernelSynth , Internal Operations Data. Curated to remove the “low-quality” samples.
Toto ( Cohen et al., 2025 )
151M
Decoder
4096
Observability Metrics, GIFT-Eval-Pretrain , Subset of Chronos , Synthetic.
FlowState ( Graf et al., 2026 )
9.1M
S5
2048
GIFT-Eval-Pretrain .
Appendix
Table 7: Comparison of recent foundation models. Pretraining corpora sources are summarized for brevity. Publicly-available data are highlighted with blue . Corpus overlap with the GIFT-Eval test cases are highlighted with red .
Comparison
MASE ratio [95% CI]
CRPS ratio [95% CI]
Zero-shot / Chronos-2
1.013 [0.987, 1.041]
0.995 [0.965, 1.022]
Zero-shot / TimesFM-2.5
1.003 [0.977, 1.032]
0.985 [0.958, 1.012]
Zero-shot / Timer-S1
1.020 [0.982, 1.062]
0.995 [0.963, 1.031]
Appendix
Table 8: Paired cluster-bootstrap 95% confidence intervals for the ratio of aggregate scores on GIFT-Eval (97 cases). A ratio below 1 favors the numerator model.
Comparison
MASE ratio [95% CI]
CRPS ratio [95% CI]
Extended / Zero-shot
0.986 [0.975, 0.995]
0.988 [0.977, 0.997]
Extended / Chronos-2
1.000 [0.986, 1.016]
1.001 [0.986, 1.017]
Extended / TimesFM-2.5
0.990 [0.976, 1.005]
0.981 [0.965, 0.998]
Extended / Timer-S1
0.943 [0.905, 0.975]
0.946 [0.911, 0.975]
Zero-shot / Chronos-2
1.014 [0.997, 1.032]
1.013 [0.997, 1.030]
Zero-shot / TimesFM-2.5
1.004 [0.987, 1.021]
0.994 [0.976, 1.012]
Appendix
Table 9: Paired cluster-bootstrap 95% confidence intervals for the ratio of aggregate scores on TIME (98 cases). A ratio below 1 favors the numerator model.
Model
MASE SD
MASE ranks
CRPS SD
CRPS ranks
Zero-shot
0.202
1/2/4/7/14
0.198
1/2/5/7/14
Chronos-2
0.179
1/2/3/6/14
0.192
1/2/4/6/14
TimesFM-2.5
0.181
1/3/5/7/11
0.204
1/3/5/8/13
Timer-S1
0.187
1/2/5/7/12
0.217
1/2/5/8/13
Appendix
Table 10: Case-level dispersion on GIFT-Eval (97 cases). SD is the sample standard deviation of the Seasonal-Naive-normalized score across cases; ranks are min/Q25/median/Q75/max among 14 models.
Model
MASE SD
MASE ranks
CRPS SD
CRPS ranks
Zero-shot
0.175
1/2/3/5/6
0.176
1/2/3/5/6
Extended
0.159
1/1.25/3/4/6
0.170
1/2/3/4/6
Chronos-2
0.150
1/1/2/4/6
0.163
1/1/3/4/6
TimesFM-2.5
0.152
1/2/4/5/6
0.163
1/3/4/5.75/6
Timer-S1
0.205
1/3/5/6/6
0.190
1/3/4/5/6
Appendix
Table 11: Case-level dispersion on TIME (98 cases). SD is the sample standard deviation of the Seasonal-Naive-normalized score across cases; ranks are min/Q25/median/Q75/max among six models.
Pretrain Data
MASE ↓
CRPS ↓
Rank ↓
KernelSynth
0.832
0.584
12.220
Real Only
0.790
0.542
7.008
Zero-shot
0.712
0.483
4.267
Extended †
0.696
0.480
3.383
Appendix
Table 12: Ablation analysis of pretraining data composition . We compare models trained on individual data sources and composite datasets, as introduced in Section 3.1 . † The Extended corpus overlaps 23 GIFT-Eval test cases (Section 3.1 ).
Figure 5: Zero-shot forecasting performance on the GIFT-Eval benchmark. The three metrics (MASE, CRPS, Rank) are aggregated with geometric mean over the 74 non-overlapping test cases, none of which is seen by either of our models. Lower values indicate better performance. The methods are sorted based on Aggregated Rank.
Num. Quantiles
MASE ↓
CRPS ↓
Rank ↓
9
0.699
0.482
3.758
21
0.700
0.481
4.087
99
0.696
0.480
3.598
Appendix
Table 13: Ablation study on output probability granularity . We compare the performance of models trained to predict 9, 21, and 99 quantiles.
Patch Size
Tokens
MASE ↓
CRPS ↓
Rank ↓
8
1024
0.714
0.487
5.495
16
512
0.719
0.487
5.340
32
256
0.724
0.497
6.722
Appendix
Table 14: Ablation study on patch size L on GIFT-Eval. Tokens is the number of patches at the default context length T=8192 . The three models form a separate set of pretraining runs; see the text for the training setup.
MASE ↓
CRPS ↓
Group
Cases
L=8
L=16
L=32
L=8
L=16
L=32
10-second
6
0.676
0.705
0.715
0.478
0.546
0.514
Minute-level
30
0.734
0.723
0.737
0.500
0.489
0.511
Hourly
31
0.693
0.696
0.697
0.434
0.432
0.439
Daily
15
0.661
0.660
0.671
0.416
0.411
0.418
Weekly
8
0.779
0.773
0.798
0.624
0.621
0.631
Appendix
Table 15: Patch-size ablation broken down by sampling frequency and forecast term. Each entry is the Seasonal-Naive-normalized geometric mean over the cases in the group. Minute-level groups the 5-, 10-, and 15-minute datasets; Monthly+ groups the monthly, quarterly, and yearly datasets.
Figure 6: Impact of training scaling on forecasting performance : We evaluate the intermediate model checkpoints during training. MASE and CRPS for both the Extended and Zero-shot variants exhibit a consistent monotonic improvement.
Figure 7: Examples of predictions on a synthetic signal with different predictions lengths.
Dataset (S/M/L)
Model
MASE-S
MASE-M
MASE-L
CRPS-S
CRPS-M
CRPS-L
Australia_Solar/H (24/72/168)
Extended
0.5746
0.6031
0.5789
0.2320
0.2102
0.1972
Zero-shot
0.6278
0.6745
0.6832
0.2322
0.2215
0.2193
Chronos-2
0.5689
0.6116
0.6182
0.2583
0.2250
0.2103
TimesFM-2.5
0.6053
0.6114
0.6012
0.2657
0.2237
0.2067
Timer-S1
0.5861
0.5986
0.5776
0.2519
0.2215
0.2038
Coastal_T_S/20T (9/72/216)
Extended
0.4701
0.9593
1.5430
0.0018
0.0040
0.0062
Appendix
Table 16: Case-level MASE and CRPS on three TIME datasets at short (S), medium (M), and long (L) horizons. Values are absolute (not normalized) case-level scores; lower is better.
Figure 8: Examples of predictions from our Extended model on datasets with significant seasonal patterns.
Figure 9: Examples of predictions from our Zero-shot model on datasets with significant seasonal patterns.
Figure 10: Examples of predictions from our Extended model on datasets with significant trend patterns.
Figure 11: Examples of predictions from our Zero-shot model on datasets with significant trend patterns.
Figure 12: Example of the predictions from models with context length (left) 2048, (middle) 4096, and (right) 8192. Shorter context lengths may fail to capture periodicity and produce poor forecasts, while longer context lengths are more robust for general usage.
Figure 13: Examples of the predictions from models pretrained with CPM Block Size (left) NCPM=1 , (middle) NCPM=4 , and (right) NCPM=8 . Larger NCPM encourages the model to infer the underlying time series dynamics, providing better forecasts for general usage.
Figure 14: Inference time of our models and other TSFMs. Benefiting from high mask ratio, single-step inference, and architecture simplicity, our models also excel in inference time which is invariant to the forecast horizon.
Learning time series foundation models has been shown to be a promising approach for zero-shot time series forecasting across diverse time series domains. Insofar as scaling has been a critical driver of performance of foundation models in other modalities such as language and vision, much recent work on time series foundation modeling has focused on scaling. This has resulted in time series foundation models with hundreds of millions of parameters that are, while performant, inefficient and expensive to use in practice. This paper describes a simple recipe for learning efficient foundation models for zero-shot time series forecasting that are orders of magnitude smaller. We show that large-scale transformers are not necessary: small hybrid models that interleave long convolution and linear RNN layers (in particular DeltaNet layers) can match the performance of larger transformer-based models while being more than a hundred times smaller. We also describe several data augmentation and inference strategies that further improve performance. This recipe results in Reverso, a family of efficient time series foundation models for zero-shot forecasting that significantly push the performance-efficiency Pareto frontier.
Xinghong Fu, Yanhong Li, Georgios Papaioannou +1
Massachusetts Institute of Technology · Allen Institute for AI · Qube Research & Technologies
Time Series Foundation Models (TSFMs) leverage extensive pretraining to accurately predict unseen time series during inference, without the need for task-specific fine-tuning. Through large-scale evaluations on standard benchmarks, we find that leading transformer-based TSFMs exhibit redundant components in their intermediate layers. We introduce a set of tools for mechanistic interpretability of TSFMs, including ablations of specific components and direct logit attribution on the residual stream. Our findings are consistent across several leading TSFMs with diverse architectures, and across a diverse set of real-world and synthetic time-series datasets. We discover that all models in our study are robust to ablations of entire layers. Furthermore, we develop a theoretical framework framing transformers as kernel regressors, motivating a purely intrinsic strategy for ablating heads based on the stable rank of the per-head projection matrices. Using this approach, we uncover the specific heads responsible for degenerate phenomena widely observed in TSFMs, such as parroting of motifs from the context and seasonality bias. Our study sheds light on the universal properties of this emerging class of architectures for continuous-time sequence modeling.
Anthony Bao, Venkata Hasith Vattikuti, Jeffrey Lai +1
ECE Department, UT Austin · Department of Physics, UT Austin · Oden Institute, UT Austin
Inspired by recent breakthroughs in large language models for natural language processing, foundation models have emerged as a promising paradigm for zero-shot time series forecasting, enabling accurate predictions on datasets never seen during pre-training. Ranging from tens to hundreds of millions of parameters, these models are pre-trained on vast and diverse collections of time series, learning generalizable representations that support both point and probabilistic forecasting. This approach alleviates the need for dataset-specific model design and manual tuning, offering a unified solution across forecasting problems. In this work, we review the main architectures, pre-training strategies, and optimization methods underpinning these models. We further investigate post-pre-training fine-tuning of selected foundation models to enhance their performance on specific datasets. Our empirical results demonstrate that this step consistently improves forecasting accuracy over the zero-shot baseline.
Morad Laglil, Bertrand Pracca, Emilie Devijver +1
Savoye, 18 bd des Gorgets, 21000, Dijon, France · Univ. Grenoble Alpes, CNRS, Grenoble INP, LIG, Grenoble, 38000, France