Learning time series foundation models has been shown to be a promising approach for zero-shot time series forecasting across diverse time series domains. Insofar as scaling has been a critical driver of performance of foundation models in other modalities such as language and vision, much recent work on time series foundation modeling has focused on scaling. This has resulted in time series foundation models with hundreds of millions of parameters that are, while performant, inefficient and expensive to use in practice. This paper describes a simple recipe for learning efficient foundation models for zero-shot time series forecasting that are orders of magnitude smaller. We show that large-scale transformers are not necessary: small hybrid models that interleave long convolution and linear RNN layers (in particular DeltaNet layers) can match the performance of larger transformer-based models while being more than a hundred times smaller. We also describe several data augmentation and inference strategies that further improve performance. This recipe results in Reverso, a family of efficient time series foundation models for zero-shot forecasting that significantly push the performance-efficiency Pareto frontier.
Figures & tables
Figure 1: Reverso architecture. Given a sequence of length S , we turn it into a length- L input t∈RL×C by taking the L,2L,…,2C−1L most recent data points and downsampling by ×1,×2,…,×2C−1 . The input t is passed through a single projection layer to obtain embedding representations x∈RL×d . Then, nlayers of sequence-mixing and channel-mixing blocks operates on x , where we alternate between long convolutions and DeltaNet for sequence mixing across length L , and use MLP layers for channel mixing across dimension d . The final output head (based on an attention-based transformation) obtains the predictions y^∈Rp .
Model
Params
Layers
Dim.
Reverso-Nano
200K
2
32
Reverso-Small
550K
4
64
Reverso
2.6M
8
128
Table 1: Architecture configurations.
Figure 2: MASE on Gift-Eval benchmark
MASE
Model
Params
Short
Medium
Long
Average
Xihe-Max
1.5B
0.623
0.718
0.763
0.701
TimesFM-2.5
200M
0.626
0.724
0.751
0.700
PatchTST-FM
260M
0.616
0.722
0.745
0.694
Flowstate-r1.1
18M
0.633
0.720
0.736
0.696
Reverso
2.6M
0.634
0.699
0.741
0.691
Table 2: Model MASE scores across forecast horizons, averaged across the 21 long sequence datasets in Gift-Eval with all three horizons available.
Figure 3: Inference efficiency on Gift-Eval. Reverso variants achieve lower latency and lower peak memory usage than larger time series foundation models while having the lowest parameters.
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Our data augmentation (left) and synthetic data generation (right) pipeline. For data augmentation we apply: downsampling, amplitude modulation, vertical flip, horizontal flip, censor, mixup. For synthetic data generation, we generate data from Gaussian process with randomly selected kernels from a kernel bank, and combine this with spike/trapezoidal patterns as well as processes sampled with trend, seasonality and irregularity.
Model
Size
UCR
Multivariate UEA
Reverso
2.6M
81%
74%
TiRex
30M
81%
74%
Chronos-Bolt
205M
79%
74%
TIC
–
80%
–
Moirai-Large
311M
80%
70%
TimesFM-2.0
500M
79%
70%
Appendix
Table 3: Classification accuracy on UCR and multivariate UEA benchmarks. Higher is better.
Model
Reverso
Reverso-Small
Reverso-Nano
Sundial-L
Super-Linear
Timer-XL
TiRex
Chronos-2
Params
2.6M
550K
200K
444M
2.6M
85M
30M
120M
ETTm1
0.367
0.376
0.382
0.369
0.389
0.392
0.365
0.359
ETTm2
0.304
0.309
0.311
0.315
0.325
0.336
0.302
0.291
ETTh1
0.404
0.404
0.416
0.420
0.416
0.417
0.417
0.405
ETTh2
0.365
0.370
0.384
0.387
0.386
0.388
0.362
0.367
Electricity
0.238
0.241
0.249
0.262
0.267
0.268
0.240
0.237
Appendix
Table 4: Zero-shot forecasting performance (MAE) on LTSF datasets: ETTm1, ETTm2, ETTh1, ETTh2, Electricity and Weather, comparing between Reverso variants against state-of-the-art foundation models. Results represent the averaged MAE across prediction lengths {96,192,336,720} . Best results are in bold , and second-best are underlined . A full set of results are shown in Table 13
Figure 5: LTSF performance vs. Parameter Count. MAE is averaged over the horizons of {96,192,336,720} for the datasets ETTh1, ETTh2, ETTm1, ETTm2, Electricity and Weather. For models which are not evaluated on all the datasets (e.g. YingLong did not report results for Electricity), we impute with the other best existing model on that dataset.
Threshold
Reverso
Timer
Anomaly Transformer
TimesNet
Detected@1%
88 (35%)
51
51
31
Detected@3%
126 (50%)
110
98
61
Detected@10%
188 (75%)
172
129
109
Appendix
Table 5: Zero-shot anomaly detection on UCR. Higher detected counts are better. Percentages in parentheses indicate the fraction of UCR anomalies detected by Reverso.
Model
ETTh1
ETTh2
ETTm1
ETTm2
Weather
Traffic
ECL
Reverso
0.285
0.147
0.149
0.084
0.081
0.433
0.151
Timer
0.312
0.209
0.413
0.158
0.158
0.477
0.152
TimesNet
0.289
0.253
0.373
0.112
0.127
0.624
0.209
Appendix
Table 6: Imputation MSE on standard time-series benchmarks. Lower is better.
Run Setting
Seed 1
Seed 2
Seed 3
Seed 4
Mean ± Std.
20% training run
0.720
0.722
0.719
0.723
0.7210±0.0018
Full training run
0.711
0.712
0.713
0.710
0.7115±0.0013
Appendix
Table 8: Gift-Eval MASE across random seeds. Lower is better.
Kernel
Formula κ(x,x′)
Hyperparameters
Constant
C
C=1
Linear
σ2+x⋅x′
σ∈{0,1,10}
RBF
exp(−2l2∥x−x′∥2)
l∈{0.1,1,10}
Rational Quadratic
(1+2α∥x−x′∥2)−α
α∈{0.1,1,10}
Matérn
Γ(ν)21−ν(2νl∥x−x′∥)νKν(2νl∥x−x′∥)
ν∈{0.5,1.5,2.5},l∈{0.1,1,10}
Periodic
exp(−2sin2(π∥x−x′∥/p))
p∈P
Appendix
Table 9: Kernel Bank K used for Synthetic Data Generation
Family
Model
Params
MASE
TimesFM
TimesFM-2.5
200M
0.705
TimesFM-2.0
500M
0.758
PatchTST-FM-r1
PatchTST-FM
260M
0.707
Xihe
Xihe-Max
1.5B
0.711
Xihe-Base
700M
0.718
Xihe-Flash
300M
0.726
Appendix
Table 11: MASE scores versus parameter size breakdowns for models compared in the Gift-Eval benchmark in Figure 2 . Note that the data leaked version of Chronos-2 has been trained on datasets within Gift-Eval and hence is not included within Figure 2 for fair zero-shot comparison, but is still a strong frame of reference for SOTA foundation models.
Figure 6: Zero-shot performance on the Gift-Eval benchmark for (a) long sequences (average length at least 2048) and (b) short sequences.
Dataset
Frequency
bitbrains_fast_storage
5T
bitbrains_rnd
5T
bizitobs_application
10S
bizitobs_l2c
5T
bizitobs_l2c
H
bizitobs_service
10S
Appendix
Table 12: Datasets used for Table 2 , a subset of GiftEval which have all three short/medium/long forecasting horizons.
Figure 7: Qualitative results visualizing zero-shot forecasts by Reverso on various tasks within Gift-Eval. Reverso is able to perform long horizon forecasting, accurately capturing patterns at multiple frequency scales. The grey vertical dotted line denotes the length of a single autoregressive prediction.
Figure 8: Downsampling Comparison. In this example, consider an input with period 4000 to a model with context length 2048, tasked with forecasting the next 720 points. In (b), the model does not have enough information to forecast the rising part of the trapezoid. However, through downsampling the input, multiple full periods now fit into the context window and the model can forecast more accurately.
Model
Long MASE
Full MASE
Reverso-Nano
0.766
0.757
w/o. multi-scale input
0.783
0.760
Reverso-Small
0.742
0.722
w/o. multi-scale input
0.754
0.726
Reverso
0.741
0.706
w/o. multi-scale input
0.749
0.711
Appendix
Table 14: Effect of multiscale downsampling on Gift-Eval MASE. Lower is better.
Sequence Module
Long MASE
Short MASE
Overall MASE
DeltaNet
0.706
0.792
0.732
Gated DeltaProduct
0.711
0.793
0.735
Gated DeltaNet
0.708
0.782
0.730
Long Convolution
0.708
0.799
0.735
Gated Linear Attention
0.726
0.817
0.753
Attention ( sin-cos )
0.729
0.840
0.762
Appendix
Table 15: Architectural ablations.
Method
MASE
Baseline
0.738
w/o mixup
0.740
w/o downsample
0.740
w/o temp rev
0.740
w/o flip
0.739
w/o censor
0.738
Appendix
Table 16: Data pipeline ablations.
Method
Short Seq
Long Seq
Short Term
Med Term
Long Term
Overall
Baseline
0.764
0.682
0.696
0.699
0.741
0.706
w/o downsampling
0.764
0.700
0.696
0.723
0.778
0.719
No flip
0.776
0.686
0.705
0.702
0.745
0.713
Flip equivariance
0.764
0.682
0.696
0.699
0.741
0.706
Appendix
Table 17: Inference strategy ablations on Gift-Eval.
Architecture
Layer Sequence
MASE
Reverso
Conv → DN → Conv → DN → Conv → DN → Conv → DN
0.720
Variant 1
DN → Conv → DN → Conv → DN → Conv → DN → Conv
0.724
Variant 2
Conv → DN → DN → DN → DN → DN → DN → DN
0.725
Variant 3
Conv → Conv → Conv → Conv → DN → DN → DN → DN
0.730
Appendix
Table 18: MASE performance for different architecture variants.
Figure 9: Filter visualizations for the first convolution layer of Reverso-Nano.
Conv layer 1
Conv layer 2
Conv layer 3
Conv layer 4
Min period
98
293
4
5
Median period
244
293
204
163
Appendix
Table 19: Minimum and median periods across convolutional layers.
Figure 10: State evolution of of the Deltanet states in Reverso-Nano. Across the 4 heads of the 8×8 state matrix St , we visualize the 32 coordinates with the highest temporal variance.
Time-series foundation models are pretrained sequence models designed to transfer across forecasting, classification, anomaly detection, and other temporal tasks.
Inspired by recent breakthroughs in large language models for natural language processing, foundation models have emerged as a promising paradigm for zero-shot time series forecasting, enabling accurate predictions on datasets never seen during pre-training. Ranging from tens to hundreds of millions of parameters, these models are pre-trained on vast and diverse collections of time series, learning generalizable representations that support both point and probabilistic forecasting. This approach alleviates the need for dataset-specific model design and manual tuning, offering a unified solution across forecasting problems. In this work, we review the main architectures, pre-training strategies, and optimization methods underpinning these models. We further investigate post-pre-training fine-tuning of selected foundation models to enhance their performance on specific datasets. Our empirical results demonstrate that this step consistently improves forecasting accuracy over the zero-shot baseline.
Morad Laglil, Bertrand Pracca, Emilie Devijver +1
Savoye, 18 bd des Gorgets, 21000, Dijon, France · Univ. Grenoble Alpes, CNRS, Grenoble INP, LIG, Grenoble, 38000, France
We introduce Timer-S1, a strong Mixture-of-Experts (MoE) time series foundation model with 8.3B total parameters, 0.75B activated parameters for each token, and a context length of 11.5K. To overcome the scalability bottleneck in existing pre-trained time series foundation models, we perform Serial Scaling in three dimensions: model architecture, dataset, and training pipeline. Timer-S1 integrates sparse TimeMoE blocks and generic TimeSTP blocks for Serial-Token Prediction (STP), a generic training objective that adheres to the serial nature of forecasting. The proposed paradigm introduces serial computations to improve long-term predictions while avoiding costly rolling-style inference and pronounced error accumulation in the standard next-token prediction. Pursuing a high-quality and unbiased training dataset, we curate TimeBench, a corpus with one trillion time points, and apply meticulous data augmentation to mitigate predictive bias. We further pioneer a post-training stage, including continued pre-training and long-context extension, to enhance short-term and long-context performance. Evaluated on the large-scale GIFT-Eval leaderboard, Timer-S1 achieves state-of-the-art forecasting performance, attaining the best MASE and CRPS scores as a pre-trained model. Timer-S1 is released to facilitate further research.
Time series forecasting with pretrained foundation models has demonstrated strong zero-shot capabilities. However, achieving optimal performance on time series with short or negligible historical data in domain-specific applications typically requires adaptation via either fine-tuning or RAG. While fine-tuning is effective, it incurs substantial computational costs. This work explores RAG within univariate time series (Retrieval Augmented Generation) as a more efficient alternative, in particular RAF (Retrieval Augmented Forecasting), and introduces RAEF (Retrieval-Augmented Extended Forecasting), a model-agnostic method built upon RAF. RAEF incorporates key refinements to the retrieval and aggregation mechanisms: (1) direct retrieval in input-space rather than embedding-space, reducing inference overhead, and (2) concatenation-based aggregation that preserves temporal structure instead of averaging. Empirical evaluation across multiple benchmark datasets demonstrates that RAEF outperforms RAF in both accuracy and inference overhead. Furthermore, comprehensive comparisons with zero-shot and fine-tuned foundation models show that RAEF achieves competitive or superior performance to fine-tuning while avoiding its computational burden, establishing it as a practical and scalable approach for domain adaptation in time series forecasting.
Juan Pablo Villa Serna, Rohan Asthana, Vasileios Belagiannis