Foundation models for zero-shot time series forecasting face challenges in efficient long-horizon prediction and reproducibility, with existing synthetic-only approaches underperforming on challenging benchmarks. This paper presents TempoPFN, a univariate time series foundation model based on linear Recurrent Neural Networks (RNNs) pre-trained exclusively on synthetic data. The model uses a GatedDeltaProduct architecture with state-weaving for fully parallelizable training across sequence lengths, eliminating the need for windowing or summarization techniques while maintaining robust temporal state-tracking. Our comprehensive synthetic data pipeline unifies diverse generators, including stochastic differential equations, Gaussian processes, and audio synthesis, with novel augmentations. In zero-shot evaluations on the Gift-Eval, fev-bench and Chronos-ZS benchmarks, TempoPFN achieves top-tier competitive performance, surpassing the majority of models trained on real-world data, while leveraging fully parallelizable training and inference. We open-source our complete data generation pipeline and training code, providing a reproducible foundation for future research.
Figures & tables
Criterion
TiRex
TabPFN-TS
Mamba4Cast
Chronos
TempoPFN
Fully open-source data pipeline
✗
✗
✓
✗
✓
Open-source training code
✗
✗
✓
✓
✓
Competitive with SOTA performance
✓
✓
✗
✓
✓
Fast training and inference
✓
✗
(✓)
(✓)
✓
Purely synthetic pretraining
✗
(✓)
✓
✗
✓
Table 1: Contributions of TempoPFN: the first fully open-source time series forecasting foundation model with competitive performance; with fully synthetic pretraining and fast training and inference.
Figure 1: (Left) Synthetic Data Generation pipeline containing a mix of novel and existing time-series generators are augmented with a diverse set of augmentations to produce the time-series used for training. (Middle) The TempoPFN architecture produces coherent quantile predictions for all future time-stamps in parallel. (Right) TempoPFN obtains competitive performance on Gift-Eval despite being trained only on synthetic time-series.
Figure 2: The TempoPFN architecture (3 blocks) uses stacked GatedDeltaProduct blocks, learnable initial states H0i and state-weaving.
Figure 3: Synthetic data augmentation pipeline. Synthetic time-series undergo probabilistic transformations across six categorical groups (Invariances, Structure, Seasonality, Analytic, Discrete, Artifacts), with optional TS-Mixup combinations and stochastic convolution filtering. Competitive selection based on change scores ensures meaningful augmentations. Example outputs demonstrate the diversity of generated temporal patterns.
Figure 4: Comparison of TempoPFN (3M iterations with a per-GPU batch size of 40) against other models on GiftEval benchmark. We compute both normalized and average ranks for CRPS and MASE. Colors represent the class of time series model.
Figure 5: Qualitative comparison between TempoPFN, TiRex and TabPFN-TS on two series from the GIFT-Eval Benchmark. (Left) Total context with prediction window in dashed grey box. (Right) Predictions between TempoPFN, TiRex, and TabPFN-TS.
Figure 6: Normalized CRPS of TempoPFN and TiRex as a function of missing values (NaN) in the data.
Rank
Model
Avg. Win Rate (%)
Skill Score (%)
Median Runtime (s)
Leakage (%)
Failed Tasks (%)
Organization
Zero-shot
1
Chronos-2
87.7
35.5
2.69
0
0
AWS
✓
2
TiRex
76.5
30.0
1.40
1
0
NX-AI
✓
3
TimesFM 2.5
74.8
30.2
16.92
10
0
Google
✓
4
Toto 1.0
66.6
28.2
90.74
8
0
Datadog
✓
5
TempoPFN
62.4
25.7
20.67
0
0
Uni Freiburg / ELLIS
✓
6
Moirai 2.0
60.4
27.3
2.46
28
0
Salesforce
✓
Table 2: fev-bench leaderboard based on MASE. Models are ranked by Win Rate and Skill Score. TempoPFN is evaluated with a sign-flip and context-length ensemble at a maximum context of 3072.
Gift-ZS Overall
Gift-ZS Short
Gift-ZS Medium
Gift-ZS Long
Model
CRPS
MASE
CRPS
MASE
CRPS
MASE
CRPS
MASE
TempoPFN (Linear RNN, 40M)
0.577
0.842
0.563
0.763
0.566
0.900
0.631
1.019
chronos-2-small (Transformer, 28M)
0.581
0.849
0.578
0.784
0.571
0.913
0.619
1.001
seasonal_naive
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
Table 3: Backbone swap. Both models are pre-trained from scratch on the same univariate synthetic corpus for the matched 500k-iteration ablation budget (as in Table 4 ). TempoPFN uses our GatedDeltaProduct linear-RNN backbone (40M parameters); chronos-2-small ( AutoGluon Team, 2025 ) is a 28M-parameter Transformer. Lower is better.
Gift-ZS Overall
Gift-ZS Short
Gift-ZS Medium
Gift-ZS Long
Ablation
Source
CRPS
MASE
CRPS
MASE
CRPS
MASE
CRPS
MASE
Base Model
–
0.577
0.842
0.563
0.763
0.566
0.900
0.631
1.019
- GP
A
0.591
0.830
0.576
0.749
0.605
0.924
0.618
0.981
- Kernel
A
0.611
0.885
0.589
0.796
0.637
0.981
0.648
1.056
- ForecastPFN
A
0.617
0.885
0.588
0.791
0.643
0.981
0.674
1.075
- Sawtooth
N
0.628
0.900
0.597
0.800
0.661
1.012
0.684
1.091
Table 4: Ablation study of synthetic data priors using a leave-one-out methodology (500k iterations). The ’Ablation’ column indicates the single prior excluded from the training mixture. Performance is measured by CRPS and MASE (lower is better). Rows are colored to indicate the performance impact on the overall CRPS when a prior is removed: High Impact ( >25% increase) and Medium Impact ( >10% increase). N = Novel prior (our contribution), A = Adapted from open-source.
Gift-ZS Overall
Gift-ZS Short
Gift-ZS Medium
Gift-ZS Long
Model
CRPS
MASE
CRPS
MASE
CRPS
MASE
CRPS
MASE
w/ Aug
0.577
0.842
0.563
0.763
0.566
0.900
0.631
1.019
w/o Aug
0.610
0.875
0.582
0.783
0.617
0.963
0.643
1.059
seasonal_naive
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
Table 5: The impact of data augmentation on model performance, shown by normalized performance values (500k iterations). Individual augmentation effects and probability tuning were not exhaustively explored due to resource constraints.
Appendix figures & tables34 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Example time series generated by CauKer
Figure 8: Example time series generated by KernelSynth
Figure 9: Example time series generated by GP
Figure 10: Example time series generated by ForecastPFN
Figure 11: Example time series generated by Sawtooth Generator
Figure 12: Example time series generated by Step Function
Figure 13: Example time series generated by Anomaly Generator
Figure 14: Example time series generated by Spike Generator
Figure 15: Example time series generated by Sine Wave Generator
Figure 16: Example time series generated by Audio Stochastic Rhythm
Figure 17: Example time series generated by Ornstein–Uhlenbeck process
Parameter
Range / Distribution
Integration step size dt
0.01
Initial value y0
N(0,22)
Regime 0 mean reversion θ(0)
[1.0,5.0]
Regime 0 mean μ(0)
N(−2.0,1.02)
Regime 0 volatility σ(0)
logN(log0.3,0.3)
Regime 0 vol. process (κv,θv,ξv)
[2.0,5.0],[0.2,0.4],[0.1,0.3]
Appendix
Table 6: Parameter ranges for the Regime-Switching OU generator.
Generator
Source
Length
Series / Sec
Cauker
A
2048
0.66
GP
A
2048
7.04
Kernel
A
2048
0.32
ForecastPFN
A
2048
35.49
Sawtooth
N
2048
242.95
Sinewave
N
2048
144.93
Appendix
Table 7: Profiling results for synthetic data generation throughput. N = Novel prior, A = Adapted from open-source.
Category
Parameter
Value
Model
Total Parameters
40M
Embedding size ( embed_size )
512
Encoder layers
10
Number of heads ( num_heads )
4
Encoder attention mode
chunk
Short convolution kernel size
32
Appendix
Table 8: Hyperparameters for main TempoPFN model.
Gift-ZS Overall
Gift-ZS Short
Gift-ZS Medium
Gift-ZS Long
Ablation
Source
CRPS
MASE
CRPS
MASE
CRPS
MASE
CRPS
MASE
Base Model
–
0.577
0.842
0.563
0.763
0.566
0.900
0.631
1.019
+ Cauker
Adapted
0.600
0.875
0.583
0.789
0.615
0.964
0.631
1.043
+ GP
Adapted
0.632
0.897
0.607
0.812
0.666
0.993
0.666
1.053
+ Kernel
Adapted
0.638
0.926
0.622
0.835
0.656
1.042
0.661
1.082
+ ForecastPFN
Adapted
0.715
1.027
0.695
0.918
0.760
1.172
0.726
1.206
Appendix
Table 9: Ablation study of single synthetic priors (trained for 500k iterations). ’Base Model’ uses all priors and augmentations. Lower values are better. Bold : best, underline : second-best. Novel priors are our contributions; Adapted are modified open-source versions.
Gift-ZS Overall
Gift-ZS Short
Gift-ZS Medium
Gift-ZS Long
Configuration
CRPS
MASE
CRPS
MASE
CRPS
MASE
CRPS
MASE
Ablating Positional Encoding:
Base Model (Sin. Pos. Enc. Off)
0.561
0.820
0.553
0.751
0.556
0.880
0.590
0.962
Sinusoidal Positional Encoding
0.648
0.937
0.596
0.809
0.731
1.120
0.715
1.152
Ablating Number of Householder Matrices (H):
H=6
0.556
0.823
0.545
0.750
0.552
0.886
0.590
0.972
Appendix
Table 10: Core architectural ablations (trained for 2M iterations with a per-GPU batch size of 40). Base config: d=512 , L=10 , conv size 16, H=4 , weaving enabled, negative eigenvalues allowed. Sorted by overall CRPS. Bold : best, underline : second-best.
Gift-ZS Overall
Gift-ZS Short
Gift-ZS Medium
Gift-ZS Long
Configuration
CRPS
MASE
CRPS
MASE
CRPS
MASE
CRPS
MASE
Base Model (d=512, L=10)
0.533
0.788
0.532
0.727
0.523
0.840
0.544
0.912
d=512, L=10, Weaving Off
0.537
0.790
0.532
0.723
0.533
0.862
0.553
0.914
d=384, L=16 (Narrower, Deeper)
0.539
0.792
0.532
0.727
0.533
0.850
0.563
0.921
d=576, L=8 (Wider, Shallower)
0.540
0.794
0.536
0.732
0.529
0.849
0.561
0.921
seasonal_naive
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
Appendix
Table 11: Ablation of model scale and depth (trained for 3M iterations with a batch size of 40). Base Model: d=512,L=10 , H=4 , conv size 32, weaving/neg eigenvalues on. Compares width vs. depth at constant parameter count. Sorted by overall CRPS. Bold : best, underline : second-best.
Gift-ZS Overall
Gift-ZS Short
Gift-ZS Medium
Gift-ZS Long
LR Scheduler
CRPS
MASE
CRPS
MASE
CRPS
MASE
CRPS
MASE
WarmupStableDecay
0.554
0.812
0.544
0.740
0.550
0.877
0.584
0.956
CosineWithWarmup
0.559
0.817
0.552
0.751
0.550
0.873
0.588
0.955
CosineWithRestarts
0.559
0.820
0.552
0.755
0.552
0.874
0.585
0.953
Cosine (no warmup)
0.561
0.820
0.553
0.751
0.556
0.880
0.590
0.962
seasonal_naive
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
Appendix
Table 12: LR scheduler ablation (trained for 2M iterations with a batch size of 40). Base architecture: d=512,L=10 , H=4 , conv size 32, weaving enabled. WarmupStableDecay : warmup (0.3%), plateau (90%), cosine decay (9.7%). CosineWithRestarts : 4 resets. Sorted by overall CRPS. Bold : best, underline : second-best.
Figure 18: Comparison of TempoPFN normalized CRPS and MASE (3M iterations with a batch size of 40) against other models on Chronos-ZS benchmark.
Model
Params (M)
Training Time
Inference Time
Memory
Parallelization
TiRex ( Auer et al., 2025 )
35
Work: O(TLd2) Depth: O(LT/w) (window w=32 )
AR: O(HLd2) Depth: O(LT/w)
O(d) (streaming)
Low (sequential sLSTM)
TabPFN-TS ( Hoo et al., 2024 )
11
O(T2d)
O(T2d)
O(Td)
Moderate (attention)
TimesFM-2.0 ( Das et al., 2024 )
500
O(T2d)
Direct: O(T2d)
O(Td)
High (transformer)
Chronos ( Ansari et al., 2024 )
9–205
O(T2d)
AR: O(H)
O(Td)
High (transformer)
MOIRAI-MoE ( Liu et al., 2025b )
14–935
O(T2d)
O(T2d)
O(Td)
High (MoE + transformer)
TempoPFN (ours)
40
Work: O(TLd2) Depth: O(LlogT)
Work: O(TLd2) Depth: O(LlogT) Horizon: O(1)
O(d) (streaming) or O(Td) (cached)
Full (sequence-parallel)
Appendix
Table 13: Comparison of time-series foundation models. Complexities are reported with respect to sequence length T , horizon H , embedding dimension d , and layers L . We distinguish total work from parallel depth. TempoPFN benefits from scan-accelerated linear recurrences enabling sublinear depth.
Figure 19: CRPS and MASE average rank of TempoPFN (3M iterations with a per-GPU batch size of 40) on Chronos-ZS benchmark.
Rank
Model
Avg. Win Rate (%)
Skill Score (%)
Median Runtime (s)
Leakage (%)
Failed Tasks (%)
Organization
Zero-shot
1
Chronos-2
90.9
47.3
2.69
0
0
AWS
✓
2
TiRex
82.1
42.6
1.40
1
0
NX-AI
✓
3
TimesFM 2.5
77.1
42.2
16.92
10
0
Google
✓
4
Toto 1.0
69.6
40.7
90.74
8
0
Datadog
✓
5
TempoPFN
65.3
38.3
20.67
0
0
Uni Freiburg / ELLIS
✓
6
Moirai 2.0
63.4
39.3
2.46
28
0
Salesforce
✓
Appendix
Table 14: fev-bench leaderboard based on Scaled Quantile Loss (SQL). Models are ranked by Win Rate and Skill Score. TempoPFN is evaluated with a sign-flip and context-length ensemble at a maximum context of 3072.
Figure 20: Head-to-head comparisons on FEV-Bench using Scaled Quantile Loss (SQL). (a) Win Rate: Percentage of tasks where the row model achieves lower error than the column model (ties count as half-wins); values > 50% indicate the row model is more accurate on average. (b) Skill Score: Average relative error reduction of the row model with respect to the column model; positive values indicate error reduction. Brackets indicate 95% confidence intervals estimated via 1000 bootstrap samples.
Figure 21: Feature-Space Distribution of Real vs. Synthetic Data. UMAP projections of time-series features extracted via tsfresh . Real-world benchmarks (GIFT-Eval, FEV-Bench, Chronos) are shown in cold colors (Blue/Teal) in the foreground, while our synthetic generators are shown in warm colors (Red/Orange/Yellow) in the background.
Figure 36Figure 37Figure 38Figure 39Figure 40
Figure 22: Qualitative comparison between TempoPFN, TiRex and TabPFN-TS on the GIFT-Eval Benchmark. (Left) Total context with prediction window in dashed grey box. (Right) Predictions between TempoPFN, TiRex, and TabPFN-TS.
Dataset
TempoPFN
TiRex
FlowState-9.1M
Toto_Open_Base_1.0
TabPFN-TS
YingLong_50m
Chronos Bolt B
TTM-R2-Finetuned
Moirai L 1.1
Moirai B 1.1
bitbrains_fast_storage/5T/long
0.555
0.558
0.701
0.568
0.752
0.586
0.635
0.796
0.608
0.622
bitbrains_fast_storage/5T/medium
0.516
0.516
0.651
0.525
0.792
0.537
0.631
0.690
0.531
0.553
bitbrains_fast_storage/5T/short
0.334
0.334
0.422
0.306
0.547
0.343
0.375
0.465
0.340
0.341
bitbrains_fast_storage/H/short
0.632
0.683
0.684
0.609
0.655
0.610
0.757
1.007
0.632
0.600
bitbrains_rnd/5T/long
0.602
0.541
0.619
0.501
0.697
0.563
0.643
0.648
0.577
0.566
bitbrains_rnd/5T/medium
0.530
0.500
0.619
0.537
0.701
0.546
0.517
0.670
0.508
0.527
Appendix
Table 15: Normalized CRPS (Continuous Ranked Probability Score) for various zero-shot models on the GiftEval benchmark. Scores are normalized against a seasonal naive baseline, with values less than 1.0 signifying superior probabilistic forecasts. The top two performing models are highlighted.
Dataset
TempoPFN
TiRex
FlowState-9.1M
Toto_Open_Base_1.0
TTM-R2-Finetuned
TabPFN-TS
Chronos Bolt B
YingLong_50m
Moirai L 1.1
Moirai B 1.1
bitbrains_fast_storage/5T/long
0.846
0.808
0.913
0.789
0.827
1.014
0.834
0.886
0.840
0.851
bitbrains_fast_storage/5T/medium
0.896
0.815
1.033
0.807
0.876
1.072
0.871
0.889
0.836
0.860
bitbrains_fast_storage/5T/short
0.716
0.609
0.883
0.591
0.648
0.879
0.662
0.719
0.728
0.697
bitbrains_fast_storage/H/short
0.910
0.825
0.851
0.728
0.927
0.912
0.824
0.856
0.839
0.909
bitbrains_rnd/5T/long
1.005
0.954
1.007
0.953
1.001
1.107
0.970
1.002
0.977
0.985
bitbrains_rnd/5T/medium
1.001
0.967
1.009
0.972
0.997
1.064
0.979
1.004
0.982
0.997
Appendix
Table 16: Normalized MASE scores of different zero-shot models on the GiftEval benchmark. Scores are relative to a seasonal naive baseline, where values below 1.0 indicate better performance. The models achieving the best and second-best scores are highlighted.
Dataset
TempoPFN
TiRex
FlowState-9.1M
Toto_Open_Base_1.0
Chronos Bolt B
TabPFN-TS
YingLong_50m
TTM-R2-Finetuned
Moirai L 1.1
Moirai B 1.1
bitbrains_fast_storage/5T/long
1.000
2.000
8.000
3.000
7.000
9.000
4.000
10.000
5.000
6.000
bitbrains_fast_storage/5T/medium
1.000
2.000
8.000
3.000
7.000
10.000
5.000
9.000
4.000
6.000
bitbrains_fast_storage/5T/short
2.000
3.000
8.000
1.000
7.000
10.000
6.000
9.000
4.000
5.000
bitbrains_fast_storage/H/short
4.000
7.000
8.000
2.000
9.000
6.000
3.000
11.000
5.000
1.000
bitbrains_rnd/5T/long
6.000
2.000
7.000
1.000
8.000
10.000
3.000
9.000
5.000
4.000
bitbrains_rnd/5T/medium
5.000
1.000
8.000
6.000
3.000
10.000
7.000
9.000
2.000
4.000
Appendix
Table 17: CRPS performance summarized by average rank for zero-shot models on the GiftEval benchmark. A lower rank signifies superior probabilistic forecasting performance. The models achieving the first and second-best overall average ranks are highlighted.
Dataset
TempoPFN
TiRex
FlowState-9.1M
Toto_Open_Base_1.0
Chronos Bolt B
TTM-R2-Finetuned
TabPFN-TS
YingLong_50m
Moirai L 1.1
Moirai B 1.1
bitbrains_fast_storage/5T/long
6.000
2.000
9.000
1.000
4.000
3.000
11.000
8.000
5.000
7.000
bitbrains_fast_storage/5T/medium
8.000
2.000
10.000
1.000
5.000
6.000
11.000
7.000
3.000
4.000
bitbrains_fast_storage/5T/short
6.000
2.000
10.000
1.000
4.000
3.000
9.000
7.000
8.000
5.000
bitbrains_fast_storage/H/short
8.000
3.000
5.000
1.000
2.000
10.000
9.000
6.000
4.000
7.000
bitbrains_rnd/5T/long
9.000
2.000
10.000
1.000
3.000
7.000
11.000
8.000
4.000
5.000
bitbrains_rnd/5T/medium
8.000
1.000
10.000
2.000
3.000
5.000
11.000
9.000
4.000
6.000
Appendix
Table 18: MASE performance presented as average ranks for zero-shot models across the GiftEval benchmark. A lower average rank indicates consistently higher accuracy across datasets. The two models with the best overall average ranks are highlighted.
Inspired by recent breakthroughs in large language models for natural language processing, foundation models have emerged as a promising paradigm for zero-shot time series forecasting, enabling accurate predictions on datasets never seen during pre-training. Ranging from tens to hundreds of millions of parameters, these models are pre-trained on vast and diverse collections of time series, learning generalizable representations that support both point and probabilistic forecasting. This approach alleviates the need for dataset-specific model design and manual tuning, offering a unified solution across forecasting problems. In this work, we review the main architectures, pre-training strategies, and optimization methods underpinning these models. We further investigate post-pre-training fine-tuning of selected foundation models to enhance their performance on specific datasets. Our empirical results demonstrate that this step consistently improves forecasting accuracy over the zero-shot baseline.
Morad Laglil, Bertrand Pracca, Emilie Devijver +1
Savoye, 18 bd des Gorgets, 21000, Dijon, France · Univ. Grenoble Alpes, CNRS, Grenoble INP, LIG, Grenoble, 38000, France
We introduce Timer-M1, a pretrained multivariate time series foundation model that learns with primitives for zero-shot forecasting. Across domains, time series share elementary temporal and relational patterns, termed primitives, yet differ in how these primitives manifest and evolve across different contexts. Despite progress in zero-shot and task-general forecasting, existing foundation models may still struggle to generalize to complex real-world scenarios. To this end, we develop a primitive-based data synthesis and pretraining pipeline. The synthesis pipeline generates series with temporal primitives shared across domains and then assembles real and generated series into multivariate samples using relational primitives. Afterwards, samples are organized into episodes by assigning distinct channel roles as target variates, past-only covariates, and known-future covariates, ensuring that the model is optimized on predictable variates using available exogenous information. Technically, Timer-M1 further adapts gated two-dimensional Transformer blocks that dynamically allocate cross-variate attention across layers. Across three large-scale forecasting benchmarks, Timer-M1 ranks first on both FEV and TIME and second on GIFT-Eval among most recent time series foundation models. These results support effective primitive-based pretraining as a route to robust general forecasting technique across domains and task settings.
Haoran Zhang, Haixuan Liu, Xingjian Su +5
School of Software, BNRist, Tsinghua University · ByteDance, Beijing, China
Learning time series foundation models has been shown to be a promising approach for zero-shot time series forecasting across diverse time series domains. Insofar as scaling has been a critical driver of performance of foundation models in other modalities such as language and vision, much recent work on time series foundation modeling has focused on scaling. This has resulted in time series foundation models with hundreds of millions of parameters that are, while performant, inefficient and expensive to use in practice. This paper describes a simple recipe for learning efficient foundation models for zero-shot time series forecasting that are orders of magnitude smaller. We show that large-scale transformers are not necessary: small hybrid models that interleave long convolution and linear RNN layers (in particular DeltaNet layers) can match the performance of larger transformer-based models while being more than a hundred times smaller. We also describe several data augmentation and inference strategies that further improve performance. This recipe results in Reverso, a family of efficient time series foundation models for zero-shot forecasting that significantly push the performance-efficiency Pareto frontier.
Xinghong Fu, Yanhong Li, Georgios Papaioannou +1
Massachusetts Institute of Technology · Allen Institute for AI · Qube Research & Technologies