Time-Aware Prior Fitted Networks for Zero-Shot Forecasting with Exogenous Variables
Organizations: Amazon Science, New York University · Amazon Science
Abstract
In many time series forecasting settings, the target time series is accompanied by exogenous covariates, such as promotions and prices in retail demand; temperature in energy load; calendar and holiday indicators for traffic or sales; and grid load or fuel costs in electricity pricing. Ignoring these exogenous signals can substantially degrade forecasting accuracy, particularly when they drive spikes, discontinuities, or regime and phase changes in the target series. Most current time series foundation models (e.g., Chronos, Sundial, TimesFM, TimeMoE, TimeLLM, and LagLlama) ignore exogenous covariates and make forecasts solely from the numerical time series history, thereby limiting their performance. In this paper, we develop ApolloPFN, a prior-data fitted network (PFN) that is time-aware (unlike prior PFNs) and that natively incorporates exogenous covariates (unlike prior univariate forecasters). Our design introduces two major advances: (i) a synthetic data generation framework that injects realistic temporal patterns, structural changes, and exogenous dependencies into the PFN prior; and (ii) time-aware architectural modifications that embed inductive biases needed to exploit temporal context. We demonstrate that ApolloPFN outperforms existing baselines across several forecasting benchmarks with exogenous information, including M5, electric price forecasting, UCI Air Quality, and Solar Energy datasets.
Figures & tables
| Level | Model | M5(D-B) | M5(W-B) | M5(M-B) | M5(D-S) | M5(W-S) | M5(M-S) |
|---|---|---|---|---|---|---|---|
| State | AutoARIMAX | 0.900 | 1.374 | 2.218 | 0.840 | 1.555 | 3.006 |
| Prophet | 0.525 | 1.685 | 3.223 | 0.859 | 1.945 | 4.593 | |
| Sundial-Base (0) | 0.572 | 2.052 | 2.398 | 0.825 | 1.614 | 3.042 | |
| Chronos-Large (0) | 0.565 | 1.223 | 2.572 | 0.894 | 1.853 | 2.989 | |
| Moirai-Large | 0.794 | 1.533 | 2.336 | 0.879 | 1.681 | 2.780 | |
| TabPFN-TS | 0.527 | 1.251 | 2.599 | 0.883 | 1.522 | 2.864 |
| sCRPS | DE(24) | NP(24) | FR(24) | BE(24) | PJM(24) | DE(48) | NP(48) | FR(48) | BE(48) | PJM(48) |
| AutoARIMAX | ||||||||||
| Prophet | ||||||||||
| Sundial-Base (0) | ||||||||||
| Chronos-Large (0) | 0.047 | |||||||||
| Moirai-Large | ||||||||||
| TabPFN-TS | 0.018 | 0.086 | 0.048 | 0.023 | 0.069 | 0.049 |
| sCRPS | Air Quality | Solar Weather |
|---|---|---|
| AutoARIMAX | ||
| Prophet | ||
| Sundial-Base (0) | ||
| Chronos-Large (0) | ||
| Moirai-Large | ||
| TabPFN-TS |
| sCRPS | M1(M) | M1(Y) | M3(M) | M3(O) | M4(D) | M4(M) | M4(Y) | Tou(M) | Tou(Y) |
|---|---|---|---|---|---|---|---|---|---|
| Sundial-Base | |||||||||
| Chronos-Large | 0.119 | 0.106 | 0.103 | ||||||
| Moirai-Large | 0.135 | 0.093 | |||||||
| TabPFN-TS | |||||||||
| ApolloPFN (ours) | 0.034 | 0.023 | 0.092 | 0.084 |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset | Source |
|---|---|
| M1 | Makridakis and Hibon (1979) |
| M3 | Makridakis and Hibon (2000) |
| M4 | Makridakis et al. (2020) |
| Tourism | Hyndman et al. (2008) |
| M5 | Makridakis et al. (2022) |
| Electricity Price | Lago et al. (2021b) |
| Model | Hyperparameter Details | Code Repo |
|---|---|---|
| AutoARIMAX | Per-series fitting via statsforecast.AutoARIMA . Search: , , , , , , season length based on frequency grain, stepwise with approximation. Covariates: exogenous regressors (NaN imputed with mean per covariate) + sin/cos seasonality features. Quantiles from prediction intervals. | statsforecast |
| Prophet | Per-series fitting with default hyperparameters. Interval width . Covariates: all available features as additional regressors (NaN imputed with mean per covariate). Quantiles from MCMC posterior samples. | prophet |
| Sundial-base | Pre-trained thuml/sundial-base-128m (128M params). 100 sample trajectories via autoregressive generation. Quantiles computed empirically from samples. Univariate only (no covariates). | HuggingFace |
| Chronos-large | Pre-trained amazon/chronos-t5-large (710M params). T5 relative position encoding. Quantiles predicted directly via predict_quantiles() . Univariate only (no covariates). | chronos-forecasting |
| Moirai-large | Pre-trained Moirai-1.1-R-large (311M params). Patch size 16, 100 Monte Carlo samples. Quantiles computed empirically from samples. Covariates: Exogenous regressors. | uni2ts |
| TabPFN-TS | Pre-trained checkpoint tabpfn-v2-regressor-v2_default.ckpt from HuggingFace (11M params). 12-layer Transformer ( , 6 heads, FFN 768, no positional encoding, no dropout). Output: Bar Distribution (5000 bins). -normalization per series. Covariates: Fourier features + exogenous regressors. Preprocessing: QuantileTransformer, StandardScaler, SVD. | Internal 2 2 2 We implement the TabPFN-v2 architecture following the publicly available design specifications and initialize models directly from the released pre-trained checkpoints. We validate our implementation against the official tabpfn-time-series library, confirming agreement in outputs within numerical tolerance. |
| Perturbation | Level | M5(D-B) | M5(W-B) | M5(M-B) | M5(D-S) | M5(W-S) | M5(M-S) |
|---|---|---|---|---|---|---|---|
| None | State | 0.500 | 1.275 | 2.088 | 0.875 | 1.497 | 2.721 |
| SNR (50%, medium) | State | 0.512 | 1.630 | 2.227 | 0.878 | 1.544 | 2.819 |
| SNR (50%, high) | State | 0.576 | 1.726 | 2.436 | 0.888 | 1.564 | 2.867 |
| Partial (50%) | State | 0.657 | 1.795 | 2.455 | 0.896 | 1.566 | 2.868 |
| Gaussian extra (50%) | State | 0.503 | 1.317 | 2.183 | 0.873 | 1.494 | 2.739 |
| AR(1) extra (50%) | State | 0.511 | 1.312 | 2.006 | 0.874 | 1.507 | 2.777 |
| Dataset | 20% PI | 40% PI | 60% PI | 80% PI | ||||
|---|---|---|---|---|---|---|---|---|
| ApolloPFN | TabPFN-TS | ApolloPFN | TabPFN-TS | ApolloPFN | TabPFN-TS | ApolloPFN | TabPFN-TS | |
| BE(24) | 0.214 | 0.148 | 0.434 | 0.325 | 0.622 | 0.521 | 0.813 | 0.747 |
| BE(48) | 0.229 | 0.209 | 0.438 | 0.412 | 0.620 | 0.616 | 0.793 | 0.799 |
| DE(24) | 0.212 | 0.177 | 0.422 | 0.375 | 0.608 | 0.563 | 0.813 | 0.778 |
| DE(48) | 0.206 | 0.201 | 0.398 | 0.387 | 0.608 | 0.583 | 0.816 | 0.782 |
| FR(24) | 0.242 | 0.181 | 0.480 | 0.359 | 0.672 | 0.542 | 0.833 | 0.753 |
| Dataset | 20% PI | 40% PI | 60% PI | 80% PI | ||||
|---|---|---|---|---|---|---|---|---|
| ApolloPFN | TabPFN-TS | ApolloPFN | TabPFN-TS | ApolloPFN | TabPFN-TS | ApolloPFN | TabPFN-TS | |
| BE(24) | 0.014 | 0.053 | 0.034 | 0.075 | 0.022 | 0.079 | 0.013 | 0.053 |
| BE(48) | 0.029 | 0.009 | 0.038 | 0.012 | 0.020 | 0.016 | 0.008 | 0.001 |
| DE(24) | 0.012 | 0.023 | 0.022 | 0.025 | 0.008 | 0.037 | 0.013 | 0.022 |
| DE(48) | 0.006 | 0.001 | 0.002 | 0.013 | 0.008 | 0.017 | 0.016 | 0.018 |
| FR(24) | 0.042 | 0.019 | 0.080 | 0.041 | 0.072 | 0.058 | 0.033 | 0.048 |