Zero-shot time-series foundation models against the Khepri v1.5 baselines for 96-hour carbon-intensity forecasts in nine Nordic bidding zones: a preregistered comparison
Organizations: EcoDeco AS, Norway
Abstract
We compare three zero-shot time-series foundation models (Timestral-5M, Timestral-209M and Chronos-2, 8192-hour context, no covariates) with the Khepri v1.5 SARIMA and GBM baselines, with their code and hyperparameters unchanged, for 96-hour forecasts of hourly carbon intensity in the Nordic bidding zones NO1-NO5 and SE1-SE4. The design, the analysis code and a minimum detectable effect (MDE) were timestamped before any foundation-model forecast by an OpenTimestamps proof (Bitcoin-confirmed) and the GPG-signed tag prereg-v1; the proofs are archived in Zenodo (DOI 10.5281/zenodo.23881315). The preregistration was also submitted to OSF Registries 2026-11-02 07:49 UTC (osf.io/hncz5; embargo requested, pending OSF approval). On the 2025 test (361 daily origins, contamination not ruled out) all three models have a lower zone-averaged MAPE than both baselines, by 0.952 to 1.347 percentage points (Diebold-Mariano with the Harvey-Leybourne-Newbold correction, all p < 0.0001), and each beats SARIMA in 9 of 9 zones. On a post-release holdout of 14 origins from 2026-10-16 the point estimates have the same sign (0.729 to 1.326 percentage points lower), but all lie within the preregistered MDE of 2.989 points and are reported as not determined. For calibration (RQ4) the preregistration locked a Diebold-Mariano-HLN test on CRPS; the MDE applies to MAPE only. Holdout CRPS point estimates (n = 14) are 1.1114 (Timestral-5M), 1.0733 (Timestral-209M) and 1.1198 (Chronos-2) against 1.4165 gCO2-eq/kWh for SARIMA (each p < 0.0001). The SARIMA quantiles are constructed from the fitted model's Gaussian predictive distribution (deviation D-03), not taken from a native v1.5 forecast, so the difference may partly reflect the SARIMA intervals. The contamination indicator does not exceed its preregistered threshold for any comparison; with 14 holdout origins it has low power (HAC standard errors 0.124 to 0.360 pp).
Figures & tables
| Arm | Specification |
|---|---|
| Seasonal naive (week) | f(o+h) = y(o+h 168 h), h = 1…96. A reference arm added for this study (D-04); it enters no test. |
| SARIMA | Khepri v1.5, unchanged: SARIMAX (2,0,1)(1,1,0,24), enforce_stationarity=False , enforce_invertibility=False , fit(disp=False, maxiter=50) once on the training data; per origin apply() on a 45-day window ending at the origin and a 96-step forecast; diurnal persistence substituted and flagged if a forecast fails. |
| GBM | Khepri v1.5, unchanged: LightGBM with 200 estimators, 31 leaves, learning rate 0.05, seeds 0, 1 and 2 averaged; features day of week, month, CI at the origin, 24 lags and horizon; trained once on the last 365 daily origins whose 96-h targets lie at or before train_end . |
| Timestral-5M | mistralai/Timestral-5M at revision bdd20cfc15a3fea0e4cb5ccef37f8d3ca7aa335f, package timestral 1.0.0 (github.com/mistralai/timestral at a534bdbd244d64f3d9d743f46c7bf6fc330ed4ff), maximum context 8192, 96-step forecast with library defaults. |
| Timestral-209M | mistralai/Timestral-209M at revision 9874d252ab0aae16f7f7c999f98f98d6ad7d8365, as above. |
| Chronos-2 | amazon/chronos-2 at revision 29ec3766d36d6f73f0696f85560a422f50e8498c, chronos-forecasting 2.3.2, predict_quantiles with prediction length 96 and context length 8192. |
| ID | Deviation | Effect |
|---|---|---|
| D-01 | Prediction-keeping driver around the unchanged v1.5 forecasting code | Validated against stored v1.5 runs (Section 3.4) |
| D-02 | NO1 SARIMA fit not bit-reproducible; cause not found | Pre-registration validation: NO1 retro SARIMA mean MAPE 2.4596 vs. 2.4656 |
| D-03 | SARIMA quantiles from the Gaussian predictive distribution | Point forecast unchanged |
| D-04 | Seasonal naive (week) added as a new reference arm | Enters no test |
| D-05 | MDE from stored v1.5 per-origin metric rows | No 2025 predictions were stored |
| D-06 | Holdout training rule mapped by parameters only | SARIMA training span 2.79 y (NO), 1.79 y (SE) vs. 3.0 and 2.0 y |
| TSFM | baseline | MAPE TSFM | MAPE baseline | diff. (pp) | HLN stat. |
|---|---|---|---|---|---|
| Timestral-5M | SARIMA | 6.253 | 7.561 | ||
| Timestral-5M | GBM | 6.253 | 7.347 | ||
| Timestral-209M | SARIMA | 6.213 | 7.561 | ||
| Timestral-209M | GBM | 6.213 | 7.347 | ||
| Chronos-2 | SARIMA | 6.395 | 7.561 | ||
| Chronos-2 | GBM | 6.395 | 7.347 |
| zone | snaive (week) | SARIMA | GBM | Timestral-5M | Timestral-209M | Chronos-2 |
|---|---|---|---|---|---|---|
| NO1 | 2.969 | 2.460 | 2.307 | 2.108 | 2.120 | 2.131 |
| NO2 | 4.194 | 3.964 | 3.423 | 3.151 | 3.111 | 3.182 |
| NO3 | 8.139 | 6.874 | 6.583 | 5.868 | 5.773 | 5.910 |
| NO4 | 10.104 | 7.242 | 10.158 | 6.171 | 6.133 | 6.339 |
| NO5 | 1.029 | 0.722 | 1.256 | 0.605 | 0.593 | 0.629 |
| SE1 | 14.322 | 11.638 | 10.758 | 10.096 | 10.031 | 10.266 |
| TSFM | baseline | TSFM | base | diff. | stat. | verdict | ||
|---|---|---|---|---|---|---|---|---|
| Timestral-5M | SARIMA | 8.415 | 9.332 | 0.0693 | 0.1385 | not determined | ||
| Timestral-5M | GBM | 8.415 | 9.223 | 0.0002 | 0.0012 | not determined | ||
| Timestral-209M | SARIMA | 8.005 | 9.332 | 0.0023 | 0.0068 | not determined | ||
| Timestral-209M | GBM | 8.005 | 9.223 | 0.0001 | 0.0001 | not determined | ||
| Chronos-2 | SARIMA | 8.494 | 9.332 | 0.0810 | 0.1385 | not determined | ||
| Chronos-2 | GBM | 8.494 | 9.223 | 0.0003 | 0.0012 | not determined |
| arm | CRPS | coverage 80 % | coverage 90 % |
|---|---|---|---|
| SARIMA | 1.4165 | 0.835 | 0.911 |
| Timestral-5M | 1.1114 | 0.838 | 0.915 |
| Timestral-209M | 1.0733 | 0.838 | 0.921 |
| Chronos-2 | 1.1198 | 0.806 | 0.905 |
Time Series Forecasting
Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.
See all 939 papers