TimeES: Probabilistic and Deterministic Time Series Forecasting via Evolutionary Spectra
Organizations: The University of Tokyo
Abstract
Real-world time series are inherently non-stationary, with trends, periodic patterns, and uncertainty evolving over time. While the Fourier domain offers a natural lens to model time series, current deep learning approaches do not explicitly model evolution and randomness in the Fourier spectra, which limits their ability to accurately predict both the expected trajectory and its uncertainty in non-stationary time series. Motivated by Evolutionary Spectra (ES) theory, we propose TimeES, a general framework that enables probabilistic and deterministic forecasting via the evolutionary spectra theory. Specifically, we derive a parameterizable evolutionary spectra formulation, recasting non-stationary random process modeling as learning an evolving representation modulated by random variables. Furthermore, we reduce the complexity of the estimated spectra from O(NM) to O(NK), where K << M/2, by exploiting Hermitian symmetry and spectral energy sparsity for frequency selection. Based on a simple linear backbone, our proposed TimeES achieves consistent state-of-the-art performance across both deterministic and probabilistic forecasting tasks, with high efficiency and interpretability. Code is available at: https://github.com/wwy155/TimeES.
Figures & tables
| Interpretation | ||
|---|---|---|
| Deterministic | ||
| Probabilistic |
| Method | ||
|---|---|---|
| iDFT | ||
| iSTFT | ||
| DES (ours) |
| Models | Datasets | ETTh1 | ETTh2 | ETTm1 | ETTm2 | ECL | EXG | ILI | Solar | Traffic |
|---|---|---|---|---|---|---|---|---|---|---|
| TimeGrad | CRPS | 0.606 | 1.212 | 0.647 | 0.775 | 0.397 | 0.826 | 1.140 | 0.293 | 0.407 |
| (2021) | MSE | 1.062 | 3.462 | 1.218 | 1.690 | 0.505 | 1.567 | 4.197 | 0.475 | 0.983 |
| CSDI | CRPS | 0.492 | 0.647 | 0.524 | 0.817 | 0.577 | 0.855 | 1.244 | 0.432 | 1.418 |
| (2022) | MSE | 0.949 | 1.226 | 1.002 | 1.723 | 1.007 | 1.701 | 4.515 | 0.763 | 1.731 |
| TimeDiff | CRPS | 0.465 | 0.471 | 0.464 | 0.316 | 0.750 | 0.433 | 1.153 | 0.700 | 0.771 |
| (2023) | MSE | 0.517 | 0.456 | 0.537 | 0.268 | 0.879 | 0.402 | 3.958 | 0.821 | 1.350 |
| Models Autoformer | FEDformer | TimesNet | FITS | FreTS | PatchTST | iTransformer | TimeES (Ours) | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Datasets | S.V. | M.V. | S.V. | M.V. | S.V. | M.V. | S.V. | M.V. | S.V. | M.V. | S.V. | M.V. | S.V. | M.V. | S.V. | M.V. |
| ETTm1 | 1.176 | 0.580 | 0.840 | 0.448 | 0.811 | 0.400 | 0.793 | 0.418 | 0.755 | 0.414 | 0.763 | 0.386 | 0.778 | 0.407 | 0.740 | 0.390 |
| ETTm2 | 0.498 | 0.327 | 0.495 | 0.305 | 0.480 | 0.291 | 0.478 | 0.285 | 0.452 | 0.323 | 0.462 | 0.281 | 0.487 | 0.288 | 0.434 | 0.278 |
| ETTh1 | 0.925 | 0.496 | 0.882 | 0.440 | 0.929 | 0.458 | 1.155 | 0.534 | 1.010 | 0.504 | 0.970 | 0.469 | 0.968 | 0.454 | 0.878 | 0.440 |
| ETTh2 | 0.700 | 0.450 | 0.713 | 0.437 | 0.659 | 0.414 | 0.656 | 0.396 | 0.596 | 0.500 | 0.633 | 0.387 | 0.682 | 0.383 | 0.583 | 0.379 |
| ECL | 0.679 | 0.227 | 0.610 | 0.214 | 0.591 | 0.193 | 0.587 | 0.328 | 0.530 | 0.205 | 0.543 | 0.216 | 0.601 | 0.178 | 0.516 | 0.197 |
| Variants | S.V. (MSE) | M.V. (MSE) | P.F. (CRPS) |
|---|---|---|---|
| w/o DES | 0.974±0.014 | 0.554±0.006 | |
| w/o E | 1.282±0.012 | 0.604±0.010 | 0.466±0.009 |
| TimeES | 0.929±0.009 | 0.485±0.003 | 0.349±0.007 |
| Model | PatchTST | +TimeES | Improvement | iTransformer | +TimeES | Improvement | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Metrics | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE |
| 96 | 0.414 | 0.419 | 0.385 | 0.402 | 6.89% | 4.15% | 0.386 | 0.405 | 0.384 | 0.402 | 0.56% | 0.78% |
| 192 | 0.460 | 0.445 | 0.434 | 0.437 | 5.65% | 1.86% | 0.441 | 0.436 | 0.433 | 0.428 | 1.88% | 1.77% |
| 336 | 0.501 | 0.466 | 0.451 | 0.457 | 9.99% | 2.02% | 0.487 | 0.458 | 0.475 | 0.451 | 2.44% | 1.51% |
| 720 | 0.500 | 0.488 | 0.482 | 0.479 | 3.68% | 1.82% | 0.503 | 0.491 | 0.474 | 0.480 | 5.83% | 2.18% |
| Avg. | 0.469 | 0.455 | 0.438 | 0.444 | 6.56% | 2.42% | 0.454 | 0.448 | 0.441 | 0.440 | 2.84% | 1.59% |
Appendix figures & tables22 assets
Supplementary material from the paper’s appendix.
Appendix
| Tasks | Benchmarks | Metrics | Input | Output |
| Long-term Forecasting (Single variate and Multivariate) | ETT{m1, m2, h1, h2}, ECL, EXG, Traffic, Weather, Solar | MSE, MAE | 96 | {96, 192, 336, 720} |
| Probabilistic Forecasting | ETT{m1, m2, h1, h2}, ECL, EXG, Traffic, ILI, Solar | CRPS, QICE, MSE, MAE | 168 | 36 (ILI), 192 (others) |
| Classification | UEA (5 subsets) | Accuracy | 315-2500 | - |
| Single-Variate Long-term Forecasting | ||||||||
|---|---|---|---|---|---|---|---|---|
| Datasets | ETTm1 | ETTm2 | ETTh1 | ETTh2 | ||||
| Variants | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE |
| TimeES | 0.893 | 0.619 | 0.590 | 0.553 | 0.929 | 0.670 | 0.618 | 0.585 |
| w/o E | 1.306 | 0.792 | 0.673 | 0.587 | 1.282 | 0.840 | 0.785 | 0.676 |
| w/o DES | 0.966 | 0.662 | 0.639 | 0.569 | 0.974 | 0.699 | 0.744 | 0.646 |
| Multi-Variate Long-term Forecasting | ||||||||
| Dataset | Horizon | KNF | Koopa | SKOLR | TimeBridge | TimeMixer | TimeXer | TimeES |
|---|---|---|---|---|---|---|---|---|
| ETTm1 | 96 | 0.691 | 0.331 | 0.342 | 0.324 | 0.344 | 0.387 | 0.328 |
| 192 | 0.707 | 0.377 | 0.397 | 0.365 | 0.368 | 0.441 | 0.364 | |
| 336 | 0.718 | 0.402 | 0.445 | 0.395 | 0.394 | 0.514 | 0.402 | |
| 720 | 0.742 | 0.466 | 0.536 | 0.457 | 0.455 | 0.592 | 0.465 | |
| ETTm2 | 96 | 0.230 | 0.182 | 0.194 | 0.178 | 0.178 | 0.364 | 0.176 |
| 192 | 0.274 | 0.245 | 0.256 | 0.245 | 0.246 | 0.526 | 0.240 |
| Autoformer | FEDformer | TimesNet | FITS | FreTS | PatchTST | iTransformer | TimeES | |
|---|---|---|---|---|---|---|---|---|
| EthanolConcentration | 31.60 | 31.20 | 35.70 | 25.10 | 24.79 | 36.42 | 31.60 | 36.81 |
| MotorImagery | 48.40 | 45.00 | 50.40 | 49.60 | 50.40 | 52.80 | 55.60 | 52.00 |
| SelfRegulationSCP1 | 84.00 | 88.70 | 91.80 | 72.70 | 89.56 | 81.78 | 88.39 | 92.83 |
| StandWalkJump | 29.33 | 30.67 | 37.33 | 38.67 | 45.33 | 41.33 | 37.33 | 46.67 |
| UWaveGestureLibrary | 85.90 | 85.30 | 85.30 | 61.81 | 85.10 | 80.19 | 87.50 | 86.50 |
| Avg. | 55.85 | 56.17 | 60.11 | 49.58 | 59.04 | 58.50 | 60.08 | 62.96 |
| Models | Datasets | ETTh1 | ETTh2 | ETTm1 | ETTm2 | ECL | EXG | ILI | Solar | Traffic |
|---|---|---|---|---|---|---|---|---|---|---|
| TimeGrad | CRPS | 0.606 | 1.212 | 0.647 | 0.775 | 0.397 | 0.826 | 1.140 | 0.293 | 0.407 |
| (2021) | QICE | 6.731 | 9.488 | 6.693 | 6.962 | 7.118 | 9.464 | 6.519 | 7.378 | 4.581 |
| CSDI | CRPS | 0.492 | 0.647 | 0.524 | 0.817 | 0.577 | 0.855 | 1.244 | 0.432 | 1.418 |
| (2022) | QICE | 3.107 | 5.331 | 2.828 | 8.106 | 7.506 | 7.864 | 7.693 | 9.957 | 13.613 |
| TimeDiff | CRPS | 0.465 | 0.471 | 0.464 | 0.316 | 0.750 | 0.433 | 1.153 | 0.700 | 0.771 |
| (2023) | QICE | 14.931 | 14.813 | 14.795 | 13.385 | 15.466 | 14.556 | 14.942 | 14.914 | 15.439 |
| Models | Datasets | ETTh1 | ETTh2 | ETTm1 | ETTm2 | ECL | EXG | ILI | Solar | Traffic |
|---|---|---|---|---|---|---|---|---|---|---|
| TimeGrad | MAE | 0.813 | 1.496 | 0.831 | 0.967 | 0.504 | 1.058 | 1.414 | 0.446 | 0.535 |
| (2021) | MSE | 1.062 | 3.462 | 1.218 | 1.690 | 0.505 | 1.567 | 4.197 | 0.475 | 0.983 |
| CSDI | MAE | 0.708 | 0.900 | 0.752 | 1.069 | 0.822 | 1.081 | 1.481 | 0.675 | 0.925 |
| (2022) | MSE | 0.949 | 1.226 | 1.002 | 1.723 | 1.007 | 1.701 | 4.515 | 0.763 | 1.731 |
| TimeDiff | MAE | 0.479 | 0.485 | 0.477 | 0.333 | 0.764 | 0.446 | 1.169 | 0.713 | 0.784 |
| (2023) | MSE | 0.517 | 0.456 | 0.537 | 0.268 | 0.879 | 0.402 | 3.958 | 0.821 | 1.350 |
| Model | Autoformer | FEDformer | TimesNet | FITS | FreTS | PatchTST | iTransformer | TimeES | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Metrics | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | |
| ETTm1 | 96 | 0.886 | 0.636 | 0.713 | 0.598 | 0.682 | 0.551 | 0.643 | 0.511 | 0.600 | 0.508 | 0.624 | 0.508 | 0.622 | 0.517 | 0.607 | 0.505 |
| 192 | 1.252 | 0.798 | 0.770 | 0.614 | 0.748 | 0.568 | 0.745 | 0.545 | 0.710 | 0.553 | 0.710 | 0.545 | 0.730 | 0.554 | 0.690 | 0.534 | |
| 336 | 0.974 | 0.681 | 0.886 | 0.664 | 0.810 | 0.592 | 0.811 | 0.573 | 0.788 | 0.592 | 0.786 | 0.585 | 0.829 | 0.609 | 0.770 | 0.568 | |
| 720 | 1.594 | 0.884 | 0.992 | 0.705 | 1.006 | 0.689 | 0.974 | 0.633 | 0.921 | 0.649 | 0.931 | 0.646 | 0.931 | 0.642 | 0.893 | 0.619 | |
| Avg | 1.177 | 0.750 | 0.840 | 0.645 | 0.812 | 0.600 | 0.793 | 0.566 | 0.755 | 0.576 | 0.763 | 0.571 | 0.778 | 0.581 | 0.740 | 0.556 | |
| Model | Autoformer | FEDformer | TimesNet | FITS | FreTS | PatchTST | iTransformer | TimeES | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Metrics | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | |
| ETTm1 | 96 | 0.475 | 0.505 | 0.379 | 0.419 | 0.338 | 0.375 | 0.353 | 0.378 | 0.352 | 0.385 | 0.329 | 0.367 | 0.334 | 0.368 | 0.328 | 0.366 |
| 192 | 0.553 | 0.496 | 0.426 | 0.441 | 0.374 | 0.387 | 0.389 | 0.400 | 0.390 | 0.407 | 0.367 | 0.385 | 0.377 | 0.391 | 0.364 | 0.384 | |
| 336 | 0.621 | 0.537 | 0.445 | 0.459 | 0.410 | 0.411 | 0.447 | 0.441 | 0.421 | 0.423 | 0.410 | 0.399 | 0.426 | 0.420 | 0.402 | 0.402 | |
| 720 | 0.671 | 0.561 | 0.543 | 0.490 | 0.478 | 0.450 | 0.482 | 0.445 | 0.495 | 0.475 | 0.439 | 0.454 | 0.491 | 0.459 | 0.465 | 0.445 | |
| Avg | 0.580 | 0.525 | 0.448 | 0.452 | 0.400 | 0.406 | 0.418 | 0.416 | 0.414 | 0.423 | 0.386 | 0.401 | 0.407 | 0.410 | 0.390 | 0.400 | |
| Single-Variates | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Model | PatchTST | +TimeES | Improvements | iTransformer | +TimeES | Improvements | |||||||
| Metrics | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | |
| ETTm1 | 96 | 0.624 | 0.508 | 0.603 | 0.505 | 3.37% | 0.63% | 0.622 | 0.517 | 0.612 | 0.496 | 1.56% | 4.02% |
| 192 | 0.710 | 0.545 | 0.704 | 0.533 | 0.88% | 2.35% | 0.730 | 0.554 | 0.721 | 0.544 | 1.35% | 1.77% | |
| 336 | 0.786 | 0.585 | 0.780 | 0.574 | 0.70% | 1.80% | 0.829 | 0.609 | 0.787 | 0.576 | 5.07% | 5.39% | |
| 720 | 0.931 | 0.646 | 0.921 | 0.635 | 1.15% | 1.63% | 0.931 | 0.642 | 0.923 | 0.634 | 0.86% | 1.23% | |
| Channel | Trend (DC) | Weekly (168h) | Daily (24h) | 12h | 8h |
|---|---|---|---|---|---|
| HUFL (high useful load) | 1.95 | 0.46 | 1.60 | 0.80 | 0.95 |
| HULL (high useless load) | 2.55 | 0.83 | 1.34 | 0.71 | 0.78 |
| MUFL (middle useful load) | 2.02 | 0.44 | 1.57 | 0.79 | 0.94 |
| MULL (middle useless load) | 2.56 | 0.84 | 1.32 | 0.70 | 0.81 |
| LUFL (low useful load) | 1.63 | 0.84 | 1.81 | 0.94 | 0.96 |
| LULL (low useless load) | 2.07 | 1.05 | 1.34 | 0.83 | 1.10 |
| Dataset | Fixed grid | Learnable |
|---|---|---|
| ETTh1 | 0.363 0.006 | 0.372 0.008 |
| Off-grid synthetic | 0.156 0.018 | 0.157 0.017 |
| Setting | Model | ETTh1 | ETTh2 | ETTm1 | ETTm2 |
|---|---|---|---|---|---|
| , | KooNPro | 0.527 | 0.351 | 0.395 | 0.277 |
| TimeES | 0.355 | 0.202 | 0.375 | 0.175 | |
| , | KooNPro | 0.398 | 0.380 | 0.352 | 0.290 |
| TimeES | 0.292 | 0.206 | 0.175 | 0.132 | |
| , | KooNPro | 3.079 | 2.922 | 3.465 | 3.656 |
| TimeES | 0.349 | 0.320 | 0.310 | 0.230 |
| Dataset | ETTh1 | ETTh2 | ETTm1 | ETTm2 | ECL | EXG | Traffic | Weather | Solar | ILI | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Coverage | 0.885 | 0.862 | 0.914 | 0.900 | 0.915 | 0.992 | 0.904 | 0.965 | 0.928 | 0.963 | 0.923 |
| Input length | 24 | 72 | 96 | 168 (default) | 336 |
|---|---|---|---|---|---|
| ETTh1 | 0.356 | 0.336 | 0.339 | 0.349 | 0.369 |
| ETTm1 | 0.398 | 0.321 | 0.298 | 0.310 | 0.299 |
Non-Stationary Time Series Forecasting
Non-stationary time series forecasting predicts future values as patterns, distributions, and relationships change over time.
See all 93 papers