Continuous Horizon Forecasting
Organizations: Forecasting Science, Amazon, New York, NY · Mohamed bin Zayed University of Artificial Intelligence, UAE · University of California, Berkeley, CA
Abstract
Multi-horizon forecasting has been dominated by discrete forecast strategies that treat model outputs as fixed-vectors or recurrently create them. In this paper, we introduce the Continuous Horizon (CH) strategy that generalizes upon recursive, direct, and multi-step strategies by redefining forecasts as a continuous operator and the forecast horizon as a query variable rather than an index. We present the theoretical foundations for the CH strategy, a Universal Operator Approximation Theorem, and a continuous-horizon bias-variance decomposition that formally proves the estimation variance improvements of CH over the discrete strategies. We validate the CH strategy empirically through a simulation study, and on 16 large-scale datasets from the M-series benchmark, showing how CH achieves competitive RMSE accuracy, while uniquely enabling parallel inference, targeted horizon querying, and interpolation and extrapolation across unseen horizons.
Figures & tables
| Property | Recursive | Direct | Multi-Step | Continuous Horizon |
| Low Bias | ✗ | ✓ | ✓ | ✓ |
| Low Variance | ✗ | ✗ | ✗ | ✓ |
| Inference parallelism | ✗ | ✓ | ✓ | ✓ |
| Targeted horizon querying | ✗ | ✓ | ✗ | ✓ |
| Unseen Horizon inter/extrapolation | ✗ | ✗ | ✗ | ✓ |
| Joint horizon modeling | ✗ | ✗ | ✓ | ✓ |
| Seen ( ) | Interpolation | Extrapolation | ||||||||
| Process | CH | MS | DIR | REC | CH | MS | DIR | REC | CH | REC |
| Mean-reverting | 0.873 | 0.934 | 0.915 | 0.939 | 0.872 | 0.932 | 0.908 | 0.937 | 1.211 | 1.280 |
| Aperiodic | 0.733 | 0.759 | 0.754 | 0.763 | 0.730 | 0.753 | 0.746 | 0.758 | 1.054 | 1.110 |
| Periodic | 0.568 | 0.581 | 0.581 | 0.617 | 0.568 | 0.576 | 0.575 | 0.613 | 0.607 | 0.649 |
| QP (slow) | 0.623 | 0.640 | 0.639 | 0.671 | 0.623 | 0.636 | 0.633 | 0.670 | 0.756 | 0.780 |
| QP (fast) | 0.689 | 0.716 | 0.712 | 0.743 | 0.688 | 0.712 | 0.705 | 0.740 | 0.940 | 0.975 |
| LSTM | CNN | Transformer | S4 | StatsForecast | |||||||||||||||
| Freq | CH | MS | DIR | REC | CH | MS | DIR | REC | CH | MS | DIR | REC | CH | MS | DIR | REC | ETS | ARIMA | |
| \csvreader []external_paper/tables/mse_main_table.csv Dataset= \dset , Frequency= \frequency , LSTM_OURS= \lstmours , LSTM_OURS_SE= \lstmoursse , LSTM_MS= \lstmms , LSTM_MS_SE= \lstmmsse , LSTM_DR= \lstmdr , LSTM_DR_SE= \lstmdrse , LSTM_RC= \lstmrc , LSTM_RC_SE= \lstmrcse , CNN_OURS= \cnnours , CNN_OURS_SE= \cnnoursse , CNN_MS= \cnnms , CNN_MS_SE= \cnnmsse , CNN_DR= \cnndr , CNN_DR_SE= \cnndrse , CNN_RC= \cnnrc , CNN_RC_SE= \cnnrcse , Transformer_OURS= \transours , Transformer_OURS_SE= \transoursse , Transformer_MS= \transms , Transformer_MS_SE= \transmsse , Transformer_DR= \transdr , Transformer_DR_SE= \transdrse , Transformer_RC= \transrc , Transformer_RC_SE= \transrcse , S4_OURS= \sfourours , S4_OURS_SE= \sfouroursse , S4_MS= \sfourms , S4_MS_SE= \sfourmsse , S4_DR= \sfourdr , S4_DR_SE= \sfourdrse , S4_RC= \sfourrc , S4_RC_SE= \sfourrcse , ETS= \ets , ARIMA= \arima \dset | \frequency | \lstmours | \lstmms | \lstmdr | \lstmrc | \cnnours | \cnnms | \cnndr | \cnnrc | \transours | \transms | \transdr | \transrc | \sfourours | \sfourms | \sfourdr | \sfourrc | \ets | \arima |
| ( \lstmoursse ) | ( \lstmmsse ) | ( \lstmdrse ) | ( \lstmrcse ) | ( \cnnoursse ) | ( \cnnmsse ) | ( \cnndrse ) | ( \cnnrcse ) | ( \transoursse ) | ( \transmsse ) | ( \transdrse ) | ( \transrcse ) | ( \sfouroursse ) | ( \sfourmsse ) | ( \sfourdrse ) | ( \sfourrcse ) | ( - ) | ( - ) | ||
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
| Regime | Kernel | |||
| Mean-reverting | 3 | 12 | 12 | |
| Aperiodic | – | – | 16 | |
| Periodic | 3 | 12 | – | |
| QP (slow) | 3 | 12 | 48 | |
| QP (fast) | 3 | 12 | 24 |
| Regime | Basis composition | Seen | Interp. | Extrap. |
| Mean-reverting | polynomial only | 1.092 | 1.100 | 1.610 |
| Fourier only | 1.092 | 1.101 | 1.226 | |
| table only | 1.091 | 1.099 | 1.153 | |
| polynomial Fourier | 1.094 | 1.103 | 1.263 | |
| polynomial table | 1.091 | 1.099 | 1.333 | |
| Fourier table | 1.091 | 1.099 | 1.188 |
| Regime | Seen | Interp. | Extrap. | |
| Mean-reverting | 1.091 | 1.102 | 1.267 | |
| 1.093 | 1.103 | 1.234 | ||
| 1.093 | 1.102 | 1.229 | ||
| 1.092 | 1.101 | 1.217 | ||
| 1.092 | 1.100 | 1.203 | ||
| Aperiodic | 0.716 | 0.716 | 1.059 |
| Regime | Frequency representation | Seen | Interp. | Extrap. | Sep. |
| Mean-reverting | fixed Fourier dictionary | 1.092 | 1.101 | 1.234 | ✓ |
| learnable Fourier (anchor) | 1.092 | 1.101 | 1.217 | ✓ | |
| frequency embedding, shared | 1.093 | 1.102 | 1.213 | ✓ | |
| frequency embedding, per-series | 1.092 | 1.100 | 1.204 | ✗ | |
| frequency embedding only (no Fourier) | 1.091 | 1.099 | 1.312 | ✓ | |
| gated composite | 1.092 | 1.100 | 1.212 | ✗ |
| Frequency | Seasonality | Horizon | Series | Min Length | Max Length | % Erratic | |
| M1 | Monthly | 12 | 18 | 617 | 48 | 150 | 0 |
| Quarterly | 4 | 8 | 203 | 18 | 114 | 0 | |
| Yearly | 1 | 6 | 181 | 15 | 58 | 0 | |
| M3 | Other | 4 | 8 | 174 | 71 | 104 | 0 |
| Monthly | 12 | 18 | 1428 | 66 | 144 | 2 | |
| Quarterly | 4 | 8 | 756 | 24 | 72 | 1 |
| Hyperparameter | Values |
|---|---|
| Single GPU SGD Batch Size * | {4, 8, 16} |
| Initial learning rate | loguniform(1e-4, 5e-2) |
| Maximum Training steps | 45,000 |
| Learning rate decay | 0.1 |
| Learning rate step size | 15,000 |
| Scaler type | Standard |
| LSTM | CNN | Transformer | S4 | |||||||||||||
| Model | CH | MS | DIR ∗ | REC | CH | MS | DIR ∗ | REC | CH | MS | DIR ∗ | REC | CH | MS | DIR ∗ | REC |
| Global model | 78,482 | 77,865 | 7,178 | 78,482 | 63,154 | 65,125 | 9,477 | 63,154 | 81,746 | 80,868 | 8,641 | 81,746 | 85,714 | 86,674 | 13,961 | 85,714 |
| LSTM | CNN | Transformer | S4 | StatsForecast | |||||||||||||||
| Freq | CH | MS | DIR | REC | CH | MS | DIR | REC | CH | MS | DIR | REC | CH | MS | DIR | REC | ETS | ARIMA | |
| \csvreader []external_paper/tables/smape_main_table.csv Dataset= \dset , Frequency= \frequency , LSTM_OURS= \lstmours , LSTM_OURS_SE= \lstmoursse , LSTM_MS= \lstmms , LSTM_MS_SE= \lstmmsse , LSTM_DR= \lstmdr , LSTM_DR_SE= \lstmdrse , LSTM_RC= \lstmrc , LSTM_RC_SE= \lstmrcse , CNN_OURS= \cnnours , CNN_OURS_SE= \cnnoursse , CNN_MS= \cnnms , CNN_MS_SE= \cnnmsse , CNN_DR= \cnndr , CNN_DR_SE= \cnndrse , CNN_RC= \cnnrc , CNN_RC_SE= \cnnrcse , Transformer_OURS= \transours , Transformer_OURS_SE= \transoursse , Transformer_MS= \transms , Transformer_MS_SE= \transmsse , Transformer_DR= \transdr , Transformer_DR_SE= \transdrse , Transformer_RC= \transrc , Transformer_RC_SE= \transrcse , S4_OURS= \sfourours , S4_OURS_SE= \sfouroursse , S4_MS= \sfourms , S4_MS_SE= \sfourmsse , S4_DR= \sfourdr , S4_DR_SE= \sfourdrse , S4_RC= \sfourrc , S4_RC_SE= \sfourrcse , ETS= \ets , ARIMA= \arima \dset | \frequency | \lstmours | \lstmms | \lstmdr | \lstmrc | \cnnours | \cnnms | \cnndr | \cnnrc | \transours | \transms | \transdr | \transrc | \sfourours | \sfourms | \sfourdr | \sfourrc | \ets | \arima |
| ( \lstmoursse ) | ( \lstmmsse ) | ( \lstmdrse ) | ( \lstmrcse ) | ( \cnnoursse ) | ( \cnnmsse ) | ( \cnndrse ) | ( \cnnrcse ) | ( \transoursse ) | ( \transmsse ) | ( \transdrse ) | ( \transrcse ) | ( \sfouroursse ) | ( \sfourmsse ) | ( \sfourdrse ) | ( \sfourrcse ) | ( - ) | ( - ) | ||
| LSTM | CNN | Transformer | S4 | StatsForecast | |||||||||||||||
| Freq | CH | MS | DIR | REC | CH | MS | DIR | REC | CH | MS | DIR | REC | CH | MS | DIR | REC | ETS | ARIMA | |
| \csvreader []external_paper/tables/mase_main_table.csv Dataset= \dset , Frequency= \frequency , LSTM_OURS= \lstmours , LSTM_OURS_SE= \lstmoursse , LSTM_MS= \lstmms , LSTM_MS_SE= \lstmmsse , LSTM_DR= \lstmdr , LSTM_DR_SE= \lstmdrse , LSTM_RC= \lstmrc , LSTM_RC_SE= \lstmrcse , CNN_OURS= \cnnours , CNN_OURS_SE= \cnnoursse , CNN_MS= \cnnms , CNN_MS_SE= \cnnmsse , CNN_DR= \cnndr , CNN_DR_SE= \cnndrse , CNN_RC= \cnnrc , CNN_RC_SE= \cnnrcse , Transformer_OURS= \transours , Transformer_OURS_SE= \transoursse , Transformer_MS= \transms , Transformer_MS_SE= \transmsse , Transformer_DR= \transdr , Transformer_DR_SE= \transdrse , Transformer_RC= \transrc , Transformer_RC_SE= \transrcse , S4_OURS= \sfourours , S4_OURS_SE= \sfouroursse , S4_MS= \sfourms , S4_MS_SE= \sfourmsse , S4_DR= \sfourdr , S4_DR_SE= \sfourdrse , S4_RC= \sfourrc , S4_RC_SE= \sfourrcse , ETS= \ets , ARIMA= \arima \dset | \frequency | \lstmours | \lstmms | \lstmdr | \lstmrc | \cnnours | \cnnms | \cnndr | \cnnrc | \transours | \transms | \transdr | \transrc | \sfourours | \sfourms | \sfourdr | \sfourrc | \ets | \arima |
| ( \lstmoursse ) | ( \lstmmsse ) | ( \lstmdrse ) | ( \lstmrcse ) | ( \cnnoursse ) | ( \cnnmsse ) | ( \cnndrse ) | ( \cnnrcse ) | ( \transoursse ) | ( \transmsse ) | ( \transdrse ) | ( \transrcse ) | ( \sfouroursse ) | ( \sfourmsse ) | ( \sfourdrse ) | ( \sfourrcse ) | ( - ) | ( - ) | ||
Time Series Forecasting
Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.
See all 947 papers