Morpheus-2.0: Multiscale Probabilistic Forecasting with Dual-Axis Attention
Abstract
A universal forecaster is trained once and applied everywhere, so it must serve tasks whose schemas, forecast horizons, and temporal resolutions all differ. The number of targets and covariates changes from dataset to dataset, horizons vary by orders of magnitude, and sampling rates range from seconds to years. Existing time-series foundation models address these only in part, since they isolate each series or visit the two grid relations in separate blocks, and they fix the horizon and resolution before training begins. We introduce Morpheus-2.0, a compact encoder-only foundation model that represents targets and covariates on a single variable-time grid. Its dual-axis attention runs both relations side by side in every layer, one across variables at a timestep and one across time within a variable; a learned gate weights the two per layer, and an axis-aware rotary encoding preserves both grid coordinates. The forecast length is sampled during training, so one set of weights spans short to long horizons at no additional inference cost. Morpheus-2.0 Multiscale extends the model to several temporal resolutions at once, weighting them according to the series being forecast. Across GIFT-Eval, which spans several domains and frequencies, the two models give complementary operating points: Morpheus-2.0 attains 0.685 relative MASE with 81M parameters, while the larger Morpheus-2.0 Multiscale attains 0.646 relative MASE and 0.455 relative CRPS, the lowest of any published system on both metrics.
Figures & tables
| Model | Params | Setup | Rel. MASE | Rel. CRPS |
|---|---|---|---|---|
| Seasonal Naive | – | Reference | 1.000 | 1.000 |
| Moirai-2.0-R-Small | 11.4M | Pretrained | 0.728 | 0.516 |
| TimesFM-2.5 | 200M | Zero-shot | 0.705 | 0.490 |
| Chronos-2 | 120M | Pretrained | 0.698 | 0.485 |
| Morpheus-2.0 | 81.5M | Pretrained | 0.685 | 0.519 |
| Toto-2.0-2.5B-FT | 2.5B | Fine-tuned | 0.679 | 0.463 |
| Attention operator | MASE | CRPS | Multivariate MASE |
|---|---|---|---|
| Full attention over the grid | 0.759 | 0.567 | 0.774 |
| Dual-axis attention, fixed gate | 0.725 | 0.550 | 0.748 |
| Dual-axis attention, learned gate | 0.685 | 0.519 | 0.703 |
| Signal family | Patch 2 | Patch 4 | Patch 8 | Average | Multiscale |
|---|---|---|---|---|---|
| High-frequency oscillations | 0.888 | 0.895 | 0.976 | 0.875 | 0.870 |
| Wide multivariate schemas | 0.405 | 0.435 | 0.427 | 0.387 | 0.379 |
| Sparse impulses | 0.873 | 0.815 | 0.970 | 0.851 | 0.809 |
| Volatility bursts | 0.644 | 0.557 | 0.776 | 0.586 | 0.563 |
| Low signal-to-noise | 0.702 | 0.473 | 0.711 | 0.602 | 0.473 |
| Regime-persistent levels | 0.531 | 0.530 | 0.486 | 0.497 | 0.484 |
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
| Field | Value |
|---|---|
| Layers | 9 dual-axis blocks |
| Hidden width | 640 |
| Attention heads | 20 |
| Head dimension | 32 |
| Feed-forward width | 4096 |
| Rotary base | 5000 |
| Domain | Partitions | Series | Obs. (B) | Size (GB) | Frequencies |
|---|---|---|---|---|---|
| Nature | 70 | 25,397,091 | 204.5 | 818.2 | H, 6H, D |
| Energy | 29 | 1,804,428 | 16.9 | 68.9 | 4S, 1min, 10min, 30min, H |
| Transport | 24 | 165,266 | 4.9 | 19.6 | 5min, 15min, 30min, H, D, W |
| Web/CloudOps | 6 | 900,863 | 2.6 | 30.5 | 5min, D, W |
| Sales | 4 | 148,654 | 0.2 | 0.8 | D, M |
| Healthcare | 3 | 1,929 | W |
| Subset | Collections | Records | Rows/record | Obs. (B) | Size (GB) |
|---|---|---|---|---|---|
| Kernel composition | 42 | 357,000 | 1–21 | 5.7 | 24.1 |
| Covariate response | 14 | 365,004 | 1–5 targets | 2.3 | 34.9 |
| Real-world emulation | 82 | 585,890 | 1–21 | 14.3 | 61.6 |
| Total | 138 | 1,307,894 | 22.3 | 120.6 |
| Design | Time geometry | Variable geometry | Both axes per layer |
|---|---|---|---|
| Flattened one-dimensional positions | Entangled with | Entangled with | Model dependent |
| Temporal RoPE + variable identifiers | Relative | Absolute | Model dependent |
| Alternating axial attention | Relative | Relative | No |
| Ours | Relative | Relative | Yes |
| All configurations | Covariate-informed | Held-out width | ||
|---|---|---|---|---|
| Position scheme | Rel. MASE | Rel. CRPS | Rel. MASE | Rel. MASE |
| Axis-aware two-dimensional RoPE | 0.685 | 0.519 | 0.703 | 0.708 |
| Temporal RoPE + variable bias | 0.695 | 0.531 | 0.713 | 0.778 |
| Flattened one-dimensional RoPE | 0.700 | 0.540 | 0.736 | 0.727 |
| No positional encoding | 1.269 | 0.927 | 1.111 | 2.527 |
| Horizon objective | Supervised span | Short | Medium | Long | Overall |
|---|---|---|---|---|---|
| Fixed prediction length | 1024 | 0.734 | 0.673 | 0.659 | 0.704 |
| Fixed prediction length | 512 | 0.701 | 0.714 | 0.716 | 0.707 |
| Lead-time loss reweighting | 1024 (weighted) | 0.721 | 0.689 | 0.683 | 0.706 |
| Horizon sampling | 8–1024 | 0.706 | 0.687 | 0.678 | 0.696 |
| Signal family | Defining statistic |
|---|---|
| High-frequency oscillations | Ten-second and five-minute sources; a third of spectral energy lies above the band a width-8 patch can represent |
| Wide multivariate schemas | 7 to 21 simultaneous targets |
| Sparse impulses | Differenced kurtosis above 300 at a jump rate below : isolated spikes on a smooth baseline |
| Volatility bursts | Absolute-difference autocorrelation near with a jump rate above |
| Low signal-to-noise | Lowest lag-1 autocorrelation and flattest spectra in the corpus |
| Regime-persistent levels | Lag-50 autocorrelation between and |
| System | Params | Rel. MASE | Rel. CRPS |
| Morpheus-2.0 , patch 2 | 84.3M | 0.696 | 0.507 |
| Morpheus-2.0 , patch 4 | 81.5M | 0.685 | 0.519 |
| Morpheus-2.0 , patch 8 | 80.1M | 0.708 | 0.526 |
| Single encoder at multiscale budget, patch 4 | 247M | 0.838 | 0.602 |
| Mixing head over three encoders, all patch 4 | 245M | 0.669 | 0.492 |
| Mixing head over patch widths 2, 4, 8 | 247M | 0.653 | 0.476 |
| Pretraining recipe | Relative MASE |
|---|---|
| Real corpus only | 0.814 |
| MySTiC, single-stage mixture | 0.759 |
| four-stage curriculum | 0.685 |
| All | Short | Medium | Long | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Model | Setup | MASE | CRPS | MASE | CRPS | MASE | CRPS | MASE | CRPS |
| Seasonal Naive | Reference | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
| Sundial-Base-128M | Zero-shot | 0.750 | 0.559 | 0.732 | 0.592 | 0.764 | 0.526 | 0.785 | 0.510 |
| Moirai-2.0-R-Small | Pretrained | 0.728 | 0.516 | 0.695 | 0.517 | 0.759 | 0.519 | 0.789 | 0.513 |
| TimesFM-2.5 | Zero-shot | 0.705 | 0.490 | 0.681 | 0.504 | 0.724 | 0.472 | 0.751 | 0.475 |
| Granite-FlowState-r1.1 | Zero-shot | 0.701 | 0.490 | 0.680 | 0.507 | 0.722 | 0.474 | 0.739 | 0.464 |
| Model | MASE | ND | MSE | MAPE | sMAPE | CRPS |
|---|---|---|---|---|---|---|
| Seasonal Naive | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
| Sundial-Base-128M | 0.750 | 0.739 | 0.546 | 0.777 | 0.945 | 0.559 |
| Moirai-2.0-R-Small | 0.728 | 0.705 | 0.545 | 0.693 | 0.925 | 0.516 |
| TimesFM-2.5 | 0.705 | 0.687 | 0.482 | 0.707 | 0.856 | 0.490 |
| Granite-FlowState-r1.1 | 0.701 | 0.681 | 0.493 | 0.671 | 0.833 | 0.490 |
| Chronos-2 | 0.698 | 0.679 | 0.495 | 0.666 | 0.886 | 0.485 |
| Model | Energy | Web/Cloud | Nature | Transport | Econ/Fin | Health | Sales | All |
|---|---|---|---|---|---|---|---|---|
| (32) | (20) | (15) | (15) | (6) | (5) | (4) | (97) | |
| Relative MASE | ||||||||
| TimesFM-3 | 0.798 | 0.582 | 0.687 | 0.543 | 0.739 | 0.535 | 0.682 | 0.667 |
| Chronos-2 | 0.816 | 0.611 | 0.723 | 0.603 | 0.772 | 0.551 | 0.679 | 0.698 |
| Morpheus-2.0 | 0.747 | 0.614 | 0.742 | 0.557 | 0.838 | 0.655 | 0.751 | 0.685 |
| Morpheus-2.0 Multiscale | 0.689 | 0.595 | 0.729 | 0.513 | 0.772 | 0.628 | 0.693 | 0.646 |
| Central interval coverage | |||
|---|---|---|---|
| System | Max. dev. | ||
| Morpheus-2.0 | 34.3 | 49.3 | 0.168 |
| Morpheus-2.0 Multiscale | 53.1 | 75.7 | 0.052 |
Time Series Forecasting
Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.
See all 861 papers