Morpheus-2.0: Multiscale Probabilistic Forecasting with Dual-Axis Attention

Aakriti; Manoj Cherukumalli; Prathamesh Patil; Leonidas Tsepenekas; Amit Varshney; Penghang Liu; Elizabeth Fons; Vamsi Potluru

cs.LGOct 16, 2026

Morpheus-2.0: Multiscale Probabilistic Forecasting with Dual-Axis Attention

Authors: Aakriti, Manoj Cherukumalli, Prathamesh Patil, Leonidas Tsepenekas, Amit Varshney, Penghang Liu, Elizabeth Fons, Vamsi Potluru

Abstract

A universal forecaster is trained once and applied everywhere, so it must serve tasks whose schemas, forecast horizons, and temporal resolutions all differ. The number of targets and covariates changes from dataset to dataset, horizons vary by orders of magnitude, and sampling rates range from seconds to years. Existing time-series foundation models address these only in part, since they isolate each series or visit the two grid relations in separate blocks, and they fix the horizon and resolution before training begins. We introduce Morpheus-2.0, a compact encoder-only foundation model that represents targets and covariates on a single variable-time grid. Its dual-axis attention runs both relations side by side in every layer, one across variables at a timestep and one across time within a variable; a learned gate weights the two per layer, and an axis-aware rotary encoding preserves both grid coordinates. The forecast length is sampled during training, so one set of weights spans short to long horizons at no additional inference cost. Morpheus-2.0 Multiscale extends the model to several temporal resolutions at once, weighting them according to the series being forecast. Across GIFT-Eval, which spans several domains and frequencies, the two models give complementary operating points: Morpheus-2.0 attains 0.685 relative MASE with 81M parameters, while the larger Morpheus-2.0 Multiscale attains 0.646 relative MASE and 0.455 relative CRPS, the lowest of any published system on both metrics.

Figures & tables

Figure 1: Overview of Morpheus-2.0 . (A) Training data span many domains, frequencies, and schemas. (B) Covariates and targets form one variable–time grid; each patch is embedded with its observation mask, so one time patch across all rows becomes one token, and future targets remain query-only. (C) Every block applies two parallel attention pathways, same-time across variables and same-variable across time with axis-aware two-dimensional RoPE, combined by a learned scalar gate. (D) The patch decoder projects each token to the native quantile grid, with one prediction length sampled per batch. Morpheus-2.0 Multiscale runs several such encoders at different patch widths and pools their forecasts (Section 4.6 ).

Model	Params	Setup	Rel. MASE $\downarrow$	Rel. CRPS $\downarrow$
Seasonal Naive	–	Reference	1.000	1.000
Moirai-2.0-R-Small	11.4M	Pretrained	0.728	0.516
TimesFM-2.5	200M	Zero-shot	0.705	0.490
Chronos-2	120M	Pretrained	0.698	0.485
Morpheus-2.0	81.5M	Pretrained	0.685	0.519
Toto-2.0-2.5B-FT	2.5B	Fine-tuned	0.679	0.463

Table 1: GIFT-Eval results over 97 configurations. Rel. MASE scores the median forecast and Rel. CRPS the full quantile forecast, both GM relative to Seasonal Naive; lower is better. Setup records whether benchmark training data are used; parameter counts are taken from the official benchmark, and ours from the released checkpoints.

Attention operator	MASE	CRPS	Multivariate MASE
Full attention over the grid	0.759	0.567	0.774
Dual-axis attention, fixed gate	0.725	0.550	0.748
Dual-axis attention, learned gate	0.685	0.519	0.703

Table 2: Attention-operator comparison. All metrics are relative; lower is better.

Signal family	Patch 2	Patch 4	Patch 8	Average	Multiscale
High-frequency oscillations	0.888	0.895	0.976	0.875	0.870
Wide multivariate schemas	0.405	0.435	0.427	0.387	0.379
Sparse impulses	0.873	0.815	0.970	0.851	0.809
Volatility bursts	0.644	0.557	0.776	0.586	0.563
Low signal-to-noise	0.702	0.473	0.711	0.602	0.473
Regime-persistent levels	0.531	0.530	0.486	0.497	0.484

Table 3: Resolution study on synthetic families, relative MASE; best single width in bold. Families are defined by measured signal statistics. Average pools the single-width encoders with equal weights; Multiscale pools the same encoders under the learned mixing head.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Field	Value
Layers	9 dual-axis blocks
Hidden width	640
Attention heads	20
Head dimension	32
Feed-forward width	4096
Rotary base $\theta$	5000

Appendix

Table 4: Shared encoder specification. Patch width, the token budget

P

, and training hyperparameters vary across the reported models.

Domain	Partitions	Series	Obs. (B)	Size (GB)	Frequencies
Nature	70	25,397,091	204.5	818.2	H, 6H, D
Energy	29	1,804,428	16.9	68.9	4S, 1min, 10min, 30min, H
Transport	24	165,266	4.9	19.6	5min, 15min, 30min, H, D, W
Web/CloudOps	6	900,863	2.6	30.5	5min, D, W
Sales	4	148,654	0.2	0.8	D, M
Healthcare	3	1,929	$<0.1$	$<0.1$	W

Appendix

Table 5: Composition of GIFT-Eval Pretrain. Series counts target rows, not items; size is on-disk Arrow. Domain assignment follows the GIFT-Eval taxonomy, and every partition is assigned to exactly one domain.

Subset	Collections	Records	Rows/record	Obs. (B)	Size (GB)
Kernel composition	42	357,000	1–21	5.7	24.1
Covariate response	14	365,004	1–5 targets	2.3	34.9
Real-world emulation	82	585,890	1–21	14.3	61.6
Total	138	1,307,894		22.3	120.6

Appendix

Table 6: MySTiC composition. A record is one multivariate series and rows are its variables.

Figure 2: Kernel composition in the MySTiC kernel subset. Each row is drawn from a Gaussian process whose covariance is assembled from base kernels under sums and products: a sum superposes independent structure, and a product modulates one component by another. Panels combine periodic, linear, exponential, and squared-exponential kernels (

\kappa_{\mathrm{per}}

\kappa_{\mathrm{lin}}

\kappa_{\mathrm{exp}}

\kappa_{\mathrm{se}}

), with the component draws in blue and the composition they produce in red. Sums give a cycle on a trend (a), a cycle on a wandering baseline (b), and two periods at once (c); products give periodic amplitude that follows a trend (d) or drifts smoothly (e), and bursts of local activity (f). Series are standardized for display.

Figure 3: Covariate-response subset. Black is the target, colored lines are the covariates that drive it, and grey lines are distractors that drive nothing; all rows are standardized for display. (a) instantaneous linear response; (b) lagged response, with the realized lag

k

annotated; (c) distributed lag, with the geometric kernel inset; (d) autoregressive response with covariates, with its persistence parameter

\phi

annotated.

Figure 4: Real-world emulation subset. (a) zero-floored pulse from the trapezoid primitive, with exact zeros between cycles; (b) a recurring profile under amplitude modulation; (c) a short intermittent record; (d) an abrupt regime break; (e) a correlated multivariate record, rows offset vertically for legibility; (f) a low-frequency series. Panels are individual records.

Figure 5: Mixing head of Morpheus-2.0 Multiscale . (A) A request is the context up to the forecast origin

T

and the horizon

H

. (B) Each target row is z-normalized, stacked with its observation mask, and encoded by an input convolution and six residual dilated-convolution blocks; masked mean and maximum pooling over time and then over targets give a fixed-width summary, which is concatenated with a Fourier embedding of

\log H

and mapped by a two-layer classifier to one weight per encoder. (C)

E

Morpheus-2.0 encoders, trained at different patch widths, each forecast the same request. (D) The forecast is the weighted linear pool of their predictive distributions. The head never reads the encoders’ forecasts, so its weights depend on the request alone.

Design	Time geometry	Variable geometry	Both axes per layer
Flattened one-dimensional positions	Entangled with $N$	Entangled with $N$	Model dependent
Temporal RoPE + variable identifiers	Relative	Absolute	Model dependent
Alternating axial attention	Relative	Relative	No
Ours	Relative	Relative	Yes

Appendix

Table 7: Structural comparison of positional and attention designs; entries summarize inductive biases.

	All configurations		Covariate-informed	Held-out width
Position scheme	Rel. MASE	Rel. CRPS	Rel. MASE	Rel. MASE
Axis-aware two-dimensional RoPE	0.685	0.519	0.703	0.708
Temporal RoPE + variable bias	0.695	0.531	0.713	0.778
Flattened one-dimensional RoPE	0.700	0.540	0.736	0.727
No positional encoding	1.269	0.927	1.111	2.527

Appendix

Table 8: Positional-encoding error on aggregate, covariate-informed, and held-out-width suites. Lower is better.

Horizon objective	Supervised span	Short	Medium	Long	Overall
Fixed prediction length	1024	0.734	0.673	0.659	0.704
Fixed prediction length	512	0.701	0.714	0.716	0.707
Lead-time loss reweighting	1024 (weighted)	0.721	0.689	0.683	0.706
Horizon sampling	8–1024	0.706	0.687	0.678	0.696

Appendix

Table 9: Horizon objectives at matched capacity. Entries are relative MASE within each horizon group; lower is better.

Signal family	Defining statistic
High-frequency oscillations	Ten-second and five-minute sources; a third of spectral energy lies above the band a width-8 patch can represent
Wide multivariate schemas	7 to 21 simultaneous targets
Sparse impulses	Differenced kurtosis above 300 at a jump rate below $0.04$ : isolated spikes on a smooth baseline
Volatility bursts	Absolute-difference autocorrelation near $0.3$ with a jump rate above $0.12$
Low signal-to-noise	Lowest lag-1 autocorrelation and flattest spectra in the corpus
Regime-persistent levels	Lag-50 autocorrelation between $0.69$ and $0.82$

Appendix

Table 10: Signal-family definitions for the resolution study. Every statistic is measured on the target series before any forecast is scored.

System	Params	Rel. MASE	Rel. CRPS
Morpheus-2.0 , patch 2	84.3M	0.696	0.507
Morpheus-2.0 , patch 4	81.5M	0.685	0.519
Morpheus-2.0 , patch 8	80.1M	0.708	0.526
Single encoder at multiscale budget, patch 4	247M	0.838	0.602
Mixing head over three encoders, all patch 4	245M	0.669	0.492
Mixing head over patch widths 2, 4, 8	247M	0.653	0.476

Appendix

Table 11: Isolating the mixing head. Every row uses the same 97 configurations, evaluation protocol, rollout policy, and benchmark quantile grid, so rows differ only in what is combined and at what capacity. Lower is better.

Pretraining recipe	Relative MASE
Real corpus only	0.814
$+$ MySTiC, single-stage mixture	0.759
$+$ four-stage curriculum	0.685

Appendix

Table 12: Contribution of the synthetic corpus and the curriculum that orders it. Entries are relative MASE over the 97 configurations; lower is better, and the last row is the full recipe.

		All		Short		Medium		Long
Model	Setup	MASE	CRPS	MASE	CRPS	MASE	CRPS	MASE	CRPS
Seasonal Naive	Reference	1.000	1.000	1.000	1.000	1.000	1.000	1.000	1.000
Sundial-Base-128M	Zero-shot	0.750	0.559	0.732	0.592	0.764	0.526	0.785	0.510
Moirai-2.0-R-Small	Pretrained	0.728	0.516	0.695	0.517	0.759	0.519	0.789	0.513
TimesFM-2.5	Zero-shot	0.705	0.490	0.681	0.504	0.724	0.472	0.751	0.475
Granite-FlowState-r1.1	Zero-shot	0.701	0.490	0.680	0.507	0.722	0.474	0.739	0.464

Appendix

Table 13: GIFT-Eval accuracy over 97 configurations, aggregate and by horizon group. Entries are normalized by Seasonal Naive within the corresponding group (55 short, 21 medium, 21 long); lower is better. Bold marks the best entry in each column.

Model	MASE	ND	MSE	MAPE	sMAPE	CRPS
Seasonal Naive	1.000	1.000	1.000	1.000	1.000	1.000
Sundial-Base-128M	0.750	0.739	0.546	0.777	0.945	0.559
Moirai-2.0-R-Small	0.728	0.705	0.545	0.693	0.925	0.516
TimesFM-2.5	0.705	0.687	0.482	0.707	0.856	0.490
Granite-FlowState-r1.1	0.701	0.681	0.493	0.671	0.833	0.490
Chronos-2	0.698	0.679	0.495	0.666	0.886	0.485

Appendix

Table 14: All distinct GIFT-Eval metrics over 97 configurations, each normalized by Seasonal Naive and aggregated as a geometric mean; lower is better. Relative MAE, RMSE and NRMSE are omitted because they are determined by ND and MSE. Bold marks the best entry in each column.

Relative MASE
Model	Energy	Web/Cloud	Nature	Transport	Econ/Fin	Health	Sales	All
	(32)	(20)	(15)	(15)	(6)	(5)	(4)	(97)
TimesFM-3	0.798	0.582	0.687	0.543	0.739	0.535	0.682	0.667
Chronos-2	0.816	0.611	0.723	0.603	0.772	0.551	0.679	0.698
Morpheus-2.0	0.747	0.614	0.742	0.557	0.838	0.655	0.751	0.685
Morpheus-2.0 Multiscale	0.689	0.595	0.729	0.513	0.772	0.628	0.693	0.646

Appendix

Table 15: GIFT-Eval accuracy by domain over 97 configurations. Entries are geometric means of the per-configuration ratio to Seasonal Naive within the domain; lower is better. Bold marks the best entry in each column within each metric.

System	$60\%$	$80\%$	Max. dev.
	Central interval coverage
Morpheus-2.0	34.3	49.3	0.168
Morpheus-2.0 Multiscale	53.1	75.7	0.052

Appendix

Table 16: Calibration over 97 configurations at the nine benchmark quantile levels. Interval columns give the empirical coverage of the central interval at the stated nominal level; the nominal value is the target. Max. deviation is the largest gap between empirical and nominal coverage over the nine levels.

Time Series Forecasting

Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.

See all 861 papers

$t_0$ : A Time-Series Foundation Model for Forecasting with Context
Sep 21, 2026Lucas Meyer, Claudio Sole, Huikan Xiang +6Time Series Forecasting Zero-Shot Time Series Forecasting
The Forecasting Company · Macrocosm
DualWeaver: Synergistic Feature Weaving Surrogates for Multivariate Forecasting with Univariate Time Series Foundation Models
Feb 25, 2026Jinpeng Li, Zhongyi Pei, Huaze Xue +3Multivariate Time Series Forecasting Time Series Forecasting
School of Software, BNRist, Tsinghua University · Tencent
Aurora-X: Built for Extreme Time Series Forecasting
Sep 25, 2026Xingjian Wu, Chenjuan Guo, Xiangfei Qiu +6Time Series Forecasting Time-Series Foundation Models
East China Normal University

Sep 21, 2026J/K move · Enter open · S save

$t_0$ : A Time-Series Foundation Model for Forecasting with Context

Lucas Meyer, Claudio Sole, Huikan Xiang, Nicolas Li+5

The Forecasting Company · Macrocosm

Time Series Forecasting Zero-Shot Time Series Forecasting Probabilistic Time Series Forecasting Multivariate Time Series Forecasting Time Series Forecasting with Covariates Time-Series Foundation Models+1

Open paper AskPDF arXiv

We present

t_0

, a family of open-weights foundation models for forecasting with multivariate context. We release its first two members:

\texttt{t0-alpha}

and

\texttt{t0-beta}

, respectively 102M and 256M parameters. Both condition their forecasts on target history, past covariates, and known-future covariates, without task-specific retraining. Their transformer layers alternate attention along time and across variates. They produce probabilistic forecasts through quantile predictions. Pretraining combines curated public data with synthetic generator families constructed to contain covariate-to-target dependencies. On GIFT-Eval,

\texttt{t0-alpha}

reaches an aggregate CRPS of 0.4941, and

\texttt{t0-beta}

a CRPS of 0.4738 and a MASE of 0.6865, third on both and within 4.0% of the best zero-shot TSFM. On fev-bench they score 42.2 and 46.7 in skill, the latter third again and 2.0 points behind the leader. We analyze

\texttt{t0-alpha}

in depth. Known-future covariates raise its skill by 6.3 percentage points across 30 tasks. The report also examines its calibration, its rollout strategy on long horizons, and its robustness to missing data. On the Victoria electricity-demand benchmark,

\texttt{t0-beta}

is among the most accurate models with a context of nearly a year. In an independent Macrocosm evaluation of hourly ERCOT prices over 29 months, both cut the MAE of the lagged-price baseline by 38%.

Abstract

Time Series Forecasting

Explore similar work

t0t_0t0​: A Time-Series Foundation Model for Forecasting with Context

DualWeaver: Synergistic Feature Weaving Surrogates for Multivariate Forecasting with Univariate Time Series Foundation Models

Aurora-X: Built for Extreme Time Series Forecasting

$t_0$ : A Time-Series Foundation Model for Forecasting with Context