Deep Generative Transformers for Probabilistic Time Series and Spatiotemporal Forecasting

Rajdeep Pathak; Rahul Goswami; Madhurima Panja; Palash Ghosh; Tanujit Chakraborty

cs.LGMar 12, 2026

Deep Generative Transformers for Probabilistic Time Series and Spatiotemporal Forecasting

Authors: Rajdeep Pathak, Rahul Goswami, Madhurima Panja, Palash Ghosh, Tanujit Chakraborty

Organizations: SAFIR, Sorbonne University Abu Dhabi, United Arab Emirates. · SCAI, Sorbonne Universit´e, Paris, France. · Indian Institute of Technology Guwahati, Assam, India.

Abstract

Reliable uncertainty quantification is paramount for forecasting multivariate time series and spatiotemporal data. While Transformer architectures excel at sequence modeling, current probabilistic approaches typically rely on restrictive parametric likelihoods or quantile-based objectives, thereby limiting their ability to capture complex joint distributions in correlated time series. To overcome these limitations, we propose Enformer and its spatiotemporal extension, GEnformer. These models synthesize the expressive power of Transformers with engression, a stochastic learning paradigm for modeling conditional distributions. By injecting stochastic noise and optimizing a strictly proper scoring objective, our frameworks directly learn conditional predictive distributions without imposing parametric assumptions. This design ensures the generation of coherent multivariate trajectories while maintaining the Transformer's efficacy in modeling long-range dependencies and cross-series interactions. The probabilistic capability of Enformer is achieved with an asymptotic overhead of only a constant factor over a deterministic Transformer with an identical configuration. We extensively evaluate our frameworks on prominent multivariate benchmarks for temporal dynamics and real-world epidemic datasets for spatiotemporal dynamics. Empirical results demonstrate that both frameworks yield calibrated probabilistic forecasts and consistently outperform state-of-the-art baselines.

Figures & tables

Figure 1: Overview of the Enformer architecture, consisting of noise injection, in-sample forecast generation, and optimization using the energy score loss. An ensemble of out-of-sample forecasts for all $D$ nodes for the next $q$ steps can be generated from the trained model by passing the $p$ -length look-back window as input.

Name	Freq.	Dim.	Training Samples	Context Length	Pred. Length	Seq. Lags	Test Windows
Solar a	1 hour	137	7009	24	24	(1, 24, 168)	7
Electricity b	1 hour	370	5833	24	24	(1, 24, 168)	7
Traffic c	1 hour	963	4001	24	24	(1, 24, 168)	7
KDD-cup d	1 hour	270	10872	48	48	(1, 24, 168)	1
Wikipedia e	1 day	2000	792	30	30	(1, 7, 14)	5
Taxi f	30 mins	1214	1488	24	24	(1, 4, 12, 24, 48)	56

Table 1: Key characteristics of the temporal benchmark datasets used in this study.

Country	Disease	Granularity	Locations	Training Samples	Context Length	Prediction Length	Ref.
Japan	TB	Monthly	47	216	36	24	a
USA	ILI	Weekly	50	328	26	13	b
Belgium	COVID-19	Daily	11	776	60	30	c
Colombia	Dengue	Weekly	33	835	26	13	d
Hungary	Chickenpox	Weekly	20	522	26	13	e

Table 2: Spatiotemporal epidemic datasets used in this study.

Method	Solar	Electricity	Traffic	KDD-cup	Taxi	Wikipedia
Vec-LSTM	$0.4825$ $(0.0027)$	$0.0949$ $(0.0175)$	$0.0915$ $(0.0197)$	$0.3560$ $(0.1667)$	$0.4794$ $(0.0343)$	$0.1254$ $(0.0174)$
GP-Scaling	$0.3802$ $(0.0052)$	$0.0499$ $(0.0031)$	$0.0753$ $(0.0152)$	$0.2983$ $(0.0448)$	$0.2265$ $(0.0210)$	$0.1351$ $(0.0612)$
GP-Copula	$0.3612$ $(0.0035)$	$0.0287$ $(0.0005)$	$0.0618$ $(0.0018)$	$0.3157$ $(0.0462)$	$0.1894$ $(0.0087)$	$0.0669$ $(0.0009)$
LSTM-MAF	$0.3427$ $(0.0082)$	$0.0312$ $(0.0046)$	$0.0526$ $(0.0021)$	$0.2919$ $(0.1486)$	$0.2295$ $(0.0082)$	$0.0763$ $(0.0051)$
Transformer-MAF	$0.3532$ $(0.0053)$	$0.0272$ $(0.0017)$	$0.0499$ $(0.0011)$	$0.2951$ $(0.0504)$	$0.1531$ $(0.0038)$	$0.0644$ $(0.0037)$
TimeGrad	$0.3335$ $(0.0653)$	$0.0232$ $(0.0035)$	$\mathbf{0.0414}$ $(0.0112)$	$0.2902$ $(0.2178)$	$0.1255$ $(0.0207)$	$\mathbf{0.0555}$ $(0.0088)$

Table 3: Forecasting performance on the temporal datasets in terms of the

\operatorname{CRPS}_{\text{sum}}

metric (lower is better). Mean and (standard error) are computed from 10 independent evaluation runs. The best results are highlighted .

Method	TB	ILI	COVID-19	Dengue	Chickenpox
NHiTS	$6.72$	$102.91$	$165.5$	$27.31$	$16.25$
Transformers	$475.54$	$104.94$	$195.33$	$22.97$	$20.15$
TCN	$578.56$	$146.24$	$457.57$	$44.02$	$24.95$
GSTAR	$6.58$	$179.34$	$190.85$	$19.15$	$20.9$
STGCN	$14.54$	$115.36$	$374.97$	$39.27$	$23.85$
DeepAR	$6.97_{\pm 0.07}$	$151.28_{\pm 1.97}$	$112.08_{\pm 9.24}$	$18.49_{\pm 0.19}$	$12.22_{\pm 0.24}$

Table 4: Forecasting performance in terms of CRPS on the spatiotemporal epidemic datasets. Mean and standard error are computed from 50 independent evaluation runs. The best results are highlighted .

Figure 2: Forecasts produced by Enformer on selected nodes of Traffic dataset on test window 5.

Figure 3: MCB test results on the (A) temporal datasets based on

\operatorname{CRPS}_{\text{sum}}

and (B) spatiotemporal datasets based on CRPS metric. On the y-axis, the notation

\nu

r

indicates that model

\nu

achieved an average rank of

r

Model	Training	Testing
Transformer-MAF	$8.524\pm 0.001$	$17.835\pm 0.002$
TimeGrad	$30.128\pm 0.002$	$44.171\pm 0.003$
Enformer (Proposed)	$\mathbf{5.380\pm 0.001}$	$\mathbf{5.560\pm 0.000}$

Table 5: Average training time per epoch and average inference time (in seconds) for Transformer-MAF, TimeGrad, and Enformer (proposed) on the Traffic dataset.

Figure 4: Probability Integral Transform (PIT) Q-Q plots evaluating the predictive calibration of Enformer across the six multivariate time series datasets. The empirical quantiles (solid blue) are plotted against the theoretical quantiles of an ideal uniform distribution,

\mathcal{U}(0,1)

(dashed black).

Figure 5: Sensitivity analysis studying the effect of the in-sample training ensemble size (

M

) on the proposed Enformer. (A) Computational overhead represented by the total training time in seconds; (B) Forecasting performance evaluated by the

\operatorname{CRPS}_{\text{sum}}

metric across the six datasets. The circular points represent the mean values and the vertical error lines indicate the standard deviation computed over 10 independent runs.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Method	Solar	Electricity	Traffic	KDD-cup	Taxi	Wikipedia
Vec-LSTM	$0.9952$ $(0.0077)$	$0.1439$ $(0.0228)$	$0.1451$ $(0.0248)$	$0.4461$ $(0.1833)$	$0.6398$ $(0.0390)$	$0.1618$ $(0.0162)$
GP-Scaling	$0.9004$ $(0.0095)$	$0.0811$ $(0.0062)$	$0.1469$ $(0.0181)$	$0.3445$ $(0.0621)$	$0.3598$ $(0.0285)$	$0.1710$ $(0.1006)$
GP-Copula	$0.8279$ $(0.0053)$	$0.0512$ $(0.0009)$	$0.1282$ $(0.0033)$	$0.2605$ $(0.0227)$	$0.3125$ $(0.0113)$	$0.0930$ $(0.0076)$
Autoformer	$0.7046$ $(0.0000)$	$0.0475$ $(0.0000)$	$0.0951$ $(0.0000)$	$0.8984$ $(0.0000)$	$0.3498$ $(0.0000)$	$0.1052$ $(0.0000)$
PatchTST	$0.7270$ $(0.0000)$	$0.0474$ $(0.0000)$	$0.1897$ $(0.0000)$	$0.5137$ $(0.0000)$	$0.3690$ $(0.0000)$	$0.0915$ $(0.0000)$
$\mathrm{D}^{3}$ VAE	$0.7472$ $(0.0508)$	$0.1640$ $(0.0928)$	$0.4722$ $(0.1197)$	$0.5628$ $(0.0419)$	$0.7624$ $(0.5598)$	$2.2094$ $(2.1646)$

Appendix

Table 6: Forecasting performance on the temporal datasets in terms of the

\operatorname{NRMSE}_{\text{sum}}

metric (lower is better). Best results are highlighted .

Probabilistic Time Series Forecasting

Probabilistic time series forecasting estimates distributions of future values or trajectories to quantify uncertainty and support risk-aware decisions.

See all 159 papers

Deep Generative Spatiotemporal Engression for Probabilistic Forecasting of Epidemics
Mar 7, 2026Rajdeep Pathak, Tanujit ChakrabortyEpidemic Forecasting Spatiotemporal Forecasting
SAFIR, Sorbonne University Abu Dhabi, United Arab Emirates · SCAI, Sorbonne Université, Paris, France
UniMamba: A Unified Spatial-Temporal Modeling Framework with State-Space and Attention Integration
Mar 6, 2026Xingsheng Chen, Xianpei Mu, Deyu Yi +6Multivariate Time Series Forecasting Long-Term Time Series Forecasting
School of Computing and Data Science, The University of Hong Kong, Hong Kong, China · School of Information Engineering, Beijing Institute of Graphic Communication, Beijing, China · Innovation Engineering College, Macau University of Science and Technology, Macau, China +1
Exploring the Potential of Probabilistic Transformer for Time Series Modeling: A Report on the ST-PT Framework
Apr 29, 2026Zhangzhi Xiong, Haoyi Wu, You Wu +3Transformer Conditional Random Fields
School of Information Science and Technology, ShanghaiTech University

Mar 7, 2026J/K move · Enter open · S save

Deep Generative Spatiotemporal Engression for Probabilistic Forecasting of Epidemics

Rajdeep Pathak, Tanujit Chakraborty

SAFIR, Sorbonne University Abu Dhabi, United Arab Emirates · SCAI, Sorbonne Université, Paris, France

Epidemic Forecasting Spatiotemporal Forecasting Probabilistic Time Series Forecasting Spatial Epidemic Modeling

Open paper AskPDF arXiv

Accurate and reliable forecasting of epidemic incidences is critical for public health preparedness, yet it remains a challenging task due to complex nonlinear temporal dependencies and heterogeneous spatial interactions. Often, point forecasts generated by spatiotemporal models are unreliable in assigning uncertainty to future epidemic events. Probabilistic forecasting of epidemics is therefore crucial for providing the best or worst-case scenarios rather than a simple, often inaccurate, point estimate. We present deep spatiotemporal engression methods to generate accurate and reliable probabilistic forecasts on low-frequency epidemic datasets. The proposed methods act as distributional lenses, and out-of-sample probabilistic forecasts are generated by sampling from the trained models. Our frameworks encapsulate lightweight deep generative architectures, wherein uncertainty is quantified endogenously, driven by a pre-additive noise component during model construction. We establish geometric ergodicity and asymptotic stationarity of the spatiotemporal engression processes under mild assumptions on the network weights and pre-additive noise process. Comprehensive evaluations across six epidemiological datasets over three forecast horizons demonstrate that the proposal consistently outperforms several temporal and spatiotemporal benchmarks in both point and probabilistic forecasting. Additionally, we explore the explainability of the proposal to enhance the models' practical application for informed, timely public health interventions.

Abstract

Probabilistic Time Series Forecasting

Explore similar work

Deep Generative Spatiotemporal Engression for Probabilistic Forecasting of Epidemics

UniMamba: A Unified Spatial-Temporal Modeling Framework with State-Space and Attention Integration

Exploring the Potential of Probabilistic Transformer for Time Series Modeling: A Report on the ST-PT Framework