DiffPTS: Rethinking Diffusion ELBO for Probabilistic Time Series Forecasting

Weiwei Ye; Dongyuan Li; Hangchen Liu; Haotong Jiang; Yoshihide Sekimoto; Renhe Jiang

cs.LGSep 26, 2026

DiffPTS: Rethinking Diffusion ELBO for Probabilistic Time Series Forecasting

Authors: Weiwei Ye, Dongyuan Li, Hangchen Liu, Haotong Jiang, Yoshihide Sekimoto, Renhe Jiang

Organizations: The University of Tokyo · Harbin Institute of Technology, Shenzhen

Abstract

Probabilistic time series forecasting requires modeling and predicting complex and time-varying distributions. Recently, Denoising Diffusion Probabilistic Model (DDPM)-based approaches have shown promise by equipping the dif- fusion process with pretrained mean and variance estimators to accommodate distributional shift. However, these methods typically follow the standard DDPM framework and consider only partial components of the evidence lower bound (ELBO), treating the training of estimators as designed regression tasks separate from the variational inference framework. To address this, we rethink the ELBO under the Location-Scale Noise Model (LSNM) and find that it naturally induces a Gaussian negative log likelihood objective for the estimators and inherently defines a joint training objective that unifies recent diffusion paradigms for probabilistic forecasting. Building on this principled ELBO reformulation, we propose Diff- PTS, a general framework that enables end-to-end optimization of all components within the ELBO. Across multiple benchmarks, DiffPTS consistently outperforms recent models, achieving state-of-the-art performance with an average CRPS/MSE reduction of over 14.53%/16.55% compared to existing diffusion-based methods. The code is available at https://github.com/wwy155/DiffPTS.

Figures & tables

Figure 1: Figure 1 illustrates the comparison of DDPM-based models on the ETTm1 dataset. The left panel displays the prediction intervals, while the right panel shows the corresponding endpoint distributions for TimeGrad, NsDiff, and DiffPTS (ours), respectively. Both TimeGrad and NsDiff utilize the standard ELBO objective from DDPM, whereas DiffPTS reformulates the ELBO to enable end-to-end training within a unified framework.

Models	Target	Endpoint	Objectives
DDPM ( [ 12 ] , [ 13 ] , [ 14 ] )	$\mathbf{Y}$	$\mathcal{N}(\mathbf{0},\mathbf{I})$	$(\epsilon_{\theta}\to\epsilon_{0})$
TMDM [ 33 ]	$\mathbf{Y}$	$\mathcal{N}(f_{\phi}(\mathbf{X}),\mathbf{I})$	$+(f_{\phi}(\mathbf{X})\to\mathbf{Y})$
D3U [ 16 ]	$\mathbf{Y}-f_{\phi}(\mathbf{X})$	$\mathcal{N}(\mathbf{0},\mathbf{I})$	$+(f_{\phi}(\mathbf{X})\to\mathbf{Y})$
RDIT [ 17 ]	$(\mathbf{Y}-f_{\phi}(\mathbf{X}))/\boldsymbol{\sigma}_{trn}$	$\mathcal{N}(\mathbf{0},\mathbf{I})$	$+(g_{\psi}(\mathbf{X})\to\sigma_{\mathbf{Y}})$
NsDiff [ 8 ]	$\mathbf{Y}$	$\mathcal{N}(f_{\phi}(\mathbf{X}),g_{\psi}({\mathbf{X}}))$	$+(g_{\psi}(\mathbf{X})\to\sigma_{\mathbf{Y}})$

Table 1: Summary of recent diffusion-based advancements for probabilistic time series forecasting. We consider the denoising target and reverse endpoint.

\to

indicates a supervised objective and

+

represents an additional objective compared to previous methods.

\mathcal{N}(f_{\phi}(\mathbf{X}),\,g_{\psi}(\mathbf{X}))

denotes

\mathcal{N}(f_{\phi}(\mathbf{X}),\,\mathrm{diag}(g_{\psi}(\mathbf{X})))

; we omit

\mathrm{diag}(\cdot)

for clarity as only diagonal variance is modeled. RDIT uses a training residual

g_{\psi}=\boldsymbol{\sigma}_{trn}

, and NsDiff estimates

\boldsymbol{\sigma}_{\mathbf{Y}}

using future sliding window variance.

Figure 2: Overview of the proposed DiffPTS framework. Training (Top): We optimize the proposed unified ELBO, encompassing: Reconstruction, Denoising, and Prior Matching. This training procedure is paradigm-agnostic and can be readily adapted to other methods in Table 1 . Inference (Bottom): The generative process initiates by sampling from the learned NLL endpoint. The model then iteratively samples from the reverse distribution $p_{\theta}(\mathbf{Y}_{t-1}|\mathbf{Y}_{t},\mathbf{X})$ to progressively recover the complex target distribution $p(\mathbf{Y}_{0}|\mathbf{X},\mathbf{Y}_{T})$ . All visualizations are based on the ETTm1 dataset.

Models	Datasets	ETTh1	ETTh2	ETTm1	ETTm2	ECL	EXG	ILI	Solar	Traffic
TimeGrad	CRPS	0.606	1.212	0.647	0.775	0.397	0.826	1.140	0.293	0.407
(2021)	MSE	1.062	3.462	1.218	1.690	0.505	1.567	4.197	0.475	0.983
CSDI	CRPS	0.492	0.647	0.524	0.817	0.577	0.855	1.244	0.432	1.418
(2022)	MSE	0.949	1.226	1.002	1.723	1.007	1.701	4.515	0.763	1.731
TimeDiff	CRPS	0.465	0.471	0.464	0.316	0.750	0.433	1.153	0.700	0.771
(2023)	MSE	0.517	0.456	0.537	0.268	0.879	0.402	3.958	0.821	1.350

Table 2: Experimental results on nine real-world datasets. Bold face indicates the best result, and underline indicates the second-best result.

Figure 3: A visualization case of a ETTm1 sample. (a) The 96% empirical distribution estimated on the samples with the same time. (b-f) The 96% prediction intervals of different methods. (g) Historical and estimated density of various methods, we use the last step of the same sample in (a-f).

Models	Datasets	ETTh1	ETTh2	ETTm1	ETTm2	ECL	EXG	ILI	Solar	Traffic
DiffPTS	CRPS	0.374	0.355	0.329	0.254	0.218	0.275	0.731	0.173	0.270
DiffPTS	MSE	0.507	0.452	0.424	0.242	0.180	0.242	2.636	0.179	0.456
w/o NLL $f_{\phi}$	CRPS	0.553	0.943	0.540	0.813	0.544	0.944	1.409	0.494	0.540
w/o NLL $f_{\phi}$	MSE	1.125	3.178	1.101	2.527	1.050	3.231	6.493	0.781	1.431
w/o NLL $g_{\psi}$	CRPS	0.416	0.407	0.330	0.279	0.389	0.356	0.800	0.324	0.428
w/o NLL $g_{\psi}$	MSE	0.631	0.538	0.446	0.294	0.361	0.312	2.657	0.211	0.653

Table 3: Ablation results across nine datasets. Bold numbers indicate the best performance.

Models	NsDiff		+DiffPTS		RDIT		+DiffPTS		D3U		+DiffPTS
Datasets	CRPS	MSE	CRPS	MSE	CRPS	MSE	CRPS	MSE	CRPS	MSE	CRPS	MSE
ETTh1	0.392	0.594	0.364	0.576	0.410	0.570	0.338	0.458	0.447	0.684	0.349	0.467
ETTh2	0.358	0.514	0.355	0.495	0.380	0.463	0.363	0.441	0.385	0.520	0.331	0.369
ETTm1	0.346	0.488	0.339	0.472	0.336	0.504	0.320	0.375	0.412	0.589	0.290	0.357
ETTm2	0.256	0.281	0.254	0.278	0.258	0.292	0.245	0.216	0.282	0.286	0.242	0.231

Table 4: Plug-and-play results on NsDiff, RDIT, and D3U. Bold face indicates the best performance.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 4 — Figure 4: Probabilistic graphical model of the conditional diffusion process in DiffPTS. Shaded nodes are observed and rectangles denote the deterministic estimators. The historical window $\mathbf{X}$ conditions the endpoint $p(\mathbf{Y}_{T}\mid\mathbf{X})=\mathcal{N}(f_{\phi}(\mathbf{X}),g_{\psi}(\mathbf{X}))$ through $f_{\phi},g_{\psi}$ , as well as every reverse transition $p_{\theta}(\mathbf{Y}_{t-1}\mid\mathbf{Y}_{t},\mathbf{X})$ (solid arrows). Dashed arrows denote the forward process $q(\mathbf{Y}_{t}\mid\mathbf{Y}_{t-1},\mathbf{X})$ in Equation ( 9 ). This conditional Markov structure yields the posterior factorization derived below.

Figure 5: The 96% prediction intervals of ETTm2, ETTh1, ETTh2 and EXG samples.

Models	Datasets	ETTh1	ETTh2	ETTm1	ETTm2	ECL	EXG	ILI	Solar	Traffic
TimeGrad	CRPS	0.606	1.212	0.647	0.775	0.397	0.826	1.140	0.293	0.407
(2021)	QICE	6.731	9.488	6.693	6.962	7.118	9.464	6.519	7.378	4.581
CSDI	CRPS	0.492	0.647	0.524	0.817	0.577	0.855	1.244	0.432	1.418
(2022)	QICE	3.107	5.331	2.828	8.106	7.506	7.864	7.693	9.957	13.613
TimeDiff	CRPS	0.465	0.471	0.464	0.316	0.750	0.433	1.153	0.700	0.771
(2023)	QICE	14.931	14.813	14.795	13.385	15.466	14.556	14.942	14.914	15.439

Appendix

Table 5: Probabilistic forecasting performance (CRPS, QICE) across nine datasets. Bold face indicates the best result, and underline indicates the second-best result.

Models	Datasets	ETTh1	ETTh2	ETTm1	ETTm2	ECL	EXG	ILI	Solar	Traffic
TimeGrad	MAE	0.813	1.496	0.831	0.967	0.504	1.058	1.414	0.446	0.535
(2021)	MSE	1.062	3.462	1.218	1.690	0.505	1.567	4.197	0.475	0.983
CSDI	MAE	0.708	0.900	0.752	1.069	0.822	1.081	1.481	0.675	0.925
(2022)	MSE	0.949	1.226	1.002	1.723	1.007	1.701	4.515	0.763	1.731
TimeDiff	MAE	0.479	0.485	0.477	0.333	0.764	0.446	1.169	0.713	0.784
(2023)	MSE	0.517	0.456	0.537	0.268	0.879	0.402	3.958	0.821	1.350

Appendix

Table 6: Point forecasting performance (MSE, MAE) across nine datasets. Bold face indicates the best result, and underline indicates the second-best result.

Variants	Datasets	ETTh1	ETTh2	ETTm1	ETTm2	-
DiffPTS	CRPS	0.374±0.009	0.355±0.007	0.329±0.005	0.254±0.007	-
	MSE	0.507±0.033	0.452±0.021	0.424±0.007	0.242±0.031	-
w/o NLL $f_{\phi}$	CRPS	0.553±0.036	0.943±0.013	0.540±0.003	0.813±0.007	-
w/o NLL $f_{\phi}$	MSE	1.125±0.003	3.178±0.053	1.101±0.007	2.527±0.057	-
w/o NLL $g_{\psi}$	CRPS	0.416±0.021	0.407±0.019	0.330±0.008	0.279±0.002	-
w/o NLL $g_{\psi}$	MSE	0.631±0.066	0.538±0.054	0.446±0.021	0.294±0.007	-

Appendix

Table 7: Ablation results with error bars over seeds

\{1,2,3\}

on nine datasets. Bold face indicates the best result.

Models	Datasets	ETTh1	ETTh2	ETTm1	ETTm2	ECL	EXG	ILI	Solar	Traffic
TMDM	CRPS	0.452	0.383	0.375	0.289	0.461	0.336	0.967	0.350	0.557
(2024)	MSE	0.696	0.512	0.494	0.315	0.257	0.334	3.636	0.250	0.679
+DiffPTS	CRPS	0.419	0.380	0.343	0.284	0.358	0.335	0.823	0.306	0.432
(Ours)	MSE	0.628	0.508	0.476	0.320	0.214	0.333	2.976	0.217	0.637
D3U	CRPS	0.447	0.385	0.412	0.282	0.439	0.354	2.097	0.488	0.507
(2025)	MSE	0.684	0.520	0.589	0.286	0.440	0.269	9.951	0.644	0.907

Appendix

Table 8: Plug-and-play results on four methods. Bold face indicates the best result.

	Memory	Average Epoch Time	Training Strategy	CRPS	MSE
TMDM	906MB	162.74ms	pretraining+DDPM	0.452	0.696
+DiffPTS	914MB	76.61ms	joint optimization	0.419	0.628
D3U	675MB	130.40ms	pretraining+DDPM	0.447	0.684
+DiffPTS	683MB	54.78ms	joint optimization	0.349	0.467
RDIT	490MB	105.75ms	pretraining+DDPM	0.410	0.570
+DiffPTS	498MB	47.66ms	joint optimization	0.338	0.458

Appendix

Table 9: Efficiency analysis results of DiffPTS on ETTh1 dataset.

Model	Training	Mean estimator $f_{\phi}$	Variance estimator $g_{\psi}$	Noise schedule
DiffPTS (Table 2 )	Joint ELBO	NS-Transformer	MLP (NLL)	Standard linear
NsDiff	Two-phase	NS-Transformer	MLP (sliding-window target)	Uncertainty-aware
NsDiff+DiffPTS	Joint ELBO	NS-Transformer	MLP (NLL)	Standard linear
RDIT	Two-phase	Mamba [ 54 ]	Fixed training residual	Standard linear
RDIT+DiffPTS	Joint ELBO	Mamba [ 54 ]	MLP (NLL)	Standard linear
D3U	Two-phase	Patch-based	None (implicit $g_{\psi}\equiv 1$ )	Standard linear

Appendix

Table 10: Components of DiffPTS (Table 2 ) and of the plug-and-play hosts with their +DiffPTS variants (Table 4 ). Joint ELBO denotes end-to-end training with Equation ( 14 ).

Training	ETTh1	ETTm1
Two-phase (pretrain, then freeze $f_{\phi}$ and $g_{\psi}$ )	0.404 / 0.647	0.331 / 0.453
Joint ELBO (DiffPTS)	0.374 / 0.507	0.329 / 0.424

Appendix

Table 11: Joint versus two-phase training under an identical architecture, noise schedule, and seeds (CRPS / MSE).

Group	Dataset	Mean / variance shift	TMDM	D3U	RDIT	NsDiff
Non-stationary	Traffic	1.59 / 35.4	22.4%	46.9%	41.6%	27.2%
	EXG	1.45 / 0.98	0.3%	8.2%	25.5%	15.1%
	ILI	1.02 / 1.18	14.9%	63.7%	4.5%	5.6%
	ECL	0.89 / 1.26	22.3%	49.4%	36.2%	24.8%
	ETTm2	1.03 / 1.00	1.7%	14.2%	5.0%	0.8%
	Average	–	12.3%	36.5%	22.6%	14.7%

Appendix

Table 12: Relative CRPS improvement of +DiffPTS over each host, grouped by non-stationarity. The mean and variance shifts are the largest deviations of the sliding-window mean from 0 and of the sliding-window variance from 1 after standardization, where larger values indicate stronger non-stationarity.

Objective	ETTh1	ETTm1	EXG
Exact-weighted ELBO ( $\gamma_{t}$ , $\bar{\alpha}_{T}/2$ )	0.406 / 0.629	0.330 / 0.455	0.281 / 0.275
Simplified weighting (DiffPTS)	0.374 / 0.507	0.329 / 0.424	0.275 / 0.242

Appendix

Table 13: Exact-weighted versus simplified ELBO objective (CRPS / MSE).

Statistic	ETTh1	ETTm1
Learned $g_{\psi}$ , minimum	$1.6\times 10^{-2}$	$5.4\times 10^{-3}$
Learned $g_{\psi}$ , median	0.11	0.10
Learned $g_{\psi}$ , maximum	4.0	2.9
Sliding-window target, minimum	$2.2\times 10^{-5}$	$6.0\times 10^{-7}$
Gaussian NLL with the sliding-window target	0.32	$1.1\times 10^{4}$

Appendix

Table 14: Test-set statistics of the learned variance

g_{\psi}

and of the sliding-window variance target of NsDiff, computed on the ground-truth future.

Dataset	$T=1$	$T=20$	$T=100$	$T=1000$
ETTh1	0.387	0.377	0.432	0.552
ECL	0.232	0.218	0.259	0.498
ETTm2	0.284	0.274	0.353	0.554
EXG	0.307	0.275	0.320	0.639

Appendix

Table 15: CRPS of DiffPTS under different numbers of diffusion steps

T

. All entries come from a separate set of runs under an identical configuration, so the

T=20

column may differ from Table 2 .

Sampler	ETTm1	ETTh1
Ancestral, $T=20$ (Algorithm 2 )	0.333 / 1.31s	0.422 / 0.68s
DPM-Solver, 10 steps	0.340 / 0.85s	0.436 / 0.64s
DPM-Solver, 5 steps	0.340 / 0.61s	0.436 / 0.32s

Appendix

Table 16: Ancestral sampling versus DPM-Solver for DiffPTS (CRPS / seconds per batch). All rows use the same checkpoints and a separate evaluation script, so values are comparable within this table.

Parameterization	ETTh1	ETTm2
$x_{0}$ -prediction	0.513 / 0.778	0.314 / 0.341
$\epsilon$ -prediction (DiffPTS)	0.374 / 0.507	0.254 / 0.242

Appendix

Table 17:

x_{0}

-prediction versus

\epsilon

-prediction for DiffPTS (CRPS / MSE).

Explore similar work

Diffusion-Driven State Space Models
Jun 19, 2026Jack Ruder, Michael WojnowiczLatent Dynamics Modeling Latent Diffusion Models
1Montana State University
When Denoising Hurts: Rethinking the Terminal Step of Diffusion Time Series Forecasters -- Extended Version
Aug 14, 2026Dat Nguyen-Cong, Luong Tran, Tung KieuTime Series Forecasting Diffusion Sampling
FPT Software AI Center, FPT Corporation · Department of Computer Science, Aalborg University, Denmark
DynG-Diff: A State-Aware Dynamic Guidance Diffusion Framework for Probabilistic Time Series Forecasting
Sep 2, 2026Zhente Zhang, Zhengwei Ni, Wei FanMultivariate Time Series Forecasting Diffusion Model Guidance
School of Information and Electronic Engineering (Sussex Artificial Intelligence Institute), Zhejiang Gongshang University, Hangzhou, Zhejiang, China · School of Computer Science, University of Auckland, Auckland, New Zealand

Jun 19, 2026, stat.MLJ/K move · Enter open · S save

Diffusion-Driven State Space Models

Jack Ruder, Michael Wojnowicz

1Montana State University

Latent Dynamics Modeling Latent Diffusion Models Time Series Forecasting State Space Models

Open paper AskPDF arXiv

In many domains, practitioners seek models that produce accurate forecasts while faithfully capturing latent system dynamics. Existing approaches typically sacrifice one of these goals: deep state space models often assume Gaussian latent transitions, limiting fit and forecasting, while diffusion models are highly expressive but lack principled inference for the underlying dynamics. To combine the strengths of both, we introduce the Diffusion-Driven State Space Model (DDSSM), which replaces the conventional Gaussian transition distribution with a diffusion model. Our DDSSM resolves the open problem of how to jointly train an autoencoder and a diffusion model on sequential data, thereby extending the literature on latent diffusion models for time series. Moreover, we find that the DDSSM empirically outperforms a state-of-the-art deep SSM at fitting and forecasting a simulated time series with multimodal transitions.

Abstract

Explore similar work

Diffusion-Driven State Space Models

When Denoising Hurts: Rethinking the Terminal Step of Diffusion Time Series Forecasters -- Extended Version

DynG-Diff: A State-Aware Dynamic Guidance Diffusion Framework for Probabilistic Time Series Forecasting