DiffPTS: Rethinking Diffusion ELBO for Probabilistic Time Series Forecasting

Weiwei Ye; Dongyuan Li; Hangchen Liu; Haotong Jiang; Yoshihide Sekimoto; Renhe Jiang

cs.LGSep 26, 2026

DiffPTS: Rethinking Diffusion ELBO for Probabilistic Time Series Forecasting

Authors: Weiwei Ye, Dongyuan Li, Hangchen Liu, Haotong Jiang, Yoshihide Sekimoto, Renhe Jiang

Organizations: The University of Tokyo · Harbin Institute of Technology, Shenzhen

Abstract

Probabilistic time series forecasting requires modeling and predicting complex and time-varying distributions. Recently, Denoising Diffusion Probabilistic Model (DDPM)-based approaches have shown promise by equipping the dif- fusion process with pretrained mean and variance estimators to accommodate distributional shift. However, these methods typically follow the standard DDPM framework and consider only partial components of the evidence lower bound (ELBO), treating the training of estimators as designed regression tasks separate from the variational inference framework. To address this, we rethink the ELBO under the Location-Scale Noise Model (LSNM) and find that it naturally induces a Gaussian negative log likelihood objective for the estimators and inherently defines a joint training objective that unifies recent diffusion paradigms for probabilistic forecasting. Building on this principled ELBO reformulation, we propose Diff- PTS, a general framework that enables end-to-end optimization of all components within the ELBO. Across multiple benchmarks, DiffPTS consistently outperforms recent models, achieving state-of-the-art performance with an average CRPS/MSE reduction of over 14.53%/16.55% compared to existing diffusion-based methods. The code is available at https://github.com/wwy155/DiffPTS.

Figures & tables

Figure 1: Figure 1 illustrates the comparison of DDPM-based models on the ETTm1 dataset. The left panel displays the prediction intervals, while the right panel shows the corresponding endpoint distributions for TimeGrad, NsDiff, and DiffPTS (ours), respectively. Both TimeGrad and NsDiff utilize the standard ELBO objective from DDPM, whereas DiffPTS reformulates the ELBO to enable end-to-end training within a unified framework.

Models	Target	Endpoint	Objectives
DDPM ( [ 12 ] , [ 13 ] , [ 14 ] )	$\mathbf{Y}$	$\mathcal{N}(\mathbf{0},\mathbf{I})$	$(\epsilon_{\theta}\to\epsilon_{0})$
TMDM [ 33 ]	$\mathbf{Y}$	$\mathcal{N}(f_{\phi}(\mathbf{X}),\mathbf{I})$	$+(f_{\phi}(\mathbf{X})\to\mathbf{Y})$
D3U [ 16 ]	$\mathbf{Y}-f_{\phi}(\mathbf{X})$	$\mathcal{N}(\mathbf{0},\mathbf{I})$	$+(f_{\phi}(\mathbf{X})\to\mathbf{Y})$
RDIT [ 17 ]	$(\mathbf{Y}-f_{\phi}(\mathbf{X}))/\boldsymbol{\sigma}_{trn}$	$\mathcal{N}(\mathbf{0},\mathbf{I})$	$+(g_{\psi}(\mathbf{X})\to\sigma_{\mathbf{Y}})$
NsDiff [ 8 ]	$\mathbf{Y}$	$\mathcal{N}(f_{\phi}(\mathbf{X}),g_{\psi}({\mathbf{X}}))$	$+(g_{\psi}(\mathbf{X})\to\sigma_{\mathbf{Y}})$

Table 1: Summary of recent diffusion-based advancements for probabilistic time series forecasting. We consider the denoising target and reverse endpoint.

\to

indicates a supervised objective and

+

represents an additional objective compared to previous methods.

\mathcal{N}(f_{\phi}(\mathbf{X}),\,g_{\psi}(\mathbf{X}))

denotes

\mathcal{N}(f_{\phi}(\mathbf{X}),\,\mathrm{diag}(g_{\psi}(\mathbf{X})))

; we omit

\mathrm{diag}(\cdot)

for clarity as only diagonal variance is modeled. RDIT uses a training residual

g_{\psi}=\boldsymbol{\sigma}_{trn}

, and NsDiff estimates

\boldsymbol{\sigma}_{\mathbf{Y}}

using future sliding window variance.

Figure 2: Overview of the proposed DiffPTS framework. Training (Top): We optimize the proposed unified ELBO, encompassing: Reconstruction, Denoising, and Prior Matching. This training procedure is paradigm-agnostic and can be readily adapted to other methods in Table 1 . Inference (Bottom): The generative process initiates by sampling from the learned NLL endpoint. The model then iteratively samples from the reverse distribution $p_{\theta}(\mathbf{Y}_{t-1}|\mathbf{Y}_{t},\mathbf{X})$ to progressively recover the complex target distribution $p(\mathbf{Y}_{0}|\mathbf{X},\mathbf{Y}_{T})$ . All visualizations are based on the ETTm1 dataset.

Models	Datasets	ETTh1	ETTh2	ETTm1	ETTm2	ECL	EXG	ILI	Solar	Traffic
TimeGrad	CRPS	0.606	1.212	0.647	0.775	0.397	0.826	1.140	0.293	0.407
(2021)	MSE	1.062	3.462	1.218	1.690	0.505	1.567	4.197	0.475	0.983
CSDI	CRPS	0.492	0.647	0.524	0.817	0.577	0.855	1.244	0.432	1.418
(2022)	MSE	0.949	1.226	1.002	1.723	1.007	1.701	4.515	0.763	1.731
TimeDiff	CRPS	0.465	0.471	0.464	0.316	0.750	0.433	1.153	0.700	0.771
(2023)	MSE	0.517	0.456	0.537	0.268	0.879	0.402	3.958	0.821	1.350

Table 2: Experimental results on nine real-world datasets. Bold face indicates the best result, and underline indicates the second-best result.

Figure 3: A visualization case of a ETTm1 sample. (a) The 96% empirical distribution estimated on the samples with the same time. (b-f) The 96% prediction intervals of different methods. (g) Historical and estimated density of various methods, we use the last step of the same sample in (a-f).

Models	Datasets	ETTh1	ETTh2	ETTm1	ETTm2	ECL	EXG	ILI	Solar	Traffic
DiffPTS	CRPS	0.374	0.355	0.329	0.254	0.218	0.275	0.731	0.173	0.270
DiffPTS	MSE	0.507	0.452	0.424	0.242	0.180	0.242	2.636	0.179	0.456
w/o NLL $f_{\phi}$	CRPS	0.553	0.943	0.540	0.813	0.544	0.944	1.409	0.494	0.540
w/o NLL $f_{\phi}$	MSE	1.125	3.178	1.101	2.527	1.050	3.231	6.493	0.781	1.431
w/o NLL $g_{\psi}$	CRPS	0.416	0.407	0.330	0.279	0.389	0.356	0.800	0.324	0.428
w/o NLL $g_{\psi}$	MSE	0.631	0.538	0.446	0.294	0.361	0.312	2.657	0.211	0.653

Table 3: Ablation results across nine datasets. Bold numbers indicate the best performance.

Models	NsDiff		+DiffPTS		RDIT		+DiffPTS		D3U		+DiffPTS
Datasets	CRPS	MSE	CRPS	MSE	CRPS	MSE	CRPS	MSE	CRPS	MSE	CRPS	MSE
ETTh1	0.392	0.594	0.364	0.576	0.410	0.570	0.338	0.458	0.447	0.684	0.349	0.467
ETTh2	0.358	0.514	0.355	0.495	0.380	0.463	0.363	0.441	0.385	0.520	0.331	0.369
ETTm1	0.346	0.488	0.339	0.472	0.336	0.504	0.320	0.375	0.412	0.589	0.290	0.357
ETTm2	0.256	0.281	0.254	0.278	0.258	0.292	0.245	0.216	0.282	0.286	0.242	0.231

Table 4: Plug-and-play results on NsDiff, RDIT, and D3U. Bold face indicates the best performance.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 4 — Figure 4: Probabilistic graphical model of the conditional diffusion process in DiffPTS. Shaded nodes are observed and rectangles denote the deterministic estimators. The historical window $\mathbf{X}$ conditions the endpoint $p(\mathbf{Y}_{T}\mid\mathbf{X})=\mathcal{N}(f_{\phi}(\mathbf{X}),g_{\psi}(\mathbf{X}))$ through $f_{\phi},g_{\psi}$ , as well as every reverse transition $p_{\theta}(\mathbf{Y}_{t-1}\mid\mathbf{Y}_{t},\mathbf{X})$ (solid arrows). Dashed arrows denote the forward process $q(\mathbf{Y}_{t}\mid\mathbf{Y}_{t-1},\mathbf{X})$ in Equation ( 9 ). This conditional Markov structure yields the posterior factorization derived below.

Figure 5: The 96% prediction intervals of ETTm2, ETTh1, ETTh2 and EXG samples.

Models	Datasets	ETTh1	ETTh2	ETTm1	ETTm2	ECL	EXG	ILI	Solar	Traffic
TimeGrad	CRPS	0.606	1.212	0.647	0.775	0.397	0.826	1.140	0.293	0.407
(2021)	QICE	6.731	9.488	6.693	6.962	7.118	9.464	6.519	7.378	4.581
CSDI	CRPS	0.492	0.647	0.524	0.817	0.577	0.855	1.244	0.432	1.418
(2022)	QICE	3.107	5.331	2.828	8.106	7.506	7.864	7.693	9.957	13.613
TimeDiff	CRPS	0.465	0.471	0.464	0.316	0.750	0.433	1.153	0.700	0.771
(2023)	QICE	14.931	14.813	14.795	13.385	15.466	14.556	14.942	14.914	15.439

Appendix

Table 5: Probabilistic forecasting performance (CRPS, QICE) across nine datasets. Bold face indicates the best result, and underline indicates the second-best result.

Models	Datasets	ETTh1	ETTh2	ETTm1	ETTm2	ECL	EXG	ILI	Solar	Traffic
TimeGrad	MAE	0.813	1.496	0.831	0.967	0.504	1.058	1.414	0.446	0.535
(2021)	MSE	1.062	3.462	1.218	1.690	0.505	1.567	4.197	0.475	0.983
CSDI	MAE	0.708	0.900	0.752	1.069	0.822	1.081	1.481	0.675	0.925
(2022)	MSE	0.949	1.226	1.002	1.723	1.007	1.701	4.515	0.763	1.731
TimeDiff	MAE	0.479	0.485	0.477	0.333	0.764	0.446	1.169	0.713	0.784
(2023)	MSE	0.517	0.456	0.537	0.268	0.879	0.402	3.958	0.821	1.350

Appendix

Table 6: Point forecasting performance (MSE, MAE) across nine datasets. Bold face indicates the best result, and underline indicates the second-best result.

Variants	Datasets	ETTh1	ETTh2	ETTm1	ETTm2	-
DiffPTS	CRPS	0.374±0.009	0.355±0.007	0.329±0.005	0.254±0.007	-
	MSE	0.507±0.033	0.452±0.021	0.424±0.007	0.242±0.031	-
w/o NLL $f_{\phi}$	CRPS	0.553±0.036	0.943±0.013	0.540±0.003	0.813±0.007	-
w/o NLL $f_{\phi}$	MSE	1.125±0.003	3.178±0.053	1.101±0.007	2.527±0.057	-
w/o NLL $g_{\psi}$	CRPS	0.416±0.021	0.407±0.019	0.330±0.008	0.279±0.002	-
w/o NLL $g_{\psi}$	MSE	0.631±0.066	0.538±0.054	0.446±0.021	0.294±0.007	-

Appendix

Table 7: Ablation results with error bars over seeds

\{1,2,3\}

on nine datasets. Bold face indicates the best result.

Models	Datasets	ETTh1	ETTh2	ETTm1	ETTm2	ECL	EXG	ILI	Solar	Traffic
TMDM	CRPS	0.452	0.383	0.375	0.289	0.461	0.336	0.967	0.350	0.557
(2024)	MSE	0.696	0.512	0.494	0.315	0.257	0.334	3.636	0.250	0.679
+DiffPTS	CRPS	0.419	0.380	0.343	0.284	0.358	0.335	0.823	0.306	0.432
(Ours)	MSE	0.628	0.508	0.476	0.320	0.214	0.333	2.976	0.217	0.637
D3U	CRPS	0.447	0.385	0.412	0.282	0.439	0.354	2.097	0.488	0.507
(2025)	MSE	0.684	0.520	0.589	0.286	0.440	0.269	9.951	0.644	0.907

Appendix

Table 8: Plug-and-play results on four methods. Bold face indicates the best result.

	Memory	Average Epoch Time	Training Strategy	CRPS	MSE
TMDM	906MB	162.74ms	pretraining+DDPM	0.452	0.696
+DiffPTS	914MB	76.61ms	joint optimization	0.419	0.628
D3U	675MB	130.40ms	pretraining+DDPM	0.447	0.684
+DiffPTS	683MB	54.78ms	joint optimization	0.349	0.467
RDIT	490MB	105.75ms	pretraining+DDPM	0.410	0.570
+DiffPTS	498MB	47.66ms	joint optimization	0.338	0.458

Appendix

Table 9: Efficiency analysis results of DiffPTS on ETTh1 dataset.

Model	Training	Mean estimator $f_{\phi}$	Variance estimator $g_{\psi}$	Noise schedule
DiffPTS (Table 2 )	Joint ELBO	NS-Transformer	MLP (NLL)	Standard linear
NsDiff	Two-phase	NS-Transformer	MLP (sliding-window target)	Uncertainty-aware
NsDiff+DiffPTS	Joint ELBO	NS-Transformer	MLP (NLL)	Standard linear
RDIT	Two-phase	Mamba [ 54 ]	Fixed training residual	Standard linear
RDIT+DiffPTS	Joint ELBO	Mamba [ 54 ]	MLP (NLL)	Standard linear
D3U	Two-phase	Patch-based	None (implicit $g_{\psi}\equiv 1$ )	Standard linear

Appendix

Table 10: Components of DiffPTS (Table 2 ) and of the plug-and-play hosts with their +DiffPTS variants (Table 4 ). Joint ELBO denotes end-to-end training with Equation ( 14 ).

Training	ETTh1	ETTm1
Two-phase (pretrain, then freeze $f_{\phi}$ and $g_{\psi}$ )	0.404 / 0.647	0.331 / 0.453
Joint ELBO (DiffPTS)	0.374 / 0.507	0.329 / 0.424

Appendix

Table 11: Joint versus two-phase training under an identical architecture, noise schedule, and seeds (CRPS / MSE).

Group	Dataset	Mean / variance shift	TMDM	D3U	RDIT	NsDiff
Non-stationary	Traffic	1.59 / 35.4	22.4%	46.9%	41.6%	27.2%
	EXG	1.45 / 0.98	0.3%	8.2%	25.5%	15.1%
	ILI	1.02 / 1.18	14.9%	63.7%	4.5%	5.6%
	ECL	0.89 / 1.26	22.3%	49.4%	36.2%	24.8%
	ETTm2	1.03 / 1.00	1.7%	14.2%	5.0%	0.8%
	Average	–	12.3%	36.5%	22.6%	14.7%

Appendix

Table 12: Relative CRPS improvement of +DiffPTS over each host, grouped by non-stationarity. The mean and variance shifts are the largest deviations of the sliding-window mean from 0 and of the sliding-window variance from 1 after standardization, where larger values indicate stronger non-stationarity.

Objective	ETTh1	ETTm1	EXG
Exact-weighted ELBO ( $\gamma_{t}$ , $\bar{\alpha}_{T}/2$ )	0.406 / 0.629	0.330 / 0.455	0.281 / 0.275
Simplified weighting (DiffPTS)	0.374 / 0.507	0.329 / 0.424	0.275 / 0.242

Appendix

Table 13: Exact-weighted versus simplified ELBO objective (CRPS / MSE).

Statistic	ETTh1	ETTm1
Learned $g_{\psi}$ , minimum	$1.6\times 10^{-2}$	$5.4\times 10^{-3}$
Learned $g_{\psi}$ , median	0.11	0.10
Learned $g_{\psi}$ , maximum	4.0	2.9
Sliding-window target, minimum	$2.2\times 10^{-5}$	$6.0\times 10^{-7}$
Gaussian NLL with the sliding-window target	0.32	$1.1\times 10^{4}$

Appendix

Table 14: Test-set statistics of the learned variance

g_{\psi}

and of the sliding-window variance target of NsDiff, computed on the ground-truth future.

Dataset	$T=1$	$T=20$	$T=100$	$T=1000$
ETTh1	0.387	0.377	0.432	0.552
ECL	0.232	0.218	0.259	0.498
ETTm2	0.284	0.274	0.353	0.554
EXG	0.307	0.275	0.320	0.639

Appendix

Table 15: CRPS of DiffPTS under different numbers of diffusion steps

T

. All entries come from a separate set of runs under an identical configuration, so the

T=20

column may differ from Table 2 .

Sampler	ETTm1	ETTh1
Ancestral, $T=20$ (Algorithm 2 )	0.333 / 1.31s	0.422 / 0.68s
DPM-Solver, 10 steps	0.340 / 0.85s	0.436 / 0.64s
DPM-Solver, 5 steps	0.340 / 0.61s	0.436 / 0.32s

Appendix

Table 16: Ancestral sampling versus DPM-Solver for DiffPTS (CRPS / seconds per batch). All rows use the same checkpoints and a separate evaluation script, so values are comparable within this table.

Parameterization	ETTh1	ETTm2
$x_{0}$ -prediction	0.513 / 0.778	0.314 / 0.341
$\epsilon$ -prediction (DiffPTS)	0.374 / 0.507	0.254 / 0.242

Appendix

Table 17:

x_{0}

-prediction versus

\epsilon

-prediction for DiffPTS (CRPS / MSE).

Explore similar work

Jun 19, 2026stat.ML

Diffusion-Driven State Space Models

In many domains, practitioners seek models that produce accurate forecasts while faithfully capturing latent system dynamics. Existing approaches typically sacrifice one of these goals: deep state space models often assume Gaussian latent transitions, limiting fit and forecasting, while diffusion models are highly expressive but lack principled inference for the underlying dynamics. To combine the strengths of both, we introduce the Diffusion-Driven State Space Model (DDSSM), which replaces the conventional Gaussian transition distribution with a diffusion model. Our DDSSM resolves the open problem of how to jointly train an autoencoder and a diffusion model on sequential data, thereby extending the literature on latent diffusion models for time series. Moreover, we find that the DDSSM empirically outperforms a state-of-the-art deep SSM at fitting and forecasting a simulated time series with multimodal transitions.

Jack Ruder, Michael Wojnowicz

1Montana State University

Latent Dynamics Modeling Latent Diffusion Models Time Series Forecasting State Space Models

Ask PDF

Aug 14, 2026cs.LG

When Denoising Hurts: Rethinking the Terminal Step of Diffusion Time Series Forecasters -- Extended Version

Diffusion models offer a natural way to model uncertainty in time series forecasting, yet their iterative sampling process is often treated as a uniformly beneficial refinement procedure. Our study challenges this view by examining how forecast quality evolves throughout reverse diffusion. We find that general temporal structure is often recovered at relatively high noise levels, whereas continued low-noise refinement can introduce statistical drift and degrade the final forecast. Our analysis further suggests that this behavior explains why prior methods often favor relatively narrow diffusion architecture and schedule design. Building on this observation, we propose a label-free global stopping criterion that detects the optimal termination point, eventually speeding up inference and improving predictive accuracy. Additionally, since early stopping terminates inference in high-noise regions, we propose a Bernoulli timestep sampler that concentrates training on this region while preserving coverage of the full diffusion process. Extensive experiments conducted across eight real-world datasets demonstrate the superior performance of our method compared to existing approaches.

Dat Nguyen-Cong, Luong Tran, Tung Kieu

FPT Software AI Center, FPT Corporation · Department of Computer Science, Aalborg University, Denmark

Time Series Forecasting Diffusion Sampling Diffusion Timestep Scheduling Diffusion Model Inference Acceleration

Ask PDF

Sep 2, 2026cs.LG

DynG-Diff: A State-Aware Dynamic Guidance Diffusion Framework for Probabilistic Time Series Forecasting

Probabilistic multivariate time series (MTS) forecasting is crucial for modeling complex dynamical systems. However, existing diffusion-based methods rely on task-specific conditional paradigms that lack flexibility and struggle with inherent "information heterogeneity"--the significantly varying noise levels and evolutionary patterns across variables. To address this, we propose DynG-Diff, a variable-sensitive dynamic guidance diffusion framework for probabilistic multivariate time-series forecasting: (1) DynG-Diff adopts a two-stage separated training strategy and uses an unconditional diffusion backbone to model the joint distribution of multivariate time series. (2) DynG-Diff introduces a lightweight state-aware policy network that adaptively infers variable reliability from real-time noisy states and one-step denoising estimates, outputting a dynamic guidance strength matrix. (3) DynG-Diff mathematically formulates this dynamic weight as the local precision of the observation distribution, enabling precise guidance for high-confidence variables during inference while filtering out interference from anomalous noise. Extensive experiments on real-world benchmarks demonstrate competitive probabilistic forecasting performance against state-of-the-art conditional diffusion models and improved robustness under severe observation corruption.The implementation code is available at: https://github.com/TT-20011031/DynG-Diff

Zhente Zhang, Zhengwei Ni, Wei Fan

School of Information and Electronic Engineering (Sussex Artificial Intelligence Institute), Zhejiang Gongshang University, Hangzhou, Zhejiang, China · School of Computer Science, University of Auckland, Auckland, New Zealand

Multivariate Time Series Forecasting Diffusion Model Guidance Time Series Forecasting Probabilistic Time Series Forecasting Diffusion Models

Ask PDF