When Denoising Hurts: Rethinking the Terminal Step of Diffusion Time Series Forecasters -- Extended Version

Dat Nguyen-Cong; Luong Tran; Tung Kieu

cs.LGAug 14, 2026

When Denoising Hurts: Rethinking the Terminal Step of Diffusion Time Series Forecasters -- Extended Version

Authors: Dat Nguyen-Cong, Luong Tran, Tung Kieu

Organizations: FPT Software AI Center, FPT Corporation · Department of Computer Science, Aalborg University, Denmark

Abstract

Diffusion models offer a natural way to model uncertainty in time series forecasting, yet their iterative sampling process is often treated as a uniformly beneficial refinement procedure. Our study challenges this view by examining how forecast quality evolves throughout reverse diffusion. We find that general temporal structure is often recovered at relatively high noise levels, whereas continued low-noise refinement can introduce statistical drift and degrade the final forecast. Our analysis further suggests that this behavior explains why prior methods often favor relatively narrow diffusion architecture and schedule design. Building on this observation, we propose a label-free global stopping criterion that detects the optimal termination point, eventually speeding up inference and improving predictive accuracy. Additionally, since early stopping terminates inference in high-noise regions, we propose a Bernoulli timestep sampler that concentrates training on this region while preserving coverage of the full diffusion process. Extensive experiments conducted across eight real-world datasets demonstrate the superior performance of our method compared to existing approaches.

Figures & tables

Panel of Figure 1 — Figure 1: (a) Performance is consistent across different numbers of sampling steps, even better with smaller steps. (b) During generation, the intermediate predicted samples drift significantly when the noise levels are small.

Panel of Figure 2 — Figure 2: Comparison of Different Schedules. (a) Signal scale decay $\sqrt{\bar{\alpha}_{t}}$ over timesteps $t$ with stability transition points. (b) Per-timestep loss curves showing convergence into the stable loss regime. (c) Candidate trajectory mean squared errors across varying noise levels.

Figure 3: Framework Overview. Breakpoint Estimation: Prediction trajectory of each sample $\mathbf{x}_{i}$ is summarized into $\mathbf{s}[\mathbf{x}_{i}]$ , where piecewise-linear fitting detects the optimal transition breakpoint $t_{\delta}*$ . Bernoulli Sampler: Training timesteps are adaptively sampled with probability $p$ focused on the high-noise region while retaining full trajectory coverage. (c) Early-Stopped Accelerated Inference: Reverse diffusion terminates early at $t_{\delta}$ , preventing error propagation and reducing compute costs.

Model	Ours		iTransformer		PatchTST		DLinear		TimesNet		TiDE		TimeMixer		CNDiff		TimeXer
Dataset	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTh1	0.426	0.426	0.459	0.456	0.499	0.481	0.437	0.443	0.534	0.502	0.444	0.441	0.444	0.441	0.454	0.461	0.456	0.451
ETTh2	0.359	0.384	0.404	0.423	0.380	0.414	0.486	0.477	0.425	0.439	0.375	0.406	0.372	0.409	0.383	0.420	0.380	0.409
ETTm1	0.358	0.368	0.374	0.396	0.363	0.390	0.368	0.385	0.531	0.487	0.372	0.384	0.365	0.389	0.488	0.452	0.363	0.389
ETTm2	0.264	0.309	0.289	0.336	0.277	0.326	0.310	0.366	0.310	0.346	0.273	0.323	0.265	0.318	0.290	0.344	0.267	0.318
Weather	0.229	0.254	0.245	0.274	0.239	0.268	0.255	0.308	0.282	0.300	0.262	0.286	0.234	0.271	0.304	0.343	0.232	0.267

Table 1: Forecasting performance comparison. The average results of all predicted lengths are listed here. Best results are highlighted in red, and second-best results are underlined in blue.

Dataset	Full	Early	CSDI	TMDM	D 3 U	NsDiff	CNDiff
ETTh1	0.303	0.306	0.555	0.385	0.314	0.347	0.421
ETTm1	0.278	0.307	0.359	0.348	0.270	0.320	0.341
Weather	0.150	0.161	0.137	0.202	0.170	0.212	0.219
Electricity	0.176	0.209	0.257	0.463	0.174	0.287	0.213
Exchange	0.178	0.183	0.582	0.228	0.219	0.218	0.197
Traffic	0.205	0.236	–	0.557	0.199	0.363	0.300

Table 2: Probabilistic performance comparison. The results are reported with prediction length

P=96

. Due to the excessive time and memory consumption of CSDI in long-term forecasting, its results are unavailable. The best results are highlighted in red , while the second-best results are blue .

Early Stop	Bernoulli	MSE	MAE	CRPS
$\times$	$\times$	0.450	0.420	0.322
$\times$	$\checkmark$	0.412	0.398	0.306
$\checkmark$	$\times$	0.339	0.358	0.309
$\checkmark$	$\checkmark$	0.333	0.357	0.335

Table 3: Ablation study of early stopping and Bernoulli timestep sampling. (Bold indicates best performance).

Panel of Figure 4 — Figure 4: Performance comparison across training epochs under different noise schedules.

Metric	Method	10	20	50	100	200	1000
MSE	Full	0.395	0.424	0.475	0.516	0.556	0.644
MSE	Early	0.346	0.346	0.347	0.350	0.350	0.354
Time	Full	1m8s	2m13s	4m48s	9m23s	17m39s	86m28s
Time	Early	59s (8s)	58s (8s)	4m1s (10s)	4m29s (10s)	8m36s (19s)	47m20s (1m14s)

Table 4: Performance comparison under different numbers of sampling steps and inference time on ETTm1 (

P=192

). Green subtext reports the breakpoint estimation time.

Dataset	0.7	0.8	0.85	0.9	0.95	0.99
ETTm1	0.363	0.336	0.344	0.333	0.358	0.336
Weather	0.206	0.203	0.205	0.205	0.206	0.205
Exchange	0.185	0.171	0.175	0.171	0.176	0.182

Table 5: Sensitivity analysis (MSE) under different Bernoulli probability

p

. (Bold indicates best performance).

Variance	GT Train		GT Val		Mean
Dataset	$t_{\delta}$	MSE	$t_{\delta}$	MSE	$t_{\delta}$	MSE
ETTm1	350	0.333	350	0.333	350	0.333
Weather	400	0.206	350	0.206	350	0.206
Exchange	600	0.173	600	0.173	550	0.172

Table 6: Comparison of stopping points estimated using different scores.

Dataset	ETTm1		Weather		Exchange
Variance	MSE	MAE	MSE	MAE	MSE	MAE
Global	0.333	0.357	0.206	0.234	0.172	0.295
Instance	0.334	0.358	0.206	0.233	0.172	0.296

Table 7: Comparison between global and instance-level stopping-point estimation. (Bold indicates best performance).

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Datasets	Dropout	Encoder	Decoder
ETTh1	{0.4,0.4,0.4,0.4}	{2,2,2,2}	{2,3,3,4}
ETTh2	{0.4,0.4,0.4,0.4}	{2,2,2,2}	{2,3,3,4}
ETTm1	{0.4,0.4,0.4,0.4}	{3,3,3,3}	{3,3,3,3}
ETTm2	{0.4,0.4,0.4,0.4}	{3,3,3,3}	{3,3,3,3}
Exchange	{0.2,0.2,0.2,0.2}	{2,2,2,2}	{2,2,2,2}
Weather	{0.4,0.4,0.4,0.2}	{3,3,3,3}	{3,3,3,4}

Appendix

Table A1: Best hyperparameters for different datasets. The parameter order corresponds to prediction length {96, 192, 336, 720}.

Dataset	Frequency	Variables	# Observations
ETTh1	Hourly	7	17,420
ETTh2	Hourly	7	17,420
ETTm1	15 minutes	7	69,680
ETTm2	15 minutes	7	69,680
Weather	10 minutes	21	52,696
Electricity	Hourly	321	26,304

Appendix

Table A2: Dataset statistics and forecasting configurations.

Panel of Figure 14 — Figure A1: Qualitative comparison of probabilistic forecasts obtained using the full and truncated reverse diffusion processes. The shaded regions represent the 95% prediction intervals.

Figure A2: Sensitivity analysis on ETTm1, Weather, and Exchange datasets with stopping steps highlighted. The noise level is scaled down by a factor of

T/M=50

. The estimated stopping point is indicated by a circle.

Dataset	Save (%)	Early	CSDI	TMDM	D 3 U	NsDiff	CNDiff
ETTh1	55	0.109	4.874	1.141	0.172	0.067	0.425
ETTm1	55	0.106	4.892	1.141	0.170	0.081	0.313
Weather	35	0.085	15.412	1.125	0.251	0.078	0.263
Exchange	55	0.079	6.020	1.137	0.170	0.070	0.275
Electricity	45	0.916	199.882	2.905	1.770	0.122	1.864
Traffic	40	2.280	635.500	5.266	2.507	1.238	4.890

Appendix

Table A3: Inference time (s) for generating 1 sample with prediction length

P=192

, using

S=100

candidates. Save denotes the percentage of reverse-sampling computation removed by early stopping relative to the complete trajectory.

			ETTm1		Weather		Exchange
Schedule	Step	Metric	Full	Ours	Full	Ours	Full	Ours
Linear	$T=20$	MSE	0.353	0.331	0.201	0.200	0.274	0.183
	$\beta_{T}=0.5$	MAE	0.373	0.356	0.240	0.234	0.394	0.314
	$T=50$	MSE	0.350	0.333	0.214	0.200	0.178	0.168
	$\beta_{T}=0.3$	MAE	0.370	0.358	0.249	0.235	0.302	0.295
	$T=100$	MSE	0.363	0.327	0.201	0.198	0.200	0.167

Appendix

Table A4: Forecasting performance under different noise schedules and diffusion steps. The prediction length is set to

P=192

. Bold values indicate better performance.

Dataset	Metric	$P$	Ours	iTransformer	PatchTST	DLinear	TimesNet	TiDE	TimeMixer	CNDiff	TimeXer
ETTh1	MSE	96	0.365	0.393	0.415	0.381	0.460	0.390	0.389	0.416	0.390
		192	0.418	0.441	0.464	0.424	0.480	0.434	0.444	0.450	0.430
		336	0.452	0.486	0.522	0.457	0.532	0.477	0.467	0.472	0.475
		720	0.468	0.516	0.593	0.484	0.663	0.473	0.475	0.477	0.530
		avg	0.426	0.459	0.499	0.437	0.534	0.444	0.444	0.454	0.456
	MAE	96	0.392	0.412	0.428	0.401	0.453	0.404	0.406	0.430	0.406

Appendix

Table A5: Full forecasting results on the ETT benchmarks. The best results are highlighted in red , while the second-best results are blue . Rankings are determined using the original unrounded values.

Dataset	Metric	$P$	Ours	iTransformer	PatchTST	DLinear	TimesNet	TiDE	TimeMixer	CNDiff	TimeXer
Weather	MSE	96	0.152	0.166	0.157	0.183	0.178	0.187	0.161	0.261	0.152
		192	0.195	0.213	0.205	0.230	0.240	0.229	0.201	0.337	0.197
		336	0.248	0.263	0.257	0.270	0.306	0.280	0.250	0.263	0.251
		720	0.322	0.339	0.335	0.337	0.403	0.350	0.325	0.355	0.326
		avg	0.229	0.245	0.239	0.255	0.282	0.262	0.234	0.304	0.232
	MAE	96	0.185	0.210	0.202	0.242	0.223	0.230	0.213	0.295	0.200

Appendix

Table A6: Full forecasting results on the remaining datasets. The best results are highlighted in red , while the second-best results are blue .

Explore similar work

Temporal Difference Learning for Diffusion Models
Jun 13, 2026Qizhen Ying, Yangchen Pan, Victor Adrian Prisacariu +1Few-Step Diffusion Sampling RL for Diffusion Models
Department of Engineering Science, University of Oxford, Oxford, United Kingdom · School of Computer Science, Carleton University, Ottawa, Canada.
DiFA: Inference-Time Forward-Process Alignment for Diffusion Models
Jul 20, 2026Shigui Li, Delu ZengDiffusion Model Alignment Diffusion Sampling
School of Mathematics, South China University of Technology, Guangzhou, China · School of Electronic and Information Engineering, South China University of Technology, Guangzhou, China
Adaptive Spectral Feature Forecasting for Diffusion Sampling Acceleration
Mar 2, 2026Jiaqi Han, Juntong Shi, Puheng Li +3Video Diffusion Models Diffusion-Based Image Generation
Stanford University · ByteDance

Jun 13, 2026, cs.LGJ/K move · Enter open · S save

Temporal Difference Learning for Diffusion Models

Qizhen Ying, Yangchen Pan, Victor Adrian Prisacariu, Junfeng Wen

Department of Engineering Science, University of Oxford, Oxford, United Kingdom · School of Computer Science, Carleton University, Ottawa, Canada.

Few-Step Diffusion Sampling RL for Diffusion Models Diffusion Models

Open paper AskPDF arXiv

Diffusion models are typically trained with objectives that focus on local denoising targets at individual time steps (or adjacent pairs), which do not enforce consistency between predictions along the denoising trajectory. This lack of cross-time consistency can degrade performance, especially for few-step samplers. We introduce a temporal difference (TD) objective that penalizes inconsistency of the model's multi-step progress along the denoising path. By reformulating the diffusion process as a Markov reward process and casting denoising as a policy evaluation problem in reinforcement learning, we derive a unified TD approach that applies to both discrete- and continuous-time diffusion formulations. We further propose a principled sample-based reweighting method that stabilizes training. Empirically, we show that using our TD training can significantly improve sample quality measured by FID, with stronger advantages when the number of sampling steps is small, highlighting its practical utility under low-computation-budget scenarios. We provide ablation studies to justify our design choices, including pairwise loss reweighting, regularization weight, and one-step stride. Overall, our TD approach can be a general drop-in that enforces cross-time consistency and improves generation quality across different diffusion generative models.

Abstract

Explore similar work

Temporal Difference Learning for Diffusion Models

DiFA: Inference-Time Forward-Process Alignment for Diffusion Models

Adaptive Spectral Feature Forecasting for Diffusion Sampling Acceleration