Organizations: School of Artificial Intelligence and Robotics, Xiamen University Malaysia, 43900 Sepang, Selangor, Malaysia · Department of Computer Science and Technology, School of Informatics, Xiamen University, Xiamen 361005, Fujian, China
Time series forecasting aims to predict the future evolution of complex systems by learning their underlying dynamics from past observations. Despite recent advances in deep learning, Transformers in particular still struggle to capture the temporal evolution of system dynamics. Existing approaches have been largely driven by architectural innovation, with comparatively little attention paid to training strategies that explicitly exploit the temporal structure governing system dynamics. In this work, we introduce DeltaTSF, a dynamics-aware forecasting framework based on discrete derivative representations of time series. Specifically, DeltaTSF jointly learns the series and their discrete temporal derivatives across both input and forecast horizons within a unified, model-agnostic framework. By coupling these representations over time, the model is encouraged to maintain temporal consistency, a property we show to induce a form of discrete Taylor consistency during training. This formulation enables the model to better reflect the underlying system dynamics, resulting in more stable and physically coherent forecasts. Notably, DeltaTSF does not require explicit derivative computation and introduces no additional computational overhead at inference time. Extensive experiments across diverse benchmarks and backbone architectures demonstrate that DeltaTSF consistently improves forecasting performance and yields more dynamically coherent predictions than strong SOTA methods. These results highlight the effectiveness of enforcing dynamic coherence through derivative representations as a model-agnostic paradigm that operates alongside and complements architectural design. Source code will be made publicly available upon acceptance.
Figures & tables
Figure 1: Overview of our DeltaTSF, a model-agnostic training paradigm for existing forecasting backbones without modifying their architectures. DeltaTSF jointly models series and their temporal derivatives within a shared architecture, coupling their predictions across time to enforce temporally structural alignment.
Dataset
PatchTST
iTransformer
TimeMixer
TimesNet
ori
+delta
ori
+delta
ori
+delta
ori
+delta
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
0.420
0.440
0.396
0.421
0.454
0.467
0.423
0.446
0.454
0.474
0.421
0.440
0.504
0.499
0.444
0.469
ETTh2
0.344
0.391
0.321
0.376
0.392
0.422
0.373
0.409
0.379
0.426
0.349
0.395
0.433
0.455
0.396
0.414
ETTm1
0.355
0.385
0.345
0.383
0.371
0.401
0.367
0.398
0.381
0.414
0.360
0.387
0.408
0.425
0.372
0.396
ETTm2
0.251
0.319
0.250
0.314
0.279
0.338
0.268
0.328
0.283
0.347
0.274
0.320
0.300
0.354
0.275
0.332
Table 2: Average long-term multivariate forecasting accuracies on six datasets with the input length set to 720. Our method DeltaTSF is applied to various forecating backbones. Full results are provided in Appendix Table 9 .
Dataset
DeltaTSF
PhaseFormer
PatchTST
iTransformer
TimeBase
PatchMLP
DLinear
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
0.396
0.421
0.403
0.415
0.420
0.440
0.454
0.467
0.404
0.416
0.483
0.485
0.437
0.448
ETTh2
0.321
0.376
0.347
0.389
0.344
0.391
0.392
0.422
0.356
0.401
0.392
0.427
0.481
0.471
ETTm1
0.345
0.383
0.347
0.374
0.355
0.385
0.371
0.401
0.357
0.381
0.372
0.396
0.361
0.383
ETTm2
0.250
0.314
0.251
0.314
0.251
0.319
0.279
0.338
0.251
0.314
0.279
0.333
0.271
0.337
Weather
0.221
0.262
0.223
0.261
0.224
0.264
0.233
0.273
0.227
0.263
0.230
0.270
0.243
0.299
Table 3: Average long-term multivariate forecasting results on six datasets.Here the input sequence length fixed to 720 for all methods. In each row, the best and second-best results for each metric are highlighted in red and blue , respectively. Full results are provided in Appendix Table 10
Figure 2: Illustration of the ablation studies in Section 4.3 , showing various approach to incorporate derivative representations into the forecasting backbone.
Table 4: Ablation studies on derivative integration and CES.
Horizon
PatchTST (MSE/MAE)
+K=1 (MSE/MAE)
+K=2 (MSE/MAE)
+K=3 (MSE/MAE)
+K=4 (MSE/MAE)
96
0.377 / 0.408
0.362 / 0.395
0.364 / 0.397
0.366 / 0.398
0.367 / 0.400
192
0.413 / 0.431
0.396 / 0.416
0.396 / 0.418
0.397 / 0.418
0.398 / 0.419
336
0.436 / 0.446
0.405 / 0.424
0.406 / 0.426
0.409 / 0.429
0.406 / 0.429
720
0.455 / 0.475
0.420 / 0.448
0.422 / 0.450
0.427 / 0.453
0.425 / 0.453
AVG
0.420 / 0.440
0.396 / 0.421
0.397 / 0.423
0.400 / 0.425
0.399 / 0.425
Table 5: Impact of higher-order temporal differences in DeltaTSF with PatchTST as the backbone on ETTh1.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Sensitivity of forecasting performance to the hyperparameter λ on the ETTm1 and ETTm2 dataset across different prediction horizons.
Dataset
Horizon
PatchTST
PatchTST+PS
DeltaTSF
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
96
0.375
0.400
0.368
0.394
0.369
0.393
192
0.414
0.421
0.407
0.415
0.405
0.413
336
0.432
0.436
0.420
0.426
0.415
0.422
720
0.450
0.466
0.425
0.449
0.423
0.447
Avg
0.418
0.431
0.405
0.421
0.403
0.419
Appendix
Table 6: Comparison of PatchTST, PatchTST+PS, and DeltaTSF on ETTh1, ETTh2, and Weather. The results of PatchTST and PatchTST+PS are taken from the Patchwise-Loss paper, and our DeltaTSF results are obtained under the same experimental setting for a fair comparison (input length 336).
Dataset
Length
O S +L S
Only I S
Only L S
Ours (I S +O S +L S )
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
96
0.377
0.409
0.370
0.403
0.367
0.401
0.362
0.395
192
0.414
0.432
0.407
0.426
0.402
0.422
0.396
0.416
336
0.429
0.443
0.417
0.434
0.414
0.432
0.405
0.424
720
0.466
0.480
0.450
0.468
0.447
0.467
0.420
0.448
AVG
0.422
0.441
0.411
0.433
0.408
0.431
0.396
0.421
Appendix
Table 7: Ablation study on injecting temporal difference information at different stages (lower is better).
Dataset
Horizon
PatchTST
+MixUp
+Delta
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
96
0.377
0.408
0.375
0.407
0.362
0.395
192
0.413
0.431
0.414
0.432
0.396
0.416
336
0.436
0.446
0.426
0.441
0.405
0.424
720
0.455
0.475
0.448
0.469
0.420
0.448
AVG
0.420
0.440
0.416
0.437
0.396
0.421
Appendix
Table 8: Comparative analysis of PatchTST with +MixUp and +Delta . Results are reported in terms of MSE / MAE (lower is better).
Dataset
PatchTST
iTransformer
TimeMixer
TimesNet
ori
+Delta
ori
+Delta
ori
+Delta
ori
+Delta
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1-96
0.377
0.408
0.362
0.395
0.389
0.421
0.373
0.406
0.410
0.441
0.391
0.420
0.437
0.454
0.401
0.433
ETTh1-192
0.413
0.431
0.396
0.416
0.424
0.446
0.409
0.431
0.448
0.465
0.415
0.431
0.454
0.469
0.425
0.453
ETTh1-336
0.436
0.446
0.405
0.424
0.456
0.469
0.423
0.440
0.482
0.490
0.426
0.440
0.494
0.494
0.466
0.487
ETTh1-720
0.455
0.475
0.420
0.448
0.545
0.532
0.486
0.505
0.475
0.500
0.450
0.468
0.632
0.578
0.485
0.502
Appendix
Table 9: Long-term multivariate forecasting results. The table reports MSE and MAE for different forecasting lengths T∈{96,192,336,720} . The input sequence length is fixed to 720 for all methods. Better results under the +delta setting are highlighted in bold. The results of the baseline models are directly obtained from PhaseFormer [ 26 ] and TimeBase [ 20 ] .
Dataset
H
DeltaTSF
PhaseFormer
PatchTST
iTransformer
TimeBase
PatchMLP
DLinear
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
96
0.362
0.395
0.359
0.382
0.377
0.408
0.389
0.421
0.365
0.387
0.417
0.440
0.378
0.402
192
0.396
0.416
0.397
0.404
0.413
0.431
0.424
0.446
0.403
0.409
0.454
0.465
0.415
0.425
336
0.405
0.424
0.425
0.424
0.436
0.446
0.456
0.469
0.409
0.419
0.514
0.503
0.449
0.449
720
0.420
0.448
0.431
0.450
0.455
0.475
0.545
0.532
0.440
0.448
0.548
0.531
0.507
0.517
AVG
0.396
0.421
0.403
0.415
0.420
0.440
0.454
0.467
0.404
0.416
0.483
0.485
0.437
0.448
Appendix
Table 10: Long-term multivariate forecasting results on six datasets. The table reports MSE and MAE for forecasting horizons H∈{96,192,336,720} . The input sequence length is fixed to 720 for all methods. In each row, the best and second-best results for each metric are highlighted in red and blue , respectively. In case of ties, priority is given to the earlier model in the table. The results of the other baseline models are directly obtained from PhaseFormer [ 26 ] and TimeBase [ 20 ] .
Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.