Organizations: School of Artificial Intelligence and Robotics, Xiamen University Malaysia, 43900 Sepang, Selangor, Malaysia · Department of Computer Science and Technology, School of Informatics, Xiamen University, Xiamen 361005, Fujian, China
Time series forecasting aims to predict the future evolution of complex systems by learning their underlying dynamics from past observations. Despite recent advances in deep learning, Transformers in particular still struggle to capture the temporal evolution of system dynamics. Existing approaches have been largely driven by architectural innovation, with comparatively little attention paid to training strategies that explicitly exploit the temporal structure governing system dynamics. In this work, we introduce DeltaTSF, a dynamics-aware forecasting framework based on discrete derivative representations of time series. Specifically, DeltaTSF jointly learns the series and their discrete temporal derivatives across both input and forecast horizons within a unified, model-agnostic framework. By coupling these representations over time, the model is encouraged to maintain temporal consistency, a property we show to induce a form of discrete Taylor consistency during training. This formulation enables the model to better reflect the underlying system dynamics, resulting in more stable and physically coherent forecasts. Notably, DeltaTSF does not require explicit derivative computation and introduces no additional computational overhead at inference time. Extensive experiments across diverse benchmarks and backbone architectures demonstrate that DeltaTSF consistently improves forecasting performance and yields more dynamically coherent predictions than strong SOTA methods. These results highlight the effectiveness of enforcing dynamic coherence through derivative representations as a model-agnostic paradigm that operates alongside and complements architectural design. Source code will be made publicly available upon acceptance.
Figures & tables
Figure 1: Overview of our DeltaTSF, a model-agnostic training paradigm for existing forecasting backbones without modifying their architectures. DeltaTSF jointly models series and their temporal derivatives within a shared architecture, coupling their predictions across time to enforce temporally structural alignment.
Dataset
PatchTST
iTransformer
TimeMixer
TimesNet
ori
+delta
ori
+delta
ori
+delta
ori
+delta
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
0.420
0.440
0.396
0.421
0.454
0.467
0.423
0.446
0.454
0.474
0.421
0.440
0.504
0.499
0.444
0.469
ETTh2
0.344
0.391
0.321
0.376
0.392
0.422
0.373
0.409
0.379
0.426
0.349
0.395
0.433
0.455
0.396
0.414
ETTm1
0.355
0.385
0.345
0.383
0.371
0.401
0.367
0.398
0.381
0.414
0.360
0.387
0.408
0.425
0.372
0.396
ETTm2
0.251
0.319
0.250
0.314
0.279
0.338
0.268
0.328
0.283
0.347
0.274
0.320
0.300
0.354
0.275
0.332
Table 2: Average long-term multivariate forecasting accuracies on six datasets with the input length set to 720. Our method DeltaTSF is applied to various forecating backbones. Full results are provided in Appendix Table 9 .
Dataset
DeltaTSF
PhaseFormer
PatchTST
iTransformer
TimeBase
PatchMLP
DLinear
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
0.396
0.421
0.403
0.415
0.420
0.440
0.454
0.467
0.404
0.416
0.483
0.485
0.437
0.448
ETTh2
0.321
0.376
0.347
0.389
0.344
0.391
0.392
0.422
0.356
0.401
0.392
0.427
0.481
0.471
ETTm1
0.345
0.383
0.347
0.374
0.355
0.385
0.371
0.401
0.357
0.381
0.372
0.396
0.361
0.383
ETTm2
0.250
0.314
0.251
0.314
0.251
0.319
0.279
0.338
0.251
0.314
0.279
0.333
0.271
0.337
Weather
0.221
0.262
0.223
0.261
0.224
0.264
0.233
0.273
0.227
0.263
0.230
0.270
0.243
0.299
Table 3: Average long-term multivariate forecasting results on six datasets.Here the input sequence length fixed to 720 for all methods. In each row, the best and second-best results for each metric are highlighted in red and blue , respectively. Full results are provided in Appendix Table 10
Figure 2: Illustration of the ablation studies in Section 4.3 , showing various approach to incorporate derivative representations into the forecasting backbone.
Table 4: Ablation studies on derivative integration and CES.
Horizon
PatchTST (MSE/MAE)
+K=1 (MSE/MAE)
+K=2 (MSE/MAE)
+K=3 (MSE/MAE)
+K=4 (MSE/MAE)
96
0.377 / 0.408
0.362 / 0.395
0.364 / 0.397
0.366 / 0.398
0.367 / 0.400
192
0.413 / 0.431
0.396 / 0.416
0.396 / 0.418
0.397 / 0.418
0.398 / 0.419
336
0.436 / 0.446
0.405 / 0.424
0.406 / 0.426
0.409 / 0.429
0.406 / 0.429
720
0.455 / 0.475
0.420 / 0.448
0.422 / 0.450
0.427 / 0.453
0.425 / 0.453
AVG
0.420 / 0.440
0.396 / 0.421
0.397 / 0.423
0.400 / 0.425
0.399 / 0.425
Table 5: Impact of higher-order temporal differences in DeltaTSF with PatchTST as the backbone on ETTh1.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Sensitivity of forecasting performance to the hyperparameter λ on the ETTm1 and ETTm2 dataset across different prediction horizons.
Dataset
Horizon
PatchTST
PatchTST+PS
DeltaTSF
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
96
0.375
0.400
0.368
0.394
0.369
0.393
192
0.414
0.421
0.407
0.415
0.405
0.413
336
0.432
0.436
0.420
0.426
0.415
0.422
720
0.450
0.466
0.425
0.449
0.423
0.447
Avg
0.418
0.431
0.405
0.421
0.403
0.419
Appendix
Table 6: Comparison of PatchTST, PatchTST+PS, and DeltaTSF on ETTh1, ETTh2, and Weather. The results of PatchTST and PatchTST+PS are taken from the Patchwise-Loss paper, and our DeltaTSF results are obtained under the same experimental setting for a fair comparison (input length 336).
Dataset
Length
O S +L S
Only I S
Only L S
Ours (I S +O S +L S )
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
96
0.377
0.409
0.370
0.403
0.367
0.401
0.362
0.395
192
0.414
0.432
0.407
0.426
0.402
0.422
0.396
0.416
336
0.429
0.443
0.417
0.434
0.414
0.432
0.405
0.424
720
0.466
0.480
0.450
0.468
0.447
0.467
0.420
0.448
AVG
0.422
0.441
0.411
0.433
0.408
0.431
0.396
0.421
Appendix
Table 7: Ablation study on injecting temporal difference information at different stages (lower is better).
Dataset
Horizon
PatchTST
+MixUp
+Delta
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
96
0.377
0.408
0.375
0.407
0.362
0.395
192
0.413
0.431
0.414
0.432
0.396
0.416
336
0.436
0.446
0.426
0.441
0.405
0.424
720
0.455
0.475
0.448
0.469
0.420
0.448
AVG
0.420
0.440
0.416
0.437
0.396
0.421
Appendix
Table 8: Comparative analysis of PatchTST with +MixUp and +Delta . Results are reported in terms of MSE / MAE (lower is better).
Dataset
PatchTST
iTransformer
TimeMixer
TimesNet
ori
+Delta
ori
+Delta
ori
+Delta
ori
+Delta
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1-96
0.377
0.408
0.362
0.395
0.389
0.421
0.373
0.406
0.410
0.441
0.391
0.420
0.437
0.454
0.401
0.433
ETTh1-192
0.413
0.431
0.396
0.416
0.424
0.446
0.409
0.431
0.448
0.465
0.415
0.431
0.454
0.469
0.425
0.453
ETTh1-336
0.436
0.446
0.405
0.424
0.456
0.469
0.423
0.440
0.482
0.490
0.426
0.440
0.494
0.494
0.466
0.487
ETTh1-720
0.455
0.475
0.420
0.448
0.545
0.532
0.486
0.505
0.475
0.500
0.450
0.468
0.632
0.578
0.485
0.502
Appendix
Table 9: Long-term multivariate forecasting results. The table reports MSE and MAE for different forecasting lengths T∈{96,192,336,720} . The input sequence length is fixed to 720 for all methods. Better results under the +delta setting are highlighted in bold. The results of the baseline models are directly obtained from PhaseFormer [ 26 ] and TimeBase [ 20 ] .
Dataset
H
DeltaTSF
PhaseFormer
PatchTST
iTransformer
TimeBase
PatchMLP
DLinear
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
96
0.362
0.395
0.359
0.382
0.377
0.408
0.389
0.421
0.365
0.387
0.417
0.440
0.378
0.402
192
0.396
0.416
0.397
0.404
0.413
0.431
0.424
0.446
0.403
0.409
0.454
0.465
0.415
0.425
336
0.405
0.424
0.425
0.424
0.436
0.446
0.456
0.469
0.409
0.419
0.514
0.503
0.449
0.449
720
0.420
0.448
0.431
0.450
0.455
0.475
0.545
0.532
0.440
0.448
0.548
0.531
0.507
0.517
AVG
0.396
0.421
0.403
0.415
0.420
0.440
0.454
0.467
0.404
0.416
0.483
0.485
0.437
0.448
Appendix
Table 10: Long-term multivariate forecasting results on six datasets. The table reports MSE and MAE for forecasting horizons H∈{96,192,336,720} . The input sequence length is fixed to 720 for all methods. In each row, the best and second-best results for each metric are highlighted in red and blue , respectively. In case of ties, priority is given to the earlier model in the table. The results of the other baseline models are directly obtained from PhaseFormer [ 26 ] and TimeBase [ 20 ] .
Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.
Foundation models for dynamical systems can forecast unseen partial differential equations (PDEs), ordinary differential equations (ODEs), and time series during inference. However, it remains an open question whether these foundation models truly learn generalizable in-context forecasting strategies. We show that looped transformers are capable of emulating propagators for dynamical systems in-context by first establishing a link between linear attention and (extended) dynamic mode decomposition. We then generalize this insight into the Test-Time Forecasting (TTF) framework in which a one-step forecaster leverages iterations of gradient descent as recurrent depth. We demonstrate that lightweight TTF models trained on ODEs and PDEs generalize to out-of-distribution system dynamics during autoregressive rollout after being trained for one-step prediction. Our results reinforce TTF as a theoretically motivated framework for designing recurrent depth models capable of forecasting unseen dynamical systems at test-time.
Jeffrey B. Lai, John Quinn, Anthony Bao +1
Oden Institute University of Texas at Austin · Department of Physics University of Texas at Austin
Time series forecasting serves as an essential tool for many real-world applications, supporting tasks such as resource optimization and decision-making. Despite significant architectural advancements, most modern models still treat forecasting task as a fixed mapping from history to target horizons. This induces temporal decoupling across future time points and limits the model's ability to adapt to the evolving context as forecasting progresses. In this work, we present LeapTS, a novel framework that reformulates time series forecasting as a dynamic scheduling process over the prediction horizon. Specifically, LeapTS organizes the forecasting process into multi-level decisions using: (1) the hierarchical controller to dynamically select the optimal prediction scale and advancement length at each step, and (2) continuous-time state evolution driven by neural controlled differential equations. Within this process, the controlled update mechanism explicitly couples the irregular temporal dynamics with discrete scheduling feedback. Extensive evaluations on both real-world and synthetic datasets demonstrate that LeapTS improves overall forecasting performance by at least 7.4% while achieving a 2.6× to 5.3× inference speedup over representative Transformer-based models. Furthermore, by explicitly tracing the scheduling trajectories, we reveal how the model autonomously adapts its forecasting behavior to capture non-stationary dynamics.
Time Series Forecasting (TSF) faces persistent challenges in modeling intricate temporal dependencies across different scales. Despite recent advances leveraging different decomposition operations and novel architectures based on CNN, MLP or Transformer, existing methods still struggle with static decomposition strategies, fragmented dependency modeling, and inflexible fusion mechanisms, limiting their ability to model intricate temporal dependencies. To explicitly solve the mentioned three problems respectively, we propose a novel Dynamic Multi-Scale Coordination Framework (DMSC) with Multi-Scale Patch Decomposition block (EMPD), Triad Interaction Block (TIB) and Adaptive Scale Routing MoE block (ASR-MoE). Specifically, EMPD is designed as a built-in component to dynamically segment sequences into hierarchical patches with exponentially scaled granularities, eliminating predefined scale constraints through input-adaptive patch adjustment. TIB then jointly models intra-patch, inter-patch, and cross-variable dependencies within each layer's decomposed representations. EMPD and TIB are jointly integrated into layers forming a multi-layer progressive cascade architecture, where coarse-grained representations from earlier layers adaptively guide fine-grained feature extraction in subsequent layers via gated pathways. And ASR-MoE dynamically fuses multi-scale predictions by leveraging specialized global and local experts with temporal-aware weighting. Comprehensive experiments on thirteen real-world benchmarks demonstrate that DMSC consistently maintains state-of-the-art (SOTA) performance and superior computational efficiency for TSF tasks. Code is available at https://github.com/1327679995/DMSC.
Haonan Yang, Jianchao Tang, Zhuo Li +1
National University of Defense Technology, Changsha, Hunan, China