Although multivariate time series forecasting has broad application prospects, effectively incorporating exogenous variables remains challenging. Prevailing discrete-time architectures often impose rigid temporal alignment across heterogeneous variables, which may limit their ability to capture continuously evolving external effects. To address this limitation, we introduce CTIA-ODE, a continuous-time forecasting framework centered on Continuous-Time Inter-Variable Attention (CTIA). Specifically, CTIA is embedded within an ordinary differential equation (ODE) solver, enabling the continuously evolving endogenous trajectory to dynamically query a static exogenous context along the integration path without requiring explicit temporal alignment. To improve the stability of the continuous latent dynamics, we further introduce a dual-stabilization strategy: architecturally, a Linear Residual Augmentation (LRA) module provides a macroscopic trend anchor, while an Outlier-Robust Hybrid Loss (ORHL) mitigates the influence of heavy-tailed anomalies during optimization. Extensive experiments across eight real-world datasets demonstrate that CTIA-ODE achieves state-of-the-art forecasting performance across diverse datasets and prediction horizons, while maintaining a favorable accuracy-efficiency trade-off with a compact memory footprint and low inference latency, particularly under long-horizon forecasting settings. Anonymous code is available at https://anonymous.4open.science/r/CTIA-ODE-15A6/.
Figures & tables
Figure 1: Motivation and empirical characteristics of CTIA-ODE. (a) Real-world exogenous variables may exhibit missing values, temporal misalignment, and frequency mismatch, motivating an asymmetric continuous-time fusion paradigm. (b) Forecasting performance on representative benchmarks. (c) Evolution of hidden-state similarity over the integration process, illustrating the difference between discrete representations and the continuous latent dynamics of CTIA-ODE. (d) t-SNE visualization of the learned latent states, providing a qualitative view of their temporal organization and representation stability.
Figure 2: The overall framework of the proposed CTIA-ODE. (A) ODE Solver drives the continuous-time evolution of hidden states via RK4 integration, dynamically querying the static exogenous field through CTIA. (B) Stable Refinement performs discrete filtering on terminal states to maintain manifold stability. (C) Linear Residual Augmentation provides a robust prediction baseline by decoupling historical fluctuations and injecting future time priors.
Models
CTIA-ODE (Ours)
WaveTS-B
S-Mamba
TimeXer
iTransformer
PatchTST
DLinear
Crossformer
TimesNet
Autoformer
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
Electricity
96
0.140
0.233
0.143
0.235
0.147
0.249
0.147
0.251
0.148
0.240
0.195
0.285
0.197
0.282
0.219
0.314
0.168
0.272
0.201
0.317
192
0.160
0.252
0.164
0.255
0.167
0.270
0.163
0.266
0.162
0.253
0.199
0.289
0.196
0.285
0.231
0.322
0.184
0.289
0.222
0.334
336
0.181
0.274
0.185
0.278
0.189
0.291
0.184
0.282
0.178
0.269
0.215
0.305
0.209
0.301
0.246
0.337
0.198
0.300
0.231
0.338
720
0.205
0.296
0.211
0.302
0.232
0.327
0.220
0.312
0.225
0.317
0.256
0.337
0.245
0.333
0.280
0.363
0.220
0.320
0.254
0.361
Avg
0.172
0.264
0.176
0.268
0.183
0.284
0.178
0.278
0.178
0.270
0.216
0.304
0.212
0.300
0.244
0.334
0.192
0.295
0.227
0.338
Table 1: Multivariate forecasting results. The look-back length L is set to 96. The best results are highlighted in bold and the second best are underlined .
Models
Electricity
ETTm1
Traffic
Weather
T=96
T=720
T=96
T=720
T=96
T=720
T=96
T=720
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
CTIA-ODE (Full)
0.140
0.233
0.205
0.296
0.326
0.351
0.472
0.434
0.411
0.253
0.474
0.285
0.153
0.191
0.351
0.342
w/o Linear (Pure ODE)
0.145
0.238
0.238
0.318
0.332
0.358
0.548
0.485
0.415
0.258
0.535
0.315
0.159
0.201
0.395
0.380
w/o ODE (Pure Linear)
0.149
0.245
0.215
0.302
0.341
0.365
0.488
0.448
0.430
0.270
0.492
0.300
0.165
0.208
0.368
0.355
w/o CTIA (Pure CI)
0.144
0.238
0.212
0.301
0.336
0.360
0.486
0.445
0.428
0.265
0.505
0.304
0.163
0.205
0.370
0.362
Table 2: Ablation study on Electricity, ETTm1, Traffic, and Weather datasets. The complete CTIA-ODE is compared against variants ablating the loss function, architecture, and attention mechanism. The best results are highlighted in bold .
Figure 3: In-depth empirical and mechanistic analysis of CTIA-ODE. (a) Pareto frontier comparing inference latency, forecasting error ( T=720 ), and peak GPU memory footprint (represented by bubble size) on the Electricity benchmark. (b) Impact of unrolling iterations ( K ) on forecasting accuracy, highlighting the optimal interaction depth at K=2 to prevent over-smoothing. (c) Resilience to heavy-tailed sensor anomalies under varying outlier contamination ratios η on the ETTm1 dataset. (d) Continuous vector field trajectory magnitude ∥dtdH(t)∥F along the RK4 integration interval t∈[0,1] .
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Domain
Variables ( M )
Timesteps
Granularity
Prediction Task
Electricity
Energy
321
26,304
1 Hour
Power consumption forecasting
Traffic
Transportation
862
17,544
1 Hour
Road occupancy rate prediction
Weather
Meteorology
21
52,696
10 Minutes
Climatological indicator tracking
Exchange
Finance
8
7,588
1 Day
Currency exchange rate prediction
ETTh1
Energy (Transformer)
7
17,420
1 Hour
Oil temperature and load forecasting
ETTh2
Energy (Transformer)
7
17,420
1 Hour
Oil temperature and load forecasting
Appendix
Table 3: Summary of the eight real-world multivariate time series datasets used in our experiments.
Models
Original (Pristine)
Mask-30% (Missing)
Shift+2 (Misaligned)
MSE
MAE
MSE
MAE
Δ MSE
MSE
MAE
Δ MSE
iTransformer
0.180
0.264
0.207
0.291
+15.1%
0.221
0.301
+23.0%
TimeXer
0.177
0.266
0.197
0.286
+11.0%
0.204
0.291
+15.1%
CTIA-ODE (Ours)
0.175
0.253
0.180
0.257
+3.0%
0.182
0.259
+4.2%
Appendix
Table 4: Performance comparison under irregular exogenous perturbations on ETTm2 ( T=96 ). The Δ indicates the relative performance degradation (higher Δ means worse robustness) compared to the pristine (Original) setting.
Parameter
Electricity
Traffic
Weather
Exchange
ETTh1/h2
ETTm1/m2
Look-back window ( L )
96
96
96
96
96
96
Latent dimension ( D )
128
128
128
128
128
128
Batch size
256
64
256
256
256
256
Learning rate
5×10−4
1×10−4
5×10−4
1×10−3
5×10−4
5×10−4
Dropout rate
0.05
0.05
0.05
0.05
0.05
0.05
ORHL δ / λ
1.0 / 5.0
1.0 / 5.0
1.0 / 5.0
1.0 / 5.0
1.0 / 5.0
1.0 / 5.0
Appendix
Table 5: Detailed hyperparameter configurations for CTIA-ODE across different datasets.
Table 7: Sensitivity analysis of ORHL hyperparameters ( δ and λ ) on the Weather dataset ( T=720 ). The optimal combination is highlighted in bold.
Figure 4: Training loss trajectories under different initialization strategies for the ODE state network. Standard random initialization triggers stiffness and stalls convergence in the early epochs, whereas our zero-initialization approach ensures stable and smooth optimization.
Figure 5: Qualitative forecasting comparison on the Weather dataset at an extreme prediction horizon ( T=720 ). Top: CTIA-ODE vs. iTransformer. The Transformer baseline degrades into high-frequency noise, whereas our model preserves temporal continuity. Bottom: CTIA-ODE vs. DLinear. The linear baseline exhibits rigid, fixed-amplitude oscillations, whereas our model adaptively captures the dynamic macroscopic envelope of the ground truth.
Dataset
Model
L=96
L=192
L=336
L=720
Weather
Autoformer
0.415 / 0.432
0.432 / 0.448
0.459 / 0.465
OOM
DLinear
0.381 / 0.405
0.380 / 0.402
0.378 / 0.401
0.385 / 0.409
CTIA-ODE
0.351 / 0.342
0.348 / 0.339
0.345 / 0.335
0.342 / 0.333
ETTh1
Autoformer
0.505 / 0.521
0.518 / 0.535
0.542 / 0.551
OOM
DLinear
0.485 / 0.510
0.482 / 0.505
0.488 / 0.512
0.501 / 0.525
CTIA-ODE
0.452 / 0.460
0.449 / 0.457
0.445 / 0.452
0.441 / 0.448
Appendix
Table 8: Forecasting performance (MSE / MAE) with varying look-back window sizes ( L ) on the Weather and ETTh1 datasets ( T=720 ).
Irregularly sampled multivariate time series arise in many real-world domains such as healthcare, climate science, and large-scale monitoring systems. Characterized by asynchronous observations and heterogeneity across variables, such time series pose significant challenges for forecasting. In this work, we introduce Cross-Variable Temporal Attention (CVTA), a short-term, continuous-time forecasting framework that learns prediction functions from its compact, interpretable state summarizing recent temporal dynamics across variables. CVTA uses Variable-Conditioned Embeddings to project heterogeneous variables into variable-specific latent spaces, and applies temporal attention to capture cross-variable dependencies from this state. Furthermore, we demonstrate that CVTA can be integrated as an auxiliary module into existing state-of-the-art prediction models through a dynamic meta-decision model. Extensive experiments on multiple real-world benchmarks show that CVTA achieves strong short-term predictive accuracy, supports efficient online inference, improves existing prediction models when used as an auxiliary module, and learns attention patterns that align well with domain knowledge.
Ankitkumar Joshi, Milos Hauskrecht
Department of Computer Science, University of Pittsburgh, Pittsburgh, PA 15260
Multivariate time series forecasting is fundamental to numerous domains such as energy, finance, and environmental monitoring, where complex temporal dependencies and cross-variable interactions pose enduring challenges. Existing Transformer-based methods capture temporal correlations through attention mechanisms but suffer from quadratic computational cost, while state-space models like Mamba achieve efficient long-context modeling yet lack explicit temporal pattern recognition. Therefore we introduce UniMamba, a unified spatial-temporal forecasting framework that integrates efficient state-space dynamics with attention-based dependency learning. UniMamba employs a Mamba Variate-Channel Encoding Layer enhanced with FFT-Laplace Transform and TCN to capture global temporal dependencies, and a Spatial Temporal Attention Layer to jointly model inter-variate correlations and temporal evolution. A Feedforward Temporal Dynamics Layer further fuses continuous and discrete contexts for accurate forecasting. Comprehensive experiments on eight public benchmark datasets demonstrate that UniMamba consistently outperforms state-of-the-art forecasting models in both forecasting accuracy and computational efficiency, establishing a scalable and robust solution for long-sequence multivariate time-series prediction.
Xingsheng Chen, Xianpei Mu, Deyu Yi +6
School of Computing and Data Science, The University of Hong Kong, Hong Kong, China · School of Information Engineering, Beijing Institute of Graphic Communication, Beijing, China · Innovation Engineering College, Macau University of Science and Technology, Macau, China +1
Irregular multivariate time series are widely encountered in applications such as healthcare monitoring, human activity recognition, and environmental sensing. Their core challenges stem from asynchronous observations, non-uniform sampling intervals, and the fact that temporal patterns themselves carry critical dynamic information. Existing approaches either rely on discretization-based preprocessing (e.g., interpolation, imputation, or aggregation), which disrupts the underlying continuous-time semantics, or adopt continuous-time modeling via ODE-based frameworks, which typically require specialized architectures and incur substantial computational overhead due to numerical solvers. To address these limitations, we propose WrapFlow, a continuous-time modeling framework for irregular time series forecasting. On the input side, WrapFlow introduces Continuous-Time Tokenization, which directly encodes raw observation events and explicitly models long unobserved intervals via gap-aware tokens. The resulting continuous-time tokens are then processed by a standard Transformer backbone to capture long-range temporal dependencies. On the output side, we develop a simulation-free training paradigm for Residual Flow Matching, which learns conditional residual vector fields around base predictions while avoiding numerical-solver simulation and backpropagation during training. This design enables high-quality continuous forecasting using only a small number of fixed rollout steps at inference. Extensive experiments on multiple real-world datasets demonstrate that WrapFlow achieves state-of-the-art performance.