CTIA-ODE: Continuous-Time Inter-Variable Attention for Long-Term Time Series Forecasting

Chengyi Pan; Huangqimei Zheng; Hao Xue

cs.LGOct 14, 2026

CTIA-ODE: Continuous-Time Inter-Variable Attention for Long-Term Time Series Forecasting

Authors: Chengyi Pan, Huangqimei Zheng, Hao Xue

Organizations: UGOD, HKUST-GZ · School of Software, Yunnan University · UGOD, The Hong Kong University of Science and Technology (Guangzhou)

Abstract

Although multivariate time series forecasting has broad application prospects, effectively incorporating exogenous variables remains challenging. Prevailing discrete-time architectures often impose rigid temporal alignment across heterogeneous variables, which may limit their ability to capture continuously evolving external effects. To address this limitation, we introduce CTIA-ODE, a continuous-time forecasting framework centered on Continuous-Time Inter-Variable Attention (CTIA). Specifically, CTIA is embedded within an ordinary differential equation (ODE) solver, enabling the continuously evolving endogenous trajectory to dynamically query a static exogenous context along the integration path without requiring explicit temporal alignment. To improve the stability of the continuous latent dynamics, we further introduce a dual-stabilization strategy: architecturally, a Linear Residual Augmentation (LRA) module provides a macroscopic trend anchor, while an Outlier-Robust Hybrid Loss (ORHL) mitigates the influence of heavy-tailed anomalies during optimization. Extensive experiments across eight real-world datasets demonstrate that CTIA-ODE achieves state-of-the-art forecasting performance across diverse datasets and prediction horizons, while maintaining a favorable accuracy-efficiency trade-off with a compact memory footprint and low inference latency, particularly under long-horizon forecasting settings. Anonymous code is available at https://anonymous.4open.science/r/CTIA-ODE-15A6/.

Figures & tables

Figure 1: Motivation and empirical characteristics of CTIA-ODE. (a) Real-world exogenous variables may exhibit missing values, temporal misalignment, and frequency mismatch, motivating an asymmetric continuous-time fusion paradigm. (b) Forecasting performance on representative benchmarks. (c) Evolution of hidden-state similarity over the integration process, illustrating the difference between discrete representations and the continuous latent dynamics of CTIA-ODE. (d) t-SNE visualization of the learned latent states, providing a qualitative view of their temporal organization and representation stability.

Figure 2: The overall framework of the proposed CTIA-ODE. (A) ODE Solver drives the continuous-time evolution of hidden states via RK4 integration, dynamically querying the static exogenous field through CTIA. (B) Stable Refinement performs discrete filtering on terminal states to maintain manifold stability. (C) Linear Residual Augmentation provides a robust prediction baseline by decoupling historical fluctuations and injecting future time priors.

Models		CTIA-ODE (Ours)		WaveTS-B		S-Mamba		TimeXer		iTransformer		PatchTST		DLinear		Crossformer		TimesNet		Autoformer
Metric		MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
Electricity	96	0.140	0.233	0.143	0.235	0.147	0.249	0.147	0.251	0.148	0.240	0.195	0.285	0.197	0.282	0.219	0.314	0.168	0.272	0.201	0.317
	192	0.160	0.252	0.164	0.255	0.167	0.270	0.163	0.266	0.162	0.253	0.199	0.289	0.196	0.285	0.231	0.322	0.184	0.289	0.222	0.334
	336	0.181	0.274	0.185	0.278	0.189	0.291	0.184	0.282	0.178	0.269	0.215	0.305	0.209	0.301	0.246	0.337	0.198	0.300	0.231	0.338
	720	0.205	0.296	0.211	0.302	0.232	0.327	0.220	0.312	0.225	0.317	0.256	0.337	0.245	0.333	0.280	0.363	0.220	0.320	0.254	0.361
	Avg	0.172	0.264	0.176	0.268	0.183	0.284	0.178	0.278	0.178	0.270	0.216	0.304	0.212	0.300	0.244	0.334	0.192	0.295	0.227	0.338

Table 1: Multivariate forecasting results. The look-back length

L

is set to 96. The best results are highlighted in bold and the second best are underlined .

Models	Electricity				ETTm1				Traffic				Weather
	$T=96$		$T=720$		$T=96$		$T=720$		$T=96$		$T=720$		$T=96$		$T=720$
	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
CTIA-ODE (Full)	0.140	0.233	0.205	0.296	0.326	0.351	0.472	0.434	0.411	0.253	0.474	0.285	0.153	0.191	0.351	0.342
w/o Linear (Pure ODE)	0.145	0.238	0.238	0.318	0.332	0.358	0.548	0.485	0.415	0.258	0.535	0.315	0.159	0.201	0.395	0.380
w/o ODE (Pure Linear)	0.149	0.245	0.215	0.302	0.341	0.365	0.488	0.448	0.430	0.270	0.492	0.300	0.165	0.208	0.368	0.355
w/o CTIA (Pure CI)	0.144	0.238	0.212	0.301	0.336	0.360	0.486	0.445	0.428	0.265	0.505	0.304	0.163	0.205	0.370	0.362

Table 2: Ablation study on Electricity, ETTm1, Traffic, and Weather datasets. The complete CTIA-ODE is compared against variants ablating the loss function, architecture, and attention mechanism. The best results are highlighted in bold .

Figure 3: In-depth empirical and mechanistic analysis of CTIA-ODE. (a) Pareto frontier comparing inference latency, forecasting error (

T=720

), and peak GPU memory footprint (represented by bubble size) on the Electricity benchmark. (b) Impact of unrolling iterations (

K

) on forecasting accuracy, highlighting the optimal interaction depth at

K=2

to prevent over-smoothing. (c) Resilience to heavy-tailed sensor anomalies under varying outlier contamination ratios

\eta

on the ETTm1 dataset. (d) Continuous vector field trajectory magnitude

\|\frac{dH(t)}{dt}\|_{F}

along the RK4 integration interval

t\in[0,1]

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Domain	Variables ( $M$ )	Timesteps	Granularity	Prediction Task
Electricity	Energy	321	26,304	1 Hour	Power consumption forecasting
Traffic	Transportation	862	17,544	1 Hour	Road occupancy rate prediction
Weather	Meteorology	21	52,696	10 Minutes	Climatological indicator tracking
Exchange	Finance	8	7,588	1 Day	Currency exchange rate prediction
ETTh1	Energy (Transformer)	7	17,420	1 Hour	Oil temperature and load forecasting
ETTh2	Energy (Transformer)	7	17,420	1 Hour	Oil temperature and load forecasting

Appendix

Table 3: Summary of the eight real-world multivariate time series datasets used in our experiments.

Models	Original (Pristine)		Mask-30% (Missing)			Shift+2 (Misaligned)
Models	MSE	MAE	MSE	MAE	$\Delta$ MSE	MSE	MAE	$\Delta$ MSE
iTransformer	0.180	0.264	0.207	0.291	$+15.1\%$	0.221	0.301	$+23.0\%$
TimeXer	0.177	0.266	0.197	0.286	$+11.0\%$	0.204	0.291	$+15.1\%$
CTIA-ODE (Ours)	0.175	0.253	0.180	0.257	+3.0%	0.182	0.259	+4.2%

Appendix

Table 4: Performance comparison under irregular exogenous perturbations on ETTm2 (

T=96

). The

\Delta

indicates the relative performance degradation (higher

\Delta

means worse robustness) compared to the pristine (Original) setting.

Parameter	Electricity	Traffic	Weather	Exchange	ETTh1/h2	ETTm1/m2
Look-back window ( $L$ )	96	96	96	96	96	96
Latent dimension ( $D$ )	128	128	128	128	128	128
Batch size	256	64	256	256	256	256
Learning rate	$5\times 10^{-4}$	$1\times 10^{-4}$	$5\times 10^{-4}$	$1\times 10^{-3}$	$5\times 10^{-4}$	$5\times 10^{-4}$
Dropout rate	0.05	0.05	0.05	0.05	0.05	0.05
ORHL $\delta$ / $\lambda$	1.0 / 5.0	1.0 / 5.0	1.0 / 5.0	1.0 / 5.0	1.0 / 5.0	1.0 / 5.0

Appendix

Table 5: Detailed hyperparameter configurations for CTIA-ODE across different datasets.

ODE Solver	Weather		ETTm2
ODE Solver	MSE	MAE	MSE	MAE
Euler (1st-order)	0.366	0.354	0.415	0.410
Midpoint (2nd-order)	0.358	0.347	0.407	0.401
RK4 (4th-order)	0.351	0.342	0.401	0.394

Appendix

Table 6: Sensitivity analysis of numerical ODE solvers (

T=720

		$\lambda$ (Gradient Saturation)
	MSE / MAE	1.0	5.0	10.0
$\delta$ (Threshold)	0.5	0.285 / 0.312	0.276 / 0.301	0.279 / 0.308
	1.0	0.268 / 0.295	0.251 / 0.283	0.259 / 0.288
	2.0	0.272 / 0.301	0.263 / 0.292	0.265 / 0.295

Appendix

Table 7: Sensitivity analysis of ORHL hyperparameters (

\delta

and

\lambda

) on the Weather dataset (

T=720

). The optimal combination is highlighted in bold.

Figure 4: Training loss trajectories under different initialization strategies for the ODE state network. Standard random initialization triggers stiffness and stalls convergence in the early epochs, whereas our zero-initialization approach ensures stable and smooth optimization.

Figure 5: Qualitative forecasting comparison on the Weather dataset at an extreme prediction horizon (

T=720

). Top: CTIA-ODE vs. iTransformer. The Transformer baseline degrades into high-frequency noise, whereas our model preserves temporal continuity. Bottom: CTIA-ODE vs. DLinear. The linear baseline exhibits rigid, fixed-amplitude oscillations, whereas our model adaptively captures the dynamic macroscopic envelope of the ground truth.

Dataset	Model	$L=96$	$L=192$	$L=336$	$L=720$
Weather	Autoformer	0.415 / 0.432	0.432 / 0.448	0.459 / 0.465	OOM
	DLinear	0.381 / 0.405	0.380 / 0.402	0.378 / 0.401	0.385 / 0.409
	CTIA-ODE	0.351 / 0.342	0.348 / 0.339	0.345 / 0.335	0.342 / 0.333
ETTh1	Autoformer	0.505 / 0.521	0.518 / 0.535	0.542 / 0.551	OOM
	DLinear	0.485 / 0.510	0.482 / 0.505	0.488 / 0.512	0.501 / 0.525
	CTIA-ODE	0.452 / 0.460	0.449 / 0.457	0.445 / 0.452	0.441 / 0.448

Appendix

Table 8: Forecasting performance (MSE / MAE) with varying look-back window sizes (

L

) on the Weather and ETTh1 datasets (

T=720

Time Series Modeling

See all 51 papers

CVTA: Cross-Variable Temporal Attention for Multivariate Irregular Time Series Prediction
Oct 18, 2026Ankitkumar Joshi, Milos HauskrechtMultivariate Time Series Forecasting Time Series Forecasting
Department of Computer Science, University of Pittsburgh, Pittsburgh, PA 15260
UniMamba: A Unified Spatial-Temporal Modeling Framework with State-Space and Attention Integration
Mar 6, 2026Xingsheng Chen, Xianpei Mu, Deyu Yi +6Multivariate Time Series Forecasting Long-Term Time Series Forecasting
School of Computing and Data Science, The University of Hong Kong, Hong Kong, China · School of Information Engineering, Beijing Institute of Graphic Communication, Beijing, China · Innovation Engineering College, Macau University of Science and Technology, Macau, China +1
Enhancing Irregular Time Series Forecasting with Continuous-Time Modeling Framework
Jul 30, 2026Tianen Shen, Zhengyu Li, Yutong Li +4Irregular Time-Series Modeling Multivariate Time Series Forecasting

Oct 18, 2026J/K move · Enter open · S save

CVTA: Cross-Variable Temporal Attention for Multivariate Irregular Time Series Prediction

Ankitkumar Joshi, Milos Hauskrecht

Department of Computer Science, University of Pittsburgh, Pittsburgh, PA 15260

Multivariate Time Series Forecasting Time Series Forecasting Irregular Time-Series Modeling Explainable Time Series Forecasting

Open paper AskPDF arXiv

Irregularly sampled multivariate time series arise in many real-world domains such as healthcare, climate science, and large-scale monitoring systems. Characterized by asynchronous observations and heterogeneity across variables, such time series pose significant challenges for forecasting. In this work, we introduce Cross-Variable Temporal Attention (CVTA), a short-term, continuous-time forecasting framework that learns prediction functions from its compact, interpretable state summarizing recent temporal dynamics across variables. CVTA uses Variable-Conditioned Embeddings to project heterogeneous variables into variable-specific latent spaces, and applies temporal attention to capture cross-variable dependencies from this state. Furthermore, we demonstrate that CVTA can be integrated as an auxiliary module into existing state-of-the-art prediction models through a dynamic meta-decision model. Extensive experiments on multiple real-world benchmarks show that CVTA achieves strong short-term predictive accuracy, supports efficient online inference, improves existing prediction models when used as an auxiliary module, and learns attention patterns that align well with domain knowledge.

Abstract

Time Series Modeling

Explore similar work

CVTA: Cross-Variable Temporal Attention for Multivariate Irregular Time Series Prediction

UniMamba: A Unified Spatial-Temporal Modeling Framework with State-Space and Attention Integration

Enhancing Irregular Time Series Forecasting with Continuous-Time Modeling Framework