Organizations: The Uncertainty Lab, Department of Applied Mechanics & Biomedical Engineering, Indian Institute of Technology Madras, India - 600036 · Center for Complex Systems & Dynamics IIT Madras, India - 600036
Accurate prediction of multivariate time series requires models that simultaneously capture variable-specific dynamics and cross-variable interactions. In this work, a Correlation-Guided and Dynamics-Trained Clustered Echo State Network (CGDT-CESN) is proposed, which comprises a modular reservoir architecture with each cluster distinctly specializing in one of the variables in the multivariate time series. An innovative reservoir design is proposed in which inter-cluster connections are enabled by empirically observed correlations among variables, providing a data-driven coupling mechanism. Apart from conventional readout training, inter-cluster connection weights informed by correlations are trained, which in turn allows the reservoir to capture time-varying cross-variable dependencies without sacrificing computational efficiency and further improves forecasting accuracy. The prediction performance of CGDT-CESN is found to be superior to that of the conventional ESN when evaluated on two fundamentally different datasets: Twitter information cascade evolution and solar wind forecasting. Also, the improvement in performance is preserved across multiple prediction horizons, even when presented with a low volume of data. By unifying cluster-specific specialization with learned inter-variable coupling, CGDT-CESN offers a principled, data-driven route to multivariate time series prediction.
Figures & tables
Figure 1: Architecture of Correlation-Guided and Dynamics Trained Clustered Echo state Network Illustrated for K=4 clusters corresponding to the four features.
Figure 2: Convergence test of the reservoir states with respect to the Uncorrelated and correlated noise sources before and after training.
Figure 3: Largest Lyapunov Exponent (LLE) computed for varying spectral radius ( ρ ) to verify the ESP for uncorrelated and correlated input signals before and after training the reservoir.
Dataset
Cascades
Twitter15
1,490
Twitter16
818
Combined
2,308
Table 1: Summary of datasets and cascade labels [ 25 ]
Figure 4: Time evolution of cascade with multiple evolving variables reflecting different aspects of a cascade such as size, breadth and depth.
Measure
Mathematical Form
Meaning
Cascade Size
S(n)=∣Z(n)∣
Total number of activated nodes in the cascade up to time n . Measures the overall spread.
Cascade Depth
D(n)=z∈Z(n)maxd(z0,z)
Maximum shortest-path distance from the source node z0 to any node in the cascade. Measures the propagation depth.
Cascade Breadth
B(n)=kmax∣Lk(n)∣
Maximum number of nodes present at any propagation level k . Measures the widest parallel spread.
Structural Virality
Zs(n)=∣Z(n)∣(∣Z(n)∣−1)1i=j∑d(zi,zj)
Average pairwise shortest-path distance among all nodes. Measures how structurally viral the cascade is.
Table 2: Time-dependent cascade descriptors and their mathematical definitions.
Hyperparameter
ESN
CG-CESN
CGDT-CESN
Reservoir size ( Nres )
200
–
–
Neurons per cluster ( Nc )
–
50
50
Number of clusters ( K )
–
4
4
Spectral radius ( ρ )
0.9
0.9
0.9
Leak rate ( α )
0.3
0.3
0.3
Reservoir connectivity ( prc )
0.3
–
–
Table 3: Hyperparameters used for different reservoir computing models
Figure 5: Validation convergence of the CGDT- C ESN model for different training data fractions ( 10% , 50% , and 100% ) over 50 Winter refinement epochs. The curves represent the mean validation NRMSE averaged over 10 independent runs, while the error bars denotes standard deviation. A rapid decrease in validation error is observed during the initial epochs, followed by convergence and saturation beyond approximately 10 epochs, indicating that additional refinement iterations provide only marginal improvement in predictive performance.
Figure 6: Effect of the leak rate parameter ( α ) on the performance of ESN ( ∙ ), CG-CESN ( ■ ), and CGDT-CESN ( ▲ ) for training sample percentages of 10% (a), 50% (b), and 100% (c). The leak rate is varied over the range ([0.1,1.0]) while all other parameters, including the number of training epochs, are kept fixed.
Figure 7: Performance (mean NRMSE, ξ ) of models as a function of training data proportion (where 100 % denotes the maximum training sample size corresponding to 80% of the original data set) evaluated at three prediction horizons (a) 50%, (b) 75% and (c) 100% for 50 realizations. The gray, blue and green bars correspond the ESN, CG-CESN and CGDT-CESN with the error bar depicting the standard deviation
Figure 8: Scatter plots of predicted versus ground truth values for the four cascade features: size (red), breadth (green), depth (blue), and structural virality (yellow), evaluated at prediction horizon 50% , across three models (columns) and three training set sizes (rows). All variables are independently normalized to [0,1] . The dashed diagonal line ( y=x ) represents perfect prediction; points lying closer to this line indicate higher predictive accuracy. Row (a): training size 10% (panels a1–a3); Row (b): training size 50% (panels b1–b3); Row (c): training size 100% (panels c1–c3). Column 1 (panels a1, b1, c1): ESN; Column 2 (panels a2, b2, c2): CG-CESN; Column 3 (panels a3, b3, c3): CGDT-CESN. The mean normalized root mean square error ( ξ , averaged across all four cascade features over 50 independent realizations) is reported in the upper left corner of each subplot. Progressively tighter clustering of points around the diagonal with increasing training size reflects improved model generalization, with CGDT-CESN consistently achieving the lowest ξ values, particularly under limited training data.
Model
ξ
ESN
0.081±0.0075
CG-CESN
0.067±0.0025
CGDT-CESN
0.063±0.0019
Table 4: Prediction performance ( ξ ) of the models on the solar wind dataset. Results are averaged over 50 realizations and reported as mean ± standard deviation of the ξ .
Figure 9: Visualization of solar wind prediction performance on the test set consisting of variables temperature, speed and density. The panels show the ground-truth depicted as thick orange curve and predicted values of ESN (dashed blue line), CG-CESN (dashed dotted red line) and CGDT-CESN (solid green line). Prediction of CGDT-CESN agrees closely with the ground truth across all variables and also exhibits its stability through time as compared to ESN.
Model
Cascade
Solar Wind
Inter-cluster Wiring
Training
Total
Inter-cluster Wiring
Training
Total
ESN
–
9.02±1.09
9.02±1.09
–
0.16±0.03
0.16±0.03
CG-CESN
341.54±13.39
8.72±1.00
350.26±13.59
0.30±0.03
0.49±0.08
0.79±0.09
CGDT-CESN
341.54±13.39
125.70±9.63
467.24±20.37
0.30±0.03
0.93±0.18
0.95±0.18
Table 5: Comparison of computational cost for training the model for the Cascade and Solar Wind datasets. Times are reported in seconds as mean ± standard deviation over 10 realizations. The epochs chosen for cascade and solar wind data sets for CGDT-CESN training is 10 and 5.
Multivariate time series forecasting presents unique challenges because future variables often co-evolve under shared system dynamics. While existing studies mainly focus on cross-variable dependencies in historical observations, dependencies among future values are much less explored. Specifically, modern forecasting models largely follow the Direct Forecasting (DF) paradigm, generating multi-step forecasts with point-wise objectives that do not explicitly constrain cross-variable structure. In this work, we show that the DF objective is mismatched in the presence of cross-variable and lagged dependencies, revealing an objective gap. To address this issue, we propose Cross-Variable Loss (CvLoss), a plug-in structural regularizer that constrains forecast residuals on a cross-variable graph. CvLoss penalizes inconsistent edge-wise residual differences over forecast patches, encouraging consistency across both synchronous and asynchronous interactions. Our experiments show that CvLoss consistently improves competitive forecasting models, outperforms representative learning objectives, and is compatible with a variety of forecasting backbones.
Kuiye Ding, Yifan Hu, Hanchen Wang +1
University of Technology Sydney · Tsinghua University · The Hong Kong University of Science and Technology (Guangzhou)
Multivariate time series forecasting is widely applied in fields such as transportation, energy, and finance. However, the data commonly suffers from issues of multi-scale characteristics, weak correlations, and noise interference, which limit the predictive performance of existing models. This paper proposes a dual-stream sparse Mixer prediction framework that extracts global trends and local dynamic features from sequences in both the frequency and time domains, respectively. It employs a sparsity mechanism to filter out invalid information, thereby enhancing the accuracy of cross-variable dependency modeling. Experimental results demonstrate that this method achieves leading performance on multiple real-world scenario datasets, validating its effectiveness and generality. The code is available at https://github.com/SDMixer/SDMixer
Xiang Ao
School of Software Engineering, Beijing Jiaotong University, Beijing, China
Irregularly sampled multivariate time series arise in many real-world domains such as healthcare, climate science, and large-scale monitoring systems. Characterized by asynchronous observations and heterogeneity across variables, such time series pose significant challenges for forecasting. In this work, we introduce Cross-Variable Temporal Attention (CVTA), a short-term, continuous-time forecasting framework that learns prediction functions from its compact, interpretable state summarizing recent temporal dynamics across variables. CVTA uses Variable-Conditioned Embeddings to project heterogeneous variables into variable-specific latent spaces, and applies temporal attention to capture cross-variable dependencies from this state. Furthermore, we demonstrate that CVTA can be integrated as an auxiliary module into existing state-of-the-art prediction models through a dynamic meta-decision model. Extensive experiments on multiple real-world benchmarks show that CVTA achieves strong short-term predictive accuracy, supports efficient online inference, improves existing prediction models when used as an auxiliary module, and learns attention patterns that align well with domain knowledge.
Ankitkumar Joshi, Milos Hauskrecht
Department of Computer Science, University of Pittsburgh, Pittsburgh, PA 15260