PRELUDE: Separating Common Components from Target-Specific Dependencies for Multivariate Time Series Forecasting
Authors: Dongqing Song, Jianyu Xu, Haobin Ding, Xinyi Guo, Jiahao Du, Ruiqi He, Ruiqi Liu, Yuntong Xie, Chao He
Organizations: School of Artificial Intelligence, South China Normal University, Foshan, China · Aberdeen Institute of Data Science and Artificial Intelligence, South China Normal University, Foshan, China · Faculty of Humanities and Arts, Macau University of Science and Technology, Macau, China
Multivariate time series forecasting often improves the prediction of a target by modeling the interactions among variables, and cross-variable modules such as cross-sectional pooling, inter-variable attention, and graph structures are used to construct such interactions. However, by intervening on the cross-variable reading of existing models at inference time, we find that although these models substantially reduce forecasting error through cross-variable interaction, their reading is nearly identical for all targets: the gain appears to come from dependencies among variables but mainly comes from the common component. We term this phenomenon spurious dependency and attribute it to common-component confounding: present in every variable, the common component affects how reading distinguishes sources, so that reading tends to be shared across targets and concentrates on a few similar averages; dependencies thus make up a small share of the read-in content, and a common component composed of several drivers is only partly captured. Theoretical analysis further shows that the common component and dependencies place different requirements on reading and should therefore be handled in different ways. Accordingly, we propose PRELUDE, a separated-reading framework that estimates the common component holistically through cross-sectional pooling and multiple common factors, and reads dependencies for each target via lead-lag relations in the innovation space obtained by prewhitening. Extensive experiments on 16 real-world datasets from four domains show that PRELUDE effectively mitigates spurious dependency and achieves the best overall performance: it attains the best MSE on 23 of the 24 short-horizon and air-quality settings and reduces the average MSE by 6.1% relative to the strongest baseline.
Figures & tables
Fig. 1: Dependencies in the data and the reading of the model (PEMS08 and PEMS07, horizon 3). (a) Relations in the data and the reading of iTransformer (target × source); the strip above each panel shows the spread of each source’s reading weights, and the boxed numbers give the target specificity κ and the gain retained under shared reading weights. (b) t-SNE of input windows.
Fig. 2: What existing models obtain from other variables. (a) Cross-variable gain under progressive replacement of the reading on PEMS08; (b) the part of the residual still predictable from the time of day, the common factors, and a single source; (c) recovery rate of the planted source on synthetic data; (d) effective rank of the read-in content as the common factors strengthen.
Fig. 3: Four stages of the reading process in existing models (PEMS08, horizon 3). (a) t-SNE of source embeddings and read-in content; (b) dependency in the data and the reading of three models; (c) response of the reading to a common change and to a single-source change; (d) learning progress of the three components on synthetic data, with the vertical dashed line marking the official training budget.
Fig. 4: Overall architecture of PRELUDE. (a) Shared-component pathway; (b) prewhitening; (c) source–lag routing; (d) transfer readout.
Dataset
Domain
N
T
Interval
Tp
Horizons
Benchmark setting
PEMS03/04/07/08
Traffic
170–883
16,992–28,224
5 min
288
12–96
Traffic
Traffic
862
17,544
1 h
24
96–720
Electricity
Energy
321
26,304
1 h
24
96–720
Solar
Energy
137
52,560
10 min
144
96–720
ETTh1/h2/m1/m2
Energy
7
17,420–69,680
1 h/15 min
24/96
96–720
TABLE I: Dataset Statistics. N is the number of variables, T the series length, and Tp the period length.
Model
PRELUDE
TQNet
EMAformer
CGTFra
Crisp
TimeEmb
DUET
CycleNet
SOFTS
iTransformer
PatchTST
LIFT
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
PEMS03
Avg
0.086
0.183
0.097
0.203
0.090
0.190
0.108
0.212
0.113
0.214
0.142
0.239
0.113
0.219
0.118
0.225
0.104
0.210
0.113
0.222
0.180
0.291
0.138
0.246
PEMS04
Avg
0.078
0.174
0.091
0.196
0.081
0.180
0.105
0.209
0.111
0.217
0.120
0.224
0.109
0.220
0.119
0.232
0.102
0.208
0.111
0.221
0.195
0.307
0.145
0.256
PEMS07
Avg
0.066
0.151
0.074
0.171
0.073
0.162
0.094
0.188
0.091
0.185
0.104
0.196
0.095
0.197
0.113
0.214
0.086
0.184
0.101
0.204
0.211
0.303
0.146
0.244
PEMS08
Avg
0.103
0.189
0.142
0.230
0.128
0.209
0.145
0.225
0.152
0.229
0.154
0.229
0.123
0.212
0.150
0.246
0.138
0.220
0.150
0.226
0.280
0.321
0.186
0.267
Traffic
Avg
0.433
0.260
0.445
0.276
0.430
0.255
0.427
0.257
0.482
0.280
0.454
0.293
0.451
0.269
0.472
0.301
0.409
0.267
0.428
0.282
0.481
0.300
0.562
0.357
TABLE II: Multivariate Forecasting Results on the Benchmark Setting (Look-Back 96, Averaged over the Four Horizons). The best results are in bold and the second best are underlined; LIFT diverges on ETTh2, ETTm1, and ETTm2 at long horizons. Per-horizon results are provided in the code repository.
Setting
Start
PRELUDE
Reduction
Periodic
Factor
Dependency
ETTm1-96
0.3085
0.2791
9.5%
99%
1%
0%
ETTm2-96
0.1705
0.1606
5.8%
103%
− 3%
0%
Weather-96
0.1613
0.1444
10.5%
98%
2%
0%
Electricity-96
0.1465
0.1298
11.4%
87%
13%
0%
Traffic-96
0.4409
0.3976
9.8%
90%
11%
0%
Solar-96
0.1990
0.1641
17.5%
68%
31%
0%
TABLE IV: Sources of Gain (Mean over Three Seeds). The three shares correspond to the periodic part and the factor part of the common component and to the source innovations of Definition 1 ; the first two are obtained by the shared-component pathway and the third by the innovation pathway. Shares are signed contributions summing to 100%, so a share may be negative or exceed 100%.
Fig. 5: Re-measurement of Section IV on PRELUDE. (a) The part of the residual still predictable; (b) CMRR of the innovation pathway; (c) innovation utilization rate; (d) CMRR on synthetic data as the common factors strengthen. P08-3 denotes PEMS08 at horizon 3, and so on.
Variant
PEMS08-3
PEMS04-3
YRD-3
BTHSA-3
PEMS08-12
Elec.-96
PRELUDE
0.0489
0.0540
0.0616
0.0522
0.0647
0.1298
w/o periodic basis
+0.09%
+0.36%
− 0.04%
− 0.79%
+1.51%
+7.15%
w/o pooling & factors
+1.27%
+2.96%
+11.98%
+14.33%
+4.90%
+1.66%
w/o common factors
–
+0.62%
+6.27%
+7.83%
− 0.48%
–
w/o innovation pathway
+2.21%
+1.50%
+1.52%
+2.02%
+0.67%
+0.00%
w/o prewhitening
+0.47%
+0.82%
− 0.19%
+0.05%
+0.22%
− 0.03%
TABLE V: Ablation Study (Mean over Three Seeds). The first row gives the MSE of PRELUDE and the other rows the relative change, where positive values indicate increased error; “–” indicates that the component is not used in that setting.
Multivariate time series forecasting is critical in many real-world systems, and thus modeling cross-channel dependencies is essential. Although existing methods improve overall accuracy by enhancing representations and cross-channel interactions, it remains challenging to reliably capture inter-variable dependencies under specific conditions. We observe that dependencies in real data are often state-dependent and noisy; in such cases, dense interactions can amplify spurious correlations and lead to representation over-smoothing, which may yield unreliable predictions in certain scenarios. Motivated by this, we propose MS-FLOW, a sparse-bottleneck framework that explicitly models inter-variable interaction as capacity-limited information flow. Specifically, MS-FLOW replaces fully connected communication with selective sparse routing, retaining only a few critical dependency paths and injecting cross-variable signals under a strict communication budget, thereby suppressing redundant connections and spurious-correlation propagation. Extensive experiments demonstrate that MS-FLOW learns more reliable multivariate correlations, achieving state-of-the-art forecasting accuracy on 12 real-world benchmarks while producing fewer yet more reliable dependencies, shifting multivariate forecasting from "more interaction" to "more effective interaction".
Fan Zhang, Shiming Fan, Hua Wang
Shandong Technology and Business University, Yantai, Shandong, China · Ludong University, Yantai, Shandong, China
Multivariate time series forecasting presents unique challenges because future variables often co-evolve under shared system dynamics. While existing studies mainly focus on cross-variable dependencies in historical observations, dependencies among future values are much less explored. Specifically, modern forecasting models largely follow the Direct Forecasting (DF) paradigm, generating multi-step forecasts with point-wise objectives that do not explicitly constrain cross-variable structure. In this work, we show that the DF objective is mismatched in the presence of cross-variable and lagged dependencies, revealing an objective gap. To address this issue, we propose Cross-Variable Loss (CvLoss), a plug-in structural regularizer that constrains forecast residuals on a cross-variable graph. CvLoss penalizes inconsistent edge-wise residual differences over forecast patches, encouraging consistency across both synchronous and asynchronous interactions. Our experiments show that CvLoss consistently improves competitive forecasting models, outperforms representative learning objectives, and is compatible with a variety of forecasting backbones.
Kuiye Ding, Yifan Hu, Hao Xue +1
University of Technology Sydney · Tsinghua University · The Hong Kong University of Science and Technology (Guangzhou)
Multivariate time series forecasting is widely applied in fields such as transportation, energy, and finance. However, the data commonly suffers from issues of multi-scale characteristics, weak correlations, and noise interference, which limit the predictive performance of existing models. This paper proposes a dual-stream sparse Mixer prediction framework that extracts global trends and local dynamic features from sequences in both the frequency and time domains, respectively. It employs a sparsity mechanism to filter out invalid information, thereby enhancing the accuracy of cross-variable dependency modeling. Experimental results demonstrate that this method achieves leading performance on multiple real-world scenario datasets, validating its effectiveness and generality. The code is available at https://github.com/SDMixer/SDMixer
Xiang Ao
School of Software Engineering, Beijing Jiaotong University, Beijing, China