VaryST: Generalizable Spatio-Temporal Forecasting Beyond the Closed-World Setting
Authors: Shengnan Guo, Jinjun Cheng, Xiaowei Mao, Jianhao Zhang, Haochen Lv, Yang Jiang, YiHeng Huang, Yan Lin, Youfang Lin, Huaiyu Wan
Organizations: School of Computer Science and Technology, Beijing Jiaotong University, Beijing, China · Key Laboratory of Big Data & Artificial Intelligence in Transportation, Ministry of Education, Beijing, China · Department of Computer Science, Aalborg University, Aalborg, Denmark · Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, Beijing, China
Spatio-temporal forecasting is fundamental to many real-world systems, yet deployment environments are inherently varying: temporal distributions may evolve, spatial domains may change, and observation availability may shift from training conditions. Existing models are often tightly coupled to their training environments through node-specific representations, predefined spatial support sets, and reliance on consistently available observations, limiting generalization to changing environments. To this end, we propose VaryST, a compact spatio-temporal forecasting model trained from scratch that achieves strong generalization by dynamically constructing predictive context from available long-term observations. To accommodate temporal changes, VaryST derives node representations from long-term histories, capturing current behavior in the context of longer-term patterns. To adapt to spatial changes, it dynamically selects functionally relevant support nodes beyond predefined neighborhoods and integrates their information with recent and long-term target states and global context. To handle varying observation availability, VaryST selectively masks historical patches on which the model relies most during training, encouraging it to exploit alternative temporal and spatial dependencies for accurate forecasting when key observations become unavailable. Extensive experiments on six real-world datasets against seventeen baselines show that VaryST achieves strong in-domain forecasting accuracy, substantially improves zero- and few-shot generalization under spatial and temporal changes, and remains robust to varying observation availability.
Figures & tables
Figure 1: ST models in varying deployment environments
Figure 2: The architecture of VaryST. It contains Long-term Observation-derived Representations, Dynamic Functional Support Node Retrieval, and Target-centric Hierarchical Contextual Decoding.
Dataset
Nodes
Time Span
Interval
Usage
PEMS03
358
09/01/2018 – 11/30/2018
5 min
RQ1/3/5/6
PEMS04
307
01/01/2018 – 02/28/2018
5 min
RQ1/3/4
PEMS07
883
05/01/2017 – 08/31/2017
5 min
RQ1/3/7
PEMS08
170
07/01/2016 – 08/31/2016
5 min
RQ1/3/4/5/6
PM2.5
12
03/01/2013 – 02/28/2017
60 min
RQ1
PEMSD3-Stream
655
2011 – 2017
5 min
RQ2
Table 1: Dataset statistics.
Model
PEMS03
PEMS04
PEMS07
PEMS08
MAE
RMSE
MAPE
MAE
RMSE
MAPE
MAE
RMSE
MAPE
MAE
RMSE
MAPE
Models with Short-term History (Input Horizon = 12)
ASTGCN
17.85
29.88
17.65%
22.42
34.75
15.87%
25.98
39.65
11.84%
18.86
28.55
12.50%
AGCRN
15.98
28.25
15.23%
19.83
32.26
12.97%
22.37
36.55
9.12%
15.95
25.22
10.09%
SSTBAN
15.90
26.11
17.27%
18.88
31.10
12.56%
20.17
33.45
8.93%
14.39
24.19
10.06%
PDFormer
14.74
25.59
15.35%
18.31
29.97
12.10%
19.83
32.87
8.53%
13.58
23.51
9.05%
Table 2: Overall prediction performance comparison on four PEMS datasets. Models are categorized into short-term, LLM based and long-term groups. Bold : best, underline : second best.
Model
MAE
RMSE
MAPE
ASTGCN
27.42
48.70
52.4%
AGCRN
28.20
49.57
50.1%
SSTBAN
30.22
53.13
53.1%
PDFormer
26.65
47.08
51.5%
iTransformer
29.56
51.80
55.9%
STD-PLM
27.55
48.88
53.3%
Table 3: Performance comparison on PM2.5.
Figure 3: Distribution-shift robustness of VaryST.
Figure 4: Ablation, hyperparameter and efficiency studies.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Year
Nodes
Time Span
Interval
2011
655
07/10/2011 – 08/31/2011
5 min
2012
715
07/10/2012 – 08/31/2012
5 min
2013
786
07/10/2013 – 08/31/2013
5 min
2014
822
07/10/2014 – 08/31/2014
5 min
2015
834
07/10/2015 – 08/31/2015
5 min
2016
850
07/10/2016 – 08/31/2016
5 min
Appendix
Table 4: Per-year statistics of PEMSD3-Stream.
Hyperparameter
Description
Value or Choices
sample_len
Input history length T
2016 (168 on PM2.5)
output_len
Prediction horizon Q
12
unit_len
Patch length Lp
36 (12 on PM2.5)
train_ratio / val_ratio
Chronological train/validation split
0.6 / 0.2
token_dim
Temporal Patch Token dimension d
96
emb_dim
backbone dim D
96
Appendix
Table 5: Default hyperparameters for the VaryST framework.
Model
MAE
RMSE
MAPE
STOP
13.01
22.44
16.91%
STD-PLM
14.40
25.06
18.55%
VaryST
10.49
18.22
14.05%
Appendix
Table 6: Average Zero-shot performance on PEMSD3-Stream.
Year
VaryST
STOP
STD-PLM
MAE
RMSE
MAPE
MAE
RMSE
MAPE
MAE
RMSE
MAPE
2011
10.64
16.47
13.84
12.34
18.96
16.12
11.96
18.28
16.21
2012
9.97
16.00
13.65
11.86
18.87
16.16
13.54
22.57
17.77
2013
10.01
16.68
13.60
12.27
20.05
16.26
14.24
24.16
18.23
2014
10.61
17.39
14.80
13.24
22.07
17.96
14.62
24.73
19.78
2015
10.55
17.80
14.53
12.99
22.40
17.22
14.32
24.97
18.87
Appendix
Table 7: Year-by-year performance on PEMSD3-Stream. 2011: In-Domain Reference
Model
Source
Target
PEMS03
PEMS04
PEMS07
PEMS08
VaryST
PEMS03
13.95 / 22.73 / 13.59
18.51 / 30.40 / 12.84
20.02 / 32.72 / 8.88
13.81 / 22.70 / 9.31
PEMS04
14.03 / 22.77 / 13.44
17.56 / 29.35 / 11.81
19.91 / 32.64 / 8.48
13.97 / 22.92 / 9.07
PEMS07
13.78 / 22.32 / 13.24
18.15 / 29.98 / 12.54
18.21 / 31.05 / 7.64
13.46 / 22.44 / 8.86
PEMS08
14.88 / 24.45 / 14.61
18.80 / 30.90 / 13.19
19.53 / 32.65 / 8.54
12.89 / 21.99 / 8.50
STD-PLM
PEMS03
14.52 / 24.69 / 14.87
30.90 / 49.01 / 20.88
34.79 / 52.85 / 20.27
25.83 / 39.86 / 25.03
Appendix
Table 8: Zero-shot cross-dataset transfer (12-step prediction). Rows: source dataset used for full training; columns: target dataset evaluated on. Diagonal cells are the in-domain (IID) baselines. Each cell: MAE / RMSE / MAPE (%).
Model
Target
1%
5%
10%
MAE
RMSE
MAPE (%)
MAE
RMSE
MAPE (%)
MAE
RMSE
MAPE (%)
Source: PEMS03
VaryST
PEMS04
18.39
30.39
12.60
18.04
29.93
12.33
17.91
29.78
12.19
PEMS07
19.16
32.03
8.16
18.60
31.48
7.85
18.63
31.50
7.85
PEMS08
13.50
22.54
8.91
13.26
22.31
8.75
13.17
22.21
8.69
STD-PLM
PEMS04
46.75
74.69
22.00
24.45
38.70
16.19
23.38
37.08
15.54
Appendix
Table 9: Few-shot cross-dataset transfer (12-step prediction) with 1%, 5% and 10% of the target training data. Each cell: MAE / RMSE / MAPE (%).
Model
10%
30%
50%
70%
MAE
RMSE
MAPE (%)
MAE
RMSE
MAPE (%)
MAE
RMSE
MAPE (%)
MAE
RMSE
MAPE (%)
PEMS04 Point Missing (RM)
FaST †
18.67
30.29
12.89
18.74
30.37
13.01
18.87
30.50
13.05
19.17
30.78
13.14
STID †
18.34
29.92
13.09
18.38
29.98
12.84
18.53
30.20
12.99
18.75
30.40
13.41
BiST †
18.47
29.94
12.88
18.53
29.94
13.25
18.53
30.10
13.22
19.00
30.42
13.69
VaryST
17.60
29.34
11.90
17.40
29.20
11.74
17.47
29.28
11.80
17.70
29.39
11.93
Appendix
Table 10: Overall prediction performance with missing observations on PEMS04 and PEMS08 under missing rates from 10% to 70%. The best and second-best results are in bold and underline , respectively.
Model
10%
30%
50%
70%
MAE
RMSE
MAPE (%)
MAE
RMSE
MAPE (%)
MAE
RMSE
MAPE (%)
MAE
RMSE
MAPE (%)
PEMS04 Point Missing (RM), source = 30%
FaST †
18.69
30.31
12.97
18.74
30.37
13.01
18.88
30.59
13.09
19.22
30.97
13.39
STID †
18.32
29.91
12.80
18.38
29.98
12.84
18.53
30.19
12.91
18.91
30.62
13.19
BiST †
18.49
29.90
13.22
18.53
29.94
13.25
18.66
30.12
13.32
18.98
30.12
13.57
VaryST
17.60
29.35
12.07
17.40
29.20
11.74
17.55
29.38
11.73
18.28
30.12
12.34
Appendix
Table 11: Generalization to unseen missing rates on PEMS04 and PEMS08. Models are trained at a 30% missing rate.
Model
PEMS04
PEMS08
RM → CM
CM → RM
RM → CM
CM → RM
MAE
RMSE
MAPE (%)
MAE
RMSE
MAPE (%)
MAE
RMSE
MAPE (%)
MAE
RMSE
MAPE (%)
FaST †
20.32
32.40
15.19
19.89
32.16
13.62
16.00
26.00
11.87
14.71
24.13
10.46
STID †
19.96
32.14
14.71
19.06
31.41
13.13
15.33
24.85
11.28
14.02
22.95
9.80
BiST †
20.02
31.87
15.06
18.98
30.86
13.38
15.63
25.06
12.33
14.42
23.37
10.31
VaryST
19.51
31.94
13.96
18.40
30.23
12.75
15.00
24.93
10.43
14.14
23.15
9.36
Appendix
Table 12: Cross-pattern generalization at 30% missing rate on PEMS04 and PEMS08.
Variant
PEMS03
PEMS08
MAE
RMSE
MAPE
MAPE 10
MAE
RMSE
MAPE
MAPE 10
VaryST (full)
13.95
22.73
13.59
11.00
12.89
21.99
8.50
7.29
w/o Laplacian PE
14.13
23.02
13.47
11.04
12.91
22.00
8.57
7.31
w/o Hop-RoPE
14.42
23.75
13.93
11.40
12.93
22.07
8.56
7.32
w/o structural encoding
14.10
23.05
13.55
11.11
12.92
22.06
8.54
7.30
w/o recent-12 token
14.10
23.00
13.35
10.99
12.86
21.97
8.47
7.29
Appendix
Table 13: Ablation study of each component on PEMS03 and PEMS08 (12-step prediction).
Setting
PEMS03
PEMS08
MAE
RMSE
MAPE
MAE
RMSE
MAPE
Input context length
36
15.30
24.37
14.88
14.78
24.91
9.67
288
14.26
23.08
13.87
13.60
22.79
9.02
864
14.31
23.37
13.65
13.40
22.53
8.90
2016 (default)
13.95
22.73
13.59
12.89
21.99
8.50
Appendix
Table 14: Hyperparameter study on PEMS03 and PEMS08. The best and second-best results are in bold and underline , respectively.
Spatial-temporal (ST) forecasting underpins many real-world systems such as traffic, climate, and energy networks. While existing methods implicitly assume strong spatiotemporal coupling, we observe that real-world ST data exhibits distinct coupling regimes, ranging from temporal-dominated and spatial-dominated to strongly coupled patterns. This mismatch causes current models to suffer from spurious dependencies and degraded performance when one correlation dominates. To overcome this limitation, we aim to dynamically modulate spatial and temporal modeling based on the data's inherent coupling structure. However, three key challenges exist: unknown coupling structure, heterogeneous coupling dynamics, and suboptimal spatial modeling. We propose AdaST, an adaptive ST forecasting framework that tackles these challenges through a decompose-recompose paradigm. AdaST factorizes inputs into components capturing different coupling patterns using heterogeneity-aware experts. Each component is processed by role-aligned modules, and a correlation-informed adaptive recomposer integrates them for final prediction. Extensive experiments confirm that AdaST significantly outperforms state-of-the-art baselines, validating the necessity of an adaptive approach.
Zhenyu Lei, Chenghao Liu, Yushun Dong +2
University of Virginia Charlottesville, VA, USA · Datadog Paris, France · Florida State University Tallahassee, FL, USA +1
Spatio-temporal forecasting is a cornerstone of logistics, urban planning, and intelligent transportation systems. However, constrained by deployment costs and maintenance resources, sensor networks often lack comprehensive spatial coverage, rendering Forecast Unobserved Node States (FUNS) a critical yet formidable challenge. Conventional models rely on historical observations and typically falter when encountering nodes without prior records. To address this, we redefine the problem as a conditional generation task on spatio-temporal graphs and propose GenST, a framework that introduces Large Language Models (LLMs) as a semantic bridge, leveraging a pre-trained LLM fine-tuned to extract rich semantic features from node descriptions, such as functional zones and road network structures, to compensate for missing spatio-temporal signals. Specifically, we design a two-stage generative architecture: a Spatio-Temporal VAE first compresses spatio-temporal dynamics into a latent space, followed by a Generative Transformer (GenT) that reconstructs the future states of unobserved nodes from noise, guided by multi-modal conditions including semantics, geographic coordinates, and neighborhood contexts. Experiments on six traffic and two non-traffic datasets show GenST significantly outperforms existing baselines in zero-shot prediction tasks, demonstrating the practical potential of semantic-guided generation for mitigating spatio-temporal data sparsity.
Shuhao Li, Weidong Yang, Changan Liu +4
Fudan University Shanghai, China · Fudan University Shanghai, China Zhuhai Fudan Innovation Research Institute Zhuhai, China · College of Computing and Data Science Nanyang Technological University Singapore +2
Continual spatio-temporal forecasting supports traffic management and environmental monitoring under evolving dynamics and expanding sensor networks. However, conventional graph-based continual learning methods tie forecasting representations to the current sensor layout, so sensor expansion can alter the representation of learned spatial relationships. Our key insight is that sensor expansion changes the evidence available about a process without necessarily changing the dynamics to be learned. We propose STFO (Spatio-Temporal Field Operator), which parameterizes forecasting knowledge as a shared field-evolution operator and handles changing sensor layouts through observation and query interfaces. Normalized coordinate-based aggregation lifts irregular sensor histories onto a fixed latent grid, enabling reuse of learned spatial maps across observation sets without sensor-specific parameters. To accommodate process drift, a spectral descriptor summarizes variation across spatial scales and conditions Fourier propagation and attention to adapt operator responses to the current spatial regime. Coordinate-based decoding queries the evolved field at sensor locations and combines spatial corrections with local-history predictions. Experiments on PEMS-Stream, CA-Stream, and AIR-Stream demonstrate state-of-the-art average forecasting performance. STFO-Large reduces average MAE over DOL by 8.4% on PEMS-Stream and 4.7% on CA-Stream. Our code is available at https://github.com/Xielewei/Spatio-Temporal-Field-Operator.
Lewei Xie, Haoyu Zhang, Jiajun Zhou +6
City University of Hong Kong (Dongguan) · City University of Hong Kong · Zhuhai College of Science and Technology +1