Organizations: Faculty of Information Science and Computing, University of Macau, Macau, China · School of Artificial Intelligence, Nanchang University, Nanchang, China · School of Computer Science Intelligence, Nanchang University, Nanchang, China · School of Computer Science and Engineering, Beihang University, Beijing, China
Spatio-temporal foundation models (STFMs) require large-scale training corpora, yet synchronized observations and spatial metadata are costly to acquire and curate, limiting the availability of suitable data. By contrast, time-series data are abundant and easier to collect. Since observations at each location form a time series, this motivates a natural question: can abundant time-series data enhance STFMs when spatio-temporal observations are scarce? To bridge this gap, we propose T2ST, a data-efficient training framework that augments limited spatio-temporal observations with selected auxiliary time series and transfers knowledge from a frozen time-series foundation model. Our key insight is that temporal pattern learning can benefit from abundant external time series, while spatial dependency learning remains grounded in real spatio-temporal observations. Guided by this insight, T2ST retrieves matched sequences to reinforce sparsely observed temporal patterns and selects reliable complementary sequences to broaden temporal coverage. Temporal relation distillation aligns relationships among historical patches, while multi-scale prediction distillation transfers forecasting behavior at fine and coarse temporal resolutions, enabling knowledge transfer across heterogeneous architectures. Using only 10% of the original spatio-temporal data, T2ST enables STFMs to achieve overall forecasting performance competitive with or superior to their full-data counterparts across six long-term and twelve short-term forecasting datasets. These results establish a new paradigm for data-efficient training of spatio-temporal foundation models. Code is available at https://anonymous.4open.science/r/T2ST-65F3/.
Figures & tables
Model
Points (B)
Spatio-temporal foundation models (STFMs)
OpenCity [ 7 ]
≈0.15
CompactST [ 4 ]
0.29
WorldST [ 9 ]
>1
Time-series foundation models (TSFMs)
TABLE I: Pre-training data scales of representative spatio-temporal and time-series foundation models. The B denotes 109 reported observations or time points.
Fig. 1: Overview of the T2ST framework. (a) Temporal data selection : a shared encoder maps spatio-temporal and auxiliary time-series windows into a common representation space. A calibrated spatio-temporal reference guides matched selection, while teacher reliability filtering and novel pattern discovery guide diversity selection. Selected embeddings identify the source windows retrieved by index to construct DM and DD . (b) Knowledge distillation : a frozen TSFM guides a STFM student through temporal relation distillation and multi-scale prediction distillation. The student is trained on DST , DM , and DD , sharing temporal modeling parameters across both input types and bypassing spatial interactions for temporal-only inputs. Snowflakes and flames indicate frozen and trainable parameters, respectively.
Fig. 2: ST reference exemplar construction and threshold calibration.
Dataset
Category
Data format
# Regions
Interval
(a) Short-term: L=12 , H=12
PEMS-03
Flow
Sensor-based
358
5 min
PEMS-04
Flow
Sensor-based
307
5 min
PEMS-07
Flow
Sensor-based
883
5 min
PEMS-08
Flow
Sensor-based
170
5 min
OCC-Pairs
Occupancy
Sensor-based
106
5 min
TABLE II: Evaluation datasets for short-term forecasting and long-term forecasting.
Model
CAD3
CAD5
PEMS07M
TrafficSH
CHI-TAXI
NYC-BIKE
MAE
RMSE
MAE
RMSE
MAE
RMSE
MAE
RMSE
MAE
RMSE
MAE
RMSE
PDFormer
20.28
36.43
12.89
21.18
4.62
8.36
0.77
1.23
4.03
12.82
7.33
13.01
STWA
21.65
37.55
14.43
24.14
4.54
8.57
1.42
2.49
3.70
11.49
7.97
14.35
MSDR
22.74
37.15
15.44
29.52
4.81
8.61
1.84
3.55
3.55
10.39
8.08
13.97
AGCRN
18.72
31.93
12.88
23.78
4.61
8.63
1.33
2.45
3.60
11.31
8.13
14.46
MTGNN
17.59
28.92
11.70
20.30
4.52
8.06
0.81
1.26
3.27
9.87
6.48
11.49
TABLE III: Long-term forecasting results. The first ten methods are full-shot baselines.Best and second-best results are shown in bold and underlined, respectively, with light purple cell backgrounds.
Model
TrafficSH
NYC-BIKE
NYC-TAXI
Tdrive-BJ
MAE
RMSE
MAPE
MAE
RMSE
MAPE
MAE
RMSE
MAPE
MAE
RMSE
MAPE
TimesFM++
0.49
1.14
7.31
3.98
7.37
65.52
8.95
20.04
49.50
13.90
30.48
18.26
Moirai
0.46
0.91
8.76
1.63
5.87
85.18
7.58
22.72
63.33
21.73
52.16
31.99
TimeMoE
0.43
0.74
6.65
3.90
8.08
64.20
10.05
23.59
89.85
21.40
36.67
30.76
Chronos
0.45
0.80
6.77
4.08
7.64
57.62
9.48
21.69
43.22
16.91
35.01
20.79
FactoST
0.40
0.70
5.84
3.77
7.11
54.36
8.84
21.68
42.33
17.26
35.23
20.61
TABLE IV: Short-term forecasting results. Best and second-best results are shown in bold and underlined, respectively, with light purple cell backgrounds.
Setting
CAD3
CAD5
PEMS07M
TrafficSH
CHI-TAXI
NYC-BIKE
MAE
RMSE
MAE
RMSE
MAE
RMSE
MAE
RMSE
MAE
RMSE
MAE
RMSE
w/o DM and DD
17.07
28.85
11.05
18.96
4.22
7.84
0.56
0.85
2.33
5.35
6.09
10.90
w/o DD
16.52
28.05
10.81
18.58
4.19
7.79
0.56
0.86
2.34
5.33
6.05
10.88
w/ random DM and DD
16.81
28.06
11.17
19.03
4.26
7.89
0.59
0.89
2.44
5.57
6.47
11.36
w/ DM and DD
16.25
27.72
10.74
18.51
4.16
7.81
0.55
0.85
2.34
5.41
6.05
10.92
TABLE V: Ablation of temporal-data composition for distillation with a fixed 10% ST-data budget. The random setting replaces both with randomly sampled temporal data. Bold and underlined values indicate the best and second-best results, respectively.
Dataset
UrbanFM (Full)
T2ST (UrbanFM)
OpenCity (Full)
T2ST (OpenCity)
MAE
RMSE
MAE
RMSE
MAE
RMSE
MAE
RMSE
OCC-Pairs
1.35
2.95
1.29
2.92
1.28
2.90
1.37
3.05
OCC-Hamburg
6.23
10.38
6.20
10.41
6.19
10.71
6.11
10.45
PEMS-Bay
2.49
5.63
2.32
5.31
2.58
6.07
2.36
5.75
METR-LA
6.17
13.33
6.05
13.15
6.41
14.29
5.86
13.67
PEMS-03
21.53
34.33
21.40
34.50
21.61
35.39
22.25
34.93
TABLE VI: Forecasting performance of UrbanFM, OpenCity, and their T2ST variants. Full denotes the full ST training set; T2ST variants use the 10% ST budget.
Dataset
TS2Vec
TS-TCC
CoST
MAE
RMSE
MAE
RMSE
MAE
RMSE
CAD3
16.25
27.72
16.57
27.94
16.47
28.13
CAD5
10.74
18.51
10.99
18.93
10.82
18.68
PEMS07M
4.16
7.81
4.29
7.95
4.17
7.73
TrafficSH
0.55
0.85
0.56
0.86
0.56
0.86
CHI-TAXI
2.34
5.41
2.32
5.33
2.31
5.23
TABLE VII: Effect of different temporal encoders.
Fig. 3: Five largest temporal clusters in (a) the 10% ST subset and (b) the diverse data DD .
Fig. 4: MAE under different ratios of ST to matched temporal batches. Horizontal dashed lines indicate the baseline for distillation using only ST data.
Fig. 5: Forecasting performance of T2ST (OpenCity) with 10%, 30%, and 60% ST-data budgets, each supplemented with auxiliary time-series data for knowledge distillation. The 60% budget denotes the full training split.
Fig. 6: Sensitivity to distillation loss weights on CAD3.
DSA Thrust, The Hong Kong University of Science and Technology (Guangzhou) · Solar System of OVB, Tencent, Guangzhou, China · INTR Thrust, The Hong Kong University of Science and Technology (Guangzhou) +4