T2ST: Data-Efficient Spatio-Temporal Foundation Models via Temporal Data and Knowledge Transfer
Organizations: Faculty of Information Science and Computing, University of Macau, Macau, China · School of Artificial Intelligence, Nanchang University, Nanchang, China · School of Computer Science Intelligence, Nanchang University, Nanchang, China · School of Computer Science and Engineering, Beihang University, Beijing, China
Abstract
Spatio-temporal foundation models (STFMs) require large-scale training corpora, yet synchronized observations and spatial metadata are costly to acquire and curate, limiting the availability of suitable data. By contrast, time-series data are abundant and easier to collect. Since observations at each location form a time series, this motivates a natural question: can abundant time-series data enhance STFMs when spatio-temporal observations are scarce? To bridge this gap, we propose T2ST, a data-efficient training framework that augments limited spatio-temporal observations with selected auxiliary time series and transfers knowledge from a frozen time-series foundation model. Our key insight is that temporal pattern learning can benefit from abundant external time series, while spatial dependency learning remains grounded in real spatio-temporal observations. Guided by this insight, T2ST retrieves matched sequences to reinforce sparsely observed temporal patterns and selects reliable complementary sequences to broaden temporal coverage. Temporal relation distillation aligns relationships among historical patches, while multi-scale prediction distillation transfers forecasting behavior at fine and coarse temporal resolutions, enabling knowledge transfer across heterogeneous architectures. Using only 10% of the original spatio-temporal data, T2ST enables STFMs to achieve overall forecasting performance competitive with or superior to their full-data counterparts across six long-term and twelve short-term forecasting datasets. These results establish a new paradigm for data-efficient training of spatio-temporal foundation models. Code is available at https://anonymous.4open.science/r/T2ST-65F3/.
Figures & tables
| Model | Points (B) |
|---|---|
| Spatio-temporal foundation models (STFMs) | |
| OpenCity [ 7 ] | |
| CompactST [ 4 ] | |
| WorldST [ 9 ] | |
| Time-series foundation models (TSFMs) | |
| Dataset | Category | Data format | # Regions | Interval |
| (a) Short-term: , | ||||
| PEMS-03 | Flow | Sensor-based | 358 | 5 min |
| PEMS-04 | Flow | Sensor-based | 307 | 5 min |
| PEMS-07 | Flow | Sensor-based | 883 | 5 min |
| PEMS-08 | Flow | Sensor-based | 170 | 5 min |
| OCC-Pairs | Occupancy | Sensor-based | 106 | 5 min |
| Model | CAD3 | CAD5 | PEMS07M | TrafficSH | CHI-TAXI | NYC-BIKE | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MAE | RMSE | MAE | RMSE | MAE | RMSE | MAE | RMSE | MAE | RMSE | MAE | RMSE | |
| PDFormer | 20.28 | 36.43 | 12.89 | 21.18 | 4.62 | 8.36 | 0.77 | 1.23 | 4.03 | 12.82 | 7.33 | 13.01 |
| STWA | 21.65 | 37.55 | 14.43 | 24.14 | 4.54 | 8.57 | 1.42 | 2.49 | 3.70 | 11.49 | 7.97 | 14.35 |
| MSDR | 22.74 | 37.15 | 15.44 | 29.52 | 4.81 | 8.61 | 1.84 | 3.55 | 3.55 | 10.39 | 8.08 | 13.97 |
| AGCRN | 18.72 | 31.93 | 12.88 | 23.78 | 4.61 | 8.63 | 1.33 | 2.45 | 3.60 | 11.31 | 8.13 | 14.46 |
| MTGNN | 17.59 | 28.92 | 11.70 | 20.30 | 4.52 | 8.06 | 0.81 | 1.26 | 3.27 | 9.87 | 6.48 | 11.49 |
| Model | TrafficSH | NYC-BIKE | NYC-TAXI | Tdrive-BJ | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MAE | RMSE | MAPE | MAE | RMSE | MAPE | MAE | RMSE | MAPE | MAE | RMSE | MAPE | |
| TimesFM++ | 0.49 | 1.14 | 7.31 | 3.98 | 7.37 | 65.52 | 8.95 | 20.04 | 49.50 | 13.90 | 30.48 | 18.26 |
| Moirai | 0.46 | 0.91 | 8.76 | 1.63 | 5.87 | 85.18 | 7.58 | 22.72 | 63.33 | 21.73 | 52.16 | 31.99 |
| TimeMoE | 0.43 | 0.74 | 6.65 | 3.90 | 8.08 | 64.20 | 10.05 | 23.59 | 89.85 | 21.40 | 36.67 | 30.76 |
| Chronos | 0.45 | 0.80 | 6.77 | 4.08 | 7.64 | 57.62 | 9.48 | 21.69 | 43.22 | 16.91 | 35.01 | 20.79 |
| FactoST | 0.40 | 0.70 | 5.84 | 3.77 | 7.11 | 54.36 | 8.84 | 21.68 | 42.33 | 17.26 | 35.23 | 20.61 |
| Setting | CAD3 | CAD5 | PEMS07M | TrafficSH | CHI-TAXI | NYC-BIKE | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MAE | RMSE | MAE | RMSE | MAE | RMSE | MAE | RMSE | MAE | RMSE | MAE | RMSE | |
| w/o and | 17.07 | 28.85 | 11.05 | 18.96 | 4.22 | 7.84 | 0.56 | 0.85 | 2.33 | 5.35 | 6.09 | 10.90 |
| w/o | 16.52 | 28.05 | 10.81 | 18.58 | 4.19 | 7.79 | 0.56 | 0.86 | 2.34 | 5.33 | 6.05 | 10.88 |
| w/ random and | 16.81 | 28.06 | 11.17 | 19.03 | 4.26 | 7.89 | 0.59 | 0.89 | 2.44 | 5.57 | 6.47 | 11.36 |
| w/ and | 16.25 | 27.72 | 10.74 | 18.51 | 4.16 | 7.81 | 0.55 | 0.85 | 2.34 | 5.41 | 6.05 | 10.92 |
| Dataset | UrbanFM (Full) | T2ST (UrbanFM) | OpenCity (Full) | T2ST (OpenCity) | ||||
|---|---|---|---|---|---|---|---|---|
| MAE | RMSE | MAE | RMSE | MAE | RMSE | MAE | RMSE | |
| OCC-Pairs | 1.35 | 2.95 | 1.29 | 2.92 | 1.28 | 2.90 | 1.37 | 3.05 |
| OCC-Hamburg | 6.23 | 10.38 | 6.20 | 10.41 | 6.19 | 10.71 | 6.11 | 10.45 |
| PEMS-Bay | 2.49 | 5.63 | 2.32 | 5.31 | 2.58 | 6.07 | 2.36 | 5.75 |
| METR-LA | 6.17 | 13.33 | 6.05 | 13.15 | 6.41 | 14.29 | 5.86 | 13.67 |
| PEMS-03 | 21.53 | 34.33 | 21.40 | 34.50 | 21.61 | 35.39 | 22.25 | 34.93 |
| Dataset | TS2Vec | TS-TCC | CoST | |||
|---|---|---|---|---|---|---|
| MAE | RMSE | MAE | RMSE | MAE | RMSE | |
| CAD3 | 16.25 | 27.72 | 16.57 | 27.94 | 16.47 | 28.13 |
| CAD5 | 10.74 | 18.51 | 10.99 | 18.93 | 10.82 | 18.68 |
| PEMS07M | 4.16 | 7.81 | 4.29 | 7.95 | 4.17 | 7.73 |
| TrafficSH | 0.55 | 0.85 | 0.56 | 0.86 | 0.56 | 0.86 |
| CHI-TAXI | 2.34 | 5.41 | 2.32 | 5.33 | 2.31 | 5.23 |
Spatiotemporal Forecasting
Spatiotemporal forecasting predicts how quantities change across locations and time, from traffic and weather to disease and environmental systems.
See all 448 papers