Organizations: Faculty of Information Science and Computing, University of Macau, Macau, China · School of Artificial Intelligence, Nanchang University, Nanchang, China · School of Computer Science and Engineering, Beihang University, Beijing, China
Urban spatio-temporal foundation models must share knowledge across diverse urban scenarios while accommodating heterogeneous node dynamics. Our analysis of 20+ datasets spanning diverse urban scenarios reveals recurring local graph structures across datasets alongside substantial node heterogeneity within local subgraphs. Motivated by these observations, we propose TopoST, which combines shared learning over local subgraphs with topology-conditioned node specialization. The key insight is to make node specialization itself transferable through a shared topology-to-parameter mapping. This mapping uses identity-independent topological representations to generate node-specific temporal and spatial dependency parameters, enabling direct specialization for unseen nodes without target-specific embeddings or model updates. Extensive experiments demonstrate that TopoST achieves remarkable zero-shot generalization across unseen cities and tasks, highlighting topology-conditioned node specialization as an effective mechanism for transferable urban spatio-temporal foundation models. Code is available at https://github.com/weikaiyun/TopoST.
Figures & tables
Fig. 1: Structure recurrence and node heterogeneity across 20+ urban datasets (3,493 subgraphs; 55,737 nodes). Subgraphs are induced by METIS partitions [ 8 ] with a target size of 16 nodes per partition. (a) Coverage of recurring structures: the percentage of whole subgraphs or node-centered neighborhoods with an exact unweighted structural match in another dataset, averaged across datasets. (b) Within-subgraph variance ratios for temporal autocorrelation and mean neighbor correlation. Autocorrelation is evaluated at one-step, one-hour, and one-day lags. For each dataset, the ratio is defined as Rvar=∑i(ai−aˉ)2∑p∑i∈Vp(ai−aˉp)2 , where ai is the statistic for node i , aˉp is its mean within subgraph Vp , and aˉ is the overall mean. Dots denote datasets; orange marks and labels indicate the mean across datasets.
Fig. 2: Overview of TopoST . (a) Spatial graphs are partitioned into local subgraphs, with observation rows for a selected subgraph highlighted in Xall . Windowing extracts historical inputs X ending at t and subsequent future targets Y for supervision; each local input unit comprises a subgraph and its historical input X . (b) Structural descriptors S=Φ(A) and a shared topology encoder Eϕ produce node codes Z , which generate node-specific temporal query/key weights WQ,WK and spatial GCN weights WG from shared parameter pools, and construct an adaptive adjacency matrix Aadp . (c) The forecasting backbone Fθ embeds normalized historical inputs through temporal patching, linear projection, and sinusoidal positional encoding. A stack of L spatio-temporal encoder blocks combines node-adaptive temporal attention with spatial graph convolution, followed by a forecast head producing Y . All learnable components are jointly pretrained using the forecasting loss Lpred(Θ) . Dataset, subgraph, forecast-origin, and block indices are omitted in this overview; each W⋆ denotes the collection of node-specific weights {W⋆,i}i=1n for ⋆∈{Q,K,G} .
Setting
Model
CAD3
CAD5
PEMS07M
TrafficSH
CHI-TAXI
NYC-BIKE
MAE
RMSE
MAPE
MAE
RMSE
MAPE
MAE
RMSE
MAPE
MAE
RMSE
MAPE
MAE
RMSE
MAPE
MAE
RMSE
MAPE
Full-shot
PDFormer
20.28
36.43
25.19
12.89
21.18
28.82
4.62
8.36
13.74
0.77
1.23
8.34
4.03
12.82
44.42
7.33
13.01
65.44
STWA
21.65
37.55
26.85
14.43
24.14
29.34
4.54
8.57
12.91
1.42
2.49
9.92
3.70
11.49
42.52
7.97
14.35
64.19
MSDR
22.74
37.15
32.37
15.44
29.52
32.49
4.81
8.61
13.90
1.84
3.55
9.91
3.55
10.39
52.88
8.08
13.97
77.56
AGCRN
18.72
31.93
25.45
12.88
23.78
26.29
4.61
8.63
12.81
1.33
2.45
8.51
3.60
11.31
46.48
8.13
14.46
78.36
MTGNN
17.59
28.92
25.22
11.70
20.30
26.75
4.52
8.06
13.11
0.81
1.26
8.29
3.27
9.87
39.38
6.48
11.49
61.52
TABLE I: Forecasting results over a one-day horizon on the OpenCity benchmark. UrbanFM and UniST use their official implementations. OpenCity and full-shot baseline results are taken from the original OpenCity paper. Lower values are better. Bold and underlined values indicate the best and second-best results in each column, respectively. MAPE is reported in percent.
Model
Metric ↓
PEMS-03
PEMS-04
PEMS-07
PEMS-08
OCC- Paris
OCC- Hamburg
PEMS- BAY
METR-LA
Traffic- SH
Bike- NYC
Taxi- NYC
TDrive- BJ
UniST
MAE
25.22 /
35.39
34.63 /
46.26
44.02 /
56.86
29.88 /
39.89
1.79 /
2.21
6.66 /
7.41
2.18 /
3.18
5.93 /
8.27
0.60 /
0.73
3.33 /
4.81
5.21 /
6.02
37.93 /
40.48
RMSE
38.10 /
55.04
50.69 /
68.29
62.03 /
82.41
42.93 /
57.78
3.16 /
4.01
10.98 /
12.12
4.76 /
6.68
12.65 /
15.75
0.84 /
1.05
6.54 /
9.99
14.36 /
17.24
72.97 /
77.80
MAPE
21.30 /
28.42
22.49 /
28.99
19.08 /
23.83
18.28 /
23.42
25.59 /
30.51
55.63 /
62.52
4.26 /
6.43
21.14 /
30.44
7.48 /
9.21
42.11 /
54.29
30.63 /
34.44
51.43 /
54.04
OpenCity
MAE
25.57 /
27.85
32.82 /
35.36
34.50 /
38.41
26.92 /
29.64
1.80 /
2.03
7.07 /
7.33
2.22 /
2.68
5.44 /
6.38
0.45 /
0.48
4.67 /
5.17
12.23 /
13.50
24.11 /
50.05
RMSE
49.25 /
50.82
58.42 /
59.94
58.19 /
61.37
47.47 /
48.89
3.99 /
4.14
11.41 /
11.68
4.71 /
5.70
10.46 /
11.37
0.82 /
0.81
11.03 /
11.79
41.34 /
40.83
71.15 /
69.42
MAPE
23.86 /
27.95
22.57 /
26.28
15.21 /
17.91
16.83 /
19.50
35.19 /
43.62
132.37 /
143.16
5.23 /
6.56
13.51 /
16.39
7.21 /
7.41
73.36 /
85.08
53.82 /
60.91
25.17 /
30.96
TABLE II: Zero-shot forecasting on the 12 target datasets of the UrbanFM benchmark. Each cell reports results for 12 → 12 / 24 → 24 input–output steps. UrbanFM, FactoST, and OpenCity results are taken from the original UrbanFM paper. UniST uses its official implementation. Lower is better; bold and underlined values indicate the best and second-best results for each dataset, metric, and horizon, respectively. MAPE is in percent.
Model
Params (M)
Batch size
Time (h/epoch)
Memory (GiB)
UniST
3.92
16
4.50
17.51
OpenCity
25.97
8
5.05
94.95
UrbanFM
2.25
16
17.74
12.90
TopoST
2.16
16
3.84
1.28
TABLE III: Model sizes and training-cost comparison over the 20 source datasets on a single NVIDIA RTX PRO 6000 Blackwell Workstation Edition GPU (96 GiB).
Variant
CAD5
CHI-TAXI
MAE
RMSE
MAPE
MAE
RMSE
MAPE
TopoST (full)
9.36
15.82
24.12
1.84
4.19
38.92
w/o temporal specialization
9.81
16.53
25.75
1.91
4.48
40.07
w/o spatial specialization
10.28
17.28
26.20
1.94
4.55
44.00
w/o node specialization
9.78
16.63
25.95
1.91
4.43
41.99
Topology as features
9.76
16.41
25.18
1.95
4.61
39.53
TABLE V: Component and full-graph ablations. The full TopoST uses SSE. Lower is better; bold marks minima, including ties. MAPE is in percent.
Fig. 3: Inference scaling on TrafficSH.
Dataset
Metric ↓
Few-shot
Zero-shot
Full-shot
TopoST
UrbanFM
OpenCity
TopoST
UrbanFM
OpenCity
PDFormer
STWA
AGCRN
MTGNN
GWN
STGCN
CD-DIDI
MAE
2.60
2.84
2.88
6.02
7.14
6.34
3.00
3.29
3.55
3.38
3.33
3.29
RMSE
3.88
4.23
4.21
7.93
9.08
9.36
4.31
4.74
5.03
4.80
4.75
4.68
SZ-DIDI
MAE
2.42
2.46
2.61
4.43
5.28
4.66
2.75
3.36
3.16
3.08
3.14
2.87
RMSE
3.75
3.93
4.04
6.18
7.22
7.10
4.06
4.74
4.54
4.32
4.47
4.07
TABLE VIII: Forecasting results on CD-DIDI and SZ-DIDI. Lower values are better; bold indicates the best result in each row.
Urban systems, as dynamic complex systems, continuously generate spatio-temporal data streams that encode the fundamental laws of human mobility and city evolution. While AI for Science has witnessed the transformative power of foundation models in disciplines like genomics and meteorology, urban computing remains fragmented due to "scenario-specific" models, which are overfitted to specific regions or tasks, hindering their generalizability. To bridge this gap and advance spatio-temporal foundation models for urban systems, we adopt scaling as the central perspective and systematically investigate two key questions: what to scale and how to scale. Grounded in first-principles analysis, we identify three critical dimensions: heterogeneity, correlation, and dynamics, aligning these principles with the fundamental scientific properties of urban spatio-temporal data. Specifically, to address heterogeneity through data scaling, we construct WorldST. This billion-scale corpus standardizes diverse physical signals, such as traffic flow and speed, from over 100 global cities into a unified data format. To enable computation scaling for modeling correlations, we introduce the MiniST unit, a novel split mechanism that discretizes continuous spatio-temporal fields into learnable computational units to unify representations of grid-based and sensor-based observations. Finally, addressing dynamics via architecture scaling, we propose UrbanFM, a minimalist self-attention architecture designed with limited inductive biases to autonomously learn dynamic spatio-temporal dependencies from massive data. Furthermore, we establish EvalST, the largest-scale urban spatio-temporal benchmark to date. Extensive experiments demonstrate that UrbanFM achieves remarkable zero-shot generalization across unseen cities and tasks, marking a pivotal first step toward large-scale urban spatio-temporal foundation models.
Spatio-temporal foundation models (STFMs) require large-scale training corpora, yet synchronized observations and spatial metadata are costly to acquire and curate, limiting the availability of suitable data. By contrast, time-series data are abundant and easier to collect. Since observations at each location form a time series, this motivates a natural question: can abundant time-series data enhance STFMs when spatio-temporal observations are scarce? To bridge this gap, we propose T2ST, a data-efficient training framework that augments limited spatio-temporal observations with selected auxiliary time series and transfers knowledge from a frozen time-series foundation model. Our key insight is that temporal pattern learning can benefit from abundant external time series, while spatial dependency learning remains grounded in real spatio-temporal observations. Guided by this insight, T2ST retrieves matched sequences to reinforce sparsely observed temporal patterns and selects reliable complementary sequences to broaden temporal coverage. Temporal relation distillation aligns relationships among historical patches, while multi-scale prediction distillation transfers forecasting behavior at fine and coarse temporal resolutions, enabling knowledge transfer across heterogeneous architectures. Using only 10% of the original spatio-temporal data, T2ST enables STFMs to achieve overall forecasting performance competitive with or superior to their full-data counterparts across six long-term and twelve short-term forecasting datasets. These results establish a new paradigm for data-efficient training of spatio-temporal foundation models. Code is available at https://anonymous.4open.science/r/T2ST-65F3/.
Yue Xu, Kaiyun Wei, Wenying Duan +3
Faculty of Information Science and Computing, University of Macau, Macau, China · School of Artificial Intelligence, Nanchang University, Nanchang, China · School of Computer Science Intelligence, Nanchang University, Nanchang, China +1
Recent advances in urban region representation learning have enabled a wide range of applications in urban analytics, yet existing methods remain limited in their capabilities to generalize across cities and analytic tasks. We aim to generalize urban representation learning beyond city- and task-specific settings, towards a foundation-style model for urban analytics. To this end, we propose UrbanVerse, a model for cross-city urban representation learning and cross-task urban analytics. For cross-city generalization, UrbanVerse focuses on features local to the target regions and structural features of the nearby regions rather than the entire city. We model regions as nodes on a graph, which enables a random walk-based procedure to form "sequences of regions" that reflect both local and neighborhood structural features for urban region representation learning. For cross-task generalization, we propose a cross-task learning module named HCondDiffCT. This module integrates region-conditioned prior knowledge and task-conditioned semantics into the diffusion process to jointly model multiple downstream urban prediction tasks. HCondDiffCT is generic. It can also be integrated with existing urban representation learning models to enhance their downstream task effectiveness. Experiments on real-world datasets show that UrbanVerse consistently outperforms state-of-the-art methods across six tasks under cross-city settings, achieving up to 35.89% improvements in prediction accuracy.
Fengze Sun, Egemen Tanin, Shanika Karunasekera +3
School of Computing and Information System, University of Melbourne, Melbourne, Australia · School of Computer Science and Engineering, University of New South Wales, Sydney, Australia