Spectral Adaptive Graph Transformer for Efficient and Scalable Traffic Forecasting

Amirhossein Ghaffari; Susanna Pirttikangas; Ekaterina Gilman

cs.AIOct 18, 2026

Spectral Adaptive Graph Transformer for Efficient and Scalable Traffic Forecasting

Authors: Amirhossein Ghaffari, Susanna Pirttikangas, Ekaterina Gilman

Organizations: Future Computing Group, University of Oulu, 90570 Oulu, Finland

Abstract

Traffic forecasting models based on spectral graph convolution offer an interpretable account of how information is mixed between sensors, but they rely on an eigendecomposition of the graph Laplacian. It costs cubic time, is fixed by the road network, and provides a symmetric operator that cannot represent the directed propagation of traffic congestion. Attention-based spatio-temporal transformers remove these limitations but generate a dense sensor-by-sensor attention map at each layer and time step, so memory, rather than accuracy, becomes the binding constraint on large networks. We introduce the spectral adaptive graph transformer, whose spatial operator is learned rather than derived from the Laplacian, directed rather than symmetric, and conditioned on the current traffic state rather than a static state, at a cost linear in the number of sensors. A sparse road graph diffusion branch provides the strictly local propagation that a low-rank operator cannot express, and a zero-initialized gate learns the balance between the two. Before spatial mixing, a unit-mean-gain frequency-bin attention module reweights each sensor's short-term periodicities according to its identity and current context. Across six public benchmarks spanning 170 to 1843 sensors, evaluated over three random seeds, the proposed model attains the lowest error of any single end-to-end model on one benchmark and remains competitive with the strongest current attention models on the others, while using about a quarter of their memory and training about twice as fast.

Figures & tables

Symbol	Meaning	Value / shape
Sizes and hyperparameters
$N$	number of sensors (nodes)	170–1843
$T$	input window length	12
$H$	forecast horizon	12
$C$	input channels	3
$d$	model (token) width	152

TABLE I: Notation and Configuration Used

Fig. 1: Architecture of SAGT. (a) Pipeline. The input

\mathcal{X}

passes through frequency-bin attention (Sec. III-C ), whose sole output is the reweighted series that substitutes for the value channel; the calendar channels are never filtered. Tokens are formed by concatenating four blocks of width

24/24/24/80

into

d=152

(Sec. III-D ), refined by

L=3

blocks of a temporal attention layer (Sec. III-E ) followed by a hybrid spatial layer (Sec. III-F ), and mapped to all

H

horizons by one linear head. Shape is preserved throughout the encoder. (b) The hybrid spatial layer, expanded. The global branch analyses with

\mathbf{U}_{k}^{\top}

, applies the input-conditioned signed filter

\boldsymbol{\theta}_{b,t}

predicted from the sensor-pooled context, and synthesizes with

\mathbf{U}_{q}\!\neq\!\mathbf{U}_{k}

; the local branch applies two sparse directed diffusions on the road graph. The effective operator

\mathbf{M}_{b,t}

is never formed: all inter-sensor mixing passes through the rank-

k

waist, so no

N\times N

quantity is stored. The fusion projection

\mathbf{W}_{f}

is the single zero-initialized gate, so the layer begins as the identity.

Variant	Spatial layer	Freq. attention
SAGT-D	single-basis, ( 37 )	no
SAGT-E	single-basis, ( 37 )	yes
SAGT-F (full)	hybrid, Sec. III-F	yes

TABLE II: Reported Variants. All Share the Temporal Backbone of Section III-E , the Embedding of Section III-D and the Head of Section III-H

Dataset	$N$	Steps	Interval	$P$	Signal
PEMS08	170	17 856	5 min	288	flow
METR-LA	207	34 272	5 min	288	speed
PEMS04	307	16 992	5 min	288	flow
PEMS-BAY	325	52 116	5 min	288	speed
PEMS07	883	28 224	5 min	288	flow
EXPY-TKY	1843	13 248	10 min	144	speed

TABLE III: Benchmark Datasets.

P

Is the Number of Time Steps per Day

Model	PEMS08			METR-LA			PEMS04
Model	MAE	RMSE	MAPE	MAE	RMSE	MAPE	MAE	RMSE	MAPE
STGCN	16.66 $\pm$ 0.07	25.85 $\pm$ 0.06	11.15 $\pm$ 0.31%	3.13 $\pm$ 0.00	6.35 $\pm$ 0.02	8.70 $\pm$ 0.07%	20.36 $\pm$ 0.13	32.16 $\pm$ 0.33	14.26 $\pm$ 0.30%
DCRNN	15.64 $\pm$ 0.06	24.66 $\pm$ 0.06	10.21 $\pm$ 0.07%	3.59 $\pm$ 0.39	7.04 $\pm$ 0.56	9.39 $\pm$ 0.31%	20.71 $\pm$ 0.09	32.44 $\pm$ 0.10	14.57 $\pm$ 0.14%
AGCRN	15.86 $\pm$ 0.11	25.15 $\pm$ 0.07	10.81 $\pm$ 0.53%	3.19 $\pm$ 0.01	6.45 $\pm$ 0.01	8.86 $\pm$ 0.06%	19.40 $\pm$ 0.05	31.26 $\pm$ 0.15	13.44 $\pm$ 0.28%
StemGNN	17.53 $\pm$ 0.55	27.52 $\pm$ 0.60	11.37 $\pm$ 0.46%	3.71 $\pm$ 0.28	7.32 $\pm$ 0.43	10.77 $\pm$ 1.00%	22.96 $\pm$ 0.51	35.59 $\pm$ 0.59	15.94 $\pm$ 0.31%
MegaCRN	15.42 $\pm$ 0.30	24.39 $\pm$ 0.37	9.95 $\pm$ 0.30%	4.19 $\pm$ 0.24	10.12 $\pm$ 0.65	10.35 $\pm$ 0.49%	19.91 $\pm$ 1.15	31.44 $\pm$ 0.90	14.64 $\pm$ 1.98%

TABLE IV: Averaged MAE, RMSE, and MAPE Over 12 Horizons on Six Benchmarks. Ranking ( Bold Best, Underlined Second) Is Computed per Metric Over Single End-to-End Models Only. † GC-MoE Composes Three Frozen Pretrained Experts and Is Shown for Completeness but Not Ranked (Sec. IV-D ); Its Cost Accounting Appears in Table IX .

Model	Horizon 3			Horizon 6			Horizon 12
Model	MAE	RMSE	MAPE	MAE	RMSE	MAPE	MAE	RMSE	MAPE
PEMS08
PDFormer	14.50 $\pm$ 0.74	22.42 $\pm$ 0.35	14.34 $\pm$ 3.33%	15.25 $\pm$ 0.85	24.01 $\pm$ 0.28	14.98 $\pm$ 3.45%	16.61 $\pm$ 0.88	26.15 $\pm$ 0.32	16.26 $\pm$ 3.68%
STAEformer	13.88 $\pm$ 0.08	22.29 $\pm$ 0.11	8.94 $\pm$ 0.12%	14.75 $\pm$ 0.12	23.96 $\pm$ 0.15	9.54 $\pm$ 0.12%	16.36 $\pm$ 0.13	26.50 $\pm$ 0.15	10.80 $\pm$ 0.21%
TESTAM	13.77 $\pm$ 0.12	22.40 $\pm$ 0.16	8.94 $\pm$ 0.03%	14.59 $\pm$ 0.13	24.11 $\pm$ 0.21	9.44 $\pm$ 0.03%	16.01 $\pm$ 0.20	26.39 $\pm$ 0.29	10.41 $\pm$ 0.12%
SAGT-F (ours)	13.43 $\pm$ 0.15	21.86 $\pm$ 0.07	8.75 $\pm$ 0.15%	14.38 $\pm$ 0.12	23.66 $\pm$ 0.02	9.27 $\pm$ 0.11%	15.89 $\pm$ 0.06	26.03 $\pm$ 0.06	10.37 $\pm$ 0.07%

TABLE V: Horizon-Resolved MAE, RMSE, and MAPE at 3, 6, and 12 Steps Ahead. For Consistency, the Same Four Ranked Models Are Shown on Every Dataset: the Three Strongest Recent Transformer Baselines and Ours. Bold Marks the Best per Column Within Each Dataset. † GC-MoE Is a Composite Method, Shown for Reference and Not Ranked

Var.	Modification introduced	MAE	RMSE	MAPE
A	Learned adjacency	25.96 $\pm$ 0.86	39.39 $\pm$ 1.42	24.06 $\pm$ 2.67%
B	Learned orthonormal basis	25.08 $\pm$ 0.50	38.80 $\pm$ 0.62	18.07 $\pm$ 0.70%
C	Input-conditioned filter	25.05 $\pm$ 0.56	38.63 $\pm$ 0.63	18.45 $\pm$ 0.78%
D	Transformer temporal backbone	19.14 $\pm$ 0.19	31.12 $\pm$ 0.30	13.38 $\pm$ 0.22%
E	+ frequency-bin attention	19.04 $\pm$ 0.13	30.98 $\pm$ 0.31	13.10 $\pm$ 0.01%
F	+ directed 2-basis, local (full)	18.89 $\pm$ 0.13	30.52 $\pm$ 0.33	13.14 $\pm$ 0.20%

TABLE VI: Incremental Construction of SAGT on PEMS04. Each Row Adds One Modification to the Row Above, Except Row F, Which Introduces Both the Directed Two-Basis Operator and the Local Branch; These Are Separated in Table VII

Configuration	PEMS04		PEMS08
Configuration	MAE	$\Delta$	MAE	$\Delta$
SAGT-F (full)	18.89 $\pm$ 0.13	–	14.39 $\pm$ 0.11	–
w/o local diffusion	18.89 $\pm$ 0.05	$0.00$	14.56 $\pm$ 0.12	$+0.17$
w/o global spectral	19.08 $\pm$ 0.23	$+0.19$	14.64 $\pm$ 0.07	$+0.25$
w/o freq. attention	19.25 $\pm$ 0.35	$+0.36$	14.38 $\pm$ 0.29	$-0.01$

TABLE VII: Component Ablations of the Full Model (Higher Is Worse).

Configuration	PEMS04		PEMS08
Configuration	MAE	$\Delta$	MAE	$\Delta$
k = 16	19.04 $\pm$ 0.09	$+0.15$	14.39 $\pm$ 0.09	$0.00$
k = 32 (default)	18.89 $\pm$ 0.13	–	14.39 $\pm$ 0.11	–
k = 64	18.92 $\pm$ 0.08	$+0.03$	14.49 $\pm$ 0.07	$+0.10$

TABLE VIII: Rank Parameter (K) Sweep Experiments (Higher Is Worse).

Model	Params	Batch size 1 (all models fit)			Batch size 8
Model	Params	Mem. (GB)	Train (ms)	Infer (ms)	Mem. (GB)	Train (ms)	Infer (ms)
STGCN	1.08 M	0.15	10.9	4.0	0.61	47.5	16.5
DCRNN	0.37 M	0.54	372.8	104.4	4.17	647.7	269.4
StemGNN	11.33 M	0.53	636.9	214.6	2.53	670.8	284.0
MegaCRN	0.45 M	4.29	987.7	466.5	7.50	1650.5	672.6
AGCRN	0.76 M	9.47	419.9	239.7	10.65	586.4	247.7

TABLE IX: Computational Cost on EXPY-TKY (

N{=}1843

). All Columns, Including Parameter Counts, Are Measured on This Benchmark; Models With Node Embeddings Grow With

N

, So These Counts Are Not Comparable With Values Reported on Smaller Networks. Per-Iteration Figures Are Measured at Two Batch Sizes; OOM Denotes Exhaustion of

16

GB.

Fig. 2: Frequency-bin attention averaged over the test split, in unit-mean gain units $\bar{w}_{v}[j]=F\bar{\alpha}_{v}[j]$ ( $1.0$ is the identity). (a,d) Per-sensor gain, sensors sorted by mass in bins $j\geq 3$ . (b,e) Gain per bin for the quietest and most variable sensor deciles, with the $5$ – $95$ th percentile band. (c,f) Mass in bins $j\geq 3$ against mean within-window variance, with Spearman $\rho$ ; dotted line marks the uniform value $4/7$ . Top: PEMS04 ( $N=307$ ). Bottom: PEMS08 ( $N=170$ ).

Spatiotemporal Forecasting

Spatiotemporal forecasting predicts how quantities change across locations and time, from traffic and weather to disease and environmental systems.

See all 410 papers

Apr 15, 2026

FAST: A Synergistic Framework of Attention and State-space Models for Spatiotemporal Traffic Prediction

Traffic forecasting requires modeling complex temporal dynamics and long-range spatial dependencies over large sensor networks. Existing methods typically face a trade-off between expressiveness and efficiency: Transformer-based models capture global dependencies well but suffer from quadratic complexity, while recent selective state-space models are computationally efficient yet less effective at modeling spatial interactions in graph-structured traffic data. We propose FAST, a unified framework that combines attention and state-space modeling for scalable spatiotemporal traffic forecasting. FAST adopts a Temporal-Spatial-Temporal architecture, where temporal attention modules capture both short- and long-term temporal patterns, and a Mamba-based spatial module models long-range inter-sensor dependencies with linear complexity. To better represent heterogeneous traffic contexts, FAST further introduces a learnable multi-source spatiotemporal embedding that integrates historical traffic flow, temporal context, and node-level information, together with a multi-level skip prediction mechanism for hierarchical feature fusion. Experiments on PeMS04, PeMS07, and PeMS08 show that FAST consistently outperforms strong baselines from Transformer-, GNN-, attention-, and Mamba-based families. In particular, FAST achieves the best MAE and RMSE on all three benchmarks, with up to 4.3% lower RMSE and 2.8% lower MAE than the strongest baseline, demonstrating a favorable balance between accuracy, scalability, and generalization.

Xinjin Li, Jinghan Cao, Mengyue Wang +5

Columbia University New York, NY, USA · San Francisco State University San Francisco, CA, USA · University of California, Berkeley Berkeley, CA, USA +5

Spatiotemporal Forecasting Traffic Forecasting

Ask PDF

May 25, 2026

ADMFormer: An Adaptive-Decomposition Transformer with Time-Varying Masked Spatial Attention for Traffic Forecasting

Accurate traffic forecasting is essential for intelligent transportation systems, supporting a wide range of real-world applications. However, it remains challenging due to two key factors:(1) Traffic series contain heterogeneous temporal patterns, where stable periodic regularities coexist with event-driven fluctuations. Existing methods often treat them within a unified representation, limiting their ability to capture fine-grained temporal dynamics.(2)Spatial dependencies among nodes are inherently dynamic and sparse, while dense all-pairs attention often introduces redundant interactions and amplifies noise. To address these issues, we propose ADMFormer, an Adaptive-Decomposition Transformer with Time-Varying Masked Spatial Attention. Specifically, ADMFormer first employs a time-node adaptive gating mechanism to decouple traffic signals into dominant regularities and residual fluctuations that vary across time and nodes. A dual-branch temporal module is then designed to separately capture global periodic dependencies and high-frequency irregular variations from these two decomposed components. Furthermore, ADMFormer introduces a time-varying masked spatial attention that sparsifies spatial interactions based on real-time traffic states, thereby effectively preserving dynamic and informative dependencies. Extensive experiments on four real-world datasets demonstrate that ADMFormer achieves state-of-the-art performance.

Ruiwen Gu, Qitai Tan, Yahao Liu +1

Shenzhen Ubiquitous Data Enabling Key Lab Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · School of Computer Science and Engineering University of Electronic Science and Technology of China, Chengdu, China

Attention Mechanisms Spatiotemporal Forecasting Traffic Forecasting Time Series Transformers

Ask PDF

Jul 14, 2026

EMAGN: Efficient Multi-Attention Graph Network via Learned Clustering for Scalable Traffic Forecasting

Traffic forecasting is highly challenging due to complex and nonlinear spatial and temporal dependencies. Self-attention mechanisms have been widely adopted to model dynamic and long-range dependencies, achieving state-of-the-art performance, but suffer from limited scalability due to quadratic computational and memory complexity. To address this, we propose an Efficient Multi-Attention Graph Network (EMAGN) that linearises the spatial attention mechanism itself, inspired by the theory of fast high-dimensional Gaussian filtering. Two learned clustering matrices C_k and C_v adaptively group key and value vectors into M super-clusters, reducing complexity from O(N^2 d) to O(NMd) without sacrificing the flexibility of attention for dynamic dependency modelling. Experimental results on PEMS-BAY and METR-LA show that EMAGN achieves accuracy within 2.7-3.2% MAE of full-attention GMAN while reducing training time by 32%, inference time by 38%, and GPU memory by 58%. Critically, at K=16 attention heads, full-attention GMAN runs out of memory on a standard 11 GB GPU entirely while EMAGN continues to operate, demonstrating a categorical expansion of feasible model configurations. EMAGN also surpasses Linformer and Performer in both accuracy and efficiency within the same backbone, owing to its traffic-network-aware adaptive clustering.

Mingxing Xu, Rakesh Chowdary Machineni, Ke Liu +7

Shanghai Jiao Tong University · University of Michigan, Ann Arbor · University of California, Berkeley +5

Efficient Neural Network Inference Graph Attention Networks Linear Attention Traffic Forecasting Spatiotemporal GNNs

Ask PDF

Abstract

Spatiotemporal Forecasting

Explore similar work

FAST: A Synergistic Framework of Attention and State-space Models for Spatiotemporal Traffic Prediction

ADMFormer: An Adaptive-Decomposition Transformer with Time-Varying Masked Spatial Attention for Traffic Forecasting

EMAGN: Efficient Multi-Attention Graph Network via Learned Clustering for Scalable Traffic Forecasting