Permutation-Equivariant 2D State Space Models: Theory and Canonical Architecture for Multivariate Time Series

Seungwoo Jeong; Heung-Il Suk

stat.MLMar 7, 2026

Permutation-Equivariant 2D State Space Models: Theory and Canonical Architecture for Multivariate Time Series

Authors: Seungwoo Jeong, Heung-Il Suk

Organizations: Department of Artificial Intelligence, Korea University, Seoul 02841, Republic of Korea

Abstract

Multivariate time series (MTS) modeling often implicitly imposes an artificial ordering over variables, violating the inherent exchangeability found in many real-world systems where no canonical variable axis exists. We formalize this limitation as a violation of the permutation symmetry principle and require state-space dynamics to be permutation-equivariant along the variable axis. In this work, we theoretically characterize the complete canonical form of linear variable coupling under this symmetry constraint. We prove that any permutation-equivariant linear 2D state-space system naturally decomposes into local self-dynamics and a global pooled interaction, rendering ordered recurrence not only unnecessary but structurally suboptimal. Motivated by this theoretical foundation, we introduce the Variable-Invariant Two-Dimensional State Space Model (VI 2D SSM), which realizes the canonical equivariant form via permutation-invariant aggregation. This formulation eliminates sequential dependency chains along the variable axis, reducing the dependency depth from $\mathcal{O}(C)$ to $\mathcal{O}(1)$ and simplifying stability analysis to two scalar modes. Furthermore, we propose VI 2D Mamba, a unified architecture integrating multi-scale temporal dynamics and spectral representations. Extensive experiments on forecasting, classification, and anomaly detection benchmarks demonstrate that our model achieves state-of-the-art performance with superior structural scalability, validating the theoretical necessity of symmetry-preserving 2D modeling.

Figures & tables

Fig. 1: Left: 1D SSM models only along the temporal axis, overlooking dependencies across variables. Middle: Conventional 2D SSM captures inter-variable correlations through sequential scans, but still imposes artificial ordering and struggles with distant relationships. Right: Our method performs global aggregation over variables, enabling simultaneous and order-free modeling of inter-variable relationships over time.

Fig. 2: Overview of the proposed Variable-Invariant 2D Mamba. Left: Variable-Invariant 2D SSM with global coupling. A permutation-invariant set aggregator enforces invariance to variable ordering, after which the 2D state space model captures jointly coupled dynamics along temporal and variable dimensions. The globally coupled state transitions enable structured long-range dependency modeling while preserving symmetry across variables. Right: Variable-Invariant 2D Mamba architecture. The framework decomposes representation learning into complementary long-term, short-term, and spectral components. These components are fused to form a unified representation that simultaneously captures global temporal dynamics, local variations, and frequency-domain structure under a variable-invariant formulation.

Dataset	Ours		Chimera		TimePro		Simba		TCN		iTransformer		RLinear		PatchTST		Crossformer		TiDE		TimesNet
Dataset	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTm1	0.345	0.378	0.355	0.381	0.391	0.400	0.383	0.396	0.351	0.381	0.407	0.410	0.414	0.407	0.387	0.400	0.513	0.496	0.419	0.419	0.400	0.406
ETTm2	0.258	0.317	0.252	0.317	0.281	0.326	0.271	0.327	0.253	0.314	0.288	0.332	0.286	0.327	0.281	0.326	0.757	0.610	0.358	0.404	0.291	0.333
ETTh1	0.397	0.419	0.408	0.425	0.438	0.438	0.441	0.432	0.404	0.420	0.454	0.447	0.446	0.434	0.469	0.454	0.529	0.522	0.541	0.507	0.458	0.450
ETTh2	0.313	0.372	0.321	0.377	0.377	0.403	0.361	0.391	0.322	0.379	0.383	0.407	0.374	0.398	0.387	0.407	0.942	0.684	0.611	0.550	0.414	0.427
ECL	0.158	0.254	0.154	0.249	0.169	0.262	0.185	0.274	0.156	0.253	0.178	0.270	0.219	0.298	0.205	0.290	0.244	0.334	0.251	0.344	0.192	0.295

TABLE I: Experimental results of long-term time-series forecasting. Best results are highlighted in bold , and second-best results are underlined .

Weighted Avg.	Ours	Chimera	ModernTCN	PatchTST	TimesNet	N-HiTS	N-BEATS	ETSformer	LightTS	DLinear	FEDformer
SMAPE	11.686	11.618	11.698	11.807	11.829	11.927	11.851	14.718	13.525	13.639	12.840
MASE	1.549	1.528	1.556	1.590	1.585	1.613	1.599	2.408	2.111	2.095	1.701
OWA	0.832	0.827	0.838	0.851	0.851	0.861	0.855	1.172	1.051	1.051	0.918

TABLE II: Experimental results of short-term time-series forecasting in the M4 dataset. Best results are highlighted in bold , and second-best results are underlined .

Fig. 3: Results of classification (accuracy) and anomaly detection (F1 score).

Fig. 4: Efficiency analysis with respect to the number of variables. The left $y$ -axis reports the training time in sec/epoch, while the right $y$ -axis reports throughput in transactions per second.

Setting	sec/epoch	MAE (std)	MAPE (std)
Ours	6.1s	0.093 (0.000)	2.007 (0.000)
$+$ Permutation	6.1s	0.093 (0.000)	2.063 (0.050)
2D-SSM	23.4s	0.093 (0.000)	1.954 (0.000)
$+$ Permutation	23.4s	0.094 (0.001)	2.341 (0.130)

TABLE III: Controlled simulation for variable-ordering robustness.

Method	# C	sec/epoch	MAE	MAPE
Ours	16	6.2s	0.106	2.465
	32	6.1s	0.098	2.146
	64	6.1s	0.093	2.007
	128	6.4s	0.096	2.313
	256	6.3s	0.097	2.663
2D-SSM	16	13.4s	0.105	2.603

TABLE IV: Controlled simulation for

C

-scaling.

Setting		ETTh1		ETTm1
Setting		MSE	MAE	MSE	MAE
I	w/o short	0.433	0.440	0.349	0.381
	w/o long	0.431	0.439	0.350	0.381
	w/o freq.	0.425	0.437	0.348	0.381
II	$[0.1,0.5]$	0.406	0.426	0.349	0.380
( $\Delta_{f}$ )	$[0.01,0.05]$	0.404	0.423	0.349	0.380

TABLE V: Ablation study on ETTh1 and ETTm1. Case I : branch removal; Case II : different

\Delta_{f}

ranges in the frequency domain.

Method	Memory (MB)	FLOPs (G)
Ours	546.62	11.99
TimePro	164.35	35.04
S-Mamba	270.76	96.13
PatchTST	675.26	85.64
iTransformer	1155.14	73.20

TABLE VI: Measured peak GPU memory and forward FLOPs.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Types	Sample Number	Variable	Prediction Length
Dataset	Types	( train/validation/test)	Dimension	Prediction Length
ETTh1, ETTh2	Long-term	(8545/2881/2881)	7	{96 192, 336, 720}
ETTm1, ETTm2		(34465/11521/11521)	7	{96 192, 336, 720}
Electricity		(18317/2633/5261)	321	{96 192, 336, 720}
Traffic		(12185/1757/3509)	862	{96 192, 336, 720}
Weather		(36792/5271/10540)	21	{96 192, 336, 720}

Appendix

TABLE VII: Data description for time series forecasting.

Dataset	Types	Sample Number	Variable	Series Length ( classification)
Dataset	Types	( train/validation/test)	Dimension	Sliding Window Length ( anomaly detection)
EthanolConcentration	Classification	(261/-/263)	3	1751
FaceDetection		(5890/-/3524)	144	62
Handwriting		(150/-/850)	3	152
Heartbeat		(204/-/205)	61	405
JapaneseVowels		(270/-/370)	12	29

Appendix

TABLE VIII: Data description for time series classification and anomaly detection.

Aggregation $\psi$	Work	Parallel Span	Peak Memory	Characteristics
Aggregation $\psi$	(per timestep)	Parallel Span	Overhead	Characteristics
Mean	$\mathcal{O}(Cd)$	$\mathcal{O}(\log C)$	$\mathcal{O}(d)$	Stable, low-variance descriptor
Sum	$\mathcal{O}(Cd)$	$\mathcal{O}(\log C)$	$\mathcal{O}(d)$	Same cost as mean, without normalization
Attention	$\mathcal{O}(Cd^{2})$	$\mathcal{O}(\log C)$	$\mathcal{O}(Cd)$	Expressive, selectively emphasizes informative variables

Appendix

TABLE IX: Computational analysis of permutation-invariant aggregation functions. The trade-off between representational expressiveness and memory/compute overhead motivates task-adaptive selection of

\psi

		Ours		Chimera		TimePro		Simba		TCN		iTransformer		RLinear		PatchTST		Crossformer		TiDE		TimesNet		DLinear		SCINet		FEDformer
		MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTm1	96	0.287	0.342	0.318	0.354	0.326	0.364	0.324	0.360	0.292	0.346	0.334	0.368	0.355	0.376	0.329	0.367	0.404	0.426	0.364	0.387	0.338	0.375	0.345	0.372	0.418	0.438	0.379	0.419
	192	0.326	0.367	0.331	0.369	0.367	0.383	0.363	0.382	0.332	0.368	0.377	0.391	0.391	0.392	0.367	0.385	0.450	0.451	0.398	0.404	0.374	0.387	0.380	0.389	0.439	0.450	0.426	0.441
	336	0.359	0.387	0.363	0.389	0.402	0.409	0.395	0.405	0.365	0.391	0.426	0.420	0.424	0.415	0.399	0.410	0.532	0.515	0.428	0.425	0.410	0.411	0.413	0.413	0.490	0.485	0.445	0.459
	720	0.409	0.416	0.409	0.415	0.469	0.446	0.451	0.437	0.416	0.417	0.491	0.459	0.487	0.450	0.454	0.439	0.666	0.589	0.487	0.461	0.478	0.450	0.474	0.453	0.595	0.550	0.543	0.490
	Avg	0.345	0.378	0.355	0.381	0.391	0.400	0.383	0.396	0.351	0.381	0.407	0.410	0.414	0.407	0.387	0.400	0.513	0.496	0.419	0.419	0.400	0.406	0.403	0.407	0.485	0.481	0.448	0.452

Appendix

TABLE X: Full results of long-term time-series forecasting.

		Ours	Chimera	ModernTCN	PatchTST	TimesNet	N-HiTS	N-BEATS	ETSformer	LightTS	DLinear	FEDformer	Stationary	Autoformer	Pyraformer	Informer	Reformer
Yearly	SMAPE	13.150	13.107	13.226	13.258	13.387	13.418	13.436	18.009	14.247	16.965	13.728	13.717	13.974	15.530	14.727	16.169
	MASE	2.950	2.902	2.957	2.985	2.996	3.045	3.043	4.487	3.109	4.283	3.048	3.078	3.134	3.711	3.418	3.800
	OWA	0.773	0.767	0.777	0.781	0.786	0.793	0.794	1.115	0.827	1.058	0.803	0.807	0.822	0.942	0.881	0.973
Quarterly	SMAPE	9.949	9.892	9.971	10.179	10.100	10.202	10.124	13.376	11.364	12.145	10.792	10.958	11.338	15.449	11.360	13.313
	MASE	1.160	1.105	1.167	0.803	1.182	1.194	1.169	1.906	1.328	1.520	1.283	1.325	1.365	2.350	1.401	1.775
	OWA	0.875	0.853	0.878	0.803	0.890	0.899	0.886	1.302	1.000	1.106	0.958	0.981	1.012	1.558	1.027	1.252

Appendix

TABLE XI: Full results of short-term forecasting in the M4 dataset.

Datasets	LSTM	LSTNet	LSSL	Reformer	Informer	Pyraformer	Autoformer	Stationary	FEDformer	ETSformer	Flowformer	DLinear	LightTS	TimesNet	PatchTST	MTCN	Chimera	Ours
EthanolConcentration	32.3	39.9	31.1	31.9	31.6	30.8	31.6	32.7	31.2	28.1	33.8	32.6	29.7	35.7	32.8	36.3	39.8	35.7
FaceDetection	57.7	65.7	66.7	68.6	67.0	65.7	68.4	68.0	66.0	66.3	67.6	68.0	67.5	68.6	68.3	70.8	70.4	69.6
Handwriting	15.2	25.8	24.6	27.4	32.8	29.4	36.7	31.6	28.0	32.5	33.8	27.0	26.1	32.1	29.6	30.6	32.9	35.3
Heartbeat	72.2	77.1	72.7	77.1	80.5	75.6	74.6	73.7	73.7	71.2	77.6	75.1	75.1	78.0	74.9	77.2	81.3	78.0
JapaneseVowels	79.7	98.1	98.4	97.8	98.9	98.4	96.2	99.2	98.4	95.9	98.9	96.2	96.2	98.4	97.5	98.8	99.1	98.4
PEMS-SF	39.9	86.7	86.1	82.7	81.5	83.2	82.7	87.3	80.9	86.0	83.8	75.1	88.4	89.6	89.3	89.1	89.5	88.1

Appendix

TABLE XII: Full results of the classification task (accuracy %)

	SMD			MSL			SMAP			SWaT			PSM			Avg F1
	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	Avg F1
LSTM	78.52	65.47	71.41	78.04	86.22	81.93	91.06	57.49	70.48	78.06	91.72	84.34	69.24	99.53	81.67	77.97
Transformer	83.58	76.13	79.56	71.57	87.37	78.68	89.37	57.12	69.70	68.84	96.53	80.37	62.75	96.56	76.07	76.88
LogTrans	83.46	70.13	76.21	73.05	87.37	79.57	89.15	57.59	69.97	68.67	97.32	80.52	63.06	98.00	76.74	76.60
TCN	84.06	79.07	81.49	75.11	82.44	78.60	86.90	59.23	70.45	76.59	95.71	85.09	54.59	99.77	70.57	77.24
Reformer	82.58	69.24	75.32	85.51	83.31	84.40	90.91	57.44	70.40	72.50	96.53	82.80	59.93	95.38	73.61	77.31

Appendix

TABLE XIII: Full results of the anomaly detection task. Metrics include Precision (P), Recall (R), and F1-score.

State Space Models

State space models represent sequences through evolving latent states, enabling efficient temporal modeling, system identification, and forecasting.

See all 295 papers

Channel-Dependent State Space Model for Multivariate Time Series Forecasting
Sep 29, 2026Yu-Cheng Wu, Fan-Keng Sun, Li-Chun Lu +1Multivariate Time Series Forecasting Selective SSMs
Massachusetts Institute of Technology · National Taiwan University
SPDM: Geometry-Modulated State Space Modeling with Manifold Constraints for Time Series Forecasting
Jun 6, 2026Xingsheng Chen, Siu-Ming YiuMultivariate Time Series Forecasting Selective SSMs
School of Computing and Data Science, The University of Hong Kong, Hong Kong SAR, China
UniMamba: A Unified Spatial-Temporal Modeling Framework with State-Space and Attention Integration
Mar 6, 2026Xingsheng Chen, Xianpei Mu, Deyu Yi +6Multivariate Time Series Forecasting Long-Term Time Series Forecasting
School of Computing and Data Science, The University of Hong Kong, Hong Kong, China · School of Information Engineering, Beijing Institute of Graphic Communication, Beijing, China · Innovation Engineering College, Macau University of Science and Technology, Macau, China +1

Sep 29, 2026J/K move · Enter open · S save

Channel-Dependent State Space Model for Multivariate Time Series Forecasting

Yu-Cheng Wu, Fan-Keng Sun, Li-Chun Lu, Duane S. Boning

Massachusetts Institute of Technology · National Taiwan University

Multivariate Time Series Forecasting Selective SSMs

Open paper AskPDF arXiv

Multivariate time series forecasting (MTSF) is critical across many real-world domains. Existing deep learning approaches fall into two paradigms with distinct limitations: channel-independent (CI) methods unconditionally ignore cross-variable dependencies and model only temporal dynamics, while channel-dependent (CD) methods consider both but typically rely on architectural compromises to mitigate overfitting and computational overhead. We therefore propose Chameleon, a specialized CD state space model (SSM) that enables data-dependent, fine-grained interactions across variables while scaling linearly with their number. By connecting selective SSMs with the Kalman filter, we leverage the missing measurement update in the former for cross-variable modeling while preserving the SSM backbone for robust temporal modeling. We further identify favorable inductive biases of GatedDeltaNet for time series, adapt it as our backbone, and improve generalization through additional techniques, including a previously unexplored stochastic perturbation of reversible instance normalization. On strongly dependent ODE and PEMS datasets, Chameleon achieves the best MSE and MAE across all settings, while its CI ablation and prior CD methods incur 61-178% higher MSE on average. Across 28 standard benchmark settings, Chameleon also achieves better MSE and MAE than each baseline in at least 27 and 22 cases, respectively. Training-time and peak-memory analyses on Traffic and ETT further demonstrate competitive efficiency and favorable memory scalability across different variable counts.

Abstract

State Space Models

Explore similar work

Channel-Dependent State Space Model for Multivariate Time Series Forecasting

SPDM: Geometry-Modulated State Space Modeling with Manifold Constraints for Time Series Forecasting

UniMamba: A Unified Spatial-Temporal Modeling Framework with State-Space and Attention Integration