STeP-TS: Shared Temporal Filters with Pruning for Time Series

Hanne Raum; Jasper Hoffmann; Joschka Boedecker

cs.LGOct 26, 2026

STeP-TS: Shared Temporal Filters with Pruning for Time Series

Authors: Hanne Raum, Jasper Hoffmann, Joschka Boedecker

Organizations: Department of Computer Science Albert-Ludwigs-Universität Freiburg · Department of Computer Science and BrainLinks-BrainTools Albert-Ludwigs-Universität Freiburg

Abstract

Multivariate time series forecasting requires capturing temporal dependencies across multiple scales while accounting for both shared and variable-specific dynamics. Compact forecasting models have emerged as an attractive alternative to increasingly large architectures, but preserving strong predictive performance within tight computational and memory budgets remains challenging. We propose a compact one-layer convolutional framework that learns a shared bank of multi-scale temporal filters of each variable. Lightweight variable-specific scalar weights determine the contribution of each temporal component, enabling the model to capture shared temporal structure while adapting to variable-specific dynamics. The learned weights and filter-output energy further provide a natural signal for structured pruning of convolutional filters, reducing computational complexity with minimal degradation in forecasting performance. We additionally introduce an amplitude-adaptive normalization mechanism that preserves variable-dependent temporal amplitudes. Experiments on multiple datasets demonstrate forecasting performance competitive with state-of-the-art methods while requiring substantially fewer parameters and inference FLOPs and achieving low per-sample latency.

Figures & tables

Figure 1: Accuracy against cost on Weather (horizon = 336). STeP-TS matches the strongest baselines in test MSE (0.235 vs 0.236 / 0.238) while using 3.1× fewer parameters and 3.0× fewer inference FLOPs than xLSTMMixer, and 10.2× / 27.3× fewer than ModernTCN. Each model is measured at its own published configuration, so lookback differs per model (given in the legend). PatchTST uses default hyperparameters, as additional reference Chronos2 ( Ansari et al., 2025b ) (zero-shot) is added

Figure 2: Overview of STeP-TS. The input features are first reshaped into the batch dimension and processed by 1D convolutional filters with different kernel sizes along the sequence length, learning general temporal patterns across the dataset. The resulting representations are reshaped into

\text{batch}\times\text{features}\times\text{channels}

, where channels are individually weighted for each feature and then aggregated. The resulting

\text{batch}\times\text{features}\times\text{sequence}

representation is finally passed to the forecasting head to generate the prediction.

Model Type	Convolutional				RNN				MLP						SSM		Transformer
Models	STeP-TS (ours)		ModTCN 2024		xLSTMix. 2025		xLSTMTime 2024		TimeMix 2024		DLinear 2023		TiDE 2024		S-Mamba 2025		PatchTST 2022		Timer-XL 2025
Dataset	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
Weather	0.216	0.246	0.224	0.264	0.219	0.250	0.222	0.255	0.222	0.262	0.246	0.300	0.236	0.282	0.251	0.276	0.241	0.264	0.256	0.294
ETTh1	0.407	0.420	0.404	0.420	0.397	0.420	0.408	0.428	0.411	0.423	0.423	0.437	0.419	0.430	0.455	0.450	0.458	0.450	0.404	0.417
ETTm1	0.353	0.369	0.351	0.381	0.339	0.366	0.347	0.372	0.348	0.376	0.357	0.379	0.355	0.378	0.398	0.405	0.353	0.382	0.373	0.392
ETTm2	0.249	0.306	0.253	0.314	0.248	0.307	0.254	0.310	0.256	0.316	0.267	0.332	0.249	0.312	0.288	0.332	0.256	0.317	0.273	0.336

Table 1: Average ( Avg. ) results across four prediction horizons 96, 192, 336, 720. Lower MSE and MAE indicate better performance. Best results are highlighted in bold red, while second-best results are shown in blue and underlined. All results are averaged over three random seeds (2021, 2022, 2023); average standard deviation of 0.001 for both MSE and MAE.

Figure 3: Computational cost of each model on Weather at

H=336

. All models are evaluated at the same three lookback windows. Left: parameter count. Centre: per-sample latency, measured as the batch-32 inference time amortised over the 32 series. Right: wall-clock inference time over the full Weather test split at batch size 32. STeP-TS is evaluated at both ends of its head range—the linear head and the published three-head configuration—as well as with a pruned variant that removes three of its eight scale channels (one low-, one mid-, and one high-scale channel) while otherwise remaining identical to the three-head model. All measurements were performed on a single RTX 3080 Ti with a batch size of 32. PatchTST is evaluated its default hyperparameters.

Weather	96				192				336				720				Avg
	-adRevIN		+adRevIN		-adRevIN		+adRevIN		-adRevIN		+adRevIN		-adRevIN		+adRevIN		-adRevIN		+adRevIN
	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
With Ensemble	0.142	0.181	0.142	0.181	0.184	0.223	0.183	0.223	0.235	0.265	0.235	0.264	0.309	0.320	0.310	0.320	0.218	0.247	0.218	0.247
W/o Ensemble	0.145	0.187	0.145	0.189	0.188	0.229	0.188	0.229	0.238	0.268	0.240	0.271	0.312	0.323	0.314	0.324	0.221	0.252	0.222	0.253
ETTm1	96				192				336				720				Avg
	-adRevIN		+adRevIN		-adRevIN		+adRevIN		-adRevIN		+adRevIN		-adRevIN		+adRevIN		-adRevIN		+adRevIN

Table 2: Ablation study of the ensemble forecasting head and adRevIN across different forecasting horizons with a fixed lookback window of 520. Results are averaged across three seeds (2021, 2022, 2023); with average standard deviation of 0.001 for both MSE and MAE.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Source	Domain	Horizons	Sampling	Variates
Weather	[ Zhou et al., 2021 ]	Weather	96–720	10 min	21
Electricity	[ Zhou et al., 2021 ]	Power Usage	96–720	1 hour	321
Traffic	[ Wu et al., 2021 ]	Traffic Load	96–720	1 hour	862
ETT	[ Zhou et al., 2021 ]	Power Production	96–720	15& 60 min	7

Appendix

Table 3: The long-term forecasting benchmark datasets

Hyperparameter	Default search space	Dataset-specific space
Learning rate ( $lr$ )	$[10^{-4},5\times 10^{-3}]$	—
Forecasting heads	$\{1,2,3\}$	—
Amplitude scale ( $s$ )	$[0,0.5]$	—
Sequence length	$[520,720]$	—
Fixed parameters
Batch size	$32$	—

Appendix

Table 4: Hyperparameter search spaces used for HPO. Deviations are indicated explicitly.

Dataset	Horizon	STeP-TS		ModTCN		xLSTM-		xLSTMTime		TimeMix		TSMixer		CycleNet		DLinear		TiDE		S-Mamba		PatchTST		TimesNet		Timer-XL		MoiraiBase
		MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
Weather	96	0.142	0.181	0.149	0.200	0.143	0.184	0.144	0.187	0.147	0.197	0.145	0.198	0.148	0.200	0.176	0.237	0.166	0.222	0.165	0.210	0.149	0.198	0.172	0.220	0.171	0.225	0.220	0.217
	192	0.183	0.222	0.196	0.245	0.186	0.226	0.192	0.236	0.189	0.239	0.191	0.242	0.190	0.240	0.220	0.282	0.209	0.263	0.214	0.252	0.194	0.241	0.219	0.261	0.221	0.271	0.271	0.259
	336	0.232	0.264	0.238	0.277	0.236	0.266	0.237	0.272	0.241	0.280	0.242	0.280	0.242	0.283	0.265	0.319	0.254	0.301	0.274	0.297	0.306	0.282	0.280	0.306	0.274	0.311	0.286	0.297
	720	0.306	0.319	0.314	0.334	0.310	0.323	0.313	0.326	0.310	0.330	0.320	0.336	0.312	0.333	0.323	0.362	0.313	0.340	0.350	0.345	0.314	0.334	0.365	0.359	0.356	0.370	0.373	0.354
	Avg	0.216	0.247	0.224	0.264	0.219	0.250	0.222	0.255	0.222	0.262	0.225	0.264	0.223	0.264	0.246	0.300	0.236	0.282	0.251	0.276	0.241	0.264	0.259	0.287	0.256	0.294	0.288	0.282

Appendix

Table 5: Full multivariate long-term forecasting results across all datasets and prediction horizons. Lower is better for both MSE and MAE. The best result in each row is shown in bold red , the second-best is underlined blue . A dash (-) indicates the model was not evaluated on that dataset. A thin rule sets each dataset’s Avg row apart from its horizon rows.

Model	MSE best	MSE 2nd	MAE best	MAE 2nd	Best (total)	2nd (total)
xLSTMix.	15	9	15	10	30	19
STeP-TS	6	3	10	12	16	15
ModTCN	4	4	4	1	8	5
TiDE	6	1	1	1	7	2
TimeMix	3	7	2	2	5	9
Timer-XL	0	1	2	1	2	2

Appendix

Table 6: Number of times each model achieves the best ( bold red ) or second-best ( underlined blue ) result in Table 5 , broken down by metric. Counted over all 35 (dataset, horizon) rows, including the per-dataset averages. “Best (total)” sums the MSE and MAE best-counts; “2nd (total)” sums the second-best counts.

Criterion	Avg. Active Channels	FLOPs	Latency	Speedup	$\Delta$ MSE%	$\Delta$ MAE%
	(of 8)	(rel.)	[ $\mu$ s/sample]
Dense	8.00	1.00	49.0	1.00	—	—
with $A_{v,c}$	5.00	0.63	42.1	1.20	+0.03	-0.01
w/o $A_{v,c}$	5.97	0.75	45.4	1.08	0.00	0.00

Appendix

Table 7: Cost–accuracy trade-off of the two channel-pruning criteria, averaged over 20 dataset/horizon pairs. Dense is the same with pruning disabled, trained from the same config, seeds, code and GPU in the same job and scored on the same epoch window. Latency is per sample at batch 32 on an L40S.

			Pruned during training			Retrained from scratch
Removed	Cells	Combos	$A_{v,c}$	Random	Random $-$ $A_{v,c}$	$A_{v,c}$	Random	Random $-$ $A_{v,c}$
0	4	4	$-0.03$	$-0.01$	$+0.02\pm 0.02$	$-0.03$	$-0.01$	$+0.02\pm 0.02$
1	5	4	$+0.01$	$+0.02$	$+0.01\pm 0.09$	$+0.04$	$-0.15$	$-0.19\pm 0.26$
2	15	11	$-0.07$	$-0.15$	$-0.08\pm 0.10$	$+0.45$	$+0.05$	$-0.40\pm 0.14$
3	13	11	$+0.06$	$+0.11$	$+0.05\pm 0.03$	$+0.44$	$+0.46$	$+0.02\pm 0.19$
4	12	10	$-0.14$	$-0.06$	$+0.08\pm 0.09$	$-0.32$	$-0.07$	$+0.25\pm 0.40$

Appendix

Table 8:

A_{v,c}

versus random channel pruning, grouped by the number of scale channels removed per seed. Entries are the change in test MSE relative to the dense control of the same seed, in percent; the difference column is random minus energy in percentage points (

\pm

one standard error over cells). Pruned is the model during iterative training and scored after the last removal; retrained is a fresh model trained from scratch with the final channel set fixed from epoch 0. Random pruning is averaged over three independent draws. Cells are (dataset, horizon, seed) triples over five datasets, so cells of one combination are not independent.

		Dense			with $A_{v,c}$						w/o $A_{v,c}$
Dataset	$H$	MSE	MAE	Lat.	Chan.	FLOPs	Lat.	Spd.	$\Delta$ MSE%	$\Delta$ MAE%	Chan.	FLOPs	Lat.	Spd.	$\Delta$ MSE%	$\Delta$ MAE%
				[ $\mu$ s]	(of 8)	(rel.)	[ $\mu$ s]				(of 8)	(rel.)	[ $\mu$ s]
ETTh1	96	0.367	0.390	49.3	7.00	0.87	49.2	1.00	+0.07	-0.09	7.33	0.91	49.8	0.97	+0.10	-0.01
	192	0.394	0.408	48.3	6.67	0.83	46.5	1.04	+0.24	+0.03	7.33	0.93	47.1	1.01	+0.53	+0.24
	336	0.427	0.427	48.5	7.00	0.85	48.2	1.01	+0.06	+0.09	6.67	0.82	47.0	1.03	+0.26	+0.13
	720	0.437	0.457	48.1	7.33	0.90	49.6	0.98	-0.09	-0.03	6.67	0.80	45.2	1.03	0.00	+0.06

Appendix

Table 9: Per-horizon pruning results. Chan. is the mean number of surviving scale channels of 8; FLOPs, latency and speedup are relative to that run’s own dense pass.

\Delta

is the change against dense on the same three seeds; bold marks the MSE changes that clear a paired two-sided

t

-test (

df=2

\alpha=0.05

) — every other

\Delta

lies inside the seed spread and should be read as no measurable change. The avg line under each dataset averages its four horizons.

$L$	Model	Params	FLOPs	Latency	Throughput	Train step	Mem ${}_{\text{inf}}$	Mem ${}_{\text{train}}$
		[K]	[M]	[ $\mu$ s/sample]	[samples/s]	[ms]	[MB]	[B]
336	STeP-TS, 8 ch, linear head	128.9	9.1	61.5	16,260	8.5	9.7	51.4
	STeP-TS, 8 ch, 1 head	128.9	9.1	71.9	13,916	8.5	17.9	51.4
	STeP-TS, 8 ch, 2 heads	242.2	13.9	73.1	13,685	8.6	18.3	53.1
	STeP-TS, 8 ch, 3 heads †	355.4	18.6	73.5	13,608	8.8	18.8	54.9
	STeP-TS, 5 ch, 3 heads (pruned)	355.1	17.0	55.0	18,193	6.3	18.8	43.2

Appendix

Table 10: Computational cost on Weather at (H=336), measured at three matched lookbacks on one RTX 3080 Ti with batch size 32. Parameters are exact counts; FLOPs are thop estimates from a batch-1 pass. Latency, throughput, training-step time, and peak memory are averaged over repeated runs after warm-up. STeP-TS uses the standard HPO configuration; pruning removes 3 of 8 scale channels on average, with all other settings unchanged. ( ‡ )PatchTST uses default hyperparameters. Batch size is fixed at 32 for comparability, so peak training memory may differ from the baseline authors’ settings.

Multivariate Time Series Forecasting

Multivariate time series forecasting predicts future values across related variables by learning their temporal and cross-variable dependencies.

See all 302 papers

UniMamba: A Unified Spatial-Temporal Modeling Framework with State-Space and Attention Integration
Mar 6, 2026Xingsheng Chen, Xianpei Mu, Deyu Yi +6Multivariate Time Series Forecasting Long-Term Time Series Forecasting
School of Computing and Data Science, The University of Hong Kong, Hong Kong, China · School of Information Engineering, Beijing Institute of Graphic Communication, Beijing, China · Innovation Engineering College, Macau University of Science and Technology, Macau, China +1
CVTA: Cross-Variable Temporal Attention for Multivariate Irregular Time Series Prediction
Oct 18, 2026Ankitkumar Joshi, Milos HauskrechtMultivariate Time Series Forecasting Time Series Forecasting
Department of Computer Science, University of Pittsburgh, Pittsburgh, PA 15260
Aurora-X: Built for Extreme Time Series Forecasting
Sep 25, 2026Xingjian Wu, Chenjuan Guo, Xiangfei Qiu +6Time Series Forecasting Time-Series Foundation Models
East China Normal University

Mar 6, 2026J/K move · Enter open · S save

UniMamba: A Unified Spatial-Temporal Modeling Framework with State-Space and Attention Integration

Xingsheng Chen, Xianpei Mu, Deyu Yi, Yilin Yuan+5

School of Computing and Data Science, The University of Hong Kong, Hong Kong, China · School of Information Engineering, Beijing Institute of Graphic Communication, Beijing, China · Innovation Engineering College, Macau University of Science and Technology, Macau, China+1

Multivariate Time Series Forecasting Long-Term Time Series Forecasting Selective SSMs Spatiotemporal Forecasting

Open paper AskPDF arXiv

Multivariate time series forecasting is fundamental to numerous domains such as energy, finance, and environmental monitoring, where complex temporal dependencies and cross-variable interactions pose enduring challenges. Existing Transformer-based methods capture temporal correlations through attention mechanisms but suffer from quadratic computational cost, while state-space models like Mamba achieve efficient long-context modeling yet lack explicit temporal pattern recognition. Therefore we introduce UniMamba, a unified spatial-temporal forecasting framework that integrates efficient state-space dynamics with attention-based dependency learning. UniMamba employs a Mamba Variate-Channel Encoding Layer enhanced with FFT-Laplace Transform and TCN to capture global temporal dependencies, and a Spatial Temporal Attention Layer to jointly model inter-variate correlations and temporal evolution. A Feedforward Temporal Dynamics Layer further fuses continuous and discrete contexts for accurate forecasting. Comprehensive experiments on eight public benchmark datasets demonstrate that UniMamba consistently outperforms state-of-the-art forecasting models in both forecasting accuracy and computational efficiency, establishing a scalable and robust solution for long-sequence multivariate time-series prediction.

Abstract

Multivariate Time Series Forecasting

Explore similar work

UniMamba: A Unified Spatial-Temporal Modeling Framework with State-Space and Attention Integration

CVTA: Cross-Variable Temporal Attention for Multivariate Irregular Time Series Prediction

Aurora-X: Built for Extreme Time Series Forecasting