cs.LGDec 16, 2024

Embedded Decomposed Transformer for explainable multivariate time series forecasting

Authors: Sanjay Chakraborty, Ibrahim Delibasoglu, Fredrik Heintz

Organizations: Department of Computer and Information Science (IDA), REAL, AIICS, Linköping University, Linköping, Sweden

Abstract

Time series forecasting is an important problem with diverse real-world applications. This paper introduces the Embedded Decomposed Transformer (EDformer) for multivariate time series forecasting. EDformer decomposes the input signal into seasonal and trend components using two complementary strategies: time-domain Average Pooling (AP) based on moving-average smoothing and Frequency-Aware (FA) decomposition based on low-pass filtering in the Fourier domain. The reconstructed multivariate seasonal component is embedded across inverted dimensions, after which attention captures cross-variate correlations and a feed-forward network learns nonlinear representations for each variable. The trend component is forecast through projection and combined with the seasonal prediction to produce the final forecast. We evaluate both decomposition variants on several real-world datasets, showing that EDformer achieves competitive performance, while their relative effectiveness varies across datasets, experimental settings, and temporal characteristics. We further conduct an ablation study isolating the contribution of decomposition, inverted embedding, and the frequency cutoff ratio; a hyperparameter sensitivity analysis over learning rate and batch size; and a statistical significance analysis using Friedman and Wilcoxon signed-rank tests to substantiate the reported performance differences. Finally, model explainability techniques are explored to improve interpretability and provide insights into the features and time steps influencing the forecasting results.

Figures & tables

Explore similar work

Oct 27, 2026

RankFormer: Rethinking Time Series Transformers through Representation Capacity Analysis and Progressive Decoding

Multivariate time series forecasting plays a critical role in Web and cyber-physical systems, where large-scale services and IoT infrastructures generate complex temporal data. Transformer-based models have achieved strong empirical performance in this domain, yet the principles governing the expressive capacity of their learned representations remain insufficiently understood. Moreover, many existing models rely on simplified decoding strategies, including channel-independent modeling and direct projection heads, which limit their ability to capture complex multivariate dynamics. To address these challenges, we propose RankFormer, a novel encoder-decoder architecture for multivariate time series forecasting. We establish a rank-based analysis framework to investigate how tokenization, embedding, and attention designs affect the representation capacity of time series Transformers. Our analysis reveals that patchification increases the potential representation capacity by aggregating multiple temporal observations into each token, while subsequent encoding operations determine how much of this capacity is realized. Guided by these insights, we develop a rank-enhancing encoder with Hourglass Attention to improve rank stability during information interaction. Furthermore, we introduce the Interactive Progressive Decoder (IPD), which improves future trajectory modeling through global-guided cross-channel interaction and progressive refinement. Extensive experiments on real-world long-term forecasting benchmarks demonstrate that RankFormer achieves competitive performance against state-of-the-art methods. Code repository: https://github.com/TangPeiwang/Rankformer.
Sep 17, 2026

SETTer: Sparse-Encoder Transformer for Long-term Multivariate Time Series Forecasting

Long-term multivariate time series plays a significant role in many application areas such as power systems, trading, etc. However, their accurate prediction is quite difficult for conventional forecasting methods as they often exhibit high dimensionality and complex relationships. Recent works show that transformer-based approaches are quite effective for long-term forecasting thanks to their attention mechanism. However, in the presence of complex high-dimensional inputs, they show evidence of oversmoothing, limited capacity, and opacity. To this end, this paper introduces SETTer, a transformer-based model that addresses these challenges by incorporating novel techniques for decoupled self-attention and hybrid masking. The proposed techniques enable SETTer to effectively capture the dominant short- and long-term patterns across the temporal and channel dimensions. In addition, we enrich the model layers with simple explainable structures that indicate the discriminative pattern of SETTer. We show that with a single-layer transformer architecture, SETTer can effectively model long-term dependencies in the presence of varying data complexities. Extensive experiments on real-word benchmark datasets for long-term multivariate time series forecasting demonstrate that SETTer outperforms state-of-the-art models in 88% of the scenarios.
Jul 24, 2026

Hopformer: Homogeneity-Pursuit Transformer for Time Series Forecasting

Forecasting multiple time-series with high-dimensional covariates presents a core challenge: unifying common temporal patterns while retaining meaningful series-specific information. We introduce Hopformer (Homogeneity-Pursuit Transformer), a two-stage framework that addresses this challenge. In the first stage, we perform a Sparsity Pattern Aggregation (SPA) scheme extracting a common low-variance trend that incorporates the covariates. This acts as a homogenization layer. In the second stage, a LoRA-fine-tuned Transformer models the remaining complex dependencies in the residual. Our method is theoretically grounded. We prove that SPA achieves a near-optimal bias-variance trade-off via an oracle inequality. We also provide generalization bounds for the second stage under dependent time series data. Hopformer sets a new state of the art, improving MASE by an average of 6.56% across synthetic and real-world forecasting benchmarks.