Overview

Time Series Transformers apply attention to ordered observations, helping model patterns over time and relationships among multiple variables for forecasting, classification, and anomaly detection. Common approaches use patches, separate temporal and channel-wise attention, or add frequency, spatial, and domain-specific structure; pretrained foundation models also aim to transfer across datasets and tasks. Work increasingly addresses long and irregular sequences, uncertainty, and efficient inference through sparse attention, adaptive tokenization, and hybrid architectures. A central tension is that added complexity does not guarantee better accuracy or generalization: performance depends on the data, forecast horizon, preprocessing, and evaluation, and simple statistical or smaller neural models can remain competitive.

Latest papers

151 papers11 papers in the last four weeks, down 21% on the four weeks before. 0.1% of all new papers.

Dec 16, 2024

Embedded Decomposed Transformer for explainable multivariate time series forecasting

Time series forecasting is an important problem with diverse real-world applications. This paper introduces the Embedded Decomposed Transformer (EDformer) for multivariate time series forecasting. EDformer decomposes the input signal into seasonal and trend components using two complementary strategies: time-domain Average Pooling (AP) based on moving-average smoothing and Frequency-Aware (FA) decomposition based on low-pass filtering in the Fourier domain. The reconstructed multivariate seasonal component is embedded across inverted dimensions, after which attention captures cross-variate correlations and a feed-forward network learns nonlinear representations for each variable. The trend component is forecast through projection and combined with the seasonal prediction to produce the final forecast. We evaluate both decomposition variants on several real-world datasets, showing that EDformer achieves competitive performance, while their relative effectiveness varies across datasets, experimental settings, and temporal characteristics. We further conduct an ablation study isolating the contribution of decomposition, inverted embedding, and the frequency cutoff ratio; a hyperparameter sensitivity analysis over learning rate and batch size; and a statistical significance analysis using Friedman and Wilcoxon signed-rank tests to substantiate the reported performance differences. Finally, model explainability techniques are explored to improve interpretability and provide insights into the features and time steps influencing the forecasting results.