Hopformer: Homogeneity-Pursuit Transformer for Time Series Forecasting

Wan Zhang; Qinjie Lin; Chan Lee; Weijian Li; Han Liu; Kai Zhang

stat.MLJul 24, 2026

Hopformer: Homogeneity-Pursuit Transformer for Time Series Forecasting

Authors: Wan Zhang, Qinjie Lin, Chan Lee, Weijian Li, Han Liu, Kai Zhang

Organizations: The AMSS Center of Forecasting Science, Chinese Academy of Sciences, Beijing, China · Department of Computer Science, Northwestern University, Evanston, IL · Department of Statistics and Data Science, Northwestern University, Evanston, IL · Department of Statistics and Operations Research, University of North Carolina, Chapel Hill, NC

Abstract

Forecasting multiple time-series with high-dimensional covariates presents a core challenge: unifying common temporal patterns while retaining meaningful series-specific information. We introduce Hopformer (Homogeneity-Pursuit Transformer), a two-stage framework that addresses this challenge. In the first stage, we perform a Sparsity Pattern Aggregation (SPA) scheme extracting a common low-variance trend that incorporates the covariates. This acts as a homogenization layer. In the second stage, a LoRA-fine-tuned Transformer models the remaining complex dependencies in the residual. Our method is theoretically grounded. We prove that SPA achieves a near-optimal bias-variance trade-off via an oracle inequality. We also provide generalization bounds for the second stage under dependent time series data. Hopformer sets a new state of the art, improving MASE by an average of 6.56% across synthetic and real-world forecasting benchmarks.

Explore similar work

UniMamba: A Unified Spatial-Temporal Modeling Framework with State-Space and Attention Integration
Mar 6, 2026Xingsheng Chen, Xianpei Mu, Deyu Yi +6Multivariate Time Series Forecasting Time Series
SETTer: Sparse-Encoder Transformer for Long-term Multivariate Time Series Forecasting
Sep 17, 2026Abraham Ezema, Chijioke Eze, Ferdinanda Ponci +1Multivariate Time Series Forecasting Transformer Architectures
TiRex-2: Generalizing TiRex to Multivariate Data and Streaming
Jul 1, 2026Patrick Podest, Marco Pichler, Elias Bürger +7Time Series Foundation Models Multivariate Forecasting

Mar 6, 2026 · cs.LGJ/K move · Enter open · S save

UniMamba: A Unified Spatial-Temporal Modeling Framework with State-Space and Attention Integration

Xingsheng Chen, Xianpei Mu, Deyu Yi, Yilin Yuan+5

School of Computing and Data Science, The University of Hong Kong, Hong Kong, China · School of Information Engineering, Beijing Institute of Graphic Communication, Beijing, China · Innovation Engineering College, Macau University of Science and Technology, Macau, China · Department of Computer Science and Technology, University of Cambridge, Cambridge, UK

Multivariate time series forecasting is fundamental to numerous domains such as energy, finance, and environmental monitoring, where complex temporal dependencies and cross-variable interactions pose enduring challenges. Existing Transformer-based methods capture temporal correlations through attention mechanisms but suffer from quadratic computational cost, while state-space models like Mamba achieve efficient long-context modeling yet lack explicit temporal pattern recognition. Therefore we introduce UniMamba, a unified spatial-temporal forecasting framework that integrates efficient state-space dynamics with attention-based dependency learning. UniMamba employs a Mamba Variate-Channel Encoding Layer enhanced with FFT-Laplace Transform and TCN to capture global temporal dependencies, and a Spatial Temporal Attention Layer to jointly model inter-variate correlations and temporal evolution. A Feedforward Temporal Dynamics Layer further fuses continuous and discrete contexts for accurate forecasting. Comprehensive experiments on eight public benchmark datasets demonstrate that UniMamba consistently outperforms state-of-the-art forecasting models in both forecasting accuracy and computational efficiency, establishing a scalable and robust solution for long-sequence multivariate time-series prediction.

Multivariate Time Series Forecasting Time Series Spatiotemporal

Open paper Ask about it PDF

Abstract

Explore similar work

UniMamba: A Unified Spatial-Temporal Modeling Framework with State-Space and Attention Integration

SETTer: Sparse-Encoder Transformer for Long-term Multivariate Time Series Forecasting

TiRex-2: Generalizing TiRex to Multivariate Data and Streaming