Time Series Transformers

Follow this topic Ask about this topic

Time Series Transformers apply attention to ordered observations, helping model patterns over time and relationships among multiple variables for forecasting, classification, and anomaly detection. Common approaches use patches, separate temporal and channel-wise attention, or add frequency, spatial, and domain-specific structure; pretrained foundation models also aim to transfer across datasets and tasks. Work increasingly addresses long and irregular sequences, uncertainty, and efficient inference through sparse attention, adaptive tokenization, and hybrid architectures. A central tension is that added complexity does not guarantee better accuracy or generalization: performance depends on the data, forecast horizon, preprocessing, and evaluation, and simple statistical or smaller neural models can remain competitive.

Latest papers

139 papers12 papers in the last four weeks, up 9% on the four weeks before. 0.1% of all new papers.

RankFormer: Rethinking Time Series Transformers through Representation Capacity Analysis and Progressive Decoding
Oct 27, 2026Peiwang Tang, Cheng HE, Xijie Liang +1Multivariate Time Series Forecasting Time Series Forecasting
University of Science and Technology of China, Hefei, Anhui, China · State Key Laboratory of Cognitive Intelligence, Hefei, Anhui, China · Shanghai Black Wing Asset Management, Shanghai, China
Learning from Points and Patches: A Pre-trained Hybrid Attention Transformer for Time Series Analysis
Oct 27, 2026Peiwang Tang, Cheng HE, Yupeng Ye +3Self-Supervised Time Series Representation Learning Time Series Transformers
University of Science and Technology of China, Hefei, Anhui, China · State Key Laboratory of Cognitive Intelligence, Hefei, Anhui, China · Shanghai Black Wing Asset Management, Shanghai, China +1
LEAP: a Latent Embedding Anomaly Patch-Transformer for time-series anomaly detection
Oct 26, 2026Cristhian Moya-Mota, Ignacio Aguilera-Martos, Diego García-Gil +1Time-Series Anomaly Detection Unsupervised Anomaly Detection
Department of Computer Science and Artificial Intelligence, University of Granada, Granada, Spain. · Andalusian Institute of Data Science and Computational Intelligence (DaSCI), University of Granada, Granada, Spain. · Department of Software Engineering, University of Granada, Granada, Spain
Extreme Low-State Quantized Transformer for Electricity-Load Forecasting
Oct 20, 2026Muhammad Sabbir Alam, Harindra Sandun Mavikumbure, Victor Cobilean +2Low-Bit Quantization Quantization-Aware Training
Department of Mechanical and Nuclear Engineering, Virginia Commonwealth University, Richmond, VA, 23284, USA · Department of Computer Science, Virginia Commonwealth University, Richmond, VA, 23284, USA · Department of Electrical and Computer Engineering, Virginia Commonwealth University, Richmond, VA, 23284, USA
Beyond Static Positional Encoding: Bayesian Time-aware Attention for Non-Stationary Long-term Time Series Forecasting
Oct 14, 2026Xinjie Lan, Yunbo Mei, Leheng Zhou +1Long-Term Time Series Forecasting Non-Stationary Time Series Forecasting
Rice University · University of Hong Kong · University of North Carolina at Charlotte
Cross-Market Meta-Learning for Electricity Price Forecasting
Oct 12, 2026Delaram Pirhayatifard, Terrence AlsupElectricity Price Forecasting Time Series Forecasting
Rice University, Houston, TX · Morgan Stanley, New York, NY
Probabilistic Ship Trajectory Forecasting using Transformer Models on AIS Data
Oct 9, 2026Frederik E. T. SchöllerVessel Trajectory Prediction Probabilistic Forecasting
Automation and Control Group, Department of Electrical Engineering, Technical University of Denmark, Kgs. Lyngby, Denmark · Technical University of Denmark
Timer-M1: A Multivariate Time Series Foundation Model via Learning Primitives
Oct 8, 2026Haoran Zhang, Haixuan Liu, Xingjian Su +5Multivariate Time Series Forecasting Time-Series Foundation Models
School of Software, BNRist, Tsinghua University · ByteDance, Beijing, China
DSTNet: Dynamic Spectral Trajectory Network for Causal Multi-Horizon Financial Forecasting
Oct 7, 2026Aashish Bohra, Lokendra VishwakarmFinancial Forecasting Time Series Forecasting
Department of Computer Science and Engineering, Indian Institute of Technology Jodhpur, Jodhpur, Rajasthan 342030, India · School of Computing and Data Sciences, FLAME University, Pune Maharashtra, India
Transformer-Based Time-Series Inference of Lindblad Dynamics in Open Quantum Systems
Oct 5, 2026Julian Guam, Jianqing LiuQuantum System Identification System Identification
Department of Computer Science, North Carolina State University, Raleigh, NC
STCFormer: Adaptive Spatio-Temporal Modeling with Dynamic Cluster Transformer for Station-based Weather Forecasting
Sep 30, 2026Rongwen Li, Haixin Xie, Mingyang Wang +5Multivariate Time Series Forecasting Clustering
Hunan University · China Meteorological Administration · Hunan Provincial Meteorological Bureau
Association profile conditioning in a set-temporal transformer for cross-session intracortical motor decoding
Sep 30, 2026Xinyuan Zhang, Handong Mo, Pengfei Wen +5Set Transformer Transformer
The University of Hong Kong, Hong Kong SAR, China · Southern University of Science and Technology, Shenzhen, China
RAE-PPG: Duration-Grounded Retain-and-Extend Pretraining for PPG Foundation Models
Sep 29, 2026Suyeong Lee, Hochang Lee, Seokyong Sheem +1Self-Supervised Pre-Training Self-Supervised Time Series Representation Learning
Korea University
GNA: Granular Neighbor Assembly for Retrieval-Augmented Multivariate Time-Series Forecasting
Sep 28, 2026Vincent UhseMultivariate Time Series Forecasting Time Series Forecasting
CERN
Universality and Generalization of Causal Transformers Across Context Lengths
Sep 28, 2026Takashi Furuya, Maarten V. de Hoop, Gabriel PeyréNeural Network Generalization Transformer Expressivity
Doshisha University, RIKEN AIP · Rice University · CNRS, ENS, PSL Université
Progressive Memory Transformer: Memory-Aware Attention for Time-Series
Sep 25, 2026Tord Sture Stangeland, Andreas Köhler, Steffen Mæland +1Self-Supervised Time Series Representation Learning Temporal Representation Learning
University of Oslo, Department of Informatics, Oslo, Norway · NORSAR, Lillestrøm, Norway · University of Tromsø, Department of Geosciences, Tromsø, Norway +1
AI-based detection of worsening heart failure from low-resolution telemonitoring data
Sep 24, 2026Erik Aerts, Yinan Yu, Annika Rosengren +5Remote Patient Monitoring Time Series Transformers
Department of Computer Science and Engineering, Chalmers Universiy of Technology, Gothenburg, Sweden · Department of Computer Science and Engineering, University of Gothenburg, Gothenburg, Sweden · Center for Digital Health, Sahlgrenska University Hospital, Gothenburg, Sweden +3
Evaluating Accuracy and Probabilistic Reliability of Zero-Shot Time Series Foundation Models
Sep 22, 2026Panagiotis Michael, Moysis Symeonides, Demetris TrihinasZero-Shot Time Series Forecasting Time Series Transformers
Department of Computer Science, University of Nicosia, Cyprus · Department of Computer Science, University of Cyprus, Cyprus
Benchmarking Hybrid Deep Learning Architectures for Predictive Maintenance in Industry 4.0
Sep 18, 2026Zhengyang, Gu, Joseph E. Hernandez +4Hybrid Transformer Architectures Predictive Maintenance
Cissy · Data Science, Liveline Technologies Livonia, MI, USA · Engineering, Liveline Technologies Livonia, MI, USA
A Lightweight Plug-in Gate for Transformer-Based Time-Series Forecasters
Sep 17, 2026Hongkai Zhuang, Tao Huang, Chen HouTime Series Forecasting Time Series Transformers
School of Computer and Big Data, Minjiang University, No. 200 Xiyuangong Road, Fuzhou, 350108, Fujian, China.
SETTer: Sparse-Encoder Transformer for Long-term Multivariate Time Series Forecasting
Sep 17, 2026Abraham Ezema, Chijioke Eze, Ferdinanda Ponci +1Multivariate Time Series Forecasting Long-Term Time Series Forecasting
Institute for Automation of Complex Power Systems, RWTH Aachen University, 52064 Aachen, Germany · RWTH Aachen University, 52064 Aachen, Germany · institute for Monitoring and Distributed Control for Power Systems Teaching and Research Area, RWTH Aachen University, 52064 Aachen, Germany +1
On the role of the tokenizer in ECG transformer models
Sep 14, 2026Jiawei Li, Fabio Bonassi, Johan Sundström +2Electrocardiogram Classification Time Series Transformers
Uppsala University, Uppsala, Sweden · Scilifelab, Uppsala, Sweden
Tabby: An Open Pretraining Recipe for Time Series Foundation Models
Sep 12, 2026Shifeng Xie, Bahaeddine Abdessalem, Zehao Xiao +9Time-Series Foundation Models Time Series Transformers
Huawei Noah’s Ark Lab, Paris, France · LIPADE, Université Paris Cité, Paris, France · École Polytechnique, France +3
TempTPI: Informer-Based trajectory prediction for maritime vessels
Sep 9, 2026Kevin Ferneding, Veronika Lietavcova, Aleksandra M. Blachowiak +1Vessel Trajectory Prediction Maritime Transportation
Technical University of Denmark Anker Engelunds Vej 101 2800 Kongens Lyngby, Denmark
Seasonality-Aware Hybrid Convolutional Transformer for Antarctic Sea Ice Concentration Forecasting
Aug 31, 2026Danyang Li, John Taylor, Thang Bui +1Hybrid CNN-Transformer Architectures Time Series Forecasting
School of Computer Science, Australian National University, Canberra, ACT 2601, Australia · Yau Mathematical Sciences Center, Tsinghua University, Beijing, China
Beat-Synchronous Tokenization for ECG Transformers
Aug 31, 2026Ahmed Sameh, Nolan Wilson, Max Enderlein +1Electrocardiography Electrocardiogram Classification
Computer Science & Engineering, University of Minnesota Twin Cities, Minneapolis, MN 55455, USA.
INTERVenE: Temporal-Abstraction-Interval Based Transformers for Short-Horizon Medical Event Prediction
Aug 30, 2026Shahar Oded, Yuval ShaharEvent Time Prediction Interpretable ML
The Stein Faculty of Computer and Information Science, Ben-Gurion University of the Negev, Beer-Sheva, Israel
A Multidimensional Data-Driven Hybrid Transformer Framework for Non-invasive Continuous Blood Pressure Prediction
Aug 24, 2026Yuexin Ma, Jingqi Hou, Yuxuan Kang +1Wearable Health Monitoring Time Series Transformers
College of Computer Science, Beijing University of Technology, Beijing, China
Time-Aware Tranformer-Based Prediction Model for AECOPD
Aug 21, 2026Weihao Qu, Ling Zheng, Dongyang Wang +2Clinical Prediction Clinical Outcome Prediction
CSSE Department, Monmouth University, West Long Branch, NJ, USA · Changzhou Yaoyuanxing Electronic Technology Co., Ltd, China
Rethinking Patch Based Multivariate Time Series Forecasting with Semantic Structured Partitioning
Aug 20, 2026Zhiquan Huang, Jiazhe Wang, Linjing Xue +3Multivariate Time Series Forecasting Time Series Forecasting
School of Information Engineering, Henan University of Science and Technology, Luoyang, 471023, Henan, China · School of Software, Henan University of Science and Technology, Luoyang, 471000, Henan, China

Oct 27, 2026J/K move · Enter open · S save

RankFormer: Rethinking Time Series Transformers through Representation Capacity Analysis and Progressive Decoding

Peiwang Tang, Cheng HE, Xijie Liang, Defu Lian

University of Science and Technology of China, Hefei, Anhui, China · State Key Laboratory of Cognitive Intelligence, Hefei, Anhui, China · Shanghai Black Wing Asset Management, Shanghai, China

Multivariate Time Series Forecasting Time Series Forecasting Time Series Transformers Long-Term Time Series Forecasting

Open paper AskPDF arXiv

Multivariate time series forecasting plays a critical role in Web and cyber-physical systems, where large-scale services and IoT infrastructures generate complex temporal data. Transformer-based models have achieved strong empirical performance in this domain, yet the principles governing the expressive capacity of their learned representations remain insufficiently understood. Moreover, many existing models rely on simplified decoding strategies, including channel-independent modeling and direct projection heads, which limit their ability to capture complex multivariate dynamics. To address these challenges, we propose RankFormer, a novel encoder-decoder architecture for multivariate time series forecasting. We establish a rank-based analysis framework to investigate how tokenization, embedding, and attention designs affect the representation capacity of time series Transformers. Our analysis reveals that patchification increases the potential representation capacity by aggregating multiple temporal observations into each token, while subsequent encoding operations determine how much of this capacity is realized. Guided by these insights, we develop a rank-enhancing encoder with Hourglass Attention to improve rank stability during information interaction. Furthermore, we introduce the Interactive Progressive Decoder (IPD), which improves future trajectory modeling through global-guided cross-channel interaction and progressive refinement. Extensive experiments on real-world long-term forecasting benchmarks demonstrate that RankFormer achieves competitive performance against state-of-the-art methods. Code repository: https://github.com/TangPeiwang/Rankformer.