Time Series Transformers

Follow this topic Ask about this topic

Time Series Transformers apply attention to ordered observations, helping model patterns over time and relationships among multiple variables for forecasting, classification, and anomaly detection. Common approaches use patches, separate temporal and channel-wise attention, or add frequency, spatial, and domain-specific structure; pretrained foundation models also aim to transfer across datasets and tasks. Work increasingly addresses long and irregular sequences, uncertainty, and efficient inference through sparse attention, adaptive tokenization, and hybrid architectures. A central tension is that added complexity does not guarantee better accuracy or generalization: performance depends on the data, forecast horizon, preprocessing, and evaluation, and simple statistical or smaller neural models can remain competitive.

Latest papers

139 papers12 papers in the last four weeks, up 9% on the four weeks before. 0.1% of all new papers.

Bayesian Transformer for Probabilistic Load Forecasting in Smart Grids
Mar 9, 2026Sajib Debnath, Md. Uzzal MiaElectricity Load Forecasting Probabilistic Time Series Forecasting
O&M Analytics, AES Clean Energ; The AES Corporation; Louisville, CO 80027, USA · Department of Information and Communication Engineering; Pabna University of Science and Technology; Rajapur, Pabna, Bangladesh
Stock Market Prediction Using Node Transformer Architecture Integrated with BERT Sentiment Analysis
Mar 6, 2026Mohammad Al Ridhawi, Mahtab Haj Ali, Hussein Al OsmanStock Price Forecasting Financial Forecasting
School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, Canada
Early Warning of Intraoperative Adverse Events via Transformer-Driven Multi-Label Learning
Mar 5, 2026Xueyao Wang, Xiuding Cai, Honglin Shang +2Clinical Risk Prediction Multi-Label Classification
Chengdu Institute of Computer Application, Chinese Academy of Sciences, Chengdu 610213, China · University of Chinese Academy of Sciences, Beijing 100049, China · China Zhenhua Research Institute Co., Ltd., Guiyang 550014, China
FaultXformer: A Transformer-Encoder Based Fault Classification and Location Identification model in PMU-Integrated Active Electrical Distribution System
Feb 27, 2026Kriti Thakur, Alivelu Manga Parimi, Mayukha PalPower System Fault Diagnosis Fault Localization
ABB Ability Innovation Center, Asea Brown Boveri Company, Hyderabad, 500084, Telangana, India · Department of Electrical and Electronics Engineering, Birla Institute of Technology and Science, Pilani- Hyderabad Campus, Hyderabad, 500078, Telangana, India
TiMi: Empower Time Series Transformers with Multimodal Mixture of Experts
Feb 25, 2026Jiafeng Lin, Yuxuan Wang, Huakun Luo +2Time Series Transformers LLM-Based Time Series Forecasting
School of Software, BNRist, Tsinghua University.
Benchmarking State Space Models, Transformers, and Recurrent Networks for US Grid Forecasting
Feb 24, 2026Sunki Hong, Jisoo LeeForecasting Benchmarks Electricity Load Forecasting
Gramm AI
From ARIMA to Attention: Power Load Forecasting Using Temporal Deep Learning
Feb 21, 2026Suhasnadh Reddy Veluru, Sai Teja Erukude, Viswa Chaitanya MarellaElectricity Load Forecasting Time Series Forecasting
Department of Computer Science and Engineering, Vellore Institute of Technology, Vellore, Tamil Nadu, India · Department of Electronics and Communication Engineering, Bharat Institute of Engineering and Technology, Hyderabad, Telangana, India
Transformers for dynamical systems learn transfer operators in-context
Feb 21, 2026Anthony Bao, Jeffrey Lai, William GilpinIn-Context Learning Dynamical Systems
Department of Electrical & Computer Engineering, The University of Texas at Austin, Austin, Texas 78712, USA · The Oden Institute, The University of Texas at Austin, Austin, Texas 78712, USA · Department of Physics, The University of Texas at Austin, Austin, Texas 78712, USA
Be Wary of Your Time Series Preprocessing
Feb 19, 2026Sofiane Ennadir, Tianze Wang, Oleg Smirnov +2Time Series Representation Learning Time Series Transformers
King AI Labs, Microsoft Gaming · Kreditz AB
A statistical perspective on transformers for small longitudinal cohort data
Feb 18, 2026Kiana Farhadyar, Maren Hackenberg, Kira Ahrens +10Permutation Testing Time Series Transformers
Institute of Medical Biometry and Statistics (IMBI), University of Freiburg, Freiburg, Germany · Freiburg Center for Data Analysis, Modeling and AI, University of Freiburg, Freiburg, Germany · Department of Psychiatry, Psychosomatic Medicine and Psychotherapy, University Hospital Frankfurt, Frankfurt, Germany +6
Structure-Aware Set Transformers: Temporal and Variable-Type Attention Biases for Asynchronous Clinical Time Series
Feb 18, 2026Joohyung Lee, Kwanhyung Lee, Changhun Kim +1Set Transformer Irregular Time-Series Modeling
AITRICS · Korea Advanced Institute of Science and Technology (KAIST)
Covariance-Aware Transformers for Quadratic Programming and Decision Making
Feb 16, 2026Kutay Tire, Yufan Zhang, Ege Onur Taga +1Quadratic Programming Portfolio Optimization
The University of Texas at Austin · University of Michigan
FTimeXer: Frequency-aware Time-series Transformer with Exogenous variables for Robust Carbon Footprint Forecasting
Feb 16, 2026Qingzhong Li, Yue Hu, Zhou Long +3Time Series Forecasting Time Series Transformers
School of Software, Xinjiang University, Urumqi, China · Xinjiang Key Laboratory of Intelligent Computing and Smart Applications, School of Software, Xinjiang University, Urumqi 830046, China
Patch-Level Tokenization with CNN Encoders and Attention for Improved Transformer Time-Series Forecasting
Jan 18, 2026Saurish Nagrath, Saroj Kumar Panigrahy, Soubhagya Sankar BarpandaMultivariate Time Series Forecasting Time Series Transformers
School of Computer Science and Engineering, VIT-AP University, Amaravati, Andhra Pradesh, 522241, India
EEG-based AI-BCI Wheelchair Advancement: Transformer-Based Learning with Motor Imagery for Brain Computer Interface
Sep 30, 2025Bipul Thapa, Biplov Paneru, Bishwash Paneru +1MI Classification Brain-Computer Interfaces
Department of Computer Science and Engineering, Kathmandu University, Kavre, Nepal · Department of Electronics and Communication Engineering, Nepal Engineering College, Pokhara University, Bhaktapur, Nepal · Department of Applied Sciences and Chemical Engineering, Institute of Engineering, Pulchowk Campus, Tribhuvan University, Lalitpur, Nepal
HyMaTE: A Hybrid Mamba and Transformer Model for EHR Representation Learning
Sep 28, 2025Md Mozaharul Mottalib, Thao-Ly T. Phan, Rahmatollah BeheshtiRepresentation Learning Hybrid Mamba-Transformer Architectures
University of Delaware Newark, Delaware, USA · Thomas Jefferson University & Nemours Children’s Health Wilmington, Delaware, USA
Dynamic Relational Priming Improves Transformer in Multivariate Time Series
Sep 15, 2025Hunjae Lee, Corey ClarkTransformer Attention Attention Mechanisms
Department of Computer Science, Southern Methodist University, Dallas TX USA · SMU Guildhall
FANTF: Fuzzy attention network based transformers for time series analysis
Mar 31, 2025Sanjay Chakraborty, Fredrik HeintzTime Series Transformers Time Series Forecasting
Department of Computer and Information Science (IDA), REAL, AIICS, Linköping University, Linköping, Sweden
Fewer yet critical: Reducing Redundant Token Dependencies for Transformer-based Time Series Forecasting
Mar 10, 2025Jianqi Zhang, Yuchan Liu, Zeen Song +2Time Series Forecasting Transformer Attention
National Key Laboratory of Space Integrated Information Systems, Institute of Software, Chinese Academy of Sciences, Beijing, China 1 · University of Chinese Academy of Sciences, Beijing, China 2 · Digital-Intelligent Operations Center, Guizhou Power Grid Co., Ltd., Guizhou, China 3 +1

Mar 9, 2026J/K move · Enter open · S save

Bayesian Transformer for Probabilistic Load Forecasting in Smart Grids

Sajib Debnath, Md. Uzzal Mia

O&M Analytics, AES Clean Energ; The AES Corporation; Louisville, CO 80027, USA · Department of Information and Communication Engineering; Pabna University of Science and Technology; Rajapur, Pabna, Bangladesh

Electricity Load Forecasting Probabilistic Time Series Forecasting Bayesian Neural Networks Uncertainty Calibration Time Series Transformers Attention Mechanisms

Open paper AskPDF arXiv

The reliable operation of modern power grids requires probabilistic load forecasts with well-calibrated uncertainty estimates. However, existing deep learning models produce overconfident point predictions that fail catastrophically under extreme weather distributional shifts. This study proposes a Bayesian Transformer (BT) framework that integrates three complementary uncertainty mechanisms into a PatchTST backbone: Monte Carlo Dropout for epistemic parameter uncertainty, variational feed-forward layers with log-uniform weight priors, and stochastic attention with learnable Gaussian noise perturbations on pre-softmax logits, representing, to the best of our knowledge, the first application of Bayesian attention to probabilistic load forecasting. A seven-level multi-quantile pinball-loss prediction head and post-training isotonic regression calibration produce sharp, near-nominally covered prediction intervals. Evaluation of five grid datasets (PJM, ERCOT, ENTSO-E Germany, France, and Great Britain) augmented with NOAA covariates across 24, 48, and 168-hour horizons demonstrates state-of-the-art performance. On the primary benchmark (PJM, H=24h), BT achieves a CRPS of 0.0289, improving 7.4% over Deep Ensembles and 29.9% over the deterministic LSTM, with 90.4% PICP at the 90% nominal level and the narrowest prediction intervals (4,960 MW) among all probabilistic baselines. During heat-wave and cold snap events, BT maintained 89.6% and 90.1% PICP respectively, versus 64.7% and 67.2% for the deterministic LSTM, confirming that Bayesian epistemic uncertainty naturally widens intervals for out-of-distribution inputs. Calibration remained stable across all horizons (89.8-90.4% PICP), while ablation confirmed that each component contributed a distinct value. The calibrated outputs directly support risk-based reserve sizing, stochastic unit commitment, and demand response activation.