Time Series Transformers

Follow this topic Ask about this topic

Momentum

8 papers in the last four weeks, up 60% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 91

All topics

On the Role of Inductive Bias in Time-Series Pretraining: A Case Study in Learning Generalizable Representations for Clinical Time Series
May 25, 2026Sharmita Dey, Diego Paez-GranadosSelf-Supervised Time Series Representation Learning Gait Analysis
ETH Zurich, Zurich, Switzerland · Swiss Paraplegic Research, Nottwil
StrTransformer: Source-Wise Structured Transformers for Unsupervised Blind Source Recovery
May 25, 2026Yuan-Hao WeiDisentangled Representation Learning Unsupervised Representation Learning
ADMFormer: An Adaptive-Decomposition Transformer with Time-Varying Masked Spatial Attention for Traffic Forecasting
May 25, 2026Ruiwen Gu, Qitai Tan, Yahao Liu +1Attention Mechanisms Spatiotemporal Forecasting
Shenzhen Ubiquitous Data Enabling Key Lab Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · School of Computer Science and Engineering University of Electronic Science and Technology of China, Chengdu, China
DAStatFormer: A Hybrid Multibranch Transformer with Statistical Feature Integration for DAS-Based Pattern Recognitions
May 22, 2026Michel Dione, Jerry Lonlac, Hélène Louis +2Distributed Acoustic Sensing Time Series Transformers
CERI SN - IMT Nord Europe · IMT Nord Europe, Institut Mines-Telecom, Univ. Lille, Centre for Digital Systems Lille, France · IMT Mines Ales, Institut Mines-Telecom, Ales, France
World Machine: Towards Generative World Modeling for Time-Series
May 21, 2026Elton Cardoso do Nascimento, Alexandre da Silva Simões, Esther Luna Colombini +2World Model Learning Latent World Models
Universidade Estadual de Campinas (UNICAMP), {FEEC, IC} · Universidade Estadual Paulista (UNESP), Instituto de Ciˆencia e Tecnologia, Sorocaba
Deep Learning Surrogates for Emulating Stochastic Climate Tipping Dynamics
May 20, 2026Adeline Hillier, Jennifer Sleeman, Jay Brett +3Neural Surrogate Modeling Multivariate Time Series Forecasting
Johns Hopkins Applied Physics Laboratory Hopkins University · Johns
Performance Monitoring of Proton Exchange Membrane Water Electrolyzer by Transformers-Based Machine Learning Model
May 18, 2026Bingqing Chen, Ivan Batalov, Qiu Chen +2Predictive Maintenance Time Series Transformers
Bosch Research & Technology Center
SAGA: A Sequence-Adaptive Generative Architecture for Multi-Horizon Probabilistic Forecasting with Adaptive Temporal Conformal Prediction
May 18, 2026Gustav Olaf Yunus Laitinen-Fredriksson Lundström-Imanov, Hafize Gonca CömertIrregular Time-Series Modeling Time Series Transformers
Department of Economics, Stockholm University, SE-106 91 Stockholm, Sweden
XCTFormer: Leveraging Cross-Channel and Cross-Time Dependencies for Enhanced Time-Series Analysis
May 18, 2026Israel Zexer, Omri AzencotMultivariate Time Series Forecasting Time Series Imputation
The Stein Faculty of Computer and Information Science Ben-Gurion University of the Negev
Olivia: Harmonizing Time Series Foundation Models with Power Spectral Density
May 17, 2026Jingru Fei, Kun Yi, Alex Xing Wang +3Zero-Shot Time Series Forecasting Frequency-Domain Feature Learning
Beijing Institute of Technology · North China Institute of Computing Technology · State Information Center +4
PRB-RUPFormer: A Recursive Unified Probabilistic Transformer for Residual PRB Forecasting
May 14, 2026Saad Masrur, Yuxuan Jiang, Matti Hiltunen +2Multivariate Time Series Forecasting Wireless Communications
AT&T RAN Technology, Bedminster, NJ · Department of Electrical and Computer Engineering, North Carolina State University, Raleigh, NC
DT-Transformer: A Foundation Model for Disease Trajectory Prediction on a Real-world Health System
May 14, 2026Yunying Zhu, Andrew R Weckstein, Kueiyu Joshua Lin +1Healthcare Clinical Prediction
Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA · Harvard T.H. Chan School of Public Health, Harvard University, Boston, MA, USA · Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University, MA, USA +2
From Index to Equity: Pre-Training Transformers for Stock Return Prediction
May 12, 2026Marie Soehl Coolsaet, Roberto Gallardo, Zhen GaoTime Series Forecasting Transfer Learning
Faculty of Engineering, McMaster University · Department* of Economics, Universidad Veracruzana, Mexico
Latent Chain-of-Thought Improves Structured-Data Transformers
May 11, 2026Carson Dudley, Samet OymakCoT Reasoning Time Series Forecasting
HEPA: A Self-Supervised Horizon-Conditioned Event Predictive Architecture for Time Series
May 11, 2026Jonas Petersen, Gian-Alessandro Lombardi, Riccardo Maggioni +3Event Time Prediction Self-Supervised Time Series Representation Learning
1ETH Zurich · 2Forgis · University of Vienna
Rethinking Random Transformers as Adaptive Sequence Smoothers for Sleep Staging
May 11, 2026Guisong Liu, Xin Gao, Martin Dresler +2Sleep Stage Classification Transformer
School of Biological Science and Medical Engineering Southeast University Nanjing, China · University of Bath Bath, England · Donders Institute for Brain, Cognition and Behaviour Radboud University Medical Center Nijmegen, The Netherlands +1
Benchmarking Transformer and xLSTM for Time-Series Forecasting of Heat Consumption
May 10, 2026Marja Wahl, Daniel R. Bayer, Sven Rausch +1Energy-Efficient ML Time Series Forecasting
RAUSCH Technology GmbH, Würzburg, Germany · Modeling and Simulation, University of Würzburg, Würzburg, Germany
Spectral Transformer Neural Processes
May 10, 2026Xianhe Chen, Hao Chen, Yingzhen LiTransformer Neural Processes
University of Cambridge · Tencent · Imperial College London
PACT: Peak-Aware Cross-Attention Graph Transformers for Efficient Storm-Surge Emulation
May 9, 2026Zesheng Liu, Doyup Kwon, Ning Lin +1PDE Surrogate Modeling Spatiotemporal Forecasting
Department of Computer Science and Engineering, Lehigh University · Department of Civil and Environmental Engineering, Princeton University · Department of Civil and Environmental Engineering, Lehigh University
Transformer-Based Wildlife Species Classification from Daily Movement Trajectories
May 7, 2026Obed Irakoze, Prasenjit MitraTime Series Classification Wildlife Monitoring
Department of Electrical & Computer Engineering Carnegie Mellon University Africa Kigali, Rwanda
Superposition Is Not Necessary: A Mechanistic Interpretability Analysis of Transformer Representations for Time Series Forecasting
May 6, 2026Alper YıldırımTransformer Interpretability Time Series Forecasting
FLUID: Continuous-Time Hyperconnected Sparse Transformer for Sink-Free Learning
May 6, 2026Waleed Razzaq, Yun-Bo ZhaoTransformer Transformer Attention
School of Automation University of Science and Technology China Hefei, Anhui
Forecasting Medium-Horizon Alzheimer's Disease Progression: Residual Gap-Aware Transformers for 24-Month CDR-SB Change from ADNI Clinical and Biomarker Histories
May 4, 2026Ran Tong, Tong Wang, Lanruo Wang +1Irregular Time-Series Modeling Alzheimer's Disease
Department of Mathematical Sciences, University of Texas at Dallas, Richardson, TX 75080, United States · Department of Plant Science and Landscape Architecture, University of Connecticut, Storrs, CT, United States · Naveen Jindal School of Management, University of Texas at Dallas, Richardson, TX 75080, United States +1
PPG-Based Affect Recognition with Long-Range Deep Models: A Measurement-Driven Comparison of CNN, Transformer, and Mamba Architectures
Apr 28, 2026Karim Alghoul, Hussein Al Osman, Abdulmotaleb El SaddikSelective SSMs Recurrent Neural Networks
School of Electrical Engineering and Computer Science University of Ottawa Ottawa, Canada
TTCD:Transformer Integrated Temporal Causal Discovery from Non-Stationary Time Series Data
Apr 27, 2026Omar Faruque, Sahara Ali, Xue Zheng +1Time-Series Causal Discovery Causal Structure Learning
Department of Information Systems University of Maryland, Baltimore County Baltimore, Maryland, USA · Department of Data Science University of North Texas Denton, Texas, USA · Climate Science Section Lawrence Livermore National Laboratory Livermore, California, USA
A Hybridizable Neural Time Integrator for Stable Autoregressive Forecasting
Apr 22, 2026Brooks Kinch, Xiaozhe Hu, Yilong Huang +6Neural Surrogate Modeling Time Series Transformers
Mechanical Engineering and Applied Mechanics, University of Pennsylvania, Philadelphia, PA, USA · Department of Mathematics, Tufts University, Medford, MA, USA · Department of Mathematics and Cybernetics, SINTEF Digital, Oslo, Norway +3
Causally-Constrained Probabilistic Forecasting for Time-Series Anomaly Detection
Apr 20, 2026Pooyan Khosravinia, João Gama, Bruno VelosoAnomaly Localization Time-Series Anomaly Detection
The Phasor Transformer: Resolving Attention Bottlenecks on the Unit Circle
Mar 18, 2026Dibakar SigdelTransformer Time Series Forecasting
Mindverse Computing LLC, WA 98087
EEG-based AI-BCI Wheelchair Advancement: Transformer-Based Learning with Motor Imagery for Brain Computer Interface
Sep 30, 2025Bipul Thapa, Biplov Paneru, Bishwash Paneru +1MI Classification Brain-Computer Interfaces
Department of Computer Science and Engineering, Kathmandu University, Kavre, Nepal · Department of Electronics and Communication Engineering, Nepal Engineering College, Pokhara University, Bhaktapur, Nepal · Department of Applied Sciences and Chemical Engineering, Institute of Engineering, Pulchowk Campus, Tribhuvan University, Lalitpur, Nepal
HyMaTE: A Hybrid Mamba and Transformer Model for EHR Representation Learning
Sep 28, 2025Md Mozaharul Mottalib, Thao-Ly T. Phan, Rahmatollah BeheshtiRepresentation Learning Longitudinal EHR Modeling
University of Delaware Newark, Delaware, USA · Thomas Jefferson University & Nemours Children’s Health Wilmington, Delaware, USA

May 25, 2026, cs.LGJ/K move · Enter open · S save

On the Role of Inductive Bias in Time-Series Pretraining: A Case Study in Learning Generalizable Representations for Clinical Time Series

Sharmita Dey, Diego Paez-Granados

ETH Zurich, Zurich, Switzerland · Swiss Paraplegic Research, Nottwil

Self-Supervised Time Series Representation Learning Gait Analysis Predictive Representation Learning Time Series Transformers

Open paper AskPDF arXiv

Clinical time-series learning is routinely constrained by small, heterogeneous cohorts and protocol drift, while its downstream use spans both classification (e.g., pathology diagnosis) and regression (e.g., temporal forecasting). These constraints make foundation-model pretraining appealing, but raises an important question of which inductive biases should the pretraining objective impose so that representations transfer across task types and subjects. We study this question in pathological gait analysis for spinal cord injury (SCI) via PathoFM, an encoder-centric transformer pretrained on multivariate gait windows with three complementary objectives: Local Completion (reconstruct contiguous masked spans to enforce local structure), Temporal Continuity (predict a masked mid-horizon continuation from an observed prefix to enforce smoothness and causal consistency), and Unsupervised In-Context Dynamics (support-query reconstruction conditioned on subject exemplar windows via attention). Empirically comparing objective families (grouping/contrastive, dynamics-based, and generative reconstruction), we find that dynamics-centric mixtures produce the most balanced transfer: grouping objectives favor discriminative margins but can degrade magnitude fidelity needed for continuous targets, whereas reconstruction-only objectives preserve waveform structure but may underperform on classification. Overall, combining local reconstruction with temporal continuity, and adding in-context conditioning when exemplar access is realistic, yields robust subject-generalizing representations.

Momentum

Related topics

Latest papers 91