Time Series Transformer Model

Time series transformer models leverage the power of transformer architectures to analyze and predict sequential data, addressing limitations of traditional methods in handling irregularly sampled data and long-range dependencies. Current research focuses on improving model interpretability, scaling to larger datasets and model sizes, adapting models across different domains, and optimizing them for resource-constrained environments through techniques like quantization-aware training. These advancements are significantly impacting fields like healthcare (e.g., disease trajectory prediction), and are enabling more accurate and efficient forecasting in various applications, from COVID-19 caseload prediction to energy consumption forecasting.

Papers

November 4, 2024

ElasTST: Towards Robust Varied-Horizon Forecasting with Elastic Time-Series Transformer
Jiawen Zhang, Shun Zheng, Xumeng Wen, Xiaofang Zhou, Jiang Bian, Jia Li
Multi Horizon Nonlinear Elasticity General Time Series Time Series Transformer Model

October 3, 2024

TrajGPT: Irregular Time-Series Representation Learning for Health Trajectory Analysis
Ziyang Song, Qingcheng Lu, He Zhu, David Buckeridge, Yue Li
Time Series Representation Learning Generative Trajectory Disease Trajectory Time Series Transformer Model

May 22, 2024

Scaling-laws for Large Time-series Models
Thomas D. P. Edwards, James Alvey, Justin Alsing, Nam H. Nguyen, Benjamin D. Wandelt
Large Language Model Large Corpus Scaling Law Large Scale Transformer Model Large Time Series Model Time Series Transformer Model

January 26, 2024

Interpreting Time Series Transformer Models and Sensitivity Analysis of Population Age Groups to COVID-19 Infections
Md Khairul Islam, Tyler Valentine, Timothy Joowon Sue, Ayush Karmacharya, Luke Neil Benham, Zhengguang Wang, Kingsley Kim, Judy Fox
Time Series Covid 19 Local Interpretability Time Series Transformer Model

January 12, 2024

Domain Adaptation for Time series Transformers using One-step fine-tuning
Subina Khanal, Seshu Tirupathi, Giulio Zizzo, Ambrish Rawat, Torben Bach Pedersen
Deep Learning Domain Adaptation Deep Learning Model Fine Tuning Time Series Pre Trained Model Time Series Transformer Model

October 4, 2023

A Study of Quantisation-aware Training on Time Series Transformer Models for Resource-constrained FPGAs
Tianheng Ling, Chao Qian, Lukas Einhaus, Gregor Schiele
Study Feature Mixed Precision Quantization Model Quantization Adaptive Quantization FPGA Resource Time Series Transformer Model

October 5, 2022

Transformer-based conditional generative adversarial network for multivariate time series generation
Abdellah Madane, Mohamed-djallel Dilmi, Florent Forest, Hanane Azzag, Mustapha Lebbah, Jerome Lacaille
Generative Adversarial Network Transformer Based Multivariate Time Series Conditional Generative Adversarial Network Time Series Transformer Model

May 20, 2022

Forecasting COVID-19 Caseloads Using Unsupervised Embedding Clusters of Social Media Posts
Felix Drinkall, Stefan Zohren, Janet B. Pierrehumbert
Social Medium Covid 19 Unsupervised Setting Sentence Level Transformer Based Language Model Different Cluster Time Series Transformer Model Epidemiological Modelling

Time Series Transformer Model

Papers

ElasTST: Towards Robust Varied-Horizon Forecasting with Elastic Time-Series Transformer

TrajGPT: Irregular Time-Series Representation Learning for Health Trajectory Analysis

Scaling-laws for Large Time-series Models

Interpreting Time Series Transformer Models and Sensitivity Analysis of Population Age Groups to COVID-19 Infections

Domain Adaptation for Time series Transformers using One-step fine-tuning

A Study of Quantisation-aware Training on Time Series Transformer Models for Resource-constrained FPGAs

Transformer-based conditional generative adversarial network for multivariate time series generation

Forecasting COVID-19 Caseloads Using Unsupervised Embedding Clusters of Social Media Posts