cs.LGFeb 6, 2026

Revisiting the Generic Transformer: Deconstructing a Strong Baseline for Time Series Foundation Models

Authors: Yunshi Wen, Wesley M. Gifford, Chandra Reddy, Lam M. Nguyen, Jayant Kalagnanam, Anak Agung Julius

Organizations: Rensselaer Polytechnic Institute · IBM Research

Abstract

The recent surge of interest in Time Series Foundation Models has rapidly advanced the field. Yet, heterogeneous training setups across studies make it difficult to determine whether performance improvements came from architectural innovations or effective data engineering. In this work, we investigate whether a standard patch Transformer architecture, paired with a straightforward training protocol, can match the zero-shot forecasting performance of state-of-the-art TSFMs. We conduct a comprehensive ablation study covering model scaling, data composition, and training techniques to examine the ingredients behind this performance. Our findings show that pretraining data composition, contiguous patch masking, and context length have a substantial effect on forecasting accuracy, and that the generic architecture itself can deliver high performance with good scalability. By strictly controlling these experimental variables, we provide comprehensive empirical results on model scaling across multiple dimensions. Furthermore, we establish that the effectiveness of this training protocol generalizes robustly across benchmarks and alternative architectures. We release our pretrained models and document the detailed training protocol and findings to establish a transparent, reproducible baseline for future research.

Figures & tables

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Reverso: Efficient Time Series Foundation Models for Zero-shot Forecasting

    Feb 19, 2026Xinghong Fu, Yanhong Li, Georgios Papaioannou +1Time-Series Foundation ModelsZero-Shot Time Series Forecasting

  2. Universal Redundancies in Time Series Foundation Models

    Feb 2, 2026Anthony Bao, Venkata Hasith Vattikuti, Jeffrey Lai +1Transformer InterpretabilityKernel Regression

  3. Foundation Models and Fine-Tuning: Toward a New Generation of Models for Time Series Forecasting

    Jul 25, 2026Morad Laglil, Bertrand Pracca, Emilie Devijver +1Zero-Shot Time Series ForecastingTime-Series Foundation Models