cs.AIMar 5, 2026

Timer-S1: A Billion-Scale Time Series Foundation Model with Serial Scaling

Authors: Yong Liu, Xingjian Su, Shiyu Wang, Haoran Zhang, Haixuan Liu, Yuxuan Wang, Zhou Ye, Yang Xiang, 

Organizations: Tsinghua University · ByteDance

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

We introduce Timer-S1, a strong Mixture-of-Experts (MoE) time series foundation model with 8.3B total parameters, 0.75B activated parameters for each token, and a context length of 11.5K. To overcome the scalability bottleneck in existing pre-trained time series foundation models, we perform Serial Scaling in three dimensions: model architecture, dataset, and training pipeline. Timer-S1 integrates sparse TimeMoE blocks and generic TimeSTP blocks for Serial-Token Prediction (STP), a generic training objective that adheres to the serial nature of forecasting. The proposed paradigm introduces serial computations to improve long-term predictions while avoiding costly rolling-style inference and pronounced error accumulation in the standard next-token prediction. Pursuing a high-quality and unbiased training dataset, we curate TimeBench, a corpus with one trillion time points, and apply meticulous data augmentation to mitigate predictive bias. We further pioneer a post-training stage, including continued pre-training and long-context extension, to enhance short-term and long-context performance. Evaluated on the large-scale GIFT-Eval leaderboard, Timer-S1 achieves state-of-the-art forecasting performance, attaining the best MASE and CRPS scores as a pre-trained model. Timer-S1 is released to facilitate further research.

Figures & tables

Figure 1: Forecasting into the long term accumulates uncertainty, as the prediction of each step depends on all preceding estimations, which positions time series forecasting as a serial problem [ liu2025serial ] . Parallel-forecasting models, which predict multiple future steps simultaneously, do not scale with sufficient serial computations to reliably capture the recurrent dependencies. Although autoregressive models mirror the serial nature of the task by "predicting step by step", their iterative rolling mechanism over the input still entails significant computational overhead.
Figure 2: The Serial Scaling of Timer-S1 is achieved by (a) serial forecasting , which efficiently produces multi-step prediction with serial computations; (b) data scaling with data augmentation applied to TimeBench [ liu2025sundial ] , a corpus of over one trillion time points; and (c) post-training that comprehensively enhances the capability of the model.
Figure 3: A timeline of representative time series forecasting models in recent years. This timeline is established according to the release date of the paper or technical report for a model. Notably, the Timer model is a continuously developed family of time series foundation models that has presents sustained scaling in model size across its generations.
Figure 4: Overall architecture of Timer-S1. The input time series is re-normalized and divided into patch tokens. These patch embeddings are fed into a decoder-only Transformer. The Transformer backbone consists of a series of TimeMoE blocks, where Pre-RMSNorm and QK-Norm [ henry2020query ] are adapted for training stability, followed by a sequence of TimeSTP blocks. TimeSTP extends TimeMoE by additionally conditioning on the initial input embeddings, iteratively refining the token embeddings from the previous block, and generating shifted-by-one token predictions. All output embeddings are projected by a shared forecasting head to produce quantile predictions. Timer-S1 enables serial forecasting, where predictions of longer horizons actually undergo more serial computations in the Transformer block.
Figure 5: Illustration of the TimeBench dataset and the training pipeline of Timer-S1. TimeBench integrates over one trillion time points from multiple domains, processed through quality-focused preprocessing, predictability assessment, and diversity-enhancing augmentation. TimeBench is loaded in a single-series sequence format for learning univariate patterns. Timer-S1’s training follows a multi-stage design: it is first pre-trained via serial-token prediction with uniform horizon weighting; then it undergoes continued pre-training using a horizon-decay objective to enhance short-term accuracy; and extends its context length from 2880 to 11520 through long-context adaptation.
Figure 6: Performance of Timer-S1 on the GIFT-Eval leaderboard.
Figure 7: Performance (MASE) on the GIFT-Eval leaderboard, grouped by the term length.
Figure 8: Performance (CRPS) on the GIFT-Eval leaderboard, grouped by the term length.
Figure 9: Performance of Timer-S1 with different training pipelines on the GIFT-Eval leaderboard.
Figure 10: Performance of the Timer model trained by next-token prediction and serial-token prediction.
Figure 11: Performance of the Timer model trained by multi-token prediction and serial-token prediction.
Figure 12: Inference time of models trained by next-token prediction, multi-token prediction, and serial-token prediction. We set the same backbone configuration except for the training objective. The input sequence length is set to 11520.
Figure 13: Performance of the pre-trained Timer-S1 with varying TimeMoE blocks (fixed 16-block TimeSTP).
Figure 14: Performance of the pre-trained Timer-S1 with varying TimeSTP blocks (fixed 24-block TimeMoE).
Figure 15: Performance of Timer-S1 with TimeSTP variants.
Figure 16: Performance of Timer-S1 with and without data augmentation.
Figure 17: Performance of of Timer-S1 on different sinusoidal signals.
Figure 18: Performance of Timer-S1 with and without the pre-training on TimeBench.
More in this topic

Time Series Forecasting

Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.

See all 839 papers

Explore similar work

CardsList

  1. Timer-M1: A Multivariate Time Series Foundation Model via Learning Primitives

    Oct 8, 2026Haoran Zhang, Haixuan Liu, Xingjian Su +5Multivariate Time Series ForecastingTime-Series Foundation Models

    School of Software, BNRist, Tsinghua University · ByteDance, Beijing, China

  2. Scale-Aware Pretraining of Time Series Foundation Models via Multi-Patch Token Alignment and Hybrid Masking

    Aug 20, 2026Taihua Chen, Xiang Ma, Yixin Zhang +3Time-Series Foundation ModelsSelf-Supervised Pre-Training

    School of Software, Shandong University, Jinan 250101, China · Joint SDU–NTU Centre for Artificial Intelligence Research (C-FAIR), Shandong University, Jinan 250101, China · Nanyang Technological University, Singapore

  3. Reverso: Efficient Time Series Foundation Models for Zero-shot Forecasting

    Feb 19, 2026Xinghong Fu, Yanhong Li, Georgios Papaioannou +1Time-Series Foundation ModelsZero-Shot Time Series Forecasting

    Massachusetts Institute of Technology · Allen Institute for AI · Qube Research & Technologies