We introduce Timer-S1, a strong Mixture-of-Experts (MoE) time series foundation model with 8.3B total parameters, 0.75B activated parameters for each token, and a context length of 11.5K. To overcome the scalability bottleneck in existing pre-trained time series foundation models, we perform Serial Scaling in three dimensions: model architecture, dataset, and training pipeline. Timer-S1 integrates sparse TimeMoE blocks and generic TimeSTP blocks for Serial-Token Prediction (STP), a generic training objective that adheres to the serial nature of forecasting. The proposed paradigm introduces serial computations to improve long-term predictions while avoiding costly rolling-style inference and pronounced error accumulation in the standard next-token prediction. Pursuing a high-quality and unbiased training dataset, we curate TimeBench, a corpus with one trillion time points, and apply meticulous data augmentation to mitigate predictive bias. We further pioneer a post-training stage, including continued pre-training and long-context extension, to enhance short-term and long-context performance. Evaluated on the large-scale GIFT-Eval leaderboard, Timer-S1 achieves state-of-the-art forecasting performance, attaining the best MASE and CRPS scores as a pre-trained model. Timer-S1 is released to facilitate further research.
Figures & tables
Figure 1: Forecasting into the long term accumulates uncertainty, as the prediction of each step depends on all preceding estimations, which positions time series forecasting as a serial problem [ liu2025serial ] . Parallel-forecasting models, which predict multiple future steps simultaneously, do not scale with sufficient serial computations to reliably capture the recurrent dependencies. Although autoregressive models mirror the serial nature of the task by "predicting step by step", their iterative rolling mechanism over the input still entails significant computational overhead.
Figure 2: The Serial Scaling of Timer-S1 is achieved by (a) serial forecasting , which efficiently produces multi-step prediction with serial computations; (b) data scaling with data augmentation applied to TimeBench [ liu2025sundial ] , a corpus of over one trillion time points; and (c) post-training that comprehensively enhances the capability of the model.
Figure 3: A timeline of representative time series forecasting models in recent years. This timeline is established according to the release date of the paper or technical report for a model. Notably, the Timer model is a continuously developed family of time series foundation models that has presents sustained scaling in model size across its generations.
Figure 4: Overall architecture of Timer-S1. The input time series is re-normalized and divided into patch tokens. These patch embeddings are fed into a decoder-only Transformer. The Transformer backbone consists of a series of TimeMoE blocks, where Pre-RMSNorm and QK-Norm [ henry2020query ] are adapted for training stability, followed by a sequence of TimeSTP blocks. TimeSTP extends TimeMoE by additionally conditioning on the initial input embeddings, iteratively refining the token embeddings from the previous block, and generating shifted-by-one token predictions. All output embeddings are projected by a shared forecasting head to produce quantile predictions. Timer-S1 enables serial forecasting, where predictions of longer horizons actually undergo more serial computations in the Transformer block.
Figure 5: Illustration of the TimeBench dataset and the training pipeline of Timer-S1. TimeBench integrates over one trillion time points from multiple domains, processed through quality-focused preprocessing, predictability assessment, and diversity-enhancing augmentation. TimeBench is loaded in a single-series sequence format for learning univariate patterns. Timer-S1’s training follows a multi-stage design: it is first pre-trained via serial-token prediction with uniform horizon weighting; then it undergoes continued pre-training using a horizon-decay objective to enhance short-term accuracy; and extends its context length from 2880 to 11520 through long-context adaptation.
Figure 6: Performance of Timer-S1 on the GIFT-Eval leaderboard.
Figure 7: Performance (MASE) on the GIFT-Eval leaderboard, grouped by the term length.
Figure 8: Performance (CRPS) on the GIFT-Eval leaderboard, grouped by the term length.
Figure 9: Performance of Timer-S1 with different training pipelines on the GIFT-Eval leaderboard.
Figure 10: Performance of the Timer model trained by next-token prediction and serial-token prediction.
Figure 11: Performance of the Timer model trained by multi-token prediction and serial-token prediction.
Figure 12: Inference time of models trained by next-token prediction, multi-token prediction, and serial-token prediction. We set the same backbone configuration except for the training objective. The input sequence length is set to 11520.
Figure 13: Performance of the pre-trained Timer-S1 with varying TimeMoE blocks (fixed 16-block TimeSTP).
Figure 14: Performance of the pre-trained Timer-S1 with varying TimeSTP blocks (fixed 24-block TimeMoE).
Figure 15: Performance of Timer-S1 with TimeSTP variants.
Figure 16: Performance of Timer-S1 with and without data augmentation.
Figure 17: Performance of of Timer-S1 on different sinusoidal signals.
Figure 18: Performance of Timer-S1 with and without the pre-training on TimeBench.
Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.
We introduce Timer-M1, a pretrained multivariate time series foundation model that learns with primitives for zero-shot forecasting. Across domains, time series share elementary temporal and relational patterns, termed primitives, yet differ in how these primitives manifest and evolve across different contexts. Despite progress in zero-shot and task-general forecasting, existing foundation models may still struggle to generalize to complex real-world scenarios. To this end, we develop a primitive-based data synthesis and pretraining pipeline. The synthesis pipeline generates series with temporal primitives shared across domains and then assembles real and generated series into multivariate samples using relational primitives. Afterwards, samples are organized into episodes by assigning distinct channel roles as target variates, past-only covariates, and known-future covariates, ensuring that the model is optimized on predictable variates using available exogenous information. Technically, Timer-M1 further adapts gated two-dimensional Transformer blocks that dynamically allocate cross-variate attention across layers. Across three large-scale forecasting benchmarks, Timer-M1 ranks first on both FEV and TIME and second on GIFT-Eval among most recent time series foundation models. These results support effective primitive-based pretraining as a route to robust general forecasting technique across domains and task settings.
Haoran Zhang, Haixuan Liu, Xingjian Su +5
School of Software, BNRist, Tsinghua University · ByteDance, Beijing, China
Pretraining time series foundation models across heterogeneous datasets necessitates effective handling of varying sampling frequencies. Current methods either employ dataset-specific patch sizes and separate FFNs, leading to fragmented representations, or enforce a fixed patch size that neglects inherent temporal variations. To address this, we propose SATS, featuring a scale-aware token alignment mechanism that treats patch size as an explicit notion of scale. By incorporating a contrastive-inspired alignment regularizer, SATS aligns representation spaces across scales while preserving distinct modeling capacities. Furthermore, a hybrid masking strategy combining random and contiguous masking is introduced to capture multi-scale temporal structures. Experimental results on LSTF benchmarks demonstrate that SATS achieves a 9.2% improvement in MSE and an 8.3% gain in GIFT-Eval MASE compared to competitive baselines. Notably, SATS consistently delivers SOTA performance while achieving a 65.6% increase in model efficiency over advanced baselines, highlighting its effectiveness and scalability in time series pretraining.
Taihua Chen, Xiang Ma, Yixin Zhang +3
School of Software, Shandong University, Jinan 250101, China · Joint SDU–NTU Centre for Artificial Intelligence Research (C-FAIR), Shandong University, Jinan 250101, China · Nanyang Technological University, Singapore
Learning time series foundation models has been shown to be a promising approach for zero-shot time series forecasting across diverse time series domains. Insofar as scaling has been a critical driver of performance of foundation models in other modalities such as language and vision, much recent work on time series foundation modeling has focused on scaling. This has resulted in time series foundation models with hundreds of millions of parameters that are, while performant, inefficient and expensive to use in practice. This paper describes a simple recipe for learning efficient foundation models for zero-shot time series forecasting that are orders of magnitude smaller. We show that large-scale transformers are not necessary: small hybrid models that interleave long convolution and linear RNN layers (in particular DeltaNet layers) can match the performance of larger transformer-based models while being more than a hundred times smaller. We also describe several data augmentation and inference strategies that further improve performance. This recipe results in Reverso, a family of efficient time series foundation models for zero-shot forecasting that significantly push the performance-efficiency Pareto frontier.
Xinghong Fu, Yanhong Li, Georgios Papaioannou +1
Massachusetts Institute of Technology · Allen Institute for AI · Qube Research & Technologies