Time series foundation models have demonstrated strong zero-shot forecasting performance, but computational efficiency remains a central architectural challenge. Temporal self-attention incurs quadratic cost in the number of temporal tokens, while nonlinear recurrent state transitions restrict parallel computation along the temporal dimension. We introduce ci-proto, a family of time series foundation models based on Mamba-3, whose linear state recurrences support chunk-wise parallel-in-time computation during training, and inference costs that scale linearly with sequence length. The architecture alternates recurrent temporal processing with cross-variate attention, supporting multivariate probabilistic forecasting with past-only and future-known covariates. Across fev-bench's 100 forecasting tasks, ci-proto XL sets a new state of the art among time series foundation models. The five ci-proto variants also make up the entire Pareto frontier for mean scaled quantile loss and mean absolute scaled error against both parameter count and median runtime.
Figures & tables
Figure 1: Arithmetic mean errors of ci-proto on fev-bench. Top seven models on the official fev-bench leaderboard shchur2026fevbench . The left panel shows the mean absolute scaled error (MASE) for point forecasting, and the right panel shows the scaled quantile loss (SQL) for probabilistic forecasting. Lower values indicate better performance in both panels.
Figure 2: Mean scaled quantile loss (SQL) on fev-bench versus parameter count . Lower SQL is better. The horizontal axis is in logarithmic scale. The dashed line marks the Pareto frontier formed by the five ci-proto variants.
Figure 3: Mean scaled quantile loss (SQL) on fev-bench versus median runtime per 100 forecasts for selected leaderboard models. We benchmark all models on identical hardware and under the same execution settings, since self-reported leaderboard timings are not directly comparable. We report the mean and standard deviation across five runs. Lower SQL is better. The horizontal axis is in logarithmic scale. The dashed line marks the Pareto frontier formed by the five ci-proto variants.
Figure 4: Skill scores of ci-proto on fev-bench. Skill score relative to the seasonal naive baseline for the top seven models on the official fev-bench leaderboard shchur2026fevbench . The left panel shows skill score based on the mean absolute scaled error (MASE) for point forecasting, and the right panel shows it based on the scaled quantile loss (SQL) for probabilistic forecasting. Higher values indicate better performance in both panels.
Figure 5: Quantile loss over a scheduled power outage for uncapped models. The outage has a single earlier occurrence at the given number of steps back as shown on the x-axis, which is logarithmic. Each model receives the smallest of power-of-two context 2k≥2,048 steps that contains the earlier outage, or all 2,048,000 steps of history if needed. ci-proto XL and the uncapped baselines read the whole context, with all other settings matching each model’s fev-bench submission. The curves show the mean and standard deviation over 40 forecast windows (lower is better). Here each step is one minute, and the top axis gives the steps as units of time.
Figure 6: Time and peak GPU memory per forecast for ci-proto XL and the uncapped baselines. The context axis is logarithmic, while the time and memory axes are linear and start at zero. Values are the median time and largest memory over 40 windows (identical to those used for Figure 5 ) on one H100 when forecasting one series at a time. All settings other than the removed context caps match the models’ fev-bench submission.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Configuration
XS
S
M
L
XL
Number of blocks
6
8
10
12
16
Hidden dimension D
128
192
320
512
640
Patch length P
32
32
32
32
32
Mamba-3 state dimension
128
128
128
128
128
Mamba-3 head dimension
64
64
64
64
64
Variate-attention heads
4
4
4
4
4
Appendix
Table 1: ci-proto model configurations. Parameter counts are reported in millions.
Setting
Value
Optimiser and coefficients
AdamW; β1 cycled between 0.95 and 0.85 by the one-cycle schedule ( 0.85 at the peak learning rate), β2=0.999 , ϵ=10−8
Learning rate and schedule
Cosine one-cycle schedule with 5,000 warm-up steps; peak LR 1.2×10−3 (3.5M–28.6M) and 6×10−4 (82.5M, 167.2M); initial LR =peak/50 , final LR =initial/104
Effective batch size
256 sampled time series groups per optimiser step
Training steps per variant
330,000;467,500;700,000;1,045,000;1,358,500
Weight decay and gradient clipping
Weight decay 0.01 , excluding Mamba parameters marked for no decay; elementwise gradient clipping to [−1,1]
Numerical precision
BF16 mixed precision with FP32 parameters
Appendix
Table 2: Pre-training settings for ci-proto. Size-dependent values follow the order 3.5M, 9.2M, 28.6M, 82.5M, and 167.2M.
Figure 7: Mean absolute scaled error (MASE) on fev-bench versus parameter count. Lower MASE is better. The horizontal axis is in logarithmic scale. The dashed line marks the Pareto frontier formed by the five ci-proto variants.
Figure 8: Mean absolute scaled error (MASE) on fev-bench versus median runtime per 100 forecasts for selected leaderboard models. We benchmark all models on identical hardware and under the same execution settings, since self-reported leaderboard timings are not directly comparable. We report the mean and standard deviation across five runs. Lower MASE is better. The horizontal axis is in logarithmic scale. The dashed line marks the Pareto frontier formed by the five ci-proto variants.
Figure 9: One minus the skill score for the scaled quantile loss (SQL) on fev-bench versus parameter count. Lower values are better. The horizontal axis is in logarithmic scale. The dashed line marks the Pareto frontier where all five ci-proto variants lie.
Figure 10: One minus skill score for the scaled quantile loss (SQL) on fev-bench versus median runtime per 100 forecasts for selected leaderboard models. We benchmark all models on identical hardware and under the same execution settings, since self-reported leaderboard timings are not directly comparable. We report the mean and standard deviation across five runs. Lower values are better. The horizontal axis is in logarithmic scale. The dashed line marks the Pareto frontier formed by the five ci-proto variants.
Figure 11: One minus the skill score for the mean absolute scaled error (MASE) on fev-bench versus parameter count. Lower values are better. The horizontal axis is in logarithmic scale. The dashed line marks the Pareto frontier where all five ci-proto variants lie.
Figure 12: One minus skill score for mean absolute scaled error (MASE) on fev-bench versus median runtime per 100 forecasts for selected leaderboard models. We benchmark all models on identical hardware and under the same execution settings, since self-reported leaderboard timings are not directly comparable. We report the mean and standard deviation across five runs. Lower values are better. The horizontal axis is in logarithmic scale. The dashed line marks the Pareto frontier formed by the five ci-proto variants.
Figure 13: Quantile loss over a scheduled power outage for models with their released context cap. The outage has a single earlier occurrence at the given number of steps back as shown on the x-axis. Each model receives the smallest of power-of-two context 2k≥2,048 steps that contains the earlier outage, or all 2,048,000 steps of history if needed. ci-proto XL and the uncapped baselines read the whole context, with all other settings matching each model’s fev-bench submission. The curves show the mean and standard deviation over 40 forecast windows (lower is better). Here each step is one minute, and the top axis gives the steps as units of time.
Time-series foundation models are pretrained sequence models designed to transfer across forecasting, classification, anomaly detection, and other temporal tasks.
Learning time series foundation models has been shown to be a promising approach for zero-shot time series forecasting across diverse time series domains. Insofar as scaling has been a critical driver of performance of foundation models in other modalities such as language and vision, much recent work on time series foundation modeling has focused on scaling. This has resulted in time series foundation models with hundreds of millions of parameters that are, while performant, inefficient and expensive to use in practice. This paper describes a simple recipe for learning efficient foundation models for zero-shot time series forecasting that are orders of magnitude smaller. We show that large-scale transformers are not necessary: small hybrid models that interleave long convolution and linear RNN layers (in particular DeltaNet layers) can match the performance of larger transformer-based models while being more than a hundred times smaller. We also describe several data augmentation and inference strategies that further improve performance. This recipe results in Reverso, a family of efficient time series foundation models for zero-shot forecasting that significantly push the performance-efficiency Pareto frontier.
Xinghong Fu, Yanhong Li, Georgios Papaioannou +1
Massachusetts Institute of Technology · Allen Institute for AI · Qube Research & Technologies
We introduce Timer-S1, a strong Mixture-of-Experts (MoE) time series foundation model with 8.3B total parameters, 0.75B activated parameters for each token, and a context length of 11.5K. To overcome the scalability bottleneck in existing pre-trained time series foundation models, we perform Serial Scaling in three dimensions: model architecture, dataset, and training pipeline. Timer-S1 integrates sparse TimeMoE blocks and generic TimeSTP blocks for Serial-Token Prediction (STP), a generic training objective that adheres to the serial nature of forecasting. The proposed paradigm introduces serial computations to improve long-term predictions while avoiding costly rolling-style inference and pronounced error accumulation in the standard next-token prediction. Pursuing a high-quality and unbiased training dataset, we curate TimeBench, a corpus with one trillion time points, and apply meticulous data augmentation to mitigate predictive bias. We further pioneer a post-training stage, including continued pre-training and long-context extension, to enhance short-term and long-context performance. Evaluated on the large-scale GIFT-Eval leaderboard, Timer-S1 achieves state-of-the-art forecasting performance, attaining the best MASE and CRPS scores as a pre-trained model. Timer-S1 is released to facilitate further research.
Time series foundation models must preserve multi-domain breadth, probabilistic output, and multiple temporal scales, but parameter count grows when each scale receives a separate representation. We introduce Fracast-0, a probabilistic forecasting foundation model that exploits temporal self-similarity to reuse one operator across scales. A parameter-free detector extracts significant seasonal structure. The encoder applies a shared local block along a geometric dilation ladder with scale conditioning, while the decoder combines context-gathered states with an explicit seasonal future state and reuses a second block along another ladder before emitting nine quantiles. Pretraining across six corpora preserves multi-domain breadth within 85,001 parameters. On 97 GIFT-Eval configurations without per-dataset fine-tuning, Fracast-0 is the smallest of 28 evaluated checkpoints and remains non-dominated in the aggregate parameter-accuracy plane with MASE 0.808 and WQL 0.564. It uses 42.0% fewer parameters than TinyCast, whose MASE and WQL are 4.2% and 3.3% lower. These results support cross-scale weight reuse as a practical route to further time series foundation model compression.
Tianxiang Zhan, Huanyao Zhang, Yuanpeng He
University of Electronic Science and Technology of China · Peking University