cs.AIOct 27, 2026
SaveTimestral
Abstract
We introduce Timestral, a family of four time series foundation models for probabilistic forecasting, with 5M, 28M, 123M, and 209M parameters. Timestral sets a new state of the art on GIFT-Eval, TIME, and fev-bench, where Timestral-209M and Timestral-123M take the top two places on every metric. We achieve this with three ingredients: Decoupled Horizon Forecasting (DHF), which separates prediction tokens from the causally encoded context, so that longer horizons are trained without masking context and each observed value is both a forecast target and context for later forecasts; a covariate framework that natively supports known-future covariates within a causal architecture; and a two-stage pipeline that pretrains exclusively on a diverse, large-scale synthetic corpus before post-training on a mix of real and synthetic data with extended context. This report describes the model design and its benchmark results. Weights for all four models are available on Hugging Face and inference code on GitHub.
Figures & tables
Figure 1: Forecast error against parameter count on GIFT-Eval, TIME, and fev-bench (lower is better; fev-bench skill scores are shown as one minus the skill score). The dashed and solid curves show the Pareto fronts without and with Timestral, respectively. All four Timestral models lie on the Pareto front across the six panels.
Figure 2: Temporal information flow during training. Input tokens enter the Transformer after embedding; output tokens are final Transformer representations supplied to the prediction head. Only outputs used for forecasting are shown. The index of denotes the forecast patch , and numbers above it give forecast distances in patches. (a) Context tokens and prediction tokens interact bidirectionally, producing at the future positions. (b) Causal attention computes at position for one-patch-ahead forecasting. Normalisation, embedding, the prediction head, and denormalisation are omitted here and shown in Figure 4 .
Figure 3: Causal decoder-only model with cpm : training and forecasting, using the token-level convention of Figure 2 . Each output is computed at position and passed to the prediction head for patch . (a) Masking patch makes serve a two-patch-ahead forecast from the context ending at patch . After visible patch , masked patches and allow forecasts for patches at distances . Their observed context contains patches , but not masked patch . (b) Context tokens followed by yield the three required outputs from positions .
Figure 4: Timestral overview: target variates, past-only covariates, and known-future covariates. Input boxes show target values and covariate values in patches of consecutive time steps. Within the Transformer, context and prediction tokens form two separate streams ( Section 3.1 ): , , and denote the context tokens of patch of a target variate, a past-only covariate, and a known-future covariate, respectively, and denotes the prediction token at the temporal position of patch . Every variate has a prediction token for each forecast patch, but only known-future covariates also have context tokens in the forecast horizon. Only the outputs at the prediction tokens of target variates are passed to the quantile head. Stacked boxes represent multiple variates.
Figure 5: Timestral training with dhf . Shared context and independent forecast blocks, with the corresponding attention mask above. Columns index queries and rows index keys. Context tokens attend causally to the context and never to prediction tokens; prediction tokens attend only to the context before their forecast block and to preceding and current prediction tokens in the same block. For readability, the red context-to-horizon arrows are drawn short: each stands for a prediction token’s attention to all context tokens before its block, as given by the red cells in the mask above. Output tokens are passed to the prediction head; numbers above them give forecast distances in patches. The blocks match the forecast distances of Figure 3(a) , but no context patch is masked to create them. Missing-data augmentation of the context is applied independently of the forecast block lengths.
| Model | Blocks | Heads | SwiGLU width | Parameters | |
|---|---|---|---|---|---|
| Timestral-5M | 192 | 8 | 3 | 512 | 5.2M |
| Timestral-28M | 384 | 12 | 6 | 1024 | 27.8M |
| Timestral-123M | 640 | 20 | 10 | 1712 | 122.6M |
| Timestral-209M | 768 | 24 | 12 | 2048 | 209.3M |
Table 1: Timestral model sizes. All models use a head dimension of 64, sigmoid-gated attention, and variate attention in every fourth block.
Figure 6: GIFT-Eval results for all four Timestral models and the ten leading competitors by aggregate mase . Both panels show the same models, ranked separately by each metric. Lower is better.
Figure 7: TIME results for all four Timestral models and the ten leading competitors by aggregate mase . Both panels show the same models, ranked separately by each metric. Lower is better on both metrics.
Figure 8: fev-bench results for all four Timestral models and the ten leading competitors by MASE skill score. Both panels show the same models, ranked separately by each metric. Higher is better for both MASE and sql skill scores.
| CPU | GPU | |||||
|---|---|---|---|---|---|---|
| Model | Parameters | Checkpoint size | 8 threads | 1 thread | series/s | series/s |
| Timestral-5M | 5.2M | 21 MB | 17 ms | 16 ms | 106 | 18,770 |
| Timestral-28M | 27.8M | 111 MB | 31 ms | 56 ms | 24 | 10,730 |
| Timestral-123M | 122.6M | 490 MB | 85 ms | 210 ms | 5.9 | 4,990 |
| Timestral-209M | 209.3M | 837 MB | 113 ms | 345 ms | 3.5 | 3,650 |
Table 2: Inference latency and throughput for forecasting 96 steps of a univariate series without covariates from a context of 2,048 steps, with symmetric averaging. CPU latency is the median of 50 runs at batch size 1. CPU throughput is per core at batch size 16, and GPU throughput is on one NVIDIA H100 at batch size 256. Checkpoint sizes are for float32 weights.
Figure 9: CPU latency per forecast against context length (left, one variate) and number of target variates (right, context of 2,048 steps), on 8 threads with the settings of Table 2 (median of 20 runs). Hollow markers: one target with seven known-future covariates. Both axes are logarithmic.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
| GIFT-Eval | TIME | fev-bench | ||||
| Model | mase | crps | mase | crps | mase skill score (%) | sql skill score (%) |
| Timestral-209M | 0.6575 | 0.4460 | 0.6262 | 0.5223 | 39.49 | 50.39 |
| Timestral-123M | 0.6589 | 0.4476 | 0.6277 | 0.5253 | 39.19 | 50.18 |
| Timestral-28M | 0.6780 | 0.4610 | 0.6382 | 0.5344 | 37.27 | 48.42 |
| Timestral-5M | 0.7049 | 0.4789 | 0.6518 | 0.5445 | 34.54 | 46.19 |
| Chronos-2 | 0.6978 | 0.4854 | 0.6620 | 0.5563 | 35.50 | 47.28 |
Table 3: Aggregate results for leading models on GIFT-Eval, TIME, and fev-bench. GIFT-Eval and TIME report geometric-mean errors relative to Seasonal Naive (lower is better); fev-bench reports MASE and sql skill scores in percent (higher is better). sql scales quantile loss by the context’s mean absolute seasonal error. Bold and underlined scores denote first and second place before rounding; dashes indicate missing scores. For TiRex-2, PT and ZS denote pretrained and zero-shot GIFT-Eval submissions, respectively; its fev-bench entry uses a benchmark-specific checkpoint.
Figure 10: IND-Eval aggregate nmase (left) and ncrps (right) for all four Timestral sizes and eleven competitors. All results are from our evaluations using public checkpoints and inference implementations under the official benchmark protocol. Both panels show the same models, ordered by their respective dataset-balanced geometric-mean error. Lower is better.
Figure 11: GIFT-Eval pairwise win rates for mase (top) and crps (bottom) across 97 dataset–frequency–horizon configurations. Rates above 50% favour the row model. crps uses the official nine-quantile approximation; TiRex-2 is the pretrained submission. Both panels order comparators by increasing aggregate mase .
Figure 12: TIME pairwise win rates for mase (top) and crps (bottom) across 98 dataset–frequency–horizon configurations. Task errors follow the official leaderboard, averaging over series, forecast windows, and variates. Configurations are equally weighted, with ties counted as half a win; rates above 50% favour the row model. Both panels order comparators by increasing aggregate mase .
Figure 13: fev-bench pairwise win rates for mase (top) and sql (bottom) across 100 official tasks. sql scales quantile loss by the context’s mean absolute seasonal error. Tasks are equally weighted, with ties counted as half a win; rates above 50% favour the row model. Both panels order comparators by decreasing MASE skill score.
Figure 14: IND-Eval pairwise win rates for nmase (top) and ncrps (bottom) across 187 forecasting tasks: all four Timestral sizes versus the eleven competitors in Figure 10 . Errors come from our evaluations under the official protocol. Tasks are equally weighted, with ties counted as half a win; aggregate errors instead weight datasets equally. Rates above 50% favour the row model. Both panels order comparators by increasing aggregate nmase .
More in this topic
Time Series Forecasting
Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.
See all 861 papers