Tabby: An Open Pretraining Recipe for Time Series Foundation Models

Shifeng Xie; Bahaeddine Abdessalem; Zehao Xiao; Youssef Attia El Hili; Ambroise Odonnat; Zhiwei Dong; Lei Zan; Themis Palpanas

cs.LGSep 12, 2026

Tabby: An Open Pretraining Recipe for Time Series Foundation Models

Authors: Shifeng Xie, Bahaeddine Abdessalem, Zehao Xiao, Youssef Attia El Hili, Ambroise Odonnat, Zhiwei Dong, Lei Zan, Themis Palpanas,

Organizations: Huawei Noah’s Ark Lab, Paris, France · LIPADE, Université Paris Cité, Paris, France · École Polytechnique, France · Centre de Recherche en Informatique, Mines Paris, PSL University, France ·

Abstract

In this report, we release Tabby, a long context probabilistic time series foundation model, together with a complete and open recipe of how it was built. Tabby adopts an encoder-only patch Transformer architecture and concentrates the contributions on the data and the training procedure. The pretraining corpus combines an extended real-world collection, GIFT-Eval-Pretrain+ and BLAST, with synthetic data from KernelSynth and CauKerV2, an online generator that composes temporal dynamics through randomly sampled structural causal models. Training couples a progressive convergence schedule, which yields reusable intermediate checkpoints, with a deep quantile supervision objective for intermediate layers. The resulting 145M parameter backbone supports contexts of up to 8,192 observations and serves forecasting, classification, and anomaly detection, while a prompt-tuning module further improves in-distribution forecasting performance with the pretrained weights frozen. Tabby achieves competitive zero-shot forecasting performance on GIFT-Eval and the out-of-distribution TIME benchmark, while the same pretrained backbone also supports classification on the UCR Archive and zero-shot anomaly detection on TSB-AD-U. We release training pipeline and model as open source at huawei-noah/trustworthyAI.

Figures & tables

Figure 1: Overview of Tabby . The model is pretrained on real and synthetic time series using deep quantile supervision and progressive convergence learning rate schedule, and supports forecasting, classification, and anomaly detection.

Generator family	Temporal characteristics	Representative domains
Compositional Kernel Gaussian Process	Smoothness and local correlations	Weather, energy, environmental sensing
Trend Seasonality Decomposition	Trends with additive or multiplicative seasonality	Retail, traffic, electricity demand
Autoregressive Integrated Moving Average	Linear dependence, integration, and seasonal autoregression	Economics, demand, climate
Regime-Switching Ornstein–Uhlenbeck Stochastic Differential Equation	Mean reversion, stochastic fluctuations, and regime transitions	Finance, physical systems
Piecewise Level and Change-Point Process	Abrupt or smooth level shifts and piecewise-stationary behavior	Industrial monitoring, operations
Spike and Event Process	Spikes, bursts, shocks, plateaus, and recovery patterns	Web traffic, energy, healthcare

Table 1: Primitive generator families used by CauKerV2. The listed application domains are representative rather than exhaustive.

Figure 2: Forecasting performance on GIFT-Eval (ID) and TIME (OOD). Models are ordered by MASE within each benchmark.

	Catch22+	TabPFN	TabICL	MOMENT	TiVIT-H	NuTime	MantisV1	Tabby
UCR	0.7969	0.7806	0.7707	0.7789	0.7943	0.7732	0.7816	0.7999

Table 2: Mean classification accuracy over all 128 datasets in the UCR Archive.

	Lag-Llama	Chronos	TimesFM	MOMENT (ZS)	MOMENT (FT)	xLSTMAD	TSPulse	Tabby
TSB-AD-U	0.27	0.27	0.30	0.38	0.39	0.40	0.48	0.43

Table 3: Mean VUS-PR over the TSB-AD-U evaluation split. Baseline numbers are taken from the TSB-AD leaderboard.

Figure 3: MASE is evaluated on Gift-Eval benchmark. Each stable phase is followed by a decay phase, producing a locally converged intermediate checkpoint.

Schedule	MASE	CRPS
WSD-S	0.7551	0.5165
Cosine	0.7456	0.5112
PCS	0.7368	0.5038

Table 4: Learning rate ablation.

Model	Start	$G_{0}$	$\Delta_{5\times 10^{-5}}$	$\Delta_{10^{-4}}$	$\Delta_{2\times 10^{-4}}$
Small	55K	0.09618	+0.42%	+1.15%	+2.49%
Small	110K	0.09464	+0.63%	+1.58%	+3.22%
Main	55K	0.09031	+0.44%	+1.25%	+2.65%
Main	110K	0.08856	+0.60%	+1.53%	+3.60%
Main	165K	0.08793	+0.69%	+1.57%	+3.63%

Table 5: Sensitivity to restart peak learning rates.

G_{0}

is the validation loss obtained with

\eta_{\mathrm{peak}}=10^{-5}

; the remaining columns report the relative increase over

G_{0}

Setting	MASE	CRPS
Without DQS	0.7476	0.5117
With DQS	0.7368	0.5038

Table 6: GIFT-Eval DQS ablation.

	$\Delta$ MASE (%)		$\Delta$ CRPS (%)
Skipped layer	w/ DQS	w/o DQS	w/ DQS	w/o DQS
6	+1.13	+0.11	+0.91	-0.05
7	+1.52	+0.09	+1.07	-0.14
8	+1.63	+0.93	+1.45	+0.92

Table 7: Functional ablation of intermediate blocks on GIFT-Eval. Positive values indicate degradation.

Radius $\rho$	B over A (%)	B over norm-matched M (%)
$0.25$	$0.00712$ $[0.00112,\,0.01499]$	$0.00663$ $[0.00124,\,0.01373]$
$0.50$	$0.01399$ $[0.00253,\,0.02873]$	$0.01296$ $[0.00251,\,0.02594]$
$1.00$	$\mathbf{0.02698}$ $[0.00470,\,0.05446]$	$\mathbf{0.02477}$ $[0.00485,\,0.04930]$

Table 8: Optimizer-aware one-step DQS intervention. Entries are the macro-average relative reduction in final validation pinball loss; a positive value favors the DQS branch B. Parentheses give 95% confidence intervals from paired original-dataset cluster bootstrap.

Setting	MASE	CRPS
Without CauKerV2	0.7536	0.5109
With CauKerV2	0.7368	0.5038

Table 9: CauKerV2 ablation.

Max. patches	MASE	CRPS
32	0.7473	0.5116
2	0.7368	0.5038

Table 10: Terminal mask ablation.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Value
Complete model window $L$	$8192$
Patch length $P$	$16$
Number of patch tokens $N=L/P$	$512$
Transformer layers $M$	$20$
Model dimension $d$	$768$
Attention heads $n_{h}$	$12$

Appendix

Table 11: Architecture configuration of Tabby .

Component	Parameters	Share
Input residual projection	$641{,}280$	$0.44\%$
Learned positional embedding	$393{,}216$	$0.27\%$
$20$ Transformer blocks	$141{,}757{,}440$	$97.20\%$
Output residual projection	$3{,}051{,}392$	$2.09\%$
Total	$145{,}843{,}328$	$100.00\%$

Appendix

Table 12: Parameter decomposition of Tabby .

Training sampling unit	Constituent source datasets	Rows
energy_covariates_small	cockatoo , covid19_energy , gfc14_load , pdb , spain , and spanish_energy_and_weather	129
macro_finance_small	bitcoin_with_missing , exchange_rate , and monash_fred_md	321
public_health_small	cdc_fluview_ilinet , cdc_fluview_who_nrevss , and covid_mobility	1,304
recovered_useful_real	HZMETRO , SHMETRO , borealis , elecdemand , elf , smart , solar_power , and wind_power	4,096

Appendix

Table 13: Source datasets consolidated into shared training sampling units. The row count is the number of stored Hugging Face samples in the composite unit.

Code	Collection component	Entries	Samples	Channels	Observations
G	Stored GIFT-Eval subset	120	3,782,761	28,855,468	228,829,433,709
T	TiRex-2 supplement	21	5,474	12,603	943,028,306
R	Restored curated subset	9	4,203	4,203	29,712,828
Total	GIFT-Eval-Pretrain+	150	3,792,438	28,872,274	229,802,174,843

Appendix

Table 14: Composition of GIFT-Eval-Pretrain+ by collection source. Mean length is computed as total observations divided by total channel sequences.

Dataset	Src.	Domain	Freq.	Type	Samples	Channels	Mean length	Observations
BEIJING_SUBWAY_30MIN	T	Transport	30 min	M	276	552	1,572.00	867,744
PEMS04	T	Transport	5 min	M	307	921	16,992.00	15,649,632
PEMS08	T	Transport	5 min	M	170	510	17,856.00	9,106,560
beijing_air_quality	T	Environment	1 h	M	12	132	35,064.00	4,628,448
china_air_quality	T	Environment	1 h	M	437	2,622	13,133.26	34,435,404
cockatoo	T	Energy	1 h	U	6	6	17,544.00	105,264

Appendix

Table 15: Complete dataset-level inventory of GIFT-Eval-Pretrain+. Source codes follow Table 14 . Samples are stored rows; channels are univariate sequences after expanding the target dimension; mean length is

P_{d}/C_{d}

; and observations count every scalar value across all channels.

Configuration	Value
Model architecture
Context length	8,192
Patch size	16
Maximum number of patch tokens	512
Number of Transformer layers	12
Hidden dimension	512

Appendix

Table 16: Complete architecture, data, masking, supervision, and optimization configuration of Tabby-Small .

Parameter	Default	Meaning
context_length	2,048 $\times$ 4	Padded model input length
min_sample_length	96	Minimum sampled training length
max_sample_length	8,192	Maximum sampled training length

Appendix

Table 17: KernelSynth reader defaults. These are implementation defaults; final launch-time overrides are not shown.

Parameter	Default	Meaning
context_length	2,048	Padded model input length
$L_{\min}$ : min_length	96	Minimum generated length
$L_{\max}$ : max_length	2,048	Maximum generated length
$P$ : num_features	8	Observed SCM channels
$K$ : num_nodes	8	Primitive root series (latent dimension)
$M$ : max_parents	6	Maximum parents of an observed node

Appendix

Table 18: Constructor defaults of OnlineCauKerIterableDataset . Values passed by the final launch command override this table.

Generator family	Weight
Compositional-kernel Gaussian process	0.10
Trend–seasonality decomposition	0.10
Regime-switching Ornstein–Uhlenbeck SDE	0.10
ARIMA/SARIMA state-space process	0.10
Piecewise level and change-point process	0.10
Spike and event process	0.30

Appendix

Table 19: Default primitive-family weights used by CauKerV2. The weights sum to one.

Family	Implemented distributions and choices
Compositional-kernel GP	Select $m\sim\operatorname{UnifInt}\{1,\ldots,7\}$ kernels with replacement from a 33-entry bank. Periodic kernels use normalized periods $\{24,48,96,168,336,672,7,14,30,60,365,730,4,26,52,4,6,12,4,40,10\}/L$ ; dot-product kernels use $\sigma_{0}\in\{0,1,10\}$ ; RBF length scales are $\{0.1,1,10\}$ ; rational-quadratic $\alpha$ values are $\{0.1,1,10\}$ ; white-noise levels are $\{0.1,1\}$ ; and one constant kernel is included. Successive kernels are combined by addition or multiplication with probability $0.5$ each. Two mean functions are sampled with replacement from zero, linear, exponential, and sparse-anomaly means and are also added or multiplied with probability $0.5$ . Linear slopes/intercepts follow $\mathcal{U}(-1,1)$ ; exponential amplitude/rate follow $\mathcal{U}(0.5,1.5)$ ; the anomaly mean contains 1–5 impulses with amplitudes $\mathcal{U}(-5,5)$ . GP covariance jitter is $10^{-6}$ .
Trend–seasonality	Trend type probabilities for none, linear, quadratic, exponential, damped, and piecewise trends are $(0.08,0.27,0.20,0.15,0.15,0.15)$ . Linear slopes are $\mathcal{U}(-3,3)$ ; quadratic coefficients use $\mathcal{U}(-4,4)$ and $\mathcal{U}(-2,2)$ ; exponential rate is $\mathcal{U}(-3,3)$ with amplitude $\mathcal{U}(0.3,2)$ ; damped rate is $\mathcal{U}(1,8)$ . The number of seasonal components is uniform in $\{0,1,2,3\}$ , with candidate periods $\{4,6,7,12,24,48,52,96,168,336\}$ , amplitude $\mathcal{U}(0.2,2)$ , and phase $\mathcal{U}(0,2\pi)$ . Seasonal waveform probabilities for sine, triangle, step, and impulse are $(0.55,0.20,0.15,0.10)$ . Irregular-component probabilities for white, colored, random-walk, and fBm-like noise are $(0.35,0.35,0.15,0.15)$ . Additive, multiplicative, and mixed composition probabilities are $(0.55,0.20,0.25)$ .
ARIMA/SARIMA	Burn-in is 300. AR and MA orders are sampled from $\{0,1,2,3\}$ with probabilities $(0.1,0.3,0.4,0.2)$ ; if both are zero, $p$ is reset to one. Differencing order $d\in\{0,1,2\}$ has probabilities $(0.50,0.40,0.10)$ . AR and MA coefficients follow $\mathcal{U}(-0.8,0.8)$ and are rescaled when their $\ell_{1}$ norm exceeds 0.92. Innovation scale follows $\operatorname{LogU}(0.02,1)$ . A seasonal AR term is added with probability 0.35, with lag in $\{4,7,12,24,52,96\}$ and coefficient $\mathcal{U}(-0.5,0.5)$ .
Regime-switching OU SDE	The number of regimes is 2, 3, or 4 with probabilities $(0.50,0.30,0.20)$ . The base mean follows $\mathcal{U}(-2,2)$ ; signed regime offsets have magnitudes $\mathcal{U}(0.5,4)$ ; mean-reversion rates follow $\mathcal{U}(0.03,3)$ ; and diffusion scales follow $\mathcal{U}(0.03,1.5)$ . Transition rows are Dirichlet distributed with diagonal concentration $\mathcal{U}(10,50)$ . A sinusoidal perturbation of the regime mean is enabled with probability 0.4, with amplitude $\mathcal{U}(0,0.8)$ and period $\mathcal{U}(24,\max(25,L/2))$ .
Piecewise level/change point	The segment count is a clipped geometric draw, $\min\{\max(2,G+1),\max(3,\lfloor L/8\rfloor)\}$ with $G\sim\operatorname{Geometric}(0.15)$ . Segment proportions are Dirichlet with concentration $\mathcal{U}(0.5,2)$ . Level modes uniform, random walk, and clustered have probabilities $(0.35,0.40,0.25)$ . Transition modes hard, ramp, and sigmoid have probabilities $(0.50,0.30,0.20)$ ; smooth-transition width is $\mathcal{U}(0.01L,0.05L)$ . White noise is added with probability 0.70 and scale $\mathcal{U}(0.02,0.30)$ ; an additional trend–seasonality component, scaled by 0.3, is added with probability 0.35.
Spike/event	Baseline probabilities for kernel GP, trend–seasonality, flat noise, and OU are $(0.25,0.45,0.15,0.15)$ . The event count is a clipped Poisson draw with rate $\max(1,L/160)$ . Event-mode probabilities for point, Gaussian, plateau, shock–recovery, and periodic events are $(0.25,0.30,0.20,0.15,0.10)$ . Signed event amplitudes have magnitude $\operatorname{LogU}(0.8,6)$ . Periodic-event spacing is sampled between 16 and approximately $L/4$ ; plateau width is sampled between 2 and approximately $L/20$ ; shock–recovery width is sampled between 4 and approximately $L/12$ . Final noise scale follows $\mathcal{U}(0.01,0.25)$ .

Appendix

Table 20: Code-level hyperparameter distributions for the ten CauKerV2 primitive generators.

\mathcal{U}(a,b)

denotes a continuous uniform distribution and

\operatorname{LogU}(a,b)

a log-uniform distribution.

Component	Distribution or value
Root scale $a_{j}$	$\operatorname{LogU}(0.6,1.8)$
Node noise scale	$\operatorname{LogU}(0.01,0.15)$
Edge weight $w_{jp}$	$\mathcal{N}(0,1)$
Activation family	Uniform over linear, ReLU, sigmoid, sine, modulo, leaky-ReLU
Linear slope	$\mathcal{U}(0.5,2)$ ; intercept fixed to zero
Modulo constant	$\mathcal{U}(1,5)$

Appendix

Table 21: Fixed hyperparameter distributions of the online CauKerV2 SCM.

	$\Delta$ MASE (%)		$\Delta$ CRPS (%)
Skipped layer	w/ DQS	w/o DQS	w/ DQS	w/o DQS
0	+25.31	+20.94	+24.02	+20.14
1	+7.67	+9.93	+7.54	+9.02
2	+5.86	+4.41	+4.82	+4.57
3	+1.82	+1.69	+2.20	+1.49
4	+1.31	+1.34	+1.54	+1.47

Appendix

Table 22: Layer-wise functional ablation on GIFT-Eval with and without DQS. Values are percentage changes relative to each checkpoint’s intact baseline; positive values indicate degradation. Each row bypasses one block without fine-tuning. Both training variants cover

97

configurations.

Table 26

$\rho$	$\Delta_{BA}$	$\Delta_{BM}$	$G_{B}$
$0.25$	$4.04$ $[0.43,8.07]$	$4.14$ $[0.72,7.94]$	$14.74$ $[0.06,33.49]$
$0.50$	$8.05$ $[1.07,15.97]$	$8.25$ $[1.51,15.72]$	$28.83$ $[-0.09,65.58]$
$1.00$	$15.99$ $[1.74,31.48]$	$16.37$ $[2.98,30.96]$	$55.19$ $[-0.52,124.86]$

Appendix

Table 25: Raw one-step loss differences, in units of

10^{-6}

. Positive values favor B. Parentheses give 95% paired original-dataset cluster-bootstrap confidence intervals.

Dataset group	B–A	B–M	Dataset group	B–A	B–M
LOOP Seattle	$0.0136$	$0.0112$	ETT1	$0.0087$	$0.0085$
M-DENSE	$0.0365$	$0.0371$	ETT2	$-0.0210$	$-0.0200$
Bitbrains Fast	$-0.0190$	$-0.0125$	Jena Weather	$0.0138$	$0.0148$
Bitbrains RND	$0.0217$	$0.0188$	KDD 2018	$0.0023$	$0.0047$
BizITObs Application	$0.1843$	$0.1578$	M4 Daily	$-0.0237$	$-0.0246$
BizITObs L2C	$0.1056$	$0.0990$	SaugeenDay	$0.0421$	$0.0431$

Appendix

Table 26: Relative one-step advantages of B at the primary radius

\rho=1

, in percent. Positive values favor B. Results are first averaged over trials and configurations belonging to the same original dataset.

Parameter group	Alignment change	Parameter group	Alignment change
Position embedding	$0.00039$ $[-0.00096,0.00169]$	Block 7	$0.00079$ $[-0.00060,0.00226]$
Block 1	$0.00197$ $[-0.00015,0.00423]$	Block 8	$0.00071$ $[-0.00060,0.00204]$
Block 2	$0.00178$ $[-0.00144,0.00519]$	Block 9	$0.00193$ $[-0.00022,0.00415]$
Block 3	$0.00218$ $[-0.00036,0.00455]$	Block 10	$0$
Block 4	$0.00208$ $[-0.00103,0.00507]$	Block 11	$0$
Block 5	$0.00145$ $[-0.00190,0.00448]$	Block 12	$0$

Appendix

Table 27: Layerwise change in gradient alignment,

A_{j}

. Parentheses give 95% original-dataset cluster-bootstrap intervals. These correlated layerwise analyses are exploratory and are not corrected for multiple comparisons.

Configuration	Tabby-Small	Tabby
Parameters	40.291M	145.843M
Transformer layers	12	20
Hidden dimension	512	768
Attention-head dimension	32	64
Context length	8,192	8,192
Patch size	16	16

Appendix

Table 28: Models and checkpoint branches used in the restart-peak experiment. The continuation batch size is the effective batch size used by the single-GPU probe, rather than the global batch size of the original distributed pretraining run.

Model	Start	$10^{-5}$	$2.5{\times}10^{-5}$	$5{\times}10^{-5}$	$10^{-4}$	$2{\times}10^{-4}$
Small	55K	0.096181	0.096301	0.096583	0.097285	0.098573
Small	110K	0.094641	0.094830	0.095235	0.096137	0.097691
Main	55K	0.090308	0.090411	0.090707	0.091437	0.092702
Main	110K	0.088560	0.088746	0.089096	0.089913	0.091745
Main	165K	0.087934	0.088084	0.088539	0.089310	0.091123

Appendix

Table 29: Complete restart-peak response. Entries are weighted geometric terminal validation losses

G_{s}(\eta)

; lower is better. Bold values indicate the best candidate for each checkpoint.

Model	Start	$2.5{\times}10^{-5}$	$5{\times}10^{-5}$	$10^{-4}$	$2{\times}10^{-4}$
Small	55K	+0.125%	+0.418%	+1.148%	+2.487%
Small	110K	+0.200%	+0.627%	+1.580%	+3.223%
Main	55K	+0.115%	+0.442%	+1.251%	+2.651%
Main	110K	+0.210%	+0.605%	+1.528%	+3.596%
Main	165K	+0.170%	+0.688%	+1.565%	+3.626%

Appendix

Table 30: Relative validation-loss increase

P_{s}(\eta)

over the

10^{-5}

continuation. Positive values indicate degradation.

Model	Start	$S_{s}$	Change from previous stage
Small	55K	0.005449	–
Small	110K	0.007098	+30.2%
Main	55K	0.005845	–
Main	110K	0.007694	+31.6%
Main	165K	0.007820	+1.6%

Appendix

Table 31: Descriptive restart sensitivity measured over the complete learning-rate grid. Larger

S_{s}

indicates a steeper validation-loss response to increasing the restart peak.

Model	Start	BLAST	CauKerV2	GIFT	KernelSynth
Small	55K	+0.922%	+1.885%	+0.959%	+5.859%
Small	110K	+1.302%	+2.234%	+0.973%	+7.870%
Main	55K	+0.882%	+1.220%	+1.980%	+5.632%
Main	110K	+1.548%	+1.925%	+2.766%	+7.117%
Main	165K	+1.690%	+1.492%	+3.360%	+6.608%

Appendix

Table 32: Source-wise validation-loss increase under

\eta_{\mathrm{peak}}=2\times 10^{-4}

relative to the

10^{-5}

continuation.

Time-Series Foundation Models

Time-series foundation models are pretrained sequence models designed to transfer across forecasting, classification, anomaly detection, and other temporal tasks.

See all 301 papers

Aug 13, 2026

Into the ORBIT for Time Series: Training Regimes for Foundation Models

Time series foundation models (TSFMs) have advanced primarily through architectural innovation, while training regimes for large-scale heterogeneous corpora remain under-explored. As a result, pre-training distributions are often poorly controlled with respect to domain imbalance, context requirements, prediction horizons, and missingness. We introduce ORBIT (Omni-Range Bootstrap Incremental Training), a training paradigm that makes this distribution explicit and controllable. ORBIT combines Bootstrap Multi-Level Sampling, which controls dataset exposure and samples records, target variables, context windows, and prediction horizons, with Omni-Range Incremental Training, which varies context lengths and prediction horizons throughout a single training stage. Under ORBIT, we train Falcon-2.0, a simple univariate encoder-only Transformer with missingness-aware triple-channel patch tokenization and parallel patch prediction. We further introduce Rank-Guided Cross-Depth Alignment, a training objective that uses late-layer representations as stop-gradient teachers for shallow layers without additional inference cost. Evaluations on GIFT-Eval and fev-bench demonstrate strong zero-shot forecasting performance across diverse domains and frequencies.

Hongjie Xia, Yiding Liu, Yifan Hu +2

Zero-Shot Time Series Forecasting Self-Supervised Pre-Training Curriculum Learning Time-Series Foundation Models

Ask PDF

Mar 5, 2026

Timer-S1: A Billion-Scale Time Series Foundation Model with Serial Scaling

We introduce Timer-S1, a strong Mixture-of-Experts (MoE) time series foundation model with 8.3B total parameters, 0.75B activated parameters for each token, and a context length of 11.5K. To overcome the scalability bottleneck in existing pre-trained time series foundation models, we perform Serial Scaling in three dimensions: model architecture, dataset, and training pipeline. Timer-S1 integrates sparse TimeMoE blocks and generic TimeSTP blocks for Serial-Token Prediction (STP), a generic training objective that adheres to the serial nature of forecasting. The proposed paradigm introduces serial computations to improve long-term predictions while avoiding costly rolling-style inference and pronounced error accumulation in the standard next-token prediction. Pursuing a high-quality and unbiased training dataset, we curate TimeBench, a corpus with one trillion time points, and apply meticulous data augmentation to mitigate predictive bias. We further pioneer a post-training stage, including continued pre-training and long-context extension, to enhance short-term and long-context performance. Evaluated on the large-scale GIFT-Eval leaderboard, Timer-S1 achieves state-of-the-art forecasting performance, attaining the best MASE and CRPS scores as a pre-trained model. Timer-S1 is released to facilitate further research.

Yong Liu, Xingjian Su, Shiyu Wang +7

Tsinghua University · ByteDance

Time Series Forecasting Time-Series Foundation Models Long-Term Time Series Forecasting Sequence Modeling Mixture-of-Experts Models

Ask PDF

Jul 25, 2026

Foundation Models and Fine-Tuning: Toward a New Generation of Models for Time Series Forecasting

Inspired by recent breakthroughs in large language models for natural language processing, foundation models have emerged as a promising paradigm for zero-shot time series forecasting, enabling accurate predictions on datasets never seen during pre-training. Ranging from tens to hundreds of millions of parameters, these models are pre-trained on vast and diverse collections of time series, learning generalizable representations that support both point and probabilistic forecasting. This approach alleviates the need for dataset-specific model design and manual tuning, offering a unified solution across forecasting problems. In this work, we review the main architectures, pre-training strategies, and optimization methods underpinning these models. We further investigate post-pre-training fine-tuning of selected foundation models to enhance their performance on specific datasets. Our empirical results demonstrate that this step consistently improves forecasting accuracy over the zero-shot baseline.

Morad Laglil, Bertrand Pracca, Emilie Devijver +1

Savoye, 18 bd des Gorgets, 21000, Dijon, France · Univ. Grenoble Alpes, CNRS, Grenoble INP, LIG, Grenoble, 38000, France

Zero-Shot Time Series Forecasting Time-Series Foundation Models Probabilistic Time Series Forecasting

Ask PDF

Abstract

Time-Series Foundation Models

Explore similar work

Into the ORBIT for Time Series: Training Regimes for Foundation Models

Timer-S1: A Billion-Scale Time Series Foundation Model with Serial Scaling

Foundation Models and Fine-Tuning: Toward a New Generation of Models for Time Series Forecasting