Cold-start forecasting, the task of forecasting a time series with little to no historical data, is a common challenge. Addressing it requires approaches that learn quickly from few datapoints and leverage information from related series, typically through static covariates, to generalize well to unseen series. While some global forecasting models can generate cold-start predictions by leveraging information shared across multiple series, they are not optimized for out-of-train-set generalization or adaptation from short histories. In this work, we formulate cold-start forecasting as a few-window learning problem and introduce Temporal Model-Agnostic Meta-Learning (TMAML), which tailors the model-agnostic meta-learning algorithm, originally developed for few-shot adaptation of neural networks, to deep time series forecasting. TMAML constructs meta-tasks as temporally consistent support-query windows and pairs them with a temporal meta-training and meta-testing procedure, yielding forecasting models that are explicitly optimized for cold-start forecasting. We instantiate TMAML on the Temporal Fusion Transformer (TFT) and present an initial empirical analysis of forecast accuracy and calibration across three cold-start scenarios: TMAML consistently outperforms or matches a standard ERM-trained TFT, yields better-calibrated forecasts than naive on two of the three scenarios, but does not consistently outperform naive on probabilistic forecast accuracy.
Figures & tables
Figure 1: Illustration of the TMAML framework. The meta-window slides across the observed series to create multiple train meta-tasks, each consisting of a support sample and a query sample. The inner model is adapted on the support sample and evaluated on the query sample, whose loss is used to update the meta-model. The figure shows the TMAML process for K=2.
Figure 2: WQL (top) and MACE (bottom) vs. K , meta-test, for Favorita, M5, and Electricity, left to right. Shaded bands are 95% bootstrap CIs.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Illustration of the cold-start forecasting problem.
Favorita
M5
Electricity
Freq.
Daily
Daily
Hourly
H
28
28
24
Max K
3
3
7
Max L
84
84
168
Train Series
50
50
20
Validation Series
500
500
85
Appendix
Table 1: Dataset, temporal split, and series-count statistics for the three cold-start scenarios. H is the forecast horizon, K the largest number of support windows evaluated, and L=K×H the corresponding maximum lookback.
Favorita
M5
Electricity
Meta-train
2015-01-02–2015-11-03
2011-01-30–2016-02-01
2014-01-01–2014-08-07
Meta-val
2015-09-09–2015-12-29
2015-12-08–2016-03-28
2014-08-01–2014-08-08
Meta-test
2015-10-07–2016-01-26
2016-01-05–2016-04-25
2014-08-02–2014-08-09
Appendix
Table 2: Calendar date range covered by each split. The meta-test window is the meta-validation window shifted forward by exactly H . Both begin before the meta-training cutoff, though never over the same series (see text).
Favorita
M5
Electricity
Known inputs x
day of month, month, store open flag, on promotion flag, day of week, national, regional, and local holidays
month, year, sell price, weekday, SNAP flag, event name and type
hour, day of week, hours from start
Observed inputs z
transactions, oil price
—
—
Static covariates s
series ID, family, perishable flag, class
series ID, department, category
series ID
Target (transform)
sales ( logy , floored at 10−8 )
sales ( log(1+y) )
power usage ( log(1+y) )
Appendix
Table 3: Input variables and target transformation per dataset. All static covariates include the anonymized series identifier in addition to the covariates listed.
Favorita
M5
Electricity
Architecture
Hidden size
240
240
160
Attention heads
4
4
4
Dropout
0.1
0.1
0.1
Quantile levels
0.1–0.9 (step 0.1)
ERM
Appendix
Table 4: Architecture and per-dataset training hyperparameters for ERM and TMAML. α / M = inner learning rate / steps, β = outer learning rate, B = meta-batch size.
Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.
Despite the success of Time Series Foundation Models (TSFMs) on broad benchmarks, their ability to internalize basic temporal logic, especially in settings supported by exogenous covariates, remains under-examined. We introduce SimpleTimeBench, a diagnostic univariate and multivariate "unit test" suite for primitives such as monotonic trends, periodic signals and leading indicator covariates, scenarios where near-perfect forecasts should be trivial. Surprisingly, prominent multivariate TSFMs (Chronos-2, Moirai and Toto) frequently produce suboptimal zero-shot forecasts for these inputs. While fine-tuning Chronos-2 improves its behaviour on specific tasks, we show that this adaptation degrades performance on other fundamental patterns rather than enhancing its generalizable foundational capabilities. This reveals a gap between pre-training scale and basic temporal reasoning, suggesting that current TSFMs could potentially lack the inductive biases needed to capture simple predictable functions. We further demonstrate that these failures are not merely synthetic curiosities: they persist in real-world sensor forecasting, where TSFMs consistently underutilize leading indicators available in observed covariates. This inability to capture simple relationships limits the practical utility and reliability of current multivariate models.
We introduce Timer-M1, a pretrained multivariate time series foundation model that learns with primitives for zero-shot forecasting. Across domains, time series share elementary temporal and relational patterns, termed primitives, yet differ in how these primitives manifest and evolve across different contexts. Despite progress in zero-shot and task-general forecasting, existing foundation models may still struggle to generalize to complex real-world scenarios. To this end, we develop a primitive-based data synthesis and pretraining pipeline. The synthesis pipeline generates series with temporal primitives shared across domains and then assembles real and generated series into multivariate samples using relational primitives. Afterwards, samples are organized into episodes by assigning distinct channel roles as target variates, past-only covariates, and known-future covariates, ensuring that the model is optimized on predictable variates using available exogenous information. Technically, Timer-M1 further adapts gated two-dimensional Transformer blocks that dynamically allocate cross-variate attention across layers. Across three large-scale forecasting benchmarks, Timer-M1 ranks first on both FEV and TIME and second on GIFT-Eval among most recent time series foundation models. These results support effective primitive-based pretraining as a route to robust general forecasting technique across domains and task settings.
Haoran Zhang, Haixuan Liu, Xingjian Su +5
School of Software, BNRist, Tsinghua University · ByteDance, Beijing, China
Time series forecasting (TSF) is evolving toward multimodal and agentic settings, yet using foundation models remains uneconomical in resource-constrained scenarios, where compact, specialized forecasters are more desirable. However, lightweight forecasters typically require substantial training data, limiting their use in domains with scarce, slowly accumulated, or privacy-sensitive time series. To address this dilemma, we investigate the challenging problem of few-shot learning for lightweight forecasters. We propose MetaCaster, a meta-harness-optimized multi-agent framework that uses agentic data generation to automatically train specialized lightweight forecasters from only a few examples and textual contexts. Our work highlights a new TSF paradigm in which agents act not as forecasters but as intermediary engineers that prepare efficient, task-specific forecasters for deployment. Experiments on 18 datasets, 23 state-of-the-art lightweight forecasters, and 14 baselines demonstrate that MetaCaster achieves both data efficiency and computational efficiency while maintaining high-quality TSF performance.
ChengAo Shen, Wenchao Yu, Fangyu Wu +6
University of Houston · NEC Labs · University of Waterloo +3