Foundations without Fundamentals: Zero-Shot Blind Spots in Time Series FMs

Nafiseh Ghoroghchian; Haipeng Zhang; Shuyi Han; Alex Labach; George Stein

cs.LGOct 1, 2026

Foundations without Fundamentals: Zero-Shot Blind Spots in Time Series FMs

Authors: Nafiseh Ghoroghchian, Haipeng Zhang, Shuyi Han, Alex Labach, George Stein

Organizations: Layer6 AI

Abstract

Despite the success of Time Series Foundation Models (TSFMs) on broad benchmarks, their ability to internalize basic temporal logic, especially in settings supported by exogenous covariates, remains under-examined. We introduce SimpleTimeBench, a diagnostic univariate and multivariate "unit test" suite for primitives such as monotonic trends, periodic signals and leading indicator covariates, scenarios where near-perfect forecasts should be trivial. Surprisingly, prominent multivariate TSFMs (Chronos-2, Moirai and Toto) frequently produce suboptimal zero-shot forecasts for these inputs. While fine-tuning Chronos-2 improves its behaviour on specific tasks, we show that this adaptation degrades performance on other fundamental patterns rather than enhancing its generalizable foundational capabilities. This reveals a gap between pre-training scale and basic temporal reasoning, suggesting that current TSFMs could potentially lack the inductive biases needed to capture simple predictable functions. We further demonstrate that these failures are not merely synthetic curiosities: they persist in real-world sensor forecasting, where TSFMs consistently underutilize leading indicators available in observed covariates. This inability to capture simple relationships limits the practical utility and reliability of current multivariate models.

Figures & tables

Figure 1 : Examples of TSFM forecasts on a subset of SimpleTimeBench datasets. Surprising failures include the exponential dataset, and Leading Covariate tests where TSFMs perform no better than random even with a perfect leading indicator covariate.

Regime	Description	Prediction Difficulty	Distributions	#
Deterministic	Low-dimensional, non-repeating growth or states.	Trivial : Solvable via simple regression.	constant , linear , staircase , exponential	4
Periodic	Fixed-frequency oscillations and synthetic patches.	Low : Phase and amplitude estimation.	sine , sawtooth , square , triangle , noise_patch , fourier	6
Evolving Periodic	Signals with time-varying frequency or amplitude.	Moderate : Tracks rate-of-change parameters.	damped_sine , growing_sine , chirp	3
I.I.D. White Noise	Independent samples from a fixed distribution.	Impossible : Optimal forecast collapses to mean/median.	normal , uniform , poisson , binary , student_t , lognormal , laplace , cauchy , skew_normal	9
Stochastic Correlated	Stochastic processes that are not independent over time, including cumulative processes or sudden impulses.	Impossible : Path-dependent; no mean-reversion.	random_walk , piecewise_constant , gbm , intermittent , impulse , logistic	6

Table 1 : Taxonomy of Time Series Generation Regimes.

	MAPE			relMAE mean			relMAE last
	Base	Finetuned	Improvement	Base	Finetuned	Improvement	Base	Finetuned	Improvement
Univariate	$0.57$	$0.51$	$4.14\%$	$0.54$	$0.43$	$1.70\%$	$0.61$	$0.61$	$1.70\%$
Multivariate	$0.56$	$0.49$	$4.51\%$	$0.52$	$0.44$	$1.20\%$	$0.67$	$0.61$	$1.20\%$
Leading-Covariate	$0.65$	$0.57$	$6.86\%$	$0.50$	$0.41$	$1.73\%$	$0.57$	$0.57$	$1.73\%$
Median	$0.58$	$0.52$	4.62%	$0.52$	$0.42$	1.38%	$0.63$	$0.61$	1.38%

Table 2 : Performance improvements obtained by fine-tuning Chronos-2 in the All-in-One experiment. Each column is first computed separately for each distribution and then aggregated; therefore, the Improvement % column is not simply the difference between the preceding columns.

Figure 2 : Comparison of Chronos-2 vs. DLinear with covariate support on real-world datasets. Forecasts use an expanding-window strategy with horizons of

H=14

(Delaware River) and

H=16

(NVIDIA). While global performance looks similar, aggregate scales mask Chronos-2’s failure to leverage the covariate channel. Granular, zoomed-in comparisons of univariate vs. covariate-support modes demonstrating localized impact are provided in Figure 14 in Appendix D .

Model	MASE (vs uni.)	Win Rate	CRPS (vs uni. )	Win Rate
Chronos-2	0.77 (0.81)	62.5%	0.59 (0.62)	56.3%
Moirai	0.93 (0.93)	59.1%	0.75 (0.74)	50.0%
ToTo	0.72 (0.73)	100%	0.65 (0.66)	87.5%

Table 3 : GIFT-Eval: Multivariate vs. Univariate.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3 : Samples from all SimpleTimeBench distributions in Univariate generation mode. Colors represent different series. Series are min-max normalized for easier visualization. Two series are shown in each panel.

Figure 4 : Samples from all SimpleTimeBench distributions in Multivariate generation mode. Colors represent different series, shades represent covariates. Series are min-max normalized for easier visualization. Two series are shown in each panel.

Figure 5 : Samples from all SimpleTimeBench distributions in Leading Covariate generation mode. Colors represent different series, shades represent covariates, where lighter shades (leading covariates) lead darker (main variate). Series are min-max normalized for easier visualization. Two series are shown in each panel.

Metric Name (Abbreviation)	What it Measures	Formula Logic	When to use	Ref.	$<$ Ref. Value (Min Bound)	$>$ Ref. Value (Max Bound)
Relative MAEs (relMAE mean , relMAE last )	Skill. Compares model error vs a naive baseline.	Ratio of Average Model Error to Average Naive Error.	Benchmarking. To verify the model has learned anything useful.	$1.0$	Good $[0,1.0)$ : Beating the baseline; extracting signal.	Bad $[1.0,\infty)$ : Worse than a naive flat-line guess.
Mean Bias Ratio (MBR)	Hedging. Frequency of predictions that are biased towards the mean.	Frequency of model landing between the historical mean and the target.	Behavior. To see if the model is “playing it safe.”	$0.5$	Aggressive $[0,0.5)$ : Model frequently overshoots the trend.	Hedging $(0.5,1.0]$ : Model tends towards the mean to avoid penalty.
Mean Attraction Ratio (MAR)	Dynamics. Ratio of target’s deviation from mean vs forecast’s deviation.	Average deviation of target from mean divided by average deviation of forecast from mean.	Volatility. When catching extreme events (spikes) is critical.	$1.0$	Hyper-Active $[0,1.0)$ : Model exaggerates peaks & valleys.	Muted $(1.0,\infty)$ : Model dampens signal; under-sizes moves.
Path Volatility Ratio (PVR)	Texture. High frequency characteristics of target vs forecast.	Sum of absolute differences in target path divided by same for forecast path.	Realism. To ensure the curve shape/noise looks realistic.	$1.0$	Hallucinating $[0,1.0)$ : Adds jitter or noise not in reality.	Too Smooth $(1.0,\infty)$ : Misses rapid changes; overly smooth.
*The noise floor is data-dependent. For deterministic signals, the reference value is $\approx 0$ .

Appendix

Table 4 : Time Series Forecasting Metrics: Interpretation Guide with Noise-Aware Bounds

Figure 6 : Samples of all distributions in SimpleTimeBench.

			MeanAttractionRatio			MeanBiasRatio			relMAE mean			PathVolatilityRatio			relMAE last
		mode	Univ	Cov	Lead	Univ	Cov	Lead	Univ	Cov	Lead	Univ	Cov	Lead	Univ	Cov	Lead
regime	model	dataset
Deterministic Trends	Chronos2	exponential	1.31	1.32	1.19	0.92	0.87	0.92	0.24	0.24	0.16	1.90	1.90	1.57	0.37	0.37	0.28
		linear	1.01	1.01	1.01	0.98	0.97	0.98	0.01	0.01	0.01	1.03	1.03	1.03	0.03	0.04	0.03
		staircase	1.01	1.01	1.01	0.72	0.67	0.69	0.01	0.01	0.01	0.70	0.74	0.74	0.05	0.06	0.06
	Moirai	exponential	1.67	1.54	1.28	0.91	0.51	0.52	0.40	1.44	1.57	2.21	1.26	0.88	0.62	2.20	2.67

Appendix

Table 5 : SimpleTimeBench Metrics on Deterministic Trends, Periodic, and Evolving Periodic. Metrics are displayed for univariate mode (Univ), past covariates from same distribution (Cov.), and past leading covariates (Lead).

			relMAE mean			relMAE last
		mode	Univ	Cov	Lead	Univ	Cov	Lead
regime	model	dataset
I.I.D. Noise	Chronos2	binary	0.78	0.78	0.78	0.80	0.80	0.79
		cauchy	0.69	0.69	0.69	0.65	0.65	0.65
		laplace	1.00	1.00	1.00	0.68	0.68	0.68
		lognormal	0.94	0.94	0.94	0.68	0.68	0.68

Appendix

Table 6 : SimpleTimeBench Metrics on IID Noise and Stochastic Correlated processes. Metrics are displayed for univariate mode (Univ), past covariates from same distribution (Cov.), and past leading covariates (Lead).

Figure 7 : Evaluation of Chronos-2 predictions when a sequence identical to the target sequence is given with no lag, either as a past-only or future covariate, using random walks. In the future covariate case, Chronos-2 is able to correctly reproduce the covariate as its prediction.

Figure 8 : Performance of Chronos-2 model on Deterministic Trends after fine-tuning in the Bias experiment. The values in each cell represent the percentage improvement of the corresponding metric calculated over 100 samples from that distribution.

Figure 9 : Performance of Chronos-2 model on exponential trends after fine-tuning in the Norm experiment. The values in each cell represent the percentage improvement of the corresponding metric calculated over 100 samples from that distribution.

Family	Distribution	MAPE			relMAE mean			relMAE last
		Base	Fine-tuned	Improvement %	Base	Fine-tuned	Improvement %	Base	Fine-tuned	Improvement %
I.I.D. Noise	binary	$0.25$	$0.17$	$32.83$	$0.77$	$0.54$	$30.02$	$0.73$	$0.51$	$30.02$
I.I.D. Noise	cauchy	$2.37$	$22.89$	$-865.51$	$0.86$	$0.98$	$-13.94$	$0.82$	$0.93$	$-13.94$
I.I.D. Noise	laplace	$1.53$	$6.35$	$-314.98$	$1.00$	$0.78$	$21.78$	$0.63$	$0.50$	$21.78$
I.I.D. Noise	lognormal	$0.52$	$0.45$	$12.75$	$0.94$	$0.71$	$24.58$	$0.73$	$0.55$	$24.58$
I.I.D. Noise	normal	$1.22$	$5.58$	$-357.85$	$1.00$	$0.72$	$28.13$	$0.74$	$0.53$	$28.13$

Appendix

Table 7 : Performance gains from fine-tuning Chronos-2 on in the Covariate-Underutilization experiment.

Figure 10 : Performance of Chronos-2 fine-tuned model on the leading covariate data in the Covariate-Underutilization experiment. The values in each cell represent the percentage improvement of the corresponding metric calculated over 100 samples from that distribution.

Panel of Figure 11 — Figure 11 : Loss trajectories of the four fine-tuning experiments.

Figure 12 : Performance of Chronos-2 fine-tuned model on the leading covariate data in the All-in-One experiment. The values in each cell represent the percentage improvement of the corresponding metric calculated over 100 samples from that distribution.

Figure 13 : Univariate vs. Multivariate performance by model and metric on GIFT-Eval. Each point represents a single dataset/term experiment.

Dataset	Model	Metric	Univariate	With Covariate	Improvement
Delaware River	Chronos-2	MSE	8.99e+06	8.05e+06	+10.5%
		MAPE	6.05	5.88	+2.8%
		MASE	7.16	6.74	+5.9%
	DLinear	MSE	1.28e+07	6.05e+06	+52.7%
		MAPE	7.16	6.07	+15.2%
		MASE	8.82	6.69	+24.1%

Appendix

Table 8 : Univariate and with covariate support detailed forecast metrics comparison of Chronos-2 v.s. DLinear on Delaware River and NVIDIA datasets

Figure 14 : Univariate and with covariate support forecast comparison of Chronos-2 v.s. DLinear on Delaware River and NVIDIA datasets zooming into critical spikes in the test period

Explore similar work

When, Not How Much: Evaluating Time-Series Foundation Models on Sparse Events
Sep 30, 2026Daniel Schoess, Florian von WangenheimTime Series Foundation Models Data-Driven Forecasting
ETH Zurich
TempusBench: An Evaluation Framework for Time-Series Forecasting
Apr 13, 2026Denizalp Goktas, Gerardo Riaño-Briceño, Omkar Tekawade +14Time Series Foundation Models
Simulacrum New York City, NY, USA
Evaluating Accuracy and Probabilistic Reliability of Zero-Shot Time Series Foundation Models
Sep 22, 2026Panagiotis Michael, Moysis Symeonides, Demetris TrihinasZero-Shot Forecasting Cnn-Lstm
Department of Computer Science, University of Nicosia, Cyprus · Department of Computer Science, University of Cyprus, Cyprus

Sep 30, 2026, cs.LGJ/K move · Enter open · S save

When, Not How Much: Evaluating Time-Series Foundation Models on Sparse Events

Daniel Schoess, Florian von Wangenheim

ETH Zurich

Time Series Foundation Models Data-Driven Forecasting Pretraining

Open paper AskPDF arXiv

Pretrained time-series foundation models (TSFMs) are evaluated as forecasters of future values, yet for sparse series many decisions depend only on which future periods contain activity. Standard benchmarks do not assess this. On five sparse datasets, we rank positions within forecast windows that contain both events and zeros. The released point forecasts of 12 TSFMs improve chance-corrected average precision over training-free references by at most 0.031, and in chance-corrected AUC the median TSFM falls below them on every dataset. With event supervision, linear probes of six frozen backbones improve on their backbone's point forecast in 29 of 30 backbone--dataset pairs. Averaging the predicted quantiles instead of taking their median improves the ranking of most TSFMs that forecast the median, and on two datasets the strongest such outputs rival the probes. The probes' advantage over raw-context learners depends on the dataset, and under the same probe, pretrained features outperform randomly initialized ones for five of six backbones. For sparse-event ranking, released point forecasts thus add little over simple references, whereas lightweight event heads on frozen TSFMs rank events better than these forecasts, and the best of them exceed gradient-boosted trees trained on the raw context on three of the five datasets. More broadly, assessing pretrained forecasters on tasks beyond value forecasting requires reporting their outputs, supervised probes of their representations, and raw-context and randomized controls side by side, since each supports a different conclusion.

Abstract

Explore similar work

When, Not How Much: Evaluating Time-Series Foundation Models on Sparse Events

TempusBench: An Evaluation Framework for Time-Series Forecasting

Evaluating Accuracy and Probabilistic Reliability of Zero-Shot Time Series Foundation Models