cs.LGNov 4, 2026

Big Models, Small Gains: Frozen Forecasters as Classifiers on MONSTER

Authors: Rishit Jain, Rishit Kar, Chetashri Bhadane, Nilesh Patil

Organizations: Department of Computer Engineering, Dwarkadas J. Sanghvi College of Engineering, University of Mumbai, India

Abstract

Forecasting foundation models are increasingly used as frozen feature extractors for time series classification, but it remains unclear whether their pretrained representations offer an advantage over strong classifiers that do not require pretraining. This raises an important question: do these representations provide useful information beyond what the model architecture alone can capture, particularly when labelled data are scarce? We present a controlled benchmarking of two forecasting models, Reverso and Chronos-Bolt, on six large datasets from the MONSTER benchmark, using 1%, 10%, and 100% of the available training labels. We compare their frozen representations against four established time series classification methods and random-weight versions of the same forecasting models. Our results show that pretraining improves classification accuracy over random-weight counterparts on five of the six datasets, with gains of up to 23 percentage points. However, the frozen representations do not reliably outperform the strongest non-pretrained baseline in any setting: they perform worse in 15 of 17 settings and tie in the remaining two. Further experiments show that restoring amplitude information recovers part of the performance gap, while fine-tuning and forecast-error features do not close it. These findings highlight the importance of strong task specific baselines and random weight controls when evaluating forecasting foundation models for classification. Our code is available at https://github.com/Rachet2006/frozen-forecasters-monster.

Figures & tables

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Jul 25, 2026

Foundation Models and Fine-Tuning: Toward a New Generation of Models for Time Series Forecasting

Inspired by recent breakthroughs in large language models for natural language processing, foundation models have emerged as a promising paradigm for zero-shot time series forecasting, enabling accurate predictions on datasets never seen during pre-training. Ranging from tens to hundreds of millions of parameters, these models are pre-trained on vast and diverse collections of time series, learning generalizable representations that support both point and probabilistic forecasting. This approach alleviates the need for dataset-specific model design and manual tuning, offering a unified solution across forecasting problems. In this work, we review the main architectures, pre-training strategies, and optimization methods underpinning these models. We further investigate post-pre-training fine-tuning of selected foundation models to enhance their performance on specific datasets. Our empirical results demonstrate that this step consistently improves forecasting accuracy over the zero-shot baseline.
Sep 30, 2026

When, Not How Much: Evaluating Time-Series Foundation Models on Sparse Events

Pretrained time-series foundation models (TSFMs) are evaluated as forecasters of future values, yet for sparse series many decisions depend only on which future periods contain activity. Standard benchmarks do not assess this. On five sparse datasets, we rank positions within forecast windows that contain both events and zeros. The released point forecasts of 12 TSFMs improve chance-corrected average precision over training-free references by at most 0.031, and in chance-corrected AUC the median TSFM falls below them on every dataset. With event supervision, linear probes of six frozen backbones improve on their backbone's point forecast in 29 of 30 backbone--dataset pairs. Averaging the predicted quantiles instead of taking their median improves the ranking of most TSFMs that forecast the median, and on two datasets the strongest such outputs rival the probes. The probes' advantage over raw-context learners depends on the dataset, and under the same probe, pretrained features outperform randomly initialized ones for five of six backbones. For sparse-event ranking, released point forecasts thus add little over simple references, whereas lightweight event heads on frozen TSFMs rank events better than these forecasts, and the best of them exceed gradient-boosted trees trained on the raw context on three of the five datasets. More broadly, assessing pretrained forecasters on tasks beyond value forecasting requires reporting their outputs, supervised probes of their representations, and raw-context and randomized controls side by side, since each supports a different conclusion.
Jul 6, 2026

When Do Foundation Models Pay Off? A Break-Even Analysis of Pretrained Time Series Forecasters

Deploying a time series foundation model (FM) requires GPU infrastructure and engineering overhead, with no guarantee of improvement over classical forecasting methods. We provide the first systematic break-even analysis of when this investment pays off. Across 30 benchmark datasets, we compare zero-shot and LoRA fine-tuned FMs (Chronos-Bolt, Moirai, Lag-Llama) against classical baselines (Naive, ETS, ARIMA, XGBoost) at six training set sizes from 2% to 100% of available data. FMs outperform classical methods at every evaluated training fraction on 16 of 30 datasets. On 6 datasets, classical methods surpass zero-shot FMs with at most 5% of the training data (21--6,324 samples); on the remaining 8, break-even ranges from 24 to 22,177 samples. No classical method dominates: ARIMA, Naive, ETS and XGBoost are each best on 6--11 datasets. One robust rule requires no model training: if n_train < 700 and seasonality is non-negligible, FM zero-shot is the right low-data choice -- below 20% of the history, classical methods either lose to it (4 of these 10 datasets) or cannot be fitted at all (6) -- and it stays best at full history on 6 of them; fine-tuning should be skipped. Contrary to common practice, LoRA fine-tuning rarely helps and degrades short series by up to 52%. We operationalise these findings as a two-step decision framework -- compute dataset length and seasonality strength, run a brief 5--10% pilot only if needed. Cheap dataset features do not predict the break-even category at this benchmark scale, so outside the rule a pilot remains necessary. Code and benchmark: https://github.com/nicolaisi/fm_breakeven.