Adapting Time Series Foundation Models through Data Mixtures

Thomas L. Lee; Edoardo M. Ponti; Amos Storkey

cs.LGMar 3, 2026

Adapting Time Series Foundation Models through Data Mixtures

Authors: Thomas L. Lee, Edoardo M. Ponti, Amos Storkey

Organizations: School of Informatics, University of Edinburgh.

Abstract

Time series foundation models (TSFMs) have become increasingly popular for zero-shot forecasting. However, for a new time series domain not fully covered by the pretraining set, performance can suffer. Therefore, when a practitioner cares about a new domain and has access to a set of related datasets, the question arises: how best to fine-tune a TSFM to improve zero-shot forecasting? A typical approach to this type of problem is to fine-tune a LoRA module on all datasets or separately on each dataset. Tuning a separate module on each dataset allows for the specialisation of the TSFM to different types of data distribution, by selecting differing combinations of per-dataset modules for different time series contexts. However, we find that, using per-dataset modules might not be optimal, since a time series dataset can contain data from several types of distributions, i.e. sub-domains. This can be due to the distribution shifting or having differing distributions for different dimensions of the time series. Hence, we propose MixFT which re-divides the data using Bayesian mixtures into sets that best represent the sub-domains present in the data, and fine-tunes separately on each of these sets. This re-division of the data ensures that each set is more homogeneous, leading to fine-tuned modules focused on specific sub-domains. Our experiments show that MixFT performs better than per-dataset methods and when fine-tuning a single module on all the data. This suggests that by re-partitioning the data to represent sub-domains we can better specialise TSFMs to improve zero-shot forecasting.

Figures & tables

Figure 1: MixFT identifies and trains separate LoRA modules on different sub-domains given in the fine-tuning datasets. This is unlike previous approaches which either separately fine-tune LoRA modules using dataset boundaries (per-dataset methods) or trained a single LoRA module on all the data (Shared) ( Ostapenko et al., 2024 ) . By training separate LoRA modules on sub-domains, MixFT aims for the training data for each LoRA to be more homogeneous, leading to more specialized and consistent LoRA modules. This also should lead to better identifiability of what LoRA modules to use when zero-shot forecasting.

		Methods
	Datasets	Base	Shared	$\mu$ Datasets	Arrow Datasets	Poly	MBC	MixFT (ours)
Chronos Bolt	CloudD1	$1.511$	$1.553_{\pm{0.024}}$	$1.580_{\pm{0.016}}$	$\mathbf{1.352_{\pm{0.006}}}$	$\underline{1.463_{\pm{0.012}}}$	$1.810_{\pm{0.266}}$	$1.485_{\pm{0.008}}$
	CloudD2	$1.572$	$1.342_{\pm{0.004}}$	$1.614_{\pm{0.029}}$	$\underline{1.341_{\pm{0.005}}}$	$\mathbf{1.320_{\pm{0.004}}}$	$1.649_{\pm{0.211}}$	$1.343_{\pm{0.004}}$
	BizITObs-L2C	$3.116$	$\underline{3.101_{\pm{0.011}}}$	$3.624_{\pm{0.084}}$	$3.680_{\pm{0.028}}$	$3.124_{\pm{0.001}}$	$3.170_{\pm{0.009}}$	$\mathbf{3.083_{\pm{0.006}}}$
	BizITObs-App	$3.494$	$\underline{1.060_{\pm{0.020}}}$	$3.691_{\pm{0.117}}$	$5.002_{\pm{0.197}}$	$1.173_{\pm{0.006}}$	$1.419_{\pm{0.346}}$	$\mathbf{0.989_{\pm{0.024}}}$
	US-Births	$1.014$	$\underline{0.967_{\pm{0.007}}}$	$1.175_{\pm{0.003}}$	$1.212_{\pm{0.003}}$	$0.979_{\pm{0.010}}$	$1.041_{\pm{0.005}}$	$\mathbf{0.942_{\pm{0.002}}}$

Table 1: MASE of fine-tuning methods. The table shows the MASE performance of fine-tuning methods on the Chronos Bolt and Moirai-1.1-R TSFMs, when zero-shot forecasting standard time series datasets. A lower MASE is better and we also provide standard errors over training runs. Importantly, the time series used for fine-tuning are related but are not the same as the time series evaluated on. We bold and underline the best and second best performing method, respectively, for each dataset and TSFM. Our results show that MixFT, in terms of average ranking and number of bolded/underlined results, performs the best.

Methods	Avg. Imp.( $\downarrow$ )	Avg. Rel. Imp.( $\uparrow$ )
Arrow-Datasets	$0.71$	$0.90$ x
MBC	$\underline{-0.06}$	$1.11$ x
Arrow-MixFT	$\mathbf{-0.17}$	$\mathbf{1.17}$ x

Table 2: Average MASE improvement over all datasets on Chronos Bolt for fine-tuning methods using arrow routing when compared to Base (i.e. not fine-tuning). We present both the average improvement in terms of absolute values (Avg. Imp) and relative values (Avg. Rel. Imp.). Relative values are calculated as the MASE of Base divided by the MASE of the fine-tuning method in question. We find that when controlling for how the LoRA modules are selected at test time, by always using arrow routing, the LoRA modules learnt by MixFT perform the best. This suggests that it finds better divisions of the fine-tuning data than per-dataset divisions.

Figure 3: Mixture membership of the fine-tuning datasets for MixFT. The plot shows, for the first channel of each fine-tuning dataset, when the time series is learnt to be of the first mixture component (purple area) or the second component (yellow area). We find that some of the channels consist of just one mixture component (CloudD3 and M4-Weekly), mostly one component (BitBrains and M4-Hourly) or exhibit periodic patterns (CloudD4 and BizITObs-Service). The plots suggest that MixFTs Bayesian mixture model finds reasonable patterns in the data. Also, as both sub-domains/mixture components are found in a given dataset, MixFT’s data divisions can not be found by per-dataset methods.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Num. of Components	Val. Avg. Rank ( $\mathbf{\downarrow}$ )
$K=2$	2.17
$K=3$	3.17
$K=4$	2.83
$K=5$	3.33
$K=10$	3.50

Appendix

Table 3: MixFT validation performance for different number of components

K

. The table contains the average rank of MixFT using different number of components. The average is taken over the validation sets of the fine-tuning datasets. The table shows that

K=2

has the lowest average rank and hence should be selected.

	MixFT with Different Component Selection Methods
Datasets	$\mu$ MixFT	Arrow MixFT	Soft MixFT	Ensemble MixFT	MixFT (ours)
CloudD1	$\mathbf{1.411_{\pm{0.003}}}$	$1.594_{\pm{0.055}}$	$1.697_{\pm{0.009}}$	$1.480_{\pm{0.003}}$	$1.485_{\pm{0.008}}$
CloudD2	$1.425_{\pm{0.001}}$	$1.384_{\pm{0.006}}$	$1.401_{\pm{0.010}}$	$1.351_{\pm{0.001}}$	$\mathbf{1.343_{\pm{0.004}}}$
BizITObs-L2C	$\underline{3.087_{\pm{0.002}}}$	$3.163_{\pm{0.013}}$	$3.237_{\pm{0.012}}$	$3.088_{\pm{0.003}}$	$\mathbf{3.083_{\pm{0.006}}}$
BizITObs-App	$1.907_{\pm{0.016}}$	$1.277_{\pm{0.038}}$	$1.168_{\pm{0.014}}$	$1.015_{\pm{0.002}}$	$\mathbf{0.989_{\pm{0.024}}}$
US-Births	$0.973_{\pm{0.001}}$	$0.977_{\pm{0.002}}$	$0.968_{\pm{0.001}}$	$0.946_{\pm{0.001}}$	$\mathbf{0.942_{\pm{0.002}}}$

Appendix

Table 4: MASE of versions of MixFT with differing component selection methods, when zero-shot forecasting using Chronos Bolt. The table shows MixFT with

\mu

and Arrow routing which are used by some of the other fine-tuning methods compared to in the main paper. Also the table shows two methods which use the probabilities given by MixFTs Bayesian GMM to perform component selection, soft and ensemble-MixFT. In the table we bold and underline the best and second best results, respectively. We find that that MixFT performs the best or second best for all time series, bar CloudD1, and achieves the lowest average rank. This shows that MixFTs way of selecting what LoRA to use for zero-shot forecasting performs well.

	MixFT with Different Prob. Models
Datasets	K-means	Topic Model	Bayesian GMM (ours)
CloudD1	$\underline{1.516_{\pm{0.005}}}$	$1.532_{\pm{0.004}}$	$\mathbf{1.485_{\pm{0.008}}}$
CloudD2	$1.363_{\pm{0.003}}$	$\mathbf{1.328_{\pm{0.009}}}$	$\underline{1.343_{\pm{0.004}}}$
BizITObs-L2C	$\underline{3.096_{\pm{0.005}}}$	$3.100_{\pm{0.002}}$	$\mathbf{3.083_{\pm{0.006}}}$
BizITObs-App	$1.048_{\pm{0.006}}$	$\underline{1.030_{\pm{0.041}}}$	$\mathbf{0.989_{\pm{0.024}}}$
US-Births	$0.967_{\pm{0.005}}$	$\underline{0.960_{\pm{0.009}}}$	$\mathbf{0.942_{\pm{0.002}}}$

Appendix

Table 5: MASE of versions of MixFT with differing probabilistic models, when using Chronos Bolt. MixFT uses a Bayesian GMM to identify and learn sub-domains. In the table we show its performance using a range of different probabilistic models, to identify and learn sub-domains. In the table we bold and underline the best and second best results, respectively. We find that using a Bayesian GMM performs best or second best for all datasets, bar M4-Daily, and achieves the lowest average rank. This shows it is a good choice for the probabilistic model of MixFT.

	MixFT with Different Numbers of Components
Datasets	$K=1$ (Shared)	$K=2$ (ours)	$K=3$	$K=4$	$K=5$	$K=10$
CloudD1	$1.553_{\pm{0.024}}$	$\underline{1.485_{\pm{0.008}}}$	$1.500_{\pm{0.010}}$	$1.489_{\pm{0.002}}$	$1.518_{\pm{0.017}}$	$\mathbf{1.448_{\pm{0.004}}}$
CloudD2	$\mathbf{1.342_{\pm{0.004}}}$	$\underline{1.343_{\pm{0.004}}}$	$1.375_{\pm{0.010}}$	$1.365_{\pm{0.003}}$	$1.384_{\pm{0.007}}$	$1.423_{\pm{0.003}}$
BizITObs-L2C	$3.101_{\pm{0.011}}$	$3.083_{\pm{0.006}}$	$\underline{3.061_{\pm{0.009}}}$	$\mathbf{3.058_{\pm{0.009}}}$	$3.068_{\pm{0.007}}$	$3.065_{\pm{0.010}}$
BizITObs-App	$1.060_{\pm{0.020}}$	$0.989_{\pm{0.024}}$	$1.001_{\pm{0.018}}$	$\mathbf{0.970_{\pm{0.013}}}$	$\underline{0.985_{\pm{0.019}}}$	$1.127_{\pm{0.062}}$
US-Births	$\underline{0.967_{\pm{0.007}}}$	$\mathbf{0.942_{\pm{0.002}}}$	$1.007_{\pm{0.005}}$	$1.034_{\pm{0.006}}$	$1.025_{\pm{0.001}}$	$1.012_{\pm{0.002}}$

Appendix

Table 6: MASE of using MixFT with differing number of mixture components, when using Chronos Bolt. In the table we show when using MixFT with one to ten mixture components. Using one is equivalent to Shared and using two is what MixFT uses in the rest of our experiments. In the table we bold and underline the best and second best results, respectively. We find that different numbers of mixture components often perform similarly and that using two is best in terms of average rank.

	Datasets	Average Classification Entropy $\mathbb{E}_{\bm{x}\sim D}[H(c\|\bm{z}=\text{TSFM}_{embed}(\bm{x}))]$
Fine-tuning	CloudD3	0.0095
	CloudD4	0.0075
	BizITObs-Service	0.0053
	BitBrains Fast Storage	0.0101
	M4-Hourly	0.0093
	M4-Weekly	0.0050

Appendix

Table 7: Confidence of MixFTs classification of a contexts sub-domain in terms of average classification entropy. The average classification entropy of a dataset

D

\mathbb{E}_{\bm{x}\sim D}[H(c|\bm{z}=\text{TSFM}_{embed}(\bm{x}))]

and measures the average uncertainty in the classification of

c

for

\bm{x}\in D

. We find that the average entropy is small across all fine-tuning and evaluation datasets, where an entropy of

0.01

bit corresponds to a

0.999

estimated probability a context is of a particular sub-domain. This shows that MixFT is on average very confident what contexts belong to what sub-domains.

Panel of Figure 4 — Figure 4: Example forecasts on BizITObs-Application given by MixFT and when using the other sub-domains LoRA module in a mixture or on its own. The two plot shows forecasts for MixFT, Ensemble MixFT (see Appendix D.2 ) and the forecast given by using the LoRA module not selected by MixFT, calling this Other Component in the plots. The plot also present the true value. In plot b) we do not show the forecasts of Ensemble MixFT as it is identical to that of MixFT—MixFT classifies with probability $1.0$ that the given context belongs to one of the sub-domains. The plots show that using the LoRA module MixFT selects gives better forecasts than using the one it does not. Furthermore, that ensembleling both forecasts can result in worse performance than MixFT because the Other Component forecast can be inaccurate given that it is forecasting an OOD context for its sub-domain.

Panel of Figure 5 — Figure 5: Plots of two contexts identified as belonging to the two mixture components of MixFT, respectively, for the first channel of the fine-tuning dataset BizITObs-service, when using Chronos Bolt. The plot shows one of the characteristics of the learnt sub-domains, in that when the end of a context is varying (i.e. is spiking in the figure) then it is seen as belonging to the second component, else it belongs to the first. This is the reason why in Figure 3 , for BizITObs-service there is a periodic pattern as it has periodic spikes. This figure demonstrates that MixFTs found sub-domains are interpretable and understandable.

Panel of Figure 6 — Figure 6: Plots of two contexts identified as belonging to the two mixture components of MixFT, respectively, for the eighth channel of the fine-tuning dataset BizITObs-service, when using Chronos Bolt. Unlike Figure 5 , these contexts do not have a clear seasonal pattern. Because of this, it is harder to identify the features used to decide the sub-domain. However, it is possible trend is used, as plot a) has a negative trend while plot b) has no discernable trend.

Figure 7: Mixture membership of the evaluation datasets for MixFT, when using Chronos Bolt. The plot shows, for the first channel of each evaluation dataset, when the time series is learnt to be of the first mixture component (purple area) or the second component (yellow area). We find, as expected, that there are less consistent patterns of mixture membership than in the fine-tuning data (see Figure 3 ). This demonstrates the difficultly of identifying sub-domains from zero-shot data. However, there are still patterns of usage. For example, for BizITObs-Application we see a periodic pattern. This suggests while it is hard to identify zero-shot mixture membership, MixFT still does a reasonable job and this is one of the reasons it performs well in our experiments.

Explore similar work

Jul 25, 2026

Foundation Models and Fine-Tuning: Toward a New Generation of Models for Time Series Forecasting

Inspired by recent breakthroughs in large language models for natural language processing, foundation models have emerged as a promising paradigm for zero-shot time series forecasting, enabling accurate predictions on datasets never seen during pre-training. Ranging from tens to hundreds of millions of parameters, these models are pre-trained on vast and diverse collections of time series, learning generalizable representations that support both point and probabilistic forecasting. This approach alleviates the need for dataset-specific model design and manual tuning, offering a unified solution across forecasting problems. In this work, we review the main architectures, pre-training strategies, and optimization methods underpinning these models. We further investigate post-pre-training fine-tuning of selected foundation models to enhance their performance on specific datasets. Our empirical results demonstrate that this step consistently improves forecasting accuracy over the zero-shot baseline.

Morad Laglil, Bertrand Pracca, Emilie Devijver +1

Savoye, 18 bd des Gorgets, 21000, Dijon, France · Univ. Grenoble Alpes, CNRS, Grenoble INP, LIG, Grenoble, 38000, France

Zero-Shot Time Series Forecasting Time-Series Foundation Models Probabilistic Time Series Forecasting

Ask PDF

Sep 26, 2026

SIFT: Enhancing Time Series Foundation Models via Semantic Invariance and Structural Fidelity Fine-Tuning

Time Series Foundation Models (TSFMs) have achieved remarkable zero-shot performance through extensive pre-training on massive time series datasets. Nevertheless, due to the low-dimensional properties and diverse structural patterns of time series data, performing naive fine-tuning on TSFMs often leads to overfitting and falling into the mean-prediction trap. To address these challenges, we propose SIFT, a robust adaptation method that enhances time series foundation models by preserving Semantic Invariance and structural Fidelity throughout the fine-Tuning process. We employ semantic-invariant adversarial augmentation, which utilizes semantic spectrum decomposition to partition the semantic space and then generates perturbations within the non-core semantic subspace to bolster the model's robustness against these perturbations, mitigating overfitting. We implement a component-based structural fidelity enhancement, which facilitates component-wise mixup and imposes a reconstruction objective to improve the model's ability to preserve structural fidelity, alleviating the mean-prediction trap. Extensive experiments on representative TSFMs covering 10 real-world datasets demonstrate that SIFT can significantly enhance the performance of TSFMs.

Yi Tang, Tengxue Zhang, Yang Shu +3

East China Normal University, Shanghai, China · Chang’an University, Xi’an, China

Time-Series Foundation Models Time Series Forecasting Time Series Data Augmentation

Ask PDF

Apr 18, 2026

TimeRFT: Stimulating Generalizable Time Series Forecasting for TSFMs via Reinforcement Finetuning

Time Series Foundation Models (TSFMs) advance generalization and data efficiency in time series forecasting by unified large-scale pretraining. But TSFMs remain lacking when adapting to specific downstream forecasting tasks for two reasons. First, the non-stationary and uncertain nature of time series data lead to inevitable temporal distribution shifts between historical training and future testing data, while current Supervised FineTuning (SFT)-based methods are prone to overfitting and may degrade generalization. Second, training data availability varies across forecasting tasks, requiring TSFMs to generalize well under diverse data regimes. To address these challenges, we introduce the Time series Reinforcement Finetuning (TimeRFT) paradigm for TSFM downstream adaptation, which consists of two task-specific training recipes: i) A forecasting quality-based temporal reward mechanism that conducts a multi-faceted evaluation of the contribution of each prediction step to overall forecasting accuracy. ii) A forecasting difficulty-based data selection strategy to identify time series samples with generalizable predictive patterns and informative training signals. Extensive experiments demonstrate TimeRFT can consistently outperform SFT-based adaptation methods across various real-world forecasting tasks and training data regimes, enhancing prediction accuracy and generalization against unforeseen distribution shifts.

Siyang Li, Yize Chen, Zijie Zhu +4

HKUST(GZ) Guangzhou, China · University of Alberta Edmonton, Canada · Alibaba Cloud Guangzhou, China +3

Time Series Forecasting OOD Generalization Non-Stationary Time Series Forecasting RL Fine-Tuning Time-Series Foundation Models

Ask PDF