Discrete Prototypical Memories for Federated Time Series Foundation Models

Liwei Deng; Qingxiang Liu; Xinhe Niu; Shengchao Chen; Sheng Sun; Yuankai Wu; Guodong Long; Yuxuan Liang

cs.LGApr 6, 2026

Discrete Prototypical Memories for Federated Time Series Foundation Models

Authors: Liwei Deng, Qingxiang Liu, Xinhe Niu, Shengchao Chen, Sheng Sun, Yuankai Wu, Guodong Long, Yuxuan Liang

Organizations: Australian Artificial Intelligence Institute, University of Technology Sydney · The Hong Kong University of Science and Technology (Guangzhou) · Shenzhen University · Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China ·

Abstract

Leveraging Large Language Models (LLMs) as federated learning (FL)-based time series foundation models offers a promising way to transfer the generalization capabilities of LLMs to time series data while preserving access to private data. However, the semantic misalignment between time-series data and the text-centric latent space of existing LLMs often leads to degraded performance. Meanwhile, the parameter-sharing mechanism in existing FL methods model heterogeneous cross-domain time-series data into a unified continuous latent space, which contradicts the fact that time-series semantics frequently manifest as discrete and recurring regimes. To address these limitations, we propose FeDPM, a federated framework for time-series foundation models based on discrete prototypical memories. Specifically, we learn local prototypical memory priors for intra-domain time-series data. We then align cross-domain memories to promote a unified discrete latent space and introduce a domain-specific memory update mechanism to balance shared and personalized prototypical knowledge. Extensive experiments demonstrate the efficiency and effectiveness of FeDPM. The code is publicly available at https://anonymous.4open.science/r/FedUnit-64D1.

Figures & tables

Figure 1: Ablation study of Time-FFM by replacing the frozen LLM backbone with trainable Transformer layers or FC layers on (a) forecasting MSE and (b) number of parameters. (Detailed settings and results in Appendix A .) (c) Performance comparison between our proposed FeDPM and FFTS.

Figure 2: The overall architecture of FeDPM .

Method	Latent Space	Limitation	Comm. Object	Comm. Efficiency	FM Construction	Params
Time-FFM	Text-centric	Semantic Misalignment	Prompts / Params	Low	Stacking Params	High
FFTS	Continuous	Feature Collapse	Model Params	Low	Stacking Params	High
FeDPM	Discrete Prototype	—	Memory Only	High	Unified Memory	Low

Table 1: Comparison of FeDPM with representative Time-FFM and FFTS.

Type		FL-FM										Cen-FM						Expert
Method		FeDPM		Time-FFM		FFTS		FL-iTransformer		FL-PatchTST		TOTEM		UniTime		Cen-PatchTST		TimeNet		Dlinear		FEDformer		iTransformer		PatchTST
Metric		MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
	96	0.391	0.407	0.406	0.411	0.417	0.445	0.473	0.453	0.459	0.457	0.402	0.405	0.397	0.418	0.433	0.422	0.384	0.402	0.386	0.400	0.376	0.419	0.387	0.405	0.414	0.419
	192	0.441	0.434	0.460	0.442	0.475	0.487	0.504	0.476	0.491	0.474	0.457	0.436	0.434	0.439	0.467	0.444	0.436	0.429	0.437	0.432	0.420	0.448	0.441	0.436	0.460	0.445
	336	0.486	0.463	0.504	0.453	0.531	0.521	0.535	0.494	0.549	0.507	0.498	0.461	0.470	0.457	0.509	0.472	0.491	0.469	0.481	0.459	0.459	0.465	0.491	0.462	0.501	0.466
ETTh1	720	0.572	0.508	0.495	0.466	0.686	0.611	0.572	0.524	0.577	0.526	0.539	0.513	0.472	0.477	0.503	0.485	0.521	0.500	0.519	0.516	0.506	0.507	0.509	0.494	0.496	0.481

Table 2: Full forecasting performance comparison results. Bold highlights the best performance across all methods, while Blue marks the best result among FL-FMs. “Comm. Params.” denotes the number of communicated parameters.

Few-shot Long-term Forecasting (5%)
Type	Method	Metric	ETTm1	ETTm2	Electricity	Weather	Exchange	$1^{st}$ Count
FL-FM	FeDPM	MSE	0.538	0.310	0.248	0.257	0.155	6
	FeDPM	MAE	0.480	0.338	0.337	0.290	0.293	6
	Time-FFM	MSE	0.567	0.293	0.324	0.292	0.167	0
	Time-FFM	MAE	0.491	0.333	0.403	0.318	0.289	0
	FFTS	MSE	0.613	0.183	0.488	0.275	0.188	2

Table 3: Few-shot forecasting performance. Comparison results under forecasting horizons

F_{i}\in\{96,192,336,720\}

. Results are averaged over the four prediction lengths. Bold indicates the best performance among all methods. Complete results are reported in Table 10 .

Method	Metric	ETTh1	ETTh2	ETTm1	ETTm2	Electricity	Weather	Exchange
Ours	MSE	0.422	3	0.353	0.216	0.209	0.185	0.142
Ours	MAE	0.424	0.368	0.376	0.281	0.303	0.229	0.283
w/ Average	MSE	0.441	0.350	0.359	0.231	0.232	0.218	0.177
w/ Average	MAE	0.429	0.373	0.383	0.291	0.319	0.255	0.303
w/ Local Memory	MSE	0.431	0.346	0.378	0.224	0.273	0.204	0.159
w/ Local Memory	MAE	0.539	0.373	0.385	0.285	0.359	0.247	0.297

Table 4: Ablation results on seven datasets with forecasting horizons

F_{i}\in\{96,192\}

. All results are averaged over the two prediction lengths. Bold denotes the best performance.

Type	Method	Metric	ETTh1	ETTh2	ETTm1	ETTm2	Electricity	Weather	Exchange
FeDPM Variants	Transformer	MSE	0.422	0.342	0.353	0.216	0.209	0.185	0.142
	Transformer	MAE	0.424	0.368	0.376	0.281	0.303	0.229	0.283
	CNN	MSE	0.427	0.344	0.363	0.219	0.220	0.187	0.144
	CNN	MAE	0.435	0.373	0.387	0.287	0.316	0.231	0.282
	FC	MSE	0.700	0.360	0.690	0.235	0.842	0.200	0.146
	FC	MAE	0.563	0.390	0.553	0.313	0.754	0.251	0.284

Table 5: Backbone models of Encoder ablation results on seven datasets with forecasting horizons

F_{i}\in\{96,192\}

. All results are averaged over the two prediction lengths. Bold denotes the best performance across all types.

Figure 3: Model efficiency comparison on ETTh1 (

F_{i}

=96) in terms of forecasting MSE, training time, and training Parameters.

Type		FeDPM $+$ Noise						Baseline (w/o Noise)
Method		Gaussian		Exponential		Laplace		FL-iTransformer		FL-PatchTST		UniTime		Cen-PatchTST
Metric		MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
	96	0.180	0.231	0.179	0.237	0.199	0.249	0.199	0.223	0.200	0.251	0.177	0.220	0.213	0.260
	192	0.250	0.293	0.221	0.275	0.219	0.264	0.275	0.279	0.254	0.294	0.224	0.260	0.269	0.300
	336	0.288	0.321	0.271	0.312	0.272	0.307	0.341	0.330	0.311	0.336	0.279	0.277	0.330	0.341
Weather	720	0.345	0.355	0.340	0.359	0.363	0.374	0.452	0.397	0.379	0.375	0.354	0.347	0.404	0.389

Table 6: Performance of FeDPM under different privacy-preserving mechanisms across forecasting horizons

F_{i}\in\{96,192,336,720\}

. Blue : best result among FeDPM variants with noise injection; Green : best result among baseline methods; Bold : best result across all methods.

Figure 4: Visualization of prototypes on the Weather dataset. (a) Input patches ( $P_{n}=4$ ) and their corresponding representative prototypes in the time domain, where thick lines denote prototypes and thin lines denote input patches. (b) Patch Representation and prototype embeddings projected into a shared latent space using t-SNE ( Maaten and Hinton, 2008 ) .

Notation	Description
Problem Definition & Data
$N$	Number of domains (clients)
$n$	Index of the domain, $n\in\{1,\dots,N\}$
$\mathcal{D}_{n}$	Local dataset of domain $n$
$\boldsymbol{X}_{n}$	Input time series sequence, $\boldsymbol{X}_{n}\in\mathbb{R}^{L_{n}\times c_{n}}$
$\boldsymbol{Y}_{n}$	Ground truth (future) sequence, $\boldsymbol{Y}_{n}\in\mathbb{R}^{F_{n}\times c_{n}}$

Table 7: Summary of Notations used in FeDPM .

Method	Dataset	ETTh1		ETTh2		ETTm1		ETTm2		Electricity		Exchange		Weather
Method	Length	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
LLMs	96	0.406	0.404	0.293	0.341	0.357	0.373	0.180	0.264	0.207	0.295	0.087	0.203	0.198	0.238
	192	0.460	0.434	0.372	0.391	0.399	0.393	0.245	0.304	0.209	0.300	0.187	0.304	0.242	0.273
	336	0.504	0.453	0.413	0.426	0.428	0.411	0.306	0.343	0.225	0.316	0.341	0.421	0.295	0.310
	720	0.495	0.466	0.419	0.440	0.490	0.444	0.404	0.398	0.264	0.344	0.891	0.714	0.370	0.358
Transformer	96	0.391	0.409	0.307	0.354	0.338	0.373	0.185	0.272	0.181	0.270	0.082	0.202	0.179	0.224

Table 8: Results of ablation experiments on Time-FFM ( Liu et al., 2024a ) . Bold denotes the best performance, and underlined results indicate improvements over LLM-based baselines.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	$c_{n}$	Dataset Size	Batch Size	Frequency	Application Domain
ETTh1	7	(8545, 2881, 2881)	32	1 hour	Electrical Asset Monitoring
ETTh2	7	(8545, 2881, 2881)	32	1 hour	Electrical Asset Monitoring
ETTm1	7	(34465, 11521, 11521)	64	15 minutes	Electrical Asset Monitoring
ETTm2	7	(34465, 11521, 11521)	64	15 minutes	Electrical Asset Monitoring
Electricity	321	(18317, 2633, 5261)	24	1 hour	Energy Consumption
Weather	21	(36792, 5271, 10540)	64	10 minutes	Weather Forecasting

Appendix

Table 9: Detailed descriptions of datasets. The dataset size is organized in (training, validation, test).

Panel of Figure 5 — Figure 5: Hyperparameter Sensitivity Analysis. We evaluate the effects of five key hyperparameters across four datasets under two forecasting horizons, $F_{i}\in\{96,192\}$ .

Few-shot Long-term Forecasting (5%)
Type		FL-FM										Cen-FM
Method		FeDPM		Time-FFM		FFTS		FL-iTransformer		FL-PatchTST		TOTEM		UniTime		Cen-PatchTST
Metric		MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTm1	96	0.472	0.441	0.515	0.459	0.538	0.492	0.879	0.601	0.866	0.548	0.928	0.693	0.576	0.498	0.559	0.477
	192	0.499	0.461	0.550	0.478	0.565	0.507	1.093	0.671	0.869	0.558	0.905	0.691	0.617	0.520	0.588	0.493
	336	0.558	0.490	0.563	0.491	0.619	0.531	1.112	0.690	0.839	0.562	0.894	0.697	0.633	0.533	0.587	0.497

Appendix

Table 10: Few-shot results of forecasting performance comparisons. Bold: the best over all types. ‘-’ means time series data is not sufficient to constitute a training set.

Time-Series Foundation Models

Time-series foundation models are pretrained sequence models designed to transfer across forecasting, classification, anomaly detection, and other temporal tasks.

See all 298 papers

Bi-level Heterogeneous Learning for Time Series Foundation Models: A Federated Learning Approach
Apr 8, 2026Shengchao Chen, Guodong Long, Dikai Liu +1Time-Series Foundation Models Time Series Forecasting
Australian AI Institute, University of Technology Sydney · Robotics Institute, University of Technology Sydney
FedChronos: Federated Fine-Tuning of Time-Series Foundation Models for Privacy-Preserving Commodity Price Forecasting
Aug 2, 2026Amit Sharma, Nitin Auluck, Akramul AzimTime Series Forecasting Federated Fine-Tuning
Indian Institute of Technology Ropar, Punjab, India · Ontario Tech University, Oshawa, Canada
EFFEKT: Efficient Federated Knowledge Transfer to Foundation Models
Aug 8, 2026Matteo Caligiuri, Francesco Barbato, Pietro Zanuttigh +1Federated Knowledge Distillation Cross-Architecture Knowledge Distillation
Department of Electrical & Computer Engineering, Northeastern University, Boston (MA), United States · Department of Information Engineering, University of Padua, Padua, Italy

Apr 8, 2026J/K move · Enter open · S save

Bi-level Heterogeneous Learning for Time Series Foundation Models: A Federated Learning Approach

Shengchao Chen, Guodong Long, Dikai Liu, Jing Jiang

Australian AI Institute, University of Technology Sydney · Robotics Institute, University of Technology Sydney

Time-Series Foundation Models Time Series Forecasting Heterogeneous Federated Learning Federated Learning Aggregation

Open paper AskPDF arXiv

Heterogeneity in time series data is more pronounced than in vision or language, as temporal dynamics vary substantially across domains and tasks. Existing efforts on training time series foundation models (TSFMs) from scratch are often trained with mixed-batch strategies that merge large-scale datasets, which can cause gradient conflicts and degrade representation quality. To address this, we propose a fine-grained learning method that distills invariant knowledge from heterogeneous series while reducing cross-domain interference. We characterize heterogeneity at two levels: inter-domain and intra-domain. To tackle this bi-level heterogeneity, we design a federated learning method that mitigates intra-domain conflicts by enforcing domain-invariant and semantically consistent representations through local regularization, and addresses inter-domain discrepancies by enhancing cross-domain collaboration via domain-aware aggregation. Experiments across diverse benchmarks show that TSFMs trained with our method consistently outperform both centralized and federated TSFM baselines in point and probabilistic forecasting, while also achieving competitive zero-shot performance at scale, offering a flexible pathway for training TSFMs from scratch in heterogeneous environments.

Abstract

Time-Series Foundation Models

Explore similar work

Bi-level Heterogeneous Learning for Time Series Foundation Models: A Federated Learning Approach

FedChronos: Federated Fine-Tuning of Time-Series Foundation Models for Privacy-Preserving Commodity Price Forecasting

EFFEKT: Efficient Federated Knowledge Transfer to Foundation Models