Learning From Static Context in Federated Time Series Forecasting

Viktor Walter; Andreas Wagner; Fabian Stricker

cs.LGNov 4, 2026

Learning From Static Context in Federated Time Series Forecasting

Authors: Viktor Walter, Andreas Wagner, Fabian Stricker

Organizations: Faculty of Management Science and Engineering, Karlsruhe University of Applied Sciences (HKA), Karlsruhe, Germany · Fraunhofer Institute for Industrial Mathematics (ITWM), Kaiserslautern, Germany · Institute of Data-Centric Software Systems (IDSS), Karlsruhe University of Applied Sciences (HKA), Karlsruhe, Germany

Abstract

Time series forecasting is shifting from locally trained, per-series models to global models trained across many heterogeneous series, which requires learning what sets the series apart. When the raw data cannot be centralized, such a global model is trained with federated learning (FL), one client per data owner. State-of-the-art forecasters such as the Temporal Fusion Transformer (TFT) tell series apart through a static input that describes each series and does not change over time. In the special case of a federated setting where each client holds a single series, this static input is constant within each client and varies only across clients. We ask whether a federated TFT can learn from it. Per-batch FedAdam, which aggregates gradients after every batch, uses the static input almost as well as central training, whereas communication-efficient schemes with local epochs (FedAvg, FedAdam+Prox) extract far less from it and gain nothing significant on either application dataset. At the same local computation, an Adam-compatible SCAFFOLD keeps the static pathway responsive on wind power and gains significantly there. On building energy its error still drops, but the static pathway falls silent, so drift and communication frequency cannot be fully separated as the cause. Only per-batch FedAdam matches or beats the per-client local models across all datasets. We confirm the effect on building-energy meter data, a wind-power dataset, and a controlled synthetic dataset. The loss is not specific to the TFT: on the synthetic and the wind data two further architectures with different static-conditioning mechanisms show it as well.

Figures & tables

Fig. 1: TFT with its temporal pathway (blue) and static pathway (orange). Encoder (observed past) and decoder (known future) inputs are separate. The static context vectors condition both at three points: variable selection, LSTM state initialization, and GRN enrichment. Both GRNs feed the shared self-attention

Scheme	Local steps per round	Aggregated	Server opt.
Per-batch FedAdam	1	gradients	Adam
FedAvg	one epoch ( $b$ )	weights	none
FedAdam+Prox	one epoch ( $b$ )	deltas	Adam
SCAFFOLD	one epoch ( $b$ )	deltas $+$ CV	none

TABLE I: Federated aggregation schemes.

b

is the number of batches in one local epoch: 214 to 314 on building energy, 45 on wind power, 7 on the synthetic data. CV: control variates

Setup	Observed	Known	Static
Synthetic	target lagged	sin/cos 24h, sin/cos 168h	entity feature
Building energy	load lagged, air temp., wind speed	hour, day of week (sin/cos), weekend flag	primary use floor area year built floor count
Wind power	power lagged	wind speed ICON-D2: 78m, 127m, 184m ECMWF HRES: 100m	turbine age power curve hub height rotor diam. altitude

TABLE II: Input variables per setup

	Synthetic	Building energy	Wind power
$d_{\text{model}}$	16	64	64
Attention heads	2	4	4
Lookback	24	72	48
Horizon	24	24	48
Dropout	0	0.1	0.1

TABLE III: Model architecture and windowing per setup

Scheme	Static	RMSE	MAE	$R^{2}$	$p(\text{static}{\downarrow})$
Central	✓	0.36	0.28	0.95	$<$ .001*
Central	–	1.42	1.25	0.22	–
Per-batch FedAdam	✓	0.35	0.28	0.95	$<$ .001*
Per-batch FedAdam	–	1.43	1.26	0.22	–
FedAdam+Prox	✓	1.00	0.75	0.62	0.026*
FedAdam+Prox	–	1.43	1.25	0.21	–

TABLE IV: Synthetic data. Mean over seeds and clients.

p

: one-sided Wilcoxon signed-rank test over per-client RMSE pairs. * and shading:

p<0.05

Fig. 2: Static-input sensitivity relative to central per dataset. Central and per-batch FedAdam stay sensitive on all three datasets. The schemes with local epochs keep a small share on the synthetic data and drop to near zero on the application datasets, except SCAFFOLD on wind power

	Synthetic		Building energy		Wind power
	PB	FA	PB	FA	PB	FA
TFT	1.01	0.11	1.01	0.00	0.96	0.00
MLP	0.17	0.01	0.09	0.00	0.23	0.08
GRU	0.68	0.01	1.21	0.37	0.75	0.00

TABLE V: Static-input sensitivity per architecture under per-batch FedAdam (PB) and under FedAvg (FA), relative to the central TFT on the same dataset as in Figure 2 . A drop to near zero means the architecture stops responding to the static input

Scheme	Static	RMSE	MAE	$R^{2}$	$p(\text{static}{\downarrow})$
Central	✓	0.382	0.222	0.883	$<$ .001*
Central	–	0.394	0.228	0.876	–
Per-batch FedAdam	✓	0.382	0.220	0.880	0.007*
Per-batch FedAdam	–	0.390	0.225	0.875	–
FedAdam+Prox	✓	0.505	0.348	0.796	0.640
FedAdam+Prox	–	0.511	0.349	0.797	–

TABLE VI: BDG2 building-energy data. Mean over seeds,

p

as in Table IV . A significant

p

says the error drops, Figure 2 says whether the static input is used

Fig. 3: Per-building RMSE improvement over the local baseline on building energy, without and with static features. Boxes mark the quartiles and the median. Values above zero beat the local model

Scheme	Static	RMSE	MAE	$R^{2}$	$p(\text{static}{\downarrow})$
Central	✓	0.094	0.055	0.761	$<$ .001*
Central	–	0.100	0.059	0.725	–
Per-batch FedAdam	✓	0.095	0.056	0.761	$<$ .001*
Per-batch FedAdam	–	0.100	0.059	0.728	–
FedAdam+Prox	✓	0.113	0.069	0.598	0.789
FedAdam+Prox	–	0.113	0.068	0.601	–

TABLE VII: Wind power data. Mean over seeds,

p

as in Table IV

	Synthetic		Building energy		Wind power
	GB/cl.	GB	GB/cl.	GB	GB/cl.	GB
Train data	0.0002	0.002	0.037	1.18	0.0028	0.090
Per-batch FedAdam	0.109	1.31	9.46	302.6	29.39	940.6
FedAvg	0.031	0.38	0.301	9.6	0.842	27.0
FedAdam+Prox	0.026	0.32	0.295	9.4	0.693	22.2
SCAFFOLD	0.038	0.45	0.360	11.5	1.375	44.0

TABLE VIII: Communication cost per scheme and dataset: data sent and received per client (GB/cl.) and total (GB) over

K

clients (

K{=}12

synthetic,

K{=}32

applications)

Time Series Forecasting

Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.

See all 910 papers

Personalized Federated Sparse Adaptation of Time-Series Foundation Models
Aug 5, 2026Priyanka Nihalchandani, Naman Srivastava, Varun Ojha +1Mixture of Experts Time Series Forecasting
Robert Bosch Centre for Cyber-Physical Systems, Indian Institute of Science, Bengaluru, India · School of Computing, Newcastle University, Newcastle upon Tyne, United Kingdom
Fed-GAME: Personalized Federated Learning with Graph Attention Mixture-of-Experts For Time-Series Forecasting
Mar 2, 2026Yi Li, Han Liu, Mingfeng Fan +3Federated Learning Federated Graph Learning
School of Automation, Central South University · Department of Electrical and Computer Engineering, National University of Singapore · Department of Mechanical Engineering, National University of Singapore +2
Bi-level Heterogeneous Learning for Time Series Foundation Models: A Federated Learning Approach
Apr 8, 2026Shengchao Chen, Guodong Long, Dikai Liu +1Time-Series Foundation Models Time Series Forecasting
Australian AI Institute, University of Technology Sydney · Robotics Institute, University of Technology Sydney

Aug 5, 2026J/K move · Enter open · S save

Personalized Federated Sparse Adaptation of Time-Series Foundation Models

Priyanka Nihalchandani, Naman Srivastava, Varun Ojha, Pandarasamy Arjunan

Robert Bosch Centre for Cyber-Physical Systems, Indian Institute of Science, Bengaluru, India · School of Computing, Newcastle University, Newcastle upon Tyne, United Kingdom

Mixture of Experts Time Series Forecasting Sparse Fine-Tuning Federated Fine-Tuning Time-Series Foundation Models Federated Learning+1

Open paper AskPDF arXiv

Federated adaptation of time-series foundation models (TSFMs) is attractive for building energy forecasting because meter data are private, distributed, and highly non-IID. However, a single parameter-sharing strategy is unlikely to serve all pretrained TSFMs or building clients: fully shared adapters can suppress building-specific temporal behavior, while fully local adaptation discards cross-building transfer. We propose a personalized federated sparse adaptation framework with a heterogeneous temporal mixture-of-experts (MoE) adapter placed after the pretrained TSFM representation. A sequence-level router maps each 168-hour context window to a top-

k

subset of experts specialized for periodicity, long-range interactions, local variation, trend-residual structure, and multi-resolution behavior. We compare global FL, local training, and personalized FL variants with globally shared or client-private expert banks. Across 50 buildings and three TSFM backbones, personalization consistently outperforms Global FL-MoE and Local MoE, while the best sparse-adaptation strategy varies by backbone and metric. Routing behavior further reveals client-level expert specialization, expert concentration, and near-uniform routing across backbones, showing that federated TSFM adaptation should be both client-aware and backbone-aware.

Abstract

Time Series Forecasting

Explore similar work

Personalized Federated Sparse Adaptation of Time-Series Foundation Models

Fed-GAME: Personalized Federated Learning with Graph Attention Mixture-of-Experts For Time-Series Forecasting

Bi-level Heterogeneous Learning for Time Series Foundation Models: A Federated Learning Approach