Learning From Static Context in Federated Time Series Forecasting
Authors: Viktor Walter, Andreas Wagner, Fabian Stricker
Organizations: Faculty of Management Science and Engineering, Karlsruhe University of Applied Sciences (HKA), Karlsruhe, Germany · Fraunhofer Institute for Industrial Mathematics (ITWM), Kaiserslautern, Germany · Institute of Data-Centric Software Systems (IDSS), Karlsruhe University of Applied Sciences (HKA), Karlsruhe, Germany
Time series forecasting is shifting from locally trained, per-series models to global models trained across many heterogeneous series, which requires learning what sets the series apart. When the raw data cannot be centralized, such a global model is trained with federated learning (FL), one client per data owner. State-of-the-art forecasters such as the Temporal Fusion Transformer (TFT) tell series apart through a static input that describes each series and does not change over time. In the special case of a federated setting where each client holds a single series, this static input is constant within each client and varies only across clients. We ask whether a federated TFT can learn from it. Per-batch FedAdam, which aggregates gradients after every batch, uses the static input almost as well as central training, whereas communication-efficient schemes with local epochs (FedAvg, FedAdam+Prox) extract far less from it and gain nothing significant on either application dataset. At the same local computation, an Adam-compatible SCAFFOLD keeps the static pathway responsive on wind power and gains significantly there. On building energy its error still drops, but the static pathway falls silent, so drift and communication frequency cannot be fully separated as the cause. Only per-batch FedAdam matches or beats the per-client local models across all datasets. We confirm the effect on building-energy meter data, a wind-power dataset, and a controlled synthetic dataset. The loss is not specific to the TFT: on the synthetic and the wind data two further architectures with different static-conditioning mechanisms show it as well.
Figures & tables
Fig. 1: TFT with its temporal pathway (blue) and static pathway (orange). Encoder (observed past) and decoder (known future) inputs are separate. The static context vectors condition both at three points: variable selection, LSTM state initialization, and GRN enrichment. Both GRNs feed the shared self-attention
Scheme
Local steps per round
Aggregated
Server opt.
Per-batch FedAdam
1
gradients
Adam
FedAvg
one epoch ( b )
weights
none
FedAdam+Prox
one epoch ( b )
deltas
Adam
SCAFFOLD
one epoch ( b )
deltas + CV
none
TABLE I: Federated aggregation schemes. b is the number of batches in one local epoch: 214 to 314 on building energy, 45 on wind power, 7 on the synthetic data. CV: control variates
Setup
Observed
Known
Static
Synthetic
target lagged
sin/cos 24h, sin/cos 168h
entity feature
Building energy
load lagged, air temp., wind speed
hour, day of week (sin/cos), weekend flag
primary use floor area year built floor count
Wind power
power lagged
wind speed ICON-D2: 78m, 127m, 184m ECMWF HRES: 100m
turbine age power curve hub height rotor diam. altitude
TABLE II: Input variables per setup
Synthetic
Building energy
Wind power
dmodel
16
64
64
Attention heads
2
4
4
Lookback
24
72
48
Horizon
24
24
48
Dropout
0
0.1
0.1
TABLE III: Model architecture and windowing per setup
Scheme
Static
RMSE
MAE
R2
p(static↓)
Central
✓
0.36
0.28
0.95
< .001*
Central
–
1.42
1.25
0.22
–
Per-batch FedAdam
✓
0.35
0.28
0.95
< .001*
Per-batch FedAdam
–
1.43
1.26
0.22
–
FedAdam+Prox
✓
1.00
0.75
0.62
0.026*
FedAdam+Prox
–
1.43
1.25
0.21
–
TABLE IV: Synthetic data. Mean over seeds and clients. p : one-sided Wilcoxon signed-rank test over per-client RMSE pairs. * and shading: p<0.05
Fig. 2: Static-input sensitivity relative to central per dataset. Central and per-batch FedAdam stay sensitive on all three datasets. The schemes with local epochs keep a small share on the synthetic data and drop to near zero on the application datasets, except SCAFFOLD on wind power
Synthetic
Building energy
Wind power
PB
FA
PB
FA
PB
FA
TFT
1.01
0.11
1.01
0.00
0.96
0.00
MLP
0.17
0.01
0.09
0.00
0.23
0.08
GRU
0.68
0.01
1.21
0.37
0.75
0.00
TABLE V: Static-input sensitivity per architecture under per-batch FedAdam (PB) and under FedAvg (FA), relative to the central TFT on the same dataset as in Figure 2 . A drop to near zero means the architecture stops responding to the static input
Scheme
Static
RMSE
MAE
R2
p(static↓)
Central
✓
0.382
0.222
0.883
< .001*
Central
–
0.394
0.228
0.876
–
Per-batch FedAdam
✓
0.382
0.220
0.880
0.007*
Per-batch FedAdam
–
0.390
0.225
0.875
–
FedAdam+Prox
✓
0.505
0.348
0.796
0.640
FedAdam+Prox
–
0.511
0.349
0.797
–
TABLE VI: BDG2 building-energy data. Mean over seeds, p as in Table IV . A significant p says the error drops, Figure 2 says whether the static input is used
Fig. 3: Per-building RMSE improvement over the local baseline on building energy, without and with static features. Boxes mark the quartiles and the median. Values above zero beat the local model
Scheme
Static
RMSE
MAE
R2
p(static↓)
Central
✓
0.094
0.055
0.761
< .001*
Central
–
0.100
0.059
0.725
–
Per-batch FedAdam
✓
0.095
0.056
0.761
< .001*
Per-batch FedAdam
–
0.100
0.059
0.728
–
FedAdam+Prox
✓
0.113
0.069
0.598
0.789
FedAdam+Prox
–
0.113
0.068
0.601
–
TABLE VII: Wind power data. Mean over seeds, p as in Table IV
Synthetic
Building energy
Wind power
GB/cl.
GB
GB/cl.
GB
GB/cl.
GB
Train data
0.0002
0.002
0.037
1.18
0.0028
0.090
Per-batch FedAdam
0.109
1.31
9.46
302.6
29.39
940.6
FedAvg
0.031
0.38
0.301
9.6
0.842
27.0
FedAdam+Prox
0.026
0.32
0.295
9.4
0.693
22.2
SCAFFOLD
0.038
0.45
0.360
11.5
1.375
44.0
TABLE VIII: Communication cost per scheme and dataset: data sent and received per client (GB/cl.) and total (GB) over K clients ( K=12 synthetic, K=32 applications)
Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.
Federated adaptation of time-series foundation models (TSFMs) is attractive for building energy forecasting because meter data are private, distributed, and highly non-IID. However, a single parameter-sharing strategy is unlikely to serve all pretrained TSFMs or building clients: fully shared adapters can suppress building-specific temporal behavior, while fully local adaptation discards cross-building transfer. We propose a personalized federated sparse adaptation framework with a heterogeneous temporal mixture-of-experts (MoE) adapter placed after the pretrained TSFM representation. A sequence-level router maps each 168-hour context window to a top-k subset of experts specialized for periodicity, long-range interactions, local variation, trend-residual structure, and multi-resolution behavior. We compare global FL, local training, and personalized FL variants with globally shared or client-private expert banks. Across 50 buildings and three TSFM backbones, personalization consistently outperforms Global FL-MoE and Local MoE, while the best sparse-adaptation strategy varies by backbone and metric. Routing behavior further reveals client-level expert specialization, expert concentration, and near-uniform routing across backbones, showing that federated TSFM adaptation should be both client-aware and backbone-aware.
Priyanka Nihalchandani, Naman Srivastava, Varun Ojha +1
Robert Bosch Centre for Cyber-Physical Systems, Indian Institute of Science, Bengaluru, India · School of Computing, Newcastle University, Newcastle upon Tyne, United Kingdom
Federated learning (FL) on graphs shows promise for distributed time-series forecasting. Yet, existing methods rely on static topologies and struggle with client heterogeneity. We propose Fed-GAME, a framework that models personalized aggregation as message passing over a learnable dynamic implicit graph. The core is a decoupled parameter difference-based update protocol, where clients transmit parameter differences between their fine-tuned private model and a shared global model. On the server, these differences are decomposed into two streams: (1) averaged difference used to updating the global model for consensus (2) the selective difference fed into a novel Graph Attention Mixture-of-Experts (GAME) aggregator for fine-grained personalization. In this aggregator, shared experts provide scoring signals while personalized gates adaptively weight selective updates to support personalized aggregation. Experiments on two real-world electric vehicle charging datasets demonstrate that Fed-GAME outperforms state-of-the-art personalized FL baselines.
Yi Li, Han Liu, Mingfeng Fan +3
School of Automation, Central South University · Department of Electrical and Computer Engineering, National University of Singapore · Department of Mechanical Engineering, National University of Singapore +2
Heterogeneity in time series data is more pronounced than in vision or language, as temporal dynamics vary substantially across domains and tasks. Existing efforts on training time series foundation models (TSFMs) from scratch are often trained with mixed-batch strategies that merge large-scale datasets, which can cause gradient conflicts and degrade representation quality. To address this, we propose a fine-grained learning method that distills invariant knowledge from heterogeneous series while reducing cross-domain interference. We characterize heterogeneity at two levels: inter-domain and intra-domain. To tackle this bi-level heterogeneity, we design a federated learning method that mitigates intra-domain conflicts by enforcing domain-invariant and semantically consistent representations through local regularization, and addresses inter-domain discrepancies by enhancing cross-domain collaboration via domain-aware aggregation. Experiments across diverse benchmarks show that TSFMs trained with our method consistently outperform both centralized and federated TSFM baselines in point and probabilistic forecasting, while also achieving competitive zero-shot performance at scale, offering a flexible pathway for training TSFMs from scratch in heterogeneous environments.
Shengchao Chen, Guodong Long, Dikai Liu +1
Australian AI Institute, University of Technology Sydney · Robotics Institute, University of Technology Sydney