cs.LGJun 29, 2025

Effective Unbounded Drift Control in Heterogeneous Federated Learning: Bayesian Fine-Tuning Using a Reference Model

Authors: Taehwan Yoon, Wesley De Neve, Bongjun Choi

Organizations: School of Computer Science and Engineering, Soongsil University, Seoul, Republic of Korea · Center for Biosystems and Biotech Data Science, Ghent University, Global Campus, Incheon, Republic of Korea

Abstract

Federated learning (FL) enables collaborative model training across distributed clients while preserving data privacy. However, data and system heterogeneity often induce unbounded drift in model updates due to each clients' different multiple local steps, partial participation, and unbounded dynamics in model updates. These challenges severely degrade predictive performance, exacerbate catastrophic forgetting, and hinder global optimization forward to optimal updates. Related works (AdaBest, FedEve, A-FedPD) defined dual drift problems and proposed each regularization term to client optimization process. However, in aspect of temporal unbounded drift, they generally reflect last round temporal drift to client optimization while forgets prior temporal unbounded drift. To address these limitations, we propose FedRef, a robust optimization framework that leverages a reference model generalized from prior global models via Bayesian fine-tuning. FedRef integrates a Maximum A Posteriori (MAP)-based regularization mechanism that calibrates global updates toward a temporally aggregated reference model, thereby effectively controlling unbounded drift and maximizing empirical stability. Moreover, unlike prior approaches that rely on complex client-side optimization heuristics, FedRef achieves superior generalization to reflect recency weighting by seamlessly blending a robust probabilistic prior with global likelihood optimization. By maintaining a temporal moving average centered on previous global trajectories, the proposed framework provides a highly reliable optimization anchor.

Figures & tables

Explore similar work

Mar 19, 2026

DriftGuard: Mitigating Asynchronous Data Drift in Federated Learning

In real-world Federated Learning (FL) deployments, data distributions on devices that participate in training evolve over time. This leads to asynchronous data drift, where different devices shift at different times and toward different distributions. Mitigating such drift is challenging: frequent retraining incurs high computational cost on resource-constrained devices, while infrequent retraining degrades performance on drifting devices. We propose DriftGuard, a federated continual learning framework that efficiently adapts to asynchronous data drift. DriftGuard adopts a Mixture-of-Experts (MoE) inspired architecture that separates shared parameters, which capture globally transferable knowledge, from local parameters that adapt to group-specific distributions. This design enables two complementary retraining strategies: (i) global retraining, which updates the shared parameters when system-wide drift is identified, and (ii) group retraining, which selectively updates local parameters for clusters of devices identified via MoE gating patterns, without sharing raw data. Experiments across multiple datasets and models show that DriftGuard matches or exceeds state-of-the-art accuracy while reducing total retraining cost by up to 83%. As a result, it achieves the highest accuracy per unit retraining cost, improving over the strongest baseline by up to 2.3x. DriftGuard is available for download from https://github.com/blessonvar/DriftGuard.
Nov 16, 2026

STORMGUIDE-FL: Stochastic Remote Guidance for Efficient Mining of Federated Streaming Data

Federated Learning enables collaborative model training across distributed edge devices ("clients") while preserving data privacy. However, real-world deployments involve clients that observe continuous data streams from evolving environments while operating under strict resource limitations. Existing solutions still face two major challenges: local concept drift, where each client's data distribution shifts over time, and global client drift, arising from asynchronous engagement and imbalanced contributions across heterogeneous training participants, formally described as dual client heterogeneity. We propose STORMGUIDE-FL, a federated optimization and data-mining framework that mitigates both types of drift using a server-guided pipeline for data acquisition and local training. For data acquisition, a lightweight client-side sampling strategy incrementally selects informative and diverse instances from non-stationary, non-IID local streams under bounded memory, compute, and labeling budget. For local training, the server's remote guidance is embedded directly into local optimization as an update-level correction, counteracting client drift without discarding scarce labeled samples. We empirically verify the effectiveness of the proposed sampling strategy on federated data streams across gradual, sudden, and recurring concept drift, and demonstrate that its gains compound with the update-level drift correction. Experimental results further show improved speed and robustness of global model convergence under resource-constrained, heterogeneous conditions.
Jun 20, 2026

FedCausal-Dyn: A Causal-Dynamic Paradigm for Federated Learning under Dynamic Feature Drift

This paper addresses the challenging problem of dynamic feature drift in federated learning, where data distributions evolve across clients and over time -- a common scenario in real-world applications like financial technology. Existing approaches often assume static drift, limiting their effectiveness in non-stationary environments. To overcome this, we propose FedCausal-Dyn, a novel federated learning framework built on a causal-dynamic paradigm. Its key innovation is causal-domain feature separation, which disentangles domain-invariant causal features from spurious, domain-specific variations via specialized projection heads and adversarial training. This enables reliable and dynamic prototype aggregation, weighting local class prototypes by estimated reliability before global aggregation. We further introduce causal-feature guided collaborative regularization, unifying prototype contrastive alignment and domain invariance into a cohesive objective. Extensive experiments on three federated domain generalization benchmarks demonstrate that FedCausal-Dyn consistently achieves state-of-the-art performance, with the highest average accuracy and the most stable results. Ablation studies confirm each component's critical contribution. Our work provides a robust and principled solution for federated learning under dynamic feature drift.