DriftGuard: Mitigating Asynchronous Data Drift in Federated Learning

Yizhou Han; Di Wu; Blesson Varghese

cs.LGMar 19, 2026

DriftGuard: Mitigating Asynchronous Data Drift in Federated Learning

Authors: Yizhou Han, Di Wu, Blesson Varghese

Organizations: School of Computer Science, University of St Andrews, UK

Abstract

In real-world Federated Learning (FL) deployments, data distributions on devices that participate in training evolve over time. This leads to asynchronous data drift, where different devices shift at different times and toward different distributions. Mitigating such drift is challenging: frequent retraining incurs high computational cost on resource-constrained devices, while infrequent retraining degrades performance on drifting devices. We propose DriftGuard, a federated continual learning framework that efficiently adapts to asynchronous data drift. DriftGuard adopts a Mixture-of-Experts (MoE) inspired architecture that separates shared parameters, which capture globally transferable knowledge, from local parameters that adapt to group-specific distributions. This design enables two complementary retraining strategies: (i) global retraining, which updates the shared parameters when system-wide drift is identified, and (ii) group retraining, which selectively updates local parameters for clusters of devices identified via MoE gating patterns, without sharing raw data. Experiments across multiple datasets and models show that DriftGuard matches or exceeds state-of-the-art accuracy while reducing total retraining cost by up to 83%. As a result, it achieves the highest accuracy per unit retraining cost, improving over the strongest baseline by up to 2.3x. DriftGuard is available for download from https://github.com/blessonvar/DriftGuard.

Figures & tables

Fig. 1 : Illustration of synchronous and asynchronous data drift.

Fig. 2 : The three steps in the FL retraining framework under asynchronous data drift. Step 1: At each time step $t$ , devices perform local inference and report observations to the server. Step 2: The server determines the retraining configuration $\pi^{t}=(Trig,S,\theta)$ , which specifies whether to retrain ( $Trig$ ), which devices participate ( $S$ ), and which parameters to update ( $\theta$ ). Step 3: If retraining is triggered, the selected devices perform FL retraining on the specified parameters.

Fig. 3 : The MoE-based architecture in DriftGuard . A branch-level soft gating network decomposes the model into a shared branch (blue) and a local branch (orange), containing shared and local parameters respectively. Within the shared branch, a layer-level hard gating network activates different subsets of neurons for inputs from different distributions.

Component Observed	Sample Value	Description
Average Local Accuracy	$0.60$	Accuracy on recent local data
Aggregated Gating Matrix	$\begin{bmatrix}0.1&\cdots&0.9\\ 0.4&\cdots&0.6\\ \vdots&\ddots&\vdots\end{bmatrix}$	Average of gate activations on recent local data

TABLE I : Example on-Device observation

o_{c}^{t}

Fig. 4 : The integrated pipeline of DriftGuard . At each time step $t$ , the server collects the set of observations $O^{t}=\{o_{c}^{t}\}$ via inference from all devices. Based on $O^{t}$ , the server performs clustering and generates the retraining configuration $\pi^{t}$ . The pipeline dynamically triggers either global retraining of shared parameters or group retraining of local parameters in specific groups.

Model	Base	Size	Architecture configuration
cResNet-S	ResNet	4.8M	layers=[1,1,1], experts=3, top-k=1
cResNet-M	ResNet	20.6M	layers=[2,2,1,1], experts=3, top-k=1
cViT-S	ViT	2.0M	patch=4, dim=128, depth=6, heads=4
cViT-M	ViT	9.1M	patch=16, dim=192, depth=12, heads=3

TABLE II : Models used in the evaluation of DriftGuard . For ResNet-based variants, layers lists the number of blocks per stage, experts is the number of expert sub-networks per layer in the shared branch, and top-k is the number of experts activated per input by the layer-level gating. For ViT-based variants, patch is the input patch size, dim is the feature dimension, depth is the number of transformer layers, and heads is the number of attention heads.

Dataset	#Domains	#Classes	#Images
DG5	5	10	215,695
PACS	4	7	9,991
DomainNet	6	7	16,824

TABLE III : Characteristics of the datasets.

Method	DG5		PACS		DomainNet
Method	cResNet-S	cViT-S	cResNet-M	cViT-M	cResNet-M	cViT-M
DriftGuard	12.14 (0.85 / 0.07) *	7.44 (0.67 / 0.09)	3.75 (0.60 / 0.16) *	3.47 (0.59 / 0.17) *	3.81 (0.61 / 0.16) *	8.57 (0.60 / 0.07) *
FCL-AveTrig	3.19 (0.86 / 0.27)	5.67 (0.68 / 0.12)	1.11 (0.60 / 0.54)	2.03 (0.59 / 0.29)	2.10 (0.61 / 0.29)	2.86 (0.60 / 0.21)
FCL-perDevice	7.82 (0.86 / 0.11)	9.29 (0.65 / 0.07)	1.25 (0.60 / 0.48)	2.19 (0.57 / 0.26)	1.42 (0.61 / 0.43)	2.32 (0.58 / 0.25)
PFL-AveTrig	4.05 (0.85 / 0.21)	6.80 (0.68 / 0.10)	1.28 (0.60 / 0.47)	2.68 (0.59 / 0.22)	2.44 (0.61 / 0.25)	3.75 (0.60 / 0.16)
PFL-perDevice	9.56 (0.86 / 0.09)	11.00 (0.66 / 0.06) *	1.62 (0.60 / 0.37)	2.50 (0.55 / 0.22)	1.76 (0.60 / 0.34)	2.90 (0.58 / 0.20)

TABLE IV : Efficiency comparison across datasets and models. Each cell reports

\mathcal{E}

(

\bar{A}

/TC), where

\mathcal{E}=\bar{A}/\text{TC}

measures accuracy achieved per unit retraining cost. * marks the best result for each experimental configuration.

Method	DG5		PACS		DomainNet
Method	cResNet-S	cViT-S	cResNet-M	cViT-M	cResNet-M	cViT-M
DriftGuard	13	19	16	21	17	21
FCL-AveTrig	15	24	14	21	13	23
FCL-perDevice	20	9	18	14	16	8
PFL-AveTrig	13	22	15	22	17	23
PFL-perDevice	14	14	17	8	13	9

TABLE V : Number of time steps each method remains above the reference threshold (out of 30 total time steps).

Fig. 5 : Mean accuracy ( $\bar{A}$ ) over time steps across datasets and models, with curves smoothed using a Gaussian filter ( $\sigma=1.5$ ). The dashed grey line denotes the reference threshold, defined as the median accuracy across all methods and time steps. The shaded region indicates the portion below the reference threshold.

Fig. 6 : Cumulative retraining cost (TC) over time steps. Markers denote different types of retraining events.

Fig. 7 : Mean accuracy versus total retraining time on the Raspberry Pi 4 prototype. Each point is annotated with efficiency $\mathcal{E}$ , defined as mean accuracy per retraining time.

Fig. 8 : Impact of the global trigger threshold $\tau_{global}$ in DriftGuard . Each point is annotated with $\mathcal{E}$ ( $\bar{A}$ /TC), where $\mathcal{E}$ measures accuracy achieved per unit retraining cost.

Fig. 9 : Impact of the clustering distance threshold. Each point is annotated with mean accuracy divided by retraining cost ( $\bar{A}/TC$ ).

Features	Classic FCL [ 17 , 18 , 19 ]	Clustering / Multi-model FCL [ 9 , 14 , 15 ]	Personalized FL [ 33 , 34 ]	DriftGuard
Adapting to asynchronous data drift	✗	✓	✗	✓
Decoupling global and personalized updates	✗	✗	✓	✓
Grouping devices with similar data distribution	✗	✓	✗	✓
Enabling lightweight adaptation	✗	✗	✗	✓

TABLE VI: Comparison of key features of existing FCL methods with DriftGuard under asynchronous drift.

Figure 16

Figure 17

Figure 18

Federated Learning

Federated learning trains shared models across data holders without requiring them to pool their raw data.

See all 921 papers

Reviving Stale Updates: Data-Free Knowledge Distillation for Asynchronous Federated Learning
Nov 1, 2025Baris Askin, Holger R. Roth, Zhenyu Sun +3Asynchronous Federated Learning Federated Learning Aggregation
Carnegie Mellon University · NVIDIA · Northwestern University
Latent Information Sharing for Accelerating Federated Learning
Oct 1, 2026Seungjun Lee, Ensieh Khazaei, Dimitrios Hatzinakos +2Federated Learning Aggregation Federated Learning
Inha University · University of Toronto · University of Birmingham
Federated Continual Learning as a Distributed Drift-Plus-Penalty Control Problem
Aug 21, 2026Nazreen Shah, Naveen Kumar Reddy Somireddy, Zubair Shaban +2Federated Continual Learning Catastrophic Forgetting
IIIT Delhi India · IIT Dharwad India

Nov 1, 2025J/K move · Enter open · S save

Reviving Stale Updates: Data-Free Knowledge Distillation for Asynchronous Federated Learning

Baris Askin, Holger R. Roth, Zhenyu Sun, Carlee Joe-Wong+2

Carnegie Mellon University · NVIDIA · Northwestern University

Asynchronous Federated Learning Federated Learning Aggregation

Open paper AskPDF arXiv

Federated learning (FL) enables collaborative model training across distributed clients without sharing raw data, yet its scalability is limited by synchronization overhead. Asynchronous federated learning (AFL) alleviates this issue by allowing clients to communicate independently, thereby improving wall-clock efficiency in large-scale, hardware-heterogeneous environments. However, asynchrony introduces updates computed on outdated global models (staleness) that can destabilize optimization and hinder convergence. We propose FedRevive, an AFL framework that revives stale updates through data-free knowledge distillation (DFKD). FedRevive integrates parameter-space aggregation with a server-side DFKD process that transfers knowledge from stale client updates to the current global model without access to data. A meta-learned generator synthesizes pseudo-samples for multi-teacher distillation. A hybrid aggregation scheme combining raw with DFKD updates effectively mitigates staleness while retaining AFL scalability. Experiments on vision and text benchmarks show that FedRevive achieves faster training by up to 29.7% and higher final accuracy by up to 11.4 percentage points than baselines.

Abstract

Federated Learning

Explore similar work

Reviving Stale Updates: Data-Free Knowledge Distillation for Asynchronous Federated Learning

Latent Information Sharing for Accelerating Federated Learning

Federated Continual Learning as a Distributed Drift-Plus-Penalty Control Problem