Federated learning (FL) enables collaborative model training across distributed clients without sharing raw data, yet its scalability is limited by synchronization overhead. Asynchronous federated learning (AFL) alleviates this issue by allowing clients to communicate independently, thereby improving wall-clock efficiency in large-scale, hardware-heterogeneous environments. However, asynchrony introduces updates computed on outdated global models (staleness) that can destabilize optimization and hinder convergence. We propose FedRevive, an AFL framework that revives stale updates through data-free knowledge distillation (DFKD). FedRevive integrates parameter-space aggregation with a server-side DFKD process that transfers knowledge from stale client updates to the current global model without access to data. A meta-learned generator synthesizes pseudo-samples for multi-teacher distillation. A hybrid aggregation scheme combining raw with DFKD updates effectively mitigates staleness while retaining AFL scalability. Experiments on vision and text benchmarks show that FedRevive achieves faster training by up to 29.7% and higher final accuracy by up to 11.4 percentage points than baselines.
Figures & tables
Figure 1: Proposed FedRevive framework.
Figure 2: Test accuracy versus simulated time on CIFAR-10 , CIFAR-100 , 20NewsGroups , and FEMNIST . On CIFAR-10 , FedRevive converges faster and reaches higher final accuracy than the baselines. On the harder CIFAR-100 task, its advantage becomes more pronounced later in training, when client models provide stronger teachers. On 20NewsGroups , FedRevive converges faster than the baselines. On FEMNIST , asynchronous methods show similar overall convergence due to the relative simplicity of the task, but FedRevive attains among the highest final accuracy.
Method
CIFAR-10
CIFAR-100
20NewsGroups
FEMNIST
FedAvg
63.18 (631.1)
45.73 (>1000)
68.46 (51.2)
63.62 (>1000)
FedBuff
66.23 (96.7)
44.46 (320.7)
67.93 (13.5)
74.43 (38.0)
AFL-DW
53.62 (—)
37.89 (525.1)
67.40 (14.3)
75.07 (35.3)
FADAS
59.89 (214.7)
31.09 (—)
67.20 (18.1)
76.90 (146.0)
FedAC
59.71 (458.9)
43.39 (540.0)
67.14 (16.6)
78.34 (70.5)
FedFa
70.08 (90.3)
45.18 (318.3)
68.00 (11.1)
75.43 (44.8)
Table 1: Final test accuracy and time-to-target, reported as accuracy% (time), where time is the mean simulated time to reach 85% of FedBuff peak accuracy. Per dataset, the best two values are bold and the third best is underlined, separately for accuracy and time. FedRevive provides the strongest accuracy and speed on most datasets.
Figure 4
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Average DKL(πi∥π^i)
CIFAR-10
0.4240 ± 0.0103
CIFAR-100
0.5539 ± 0.0174
20NewsGroups
0.5889 ± 0.0312
FEMNIST
0.4561 ± 0.0554
Appendix
Table 2: KL divergence between true client label proportions and the server-derived proxy proportions, reported as mean ± std across seeds. We report the average client KL after the second received update.
Figure 5: FedRevive with estimated client label proportions on CIFAR-10 , CIFAR-100 , and FEMNIST . Replacing ground-truth client label proportions with our probing-based proxies yields similar training dynamics and final performance across datasets.
Dataset
Nb. of classes
Top-1 acc.
Top-2 acc.
CIFAR-10
10
0.456±0.022
0.673±0.032
CIFAR-100
100
0.135±0.001
0.219±0.010
20NewsGroups
20
0.198±0.015
0.276±0.062
FEMNIST
62
0.175±0.045
0.338±0.014
Appendix
Table 3: Top-1 and top-2 dominant-class recovery of the server-derived proxy proportions, reported as mean ± std across seeds. Top-1 accuracy checks whether argmaxjπ^i,j matches the true dominant class of client i ; top-2 accuracy checks whether the true dominant class appears in the proxy’s top-2 classes.
Dataset
Metric
Dirichlet concentration α
0.05
0.10
0.30
0.50
1.00
5.00
CIFAR-10
Top-1
0.930
0.899
0.822
0.741
0.691
0.437
Top-2
0.995
0.989
0.960
0.915
0.872
0.611
CIFAR-100
Top-1
0.708
0.671
0.528
0.472
0.382
0.289
Top-2
0.876
0.847
0.679
0.626
0.551
0.468
Appendix
Table 4: Dominant-class recovery across Dirichlet concentration levels after one local training round from random initialization. Entries are recovery rates across 1000 clients; smaller α indicates stronger label skew.
Table 5: Compute overhead breakdown (FLOPs) per sample for CIFAR-10 experiments. FLOPs (G) for dominant components. c denotes the teacher buffer size. † Rows report total compute except the last row, which reports effective blocking compute with concurrent implementation (total FLOPs unchanged).
Figure 6: Distribution of update staleness across server rounds for a fully asynchronous setup. The distribution aligns with real-world AFL systems reported by Nguyen et al. [38] .
Figure 7: Wall-clock evaluation with server overhead on CIFAR-10 under different hardware assumptions. We adjust the CIFAR-10 timelines by adding measured and converted per-aggregation server runtimes for several server configurations (H200, H100, Tesla T4, CPU-32, and CPU-16), while leaving the underlying update sequence and training dynamics unchanged. With strong accelerators, the conclusions are essentially unchanged; with weaker servers, FedRevive and DD-FedRevive slow down in wall-clock time but still retain a clear accuracy advantage over the asynchronous baselines.
Figure 8: Staleness distribution induced by the modified delay schedule used in the experiment for robustness to change in delay scheme in Section 3.2 . For reference, the default distribution is shown in Fig. 6 .
Figure 9: Test accuracy over simulated time on CIFAR-10 , CIFAR-100 , and 20NewsGroups under milder heterogeneity ( α=1 ). FedRevive converges faster than FedBuff and achieves higher final accuracy across datasets.
Figure 10: Raw accuracy curves across datasets ( α=0.5 ).
Figure 11: Update contributions and the role of β(τ) on CIFAR-10 . Left: Per-round effect of applying only the stale client update, only the KD-revived update, or their combination under the adaptive β(τ) schedule. Right: Test accuracy vs. simulated time under adaptive and constant ( 0.75 and 1 ) β(τ) . Constant schedules underweight the client update and degrade performance.
Figure 12: CIFAR-10 update-component ablation under adaptive and fixed mixing-weight schedules.
Figure 13: Comparison of different β(τ) functions. Training curves under different adaptive and constant mixing schedules on 20NewsGroups and FEMNIST . The method is generally robust to the choice of β , while adaptive schedules provide the best overall trade-off by relying more on the raw client update at low staleness and shifting more weight to the KD-revived update as staleness increases.
Figure 14: Representative synthesized pseudo-samples used by FedRevive . The samples are generated on the server for data-free knowledge distillation and are not intended to be realistic reconstructions of private client data. In our setting, regular asynchronous FL updates appear to provide enough signal for knowledge transfer, but not enough to recover visually realistic client images.
Dataset
Method
ηc
ηs
Buffer / Window
CIFAR-10
FedAvg
0.0003
1.4
–
FedBuff
0.0003
0.05
2
FADAS
0.0003
0.004
2
FedAC
0.0003
0.0001
2
FedFa
0.0003
0.03
2
AFL-DW
0.0003
0.10
–
Appendix
Table 6: Client and server learning rates used in each experiment, together with the buffer / window size for buffered async methods.
Dataset
KD learning rate
Distillation temperature
CIFAR-10
0.0003
2.0
CIFAR-100
0.0001
2.0
FEMNIST
0.0003
2.0
20NewsGroups
0.0001
2.0
Appendix
Table 7: Hyperparameters for DD-FedRevive . Shared: 10 distillation steps per server update, distillation batch size 32 , teacher buffer size c=8 , and β=0.5 (constant for DD-FedRevive ).
Dataset
KD LR
Generator LR
Latent noise LR
αfeature
CIFAR-10
1e-4
0.003
0.001
0.003
CIFAR-100
3e-5
0.003
0.001
0.0003
FEMNIST
1e-4
0.01
0.003
0.3
20NewsGroups
3e-4
0.01
0.1
0.0001
Appendix
Table 8: Dataset-specific DFKD hyperparameters for FedRevive . Shared: synthesis batch size 64 , KD batch size 32 , teacher buffer size 8 , KD temperature T=1 , 2 generator steps every 10 server updates, and 10 KD steps per server update. Learning rates (LR) are presented in the table.
Figure 15: Different candidate β scheduling functions controlling the mix between parameter aggregation and DFKD updates. 1−cosine schedule is chosen, as it empirically shows the best performance on FEMNIST and 20NewsGroups .
Asynchronous federated learning lets distributed clients train shared models without waiting for every participant, despite delays, churn, and uneven data.
Asynchronous Federated Learning (AFL) has emerged as a significant research area in recent years. By not waiting for slower clients and executing the training process concurrently, it achieves faster training speed compared to traditional federated learning. However, due to the staleness introduced by the asynchronous process, its performance may degrade in some scenarios. Existing methods often use the round difference between the current model and the global model as the sole measure of staleness, which is coarse-grained and lacks observation of the model itself, thereby limiting the performance ceiling of asynchronous methods. In this paper, we propose FedPSA (Parameter Sensitivity-based Asynchronous Federated Learning), a more fine-grained AFL framework that leverages parameter sensitivity to measure model obsolescence and establishes a dynamic momentum queue to assess the current training phase in real time, thereby adjusting the tolerance for outdated information dynamically. Extensive experiments on multiple datasets and comparisons with various methods demonstrate the superior performance of FedPSA, achieving up to 6.37% improvement over baseline methods and 1.93% over the current state-of-the-art method.
In real-world Federated Learning (FL) deployments, data distributions on devices that participate in training evolve over time. This leads to asynchronous data drift, where different devices shift at different times and toward different distributions. Mitigating such drift is challenging: frequent retraining incurs high computational cost on resource-constrained devices, while infrequent retraining degrades performance on drifting devices. We propose DriftGuard, a federated continual learning framework that efficiently adapts to asynchronous data drift. DriftGuard adopts a Mixture-of-Experts (MoE) inspired architecture that separates shared parameters, which capture globally transferable knowledge, from local parameters that adapt to group-specific distributions. This design enables two complementary retraining strategies: (i) global retraining, which updates the shared parameters when system-wide drift is identified, and (ii) group retraining, which selectively updates local parameters for clusters of devices identified via MoE gating patterns, without sharing raw data. Experiments across multiple datasets and models show that DriftGuard matches or exceeds state-of-the-art accuracy while reducing total retraining cost by up to 83%. As a result, it achieves the highest accuracy per unit retraining cost, improving over the strongest baseline by up to 2.3x. DriftGuard is available for download from https://github.com/blessonvar/DriftGuard.
Yizhou Han, Di Wu, Blesson Varghese
School of Computer Science, University of St Andrews, UK
In asynchronous federated learning (FL), client devices send updates to a central server at varying times based on their computational speed, often using stale versions of the global model. This staleness can degrade the convergence and accuracy of the global model. Previous work, such as AsyncFedED, proposed an adaptive aggregation method using Euclidean distance to measure staleness. In this paper, we extend this approach by exploring alternative distance metrics to more accurately capture the effect of gradient staleness. We integrate these metrics into the aggregation process and evaluate their impact on convergence speed, model performance, and training stability under heterogeneous clients and non-IID data settings. Our results demonstrate that certain metrics lead to more robust and efficient asynchronous FL training, offering a stronger foundation for practical deployment.
Patrick Wilhelm, Odej Kao
BIFOLD, Technische Universität Berlin, Berlin, Germany · Technische Universität Berlin, Berlin, Germany