cs.LGJan 29, 2026

FedAdaVR: Adaptive Variance Reduction for Robust Federated Learning under Limited Client Participation

Authors: S M Ruhul Kabir Howlader, Xiao Chen, Yifei Xie, Lu Liu

Organizations: School of Computing and Mathematical Sciences, University of Leicester, Leicester, United Kingdom · School of Informatics, University of Edinburgh, Edinburgh, United Kingdom · Department of Computer Science, University of Exeter, Exeter, United Kingdom

Abstract

Federated learning (FL) encounters substantial challenges due to heterogeneity, leading to gradient noise, client drift, and partial client participation errors, the last of which is the most pervasive but remains insufficiently addressed in current literature. In this paper, we propose FedAdaVR, a novel FL algorithm aimed at solving heterogeneity issues caused by partial client participation by incorporating an adaptive optimiser with a variance reduction technique. This method takes advantage of the most recent stored updates from clients, even when they are absent from the current training round, thereby emulating their presence. Furthermore, we propose FedAdaVR-Quant, a server-side memory compression variant that stores historical client updates in quantised form (while keeping client-server communication in full precision), reducing the server memory footprint of FedAdaVR by 50% (FP16), 75% (Int8), and 87.5% (Int4) while maintaining highly competitive, and occasionally superior model performance. We analyse the non-convex convergence behaviour of FedAdaVR with Adagrad and prove that our proposed framework eliminates the first-order partial client participation bias while improving monotonically with cohort size. Extensive experiments conducted on multiple vision and NLP datasets demonstrate that FedAdaVR consistently outperforms state-of-the-art baseline methods.

Figures & tables

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. FedSteer: Taming Extreme Gradient Staleness in Federated Learning with Corrective Projections and Caching

    Jun 8, 2026Haoran Zhang, Cainã Figueiredo Pereira, Marie Siew +3Federated Learning AggregationFederated Learning