Client Drift

Client drift in federated learning (FL) refers to the divergence of locally trained models on individual devices from a globally shared model, hindering the overall learning process due to data heterogeneity across clients. Current research focuses on mitigating this drift through techniques like gradient correction, adaptive bias estimation, and self-distillation, often integrated into existing FL algorithms such as FedAvg, or employing novel approaches like gradual model unfreezing. Addressing client drift is crucial for improving the accuracy and efficiency of FL, enabling robust and privacy-preserving distributed machine learning across diverse datasets and applications.

Papers

September 27, 2024

Hierarchical Federated Learning with Multi-Timescale Gradient Correction
Wenzhi Fang, Dong-Jun Han, Evan Chen, Shiqiang Wang, Christopher G. Brinton
Hierarchical Federated Learning Client Drift Multi Timescale

September 1, 2023

Jointly Exploring Client Drift and Catastrophic Forgetting in Dynamic Learning
Niklas Babendererde, Moritz Fuchs, Camila Gonzalez, Yuri Tolkach, Anirban Mukhopadhyay
Deep Learning Continual LEArning Catastrophic Forgetting Temporal Shift Dynamic Learning Client Drift

August 20, 2023

Rethinking Client Drift in Federated Learning: A Logit Perspective
Yunlu Yan, Chun-Mei Feng, Mang Ye, Wangmeng Zuo, Ping Li, Rick Siow Mong Goh, Lei Zhu, C. L. Philip Chen
Unlearned Model Second Ranked Logits Client Drift Logit Vector

July 19, 2023

FedBug: A Bottom-Up Gradual Unfreezing Framework for Federated Learning
Chia-Hsiang Kao, Yu-Chiang Frank Wang
Federated Learning Collaborative Training Client Drift Gradual Unfreezing

May 31, 2023

Adaptive Self-Distillation for Minimizing Client Drift in Heterogeneous Federated Learning
M. Yashwanth, Gaurav Kumar Nayak, Arya Singh, Yogesh Simmhan, Anirban Chakraborty
Regularization Model Self Distillation Heterogeneous Federated Learning Local Datasets Client Drift

December 5, 2022

On the effectiveness of partial variance reduction in federated learning with heterogeneous data
Bo Li, Mikkel N. Schmidt, Tommy S. Alstrøm, Sebastian U. Stich
LeArning Abstract Data Heterogeneity Heterogeneous Data Variance Reduction FedAvg Algorithm Gradient Update Client Drift

April 27, 2022

AdaBest: Minimizing Client Drift in Federated Learning via Adaptive Bias Estimation
Farshid Varno, Marzie Saghayi, Laya Rafiee Sevyeri, Sharut Gupta, Stan Matwin, Mohammad Havaei
Federated Learning Decentralized Training Variance Reduction Technique Client Drift

March 24, 2022

Addressing Client Drift in Federated Continual Learning with Adaptive Optimization
Yeshwanth Venkatesha, Youngeun Kim, Hyoungseob Park, Yuhang Li, Priyadarshini Panda
Continual LEArning Federated Learning Federated Continual Learning Adaptive Optimization Client Drift