Non-IID Federated Learning

Follow this topic Ask about this topic

Non-IID federated learning studies how clients can train a shared model when their data differ in labels, features, domains, or changing conditions, while keeping raw data local. Methods include optimization corrections such as FedProx and SCAFFOLD, clustering and personalization, and knowledge distillation or prototypes to share useful information without simply averaging every update. Client selection, compression, privacy mechanisms, and asynchronous or decentralized training address practical limits such as communication, device capacity, and unreliable participation. A central tension remains between a broadly useful global model and accurate, fair models for individual clients; gains depend on the kind of heterogeneity and must be weighed against communication, privacy, and worst-client performance.

Latest papers

211 papers17 papers in the last four weeks, down 37% on the four weeks before. 0.1% of all new papers.

StoCFL: A Stochastically Clustered Federated Learning Framework for Non-IID Data with Dynamic Client Participation
Mar 2, 2023Dun Zeng, Xiangjing Hu, Shiyu Liu +3Non-IID Federated Learning Clustered Federated Learning
University of Electronic Science and Technology of China · Peng Cheng Lab · Harbin Institute of Technology, Shenzhen +1

Mar 2, 2023J/K move · Enter open · S save

StoCFL: A Stochastically Clustered Federated Learning Framework for Non-IID Data with Dynamic Client Participation

Dun Zeng, Xiangjing Hu, Shiyu Liu, Yue Yu+2

University of Electronic Science and Technology of China · Peng Cheng Lab · Harbin Institute of Technology, Shenzhen+1

Non-IID Federated Learning Clustered Federated Learning Federated Learning Data-Efficient Learning

Open paper AskPDF arXiv

Federated learning is a distributed learning framework that takes full advantage of private data samples kept on edge devices. In real-world federated learning systems, these data samples are often decentralized and Non-Independently Identically Distributed (Non-IID), causing divergence and performance degradation in the federated learning process. As a new solution, clustered federated learning groups federated clients with similar data distributions to impair the Non-IID effects and train a better model for every cluster. However, existing CFL algorithms are ineffective because they lack an information-sharing mechanism across clusters resulting in low data efficiency and model performance. Meanwhile, their performance is highly subjected to ideal client clustering results which are practically unavailable. This paper proposes StoCFL, a novel clustered federated learning framework for generic Non-IID issues. In detail, StoCFL implements a flexible CFL framework that supports an arbitrary proportion of client participation and newly joined clients for a varying FL system, while maintaining a great improvement in model performance. The intensive experiments are conducted by using four basic Non-IID settings and a real-world dataset. The results show that StoCFL could obtain promising cluster results even when the number of clusters is unknown. Based on the client clustering results, models trained with StoCFL outperform baseline approaches in a variety of scenarios.