System Heterogeneity

System heterogeneity in federated learning (FL) refers to the variability in computational resources and network conditions across participating devices, hindering efficient and equitable model training. Current research focuses on developing algorithms that adapt to this heterogeneity, including methods employing clustering, model pruning, and gradient approximation techniques to improve both training speed and model accuracy. These advancements are crucial for realizing the full potential of FL in real-world applications, where diverse devices with varying capabilities are commonplace, and for ensuring fair participation of all clients in the learning process. Addressing system heterogeneity is vital for the scalability and robustness of FL systems.

Papers

July 6, 2024

FedTSA: A Cluster-based Two-Stage Aggregation Method for Model-heterogeneous Federated Learning
Boyu Fan, Chenrui Wu, Xiang Su, Pan Hui
Data Heterogeneity Heterogeneous Model Heterogeneous Source System Heterogeneity Two Stage Aggregation

April 15, 2024

FedP3: Federated Personalized and Privacy-friendly Network Pruning under Model Heterogeneity
Kai Yi, Nidham Gazagnadou, Peter Richtárik, Lingjuan Lyu
Model Heterogeneity System Heterogeneity Personalized Federated

February 2, 2024

FedShift: Tackling Dual Heterogeneity Problem of Federated Learning via Weight Shift Aggregation
Jungwon Seo, Chunming Rong, Minhoe Kim
Weight Aggregation Heterogeneity Challenge System Heterogeneity Unknown Heterogeneity Heterogeneity Environment

August 22, 2023

Internal Cross-layer Gradients for Extending Homogeneity to Heterogeneity in Federated Learning
Yun-Hin Chan, Rui Zhou, Running Zhao, Zhihan Jiang, Edith C. -H. Ngai
Heterogeneous Federated Learning Homogeneity Bias System Heterogeneity Layer Similarity

August 15, 2023

NeFL: Nested Model Scaling for Federated Learning with System Heterogeneous Clients
Honggu Kang, Seohyeon Cha, Jinwoo Shin, Jongmyeong Lee, Joonhyuk Kang
Pre Trained Model Worst Case Heterogeneous Computing Heterogeneous Client System Heterogeneity

August 3, 2023

Hierarchical Federated Learning in Wireless Networks: Pruning Tackles Bandwidth Scarcity and System Heterogeneity
Md Ferdous Pervej, Richeng Jin, Huaiyu Dai
Federated Learning Wireless Network Model Pruning Hierarchical Federated Learning Heterogeneous Network System Heterogeneity

November 25, 2022

FedSysID: A Federated Approach to Sample-Efficient System Identification
Han Wang, Leonardo F. Toso, James Anderson
Dynamical System Learning Algorithm System Identification Dynamical Model System Heterogeneity Federated Approach

October 14, 2022

Where to Begin? On the Impact of Pre-Training and Initialization in Federated Learning
John Nguyen, Jianyu Wang, Kshitiz Malik, Maziar Sanjabi, Michael Rabbat
Global Impact Pre Trained Pre Training New Initialization Unobserved Heterogeneity System Heterogeneity

October 10, 2022

A Survey on Heterogeneous Federated Learning
Dashan Gao, Xin Yao, Qiang Yang
Timely Survey Heterogeneous Data Heterogeneous Federated Learning Model Heterogeneity System Heterogeneity

August 30, 2022

Reducing Impacts of System Heterogeneity in Federated Learning using Weight Update Magnitudes
Irene Wang
Impact Assessment Weight Update Straggler Mitigation System Heterogeneity Invariant Dropout

June 30, 2022

Where to Begin? On the Impact of Pre-Training and Initialization in Federated Learning
John Nguyen, Jianyu Wang, Kshitiz Malik, Maziar Sanjabi, Michael Rabbat
Global Impact Pre Trained Pre Training New Initialization System Heterogeneity

June 22, 2022

FedorAS: Federated Architecture Search under system heterogeneity
Lukasz Dudziak, Stefanos Laskaridis, Javier Fernandez-Marques
Neural Architecture Search Federated Edge Learning Heterogeneous Environment System Heterogeneity

June 3, 2022

Supernet Training for Federated Image Classification under System Heterogeneity
Taehyeon Kim, Se-Young Yun
Supernet Training Weight Sharing Federated Image Classification System Heterogeneity Heterogeneous Training

April 7, 2022

FedADMM: A Robust Federated Deep Learning Framework with Adaptivity to System Heterogeneity
Yonghai Gong, Yichuan Li, Nikolaos M. Freris
Adaptive Importance Primal Dual Heterogeneous Data Non Convex Objective Robust Federated Learning System Heterogeneity

December 22, 2021

FedLGA: Towards System-Heterogeneity of Federated Learning via Local Gradient Approximation
Xingyu Li, Zhe Qu, Bo Tang, Zhuo Lu
Data Heterogeneity Gradient Approximation Device Heterogeneity System Heterogeneity

System Heterogeneity

Papers

FedTSA: A Cluster-based Two-Stage Aggregation Method for Model-heterogeneous Federated Learning

FedP3: Federated Personalized and Privacy-friendly Network Pruning under Model Heterogeneity

FedShift: Tackling Dual Heterogeneity Problem of Federated Learning via Weight Shift Aggregation

Internal Cross-layer Gradients for Extending Homogeneity to Heterogeneity in Federated Learning

NeFL: Nested Model Scaling for Federated Learning with System Heterogeneous Clients

Hierarchical Federated Learning in Wireless Networks: Pruning Tackles Bandwidth Scarcity and System Heterogeneity

FedSysID: A Federated Approach to Sample-Efficient System Identification

Where to Begin? On the Impact of Pre-Training and Initialization in Federated Learning

A Survey on Heterogeneous Federated Learning

Reducing Impacts of System Heterogeneity in Federated Learning using Weight Update Magnitudes

Where to Begin? On the Impact of Pre-Training and Initialization in Federated Learning

FedorAS: Federated Architecture Search under system heterogeneity

Supernet Training for Federated Image Classification under System Heterogeneity

FedADMM: A Robust Federated Deep Learning Framework with Adaptivity to System Heterogeneity

FedLGA: Towards System-Heterogeneity of Federated Learning via Local Gradient Approximation