Data Heterogeneity

Data heterogeneity, the variability in data distributions across different sources in federated learning, significantly hinders model accuracy and convergence. Current research focuses on mitigating this challenge through various techniques, including personalized model architectures (e.g., using adaptors or subnetworks), robust aggregation methods (e.g., weighted averaging based on client performance or data characteristics), and innovative training strategies (e.g., warmup phases, loss decomposition, and adversarial training). Addressing data heterogeneity is crucial for advancing federated learning's applicability in diverse real-world scenarios, particularly in healthcare, industrial IoT, and other domains with decentralized, privacy-sensitive data.

Papers

December 10, 2023

December 7, 2023

Factor-Assisted Federated Learning for Personalized Optimization with Heterogeneous Data
Feifei Wang, Huiyun Tang, Yang Li
Data Heterogeneity Heterogeneous Data Machine Learning Framework Tucker Decomposition

November 22, 2023

FedFN: Feature Normalization for Alleviating Data Heterogeneity Problem in Federated Learning
Seongyoon Kim, Gihun Lee, Jaehoon Oh, Se-Young Yun
Data Heterogeneity Federated Averaging Feature Normalization Feature Norm

October 26, 2023

Navigating Data Heterogeneity in Federated Learning A Semi-Supervised Federated Object Detection
Taehyeon Kim, Eric Lin, Junu Lee, Christian Lau, Vaikkunth Mugunthan
Pseudo Label Pseudo Labeling Data Heterogeneity Centralized Training Federated Object Detection

October 23, 2023

Handling Data Heterogeneity via Architectural Design for Federated Visual Recognition
Sara Pieri, Jose Renato Restom, Samuel Horvath, Hisham Cholakkal
Convolutional Neural Network Data Heterogeneity Architecture Design Visual Recognition Federated Object Detection

October 11, 2023

FedSym: Unleashing the Power of Entropy for Benchmarking the Algorithms for Federated Learning
Ensiye Kiyamousavi, Boris Kraychev, Ivan Koychev
Practical Algorithm Real Power Data Heterogeneity Decentralized Machine Learning Stable Entropy Data Diversity

October 9, 2023

Find Your Optimal Assignments On-the-fly: A Holistic Framework for Clustered Federated Learning
Yongxin Guo, Xiaoying Tang, Tao Lin
Data Heterogeneity Client Heterogeneity Clustered Federated Learning Clustering Framework Optimal Assignment

October 8, 2023

FedFed: Feature Distillation against Data Heterogeneity in Federated Learning
Zhiqin Yang, Yonggang Zhang, Yu Zheng, Xinmei Tian, Hao Peng, Tongliang Liu, Bo Han
Data Heterogeneity Data Sharing Feature Distillation

October 6, 2023

FedConv: Enhancing Convolutional Neural Networks for Handling Data Heterogeneity in Federated Learning
Peiran Xu, Zeyu Wang, Jieru Mei, Liangqiong Qu, Alan Yuille, Cihang Xie, Yuyin Zhou
Convolutional Neural Network Vision Transformer Data Heterogeneity

September 29, 2023

Mode Connectivity and Data Heterogeneity of Federated Learning
Tailin Zhou, Jun Zhang, Danny H. K. Tsang
Data Heterogeneity Linear Mode Connectivity Mode Connectivity

September 24, 2023

September 3, 2023

A Comparative Evaluation of FedAvg and Per-FedAvg Algorithms for Dirichlet Distributed Heterogeneous Data
Hamza Reguieg, Mohammed El Hanjri, Mohamed El Kamili, Abdellatif Kobbane
Model Training Data Heterogeneity Heterogeneous Data Federated Averaging Comparative Evaluation FedAvg Algorithm FedAvg Converges Ordered Matrix Dirichlet

August 30, 2023

Federated Two Stage Decoupling With Adaptive Personalization Layers
Hangyu Zhu, Yuxiang Fan, Zhenping Xie
Data Heterogeneity Federated Prompt Cooperation Cluster Structure Client Heterogeneity Light Stage Client Clustering Personalization Layer

August 22, 2023

EM for Mixture of Linear Regression with Clustered Data
Amirhossein Reisizadeh, Khashayar Gatmiry, Asuman Ozdaglar
Mixture Component Latent Variable Data Heterogeneity Heterogeneous Data Expectation Maximization Linear Regression

August 4, 2023

Analysis and Optimization of Wireless Federated Learning with Data Heterogeneity
Xuefeng Han, Jun Li, Wen Chen, Zhen Mei, Kang Wei, Ming Ding, H. Vincent Poor
General Analysis Federated Learning Optimization Purpose Resource Allocation Data Heterogeneity Heterogeneous Data Loss Minimization

July 31, 2023

Federated Learning for Data and Model Heterogeneity in Medical Imaging
Hussain Ahmad Madni, Rao Muhammad Umer, Gian Luca Foresti
Knowledge Distillation Raw Data Medical Imaging Data Heterogeneity Model Heterogeneity Heterogeneous Label

July 28, 2023

Data Heterogeneity

Papers

Fake It Till Make It: Federated Learning with Consensus-Oriented Generation

Federated Learning Empowered by Generative Content

Factor-Assisted Federated Learning for Personalized Optimization with Heterogeneous Data

FedFN: Feature Normalization for Alleviating Data Heterogeneity Problem in Federated Learning

Navigating Data Heterogeneity in Federated Learning A Semi-Supervised Federated Object Detection

Handling Data Heterogeneity via Architectural Design for Federated Visual Recognition

FedSym: Unleashing the Power of Entropy for Benchmarking the Algorithms for Federated Learning

Find Your Optimal Assignments On-the-fly: A Holistic Framework for Clustered Federated Learning

FedFed: Feature Distillation against Data Heterogeneity in Federated Learning

FedConv: Enhancing Convolutional Neural Networks for Handling Data Heterogeneity in Federated Learning

Mode Connectivity and Data Heterogeneity of Federated Learning

Robust Distributed Learning: Tight Error Bounds and Breakdown Point under Data Heterogeneity

Tackling the Unlimited Staleness in Federated Learning with Intertwined Data and Device Heterogeneities

A Comparative Evaluation of FedAvg and Per-FedAvg Algorithms for Dirichlet Distributed Heterogeneous Data

Federated Two Stage Decoupling With Adaptive Personalization Layers

EM for Mixture of Linear Regression with Clustered Data

Analysis and Optimization of Wireless Federated Learning with Data Heterogeneity

Federated Learning for Data and Model Heterogeneity in Medical Imaging

Mean Estimation with User-level Privacy under Data Heterogeneity

A Practical Recipe for Federated Learning Under Statistical Heterogeneity Experimental Design