Data Heterogeneity

Data heterogeneity, the variability in data distributions across different sources in federated learning, significantly hinders model accuracy and convergence. Current research focuses on mitigating this challenge through various techniques, including personalized model architectures (e.g., using adaptors or subnetworks), robust aggregation methods (e.g., weighted averaging based on client performance or data characteristics), and innovative training strategies (e.g., warmup phases, loss decomposition, and adversarial training). Addressing data heterogeneity is crucial for advancing federated learning's applicability in diverse real-world scenarios, particularly in healthcare, industrial IoT, and other domains with decentralized, privacy-sensitive data.

Papers

July 7, 2022

June 27, 2022

An Empirical Study of Personalized Federated Learning
Koji Matsuda, Yuya Sasaki, Chuan Xiao, Makoto Onizuka
Federated Learning Empirical Study Data Heterogeneity Personalized Federated Learning

June 21, 2022

FedHiSyn: A Hierarchical Synchronous Federated Learning Framework for Resource and Data Heterogeneity
Guanghao Li, Yue Hu, Miao Zhang, Ji Liu, Quanjun Yin, Yong Peng, Dejing Dou
Federated Learning Data Heterogeneity New Resource Decentralized Data

June 20, 2022

June 17, 2022

Federated learning with incremental clustering for heterogeneous data
Fabiola Espinoza Castellon, Aurelien Mayoue, Jacques-Henri Sublemontier, Cedric Gouy-Pailler
Data Heterogeneity Heterogeneous Data Robust Defense Malicious Client Incremental Clustering

June 16, 2022

On Privacy and Personalization in Cross-Silo Federated Learning
Ziyu Liu, Shengyuan Hu, Zhiwei Steven Wu, Virginia Smith
Differential Privacy Privacy Policy Data Heterogeneity Cross Silo Federated Learning

June 6, 2022

Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learning
Zhenheng Tang, Yonggang Zhang, Shaohuai Shi, Xin He, Bo Han, Xiaowen Chu
Data Heterogeneity Provable Generalization Virtual Dataset

June 5, 2022

Straggler-Resilient Personalized Federated Learning
Isidoros Tziotis, Zebang Shen, Ramtin Pedarsani, Hamed Hassani, Aryan Mokhtari
Representation Learning Sample Complexity Data Heterogeneity Linear Speedup

June 1, 2022

Federated Learning in Non-IID Settings Aided by Differentially Private Synthetic Data
Huancheng Chen, Haris Vikalo
Synthetic Data Latent Representation Data Heterogeneity Federated Prompt Cooperation Variational Auto

May 30, 2022

Towards Fair Federated Recommendation Learning: Characterizing the Inter-Dependence of System and Data Heterogeneity
Kiwan Maeng, Haiyu Lu, Luca Melis, John Nguyen, Mike Rabbat, Carole-Jean Wu
Recommender System System Description Data Heterogeneity Dependency Modeling

May 17, 2022

Label-Efficient Self-Supervised Federated Learning for Tackling Data Heterogeneity in Medical Imaging
Rui Yan, Liangqiong Qu, Qingyue Wei, Shih-Cheng Huang, Liyue Shen, Daniel Rubin, Lei Xing, Yuyin Zhou
Medical Imaging Data Heterogeneity Self Supervised Transformer Large Scale Medical Federated Self Supervised Learning

April 27, 2022

Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learning
Yae Jee Cho, Andre Manoel, Gauri Joshi, Robert Sim, Dimitrios Dimitriadis
Ensemble Learning Large Model Data Heterogeneity Heterogeneous Ensemble

April 18, 2022

FedKL: Tackling Data Heterogeneity in Federated Reinforcement Learning by Penalizing KL Divergence
Zhijie Xie, S. H. Song
Reinforcement Learning Policy Gradient Data Heterogeneity Kullback Leibler Divergence Federated Reinforcement Learning

April 16, 2022

DRFLM: Distributionally Robust Federated Learning with Inter-client Noise via Local Mixup
Bingzhe Wu, Zhipeng Liang, Yuxuan Han, Yatao Bian, Peilin Zhao, Junzhou Huang
Robust Optimization Data Heterogeneity Robust Federated Learning Learning Probability Distribution Local Mixup Drug Target Affinity Prediction

April 13, 2022

Data-heterogeneity-aware Mixing for Decentralized Learning
Yatin Dandi, Anastasia Koloskova, Martin Jaggi, Sebastian U. Stich
Convergence Rate Data Heterogeneity Decentralized Learning Graph Topology Communication Graph Constrained Convex Optimization

April 7, 2022

FedCos: A Scene-adaptive Federated Optimization Enhancement for Performance Improvement
Hao Zhang, Tingting Wu, Siyao Cheng, Jie Liu
Data Heterogeneity Performance Improvement Global Model Centralized Training Federated Optimization Iterative Method

February 4, 2022

Functional Mixtures-of-Experts
Faïcel Chamroukhi, Nhat Thien Pham, Van Hà Hoang, Geoffrey J. McLachlan
Mixture of Expert Data Heterogeneity Standard LASSO Non Functional Property

December 22, 2021

FedLGA: Towards System-Heterogeneity of Federated Learning via Local Gradient Approximation
Xingyu Li, Zhe Qu, Bo Tang, Zhuo Lu
Data Heterogeneity Gradient Approximation Device Heterogeneity System Heterogeneity

Data Heterogeneity

Papers

Adaptive Personlization in Federated Learning for Highly Non-i.i.d. Data

Towards the Practical Utility of Federated Learning in the Medical Domain

An Empirical Study of Personalized Federated Learning

FedHiSyn: A Hierarchical Synchronous Federated Learning Framework for Resource and Data Heterogeneity

Communication-Efficient Federated Learning With Data and Client Heterogeneity

Mitigating Data Heterogeneity in Federated Learning with Data Augmentation

Federated learning with incremental clustering for heterogeneous data

On Privacy and Personalization in Cross-Silo Federated Learning

Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learning

Straggler-Resilient Personalized Federated Learning

Federated Learning in Non-IID Settings Aided by Differentially Private Synthetic Data

Towards Fair Federated Recommendation Learning: Characterizing the Inter-Dependence of System and Data Heterogeneity

Label-Efficient Self-Supervised Federated Learning for Tackling Data Heterogeneity in Medical Imaging

Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learning

FedKL: Tackling Data Heterogeneity in Federated Reinforcement Learning by Penalizing KL Divergence

DRFLM: Distributionally Robust Federated Learning with Inter-client Noise via Local Mixup

Data-heterogeneity-aware Mixing for Decentralized Learning

FedCos: A Scene-adaptive Federated Optimization Enhancement for Performance Improvement

Functional Mixtures-of-Experts

FedLGA: Towards System-Heterogeneity of Federated Learning via Local Gradient Approximation