Statistical Heterogeneity

Statistical heterogeneity in federated learning (FL) arises from the diverse data distributions across participating devices, hindering the efficient training of a shared model. Current research focuses on mitigating this heterogeneity through improved aggregation techniques (e.g., FedAvg, FedProx), personalized model approaches, and adaptive client selection strategies that prioritize informative updates. These advancements aim to enhance model accuracy, robustness, and fairness in FL systems, impacting various applications, particularly those involving sensitive, decentralized data like medical imaging or industrial manufacturing. The ultimate goal is to enable effective collaborative learning while preserving data privacy and addressing the challenges posed by non-identically distributed data.

Papers

November 7, 2024

Towards Robust Federated Analytics via Differentially Private Measurements of Statistical Heterogeneity
Mary Scott, Graham Cormode, Carsten Maple
Differential Privacy Homogeneity Bias Statistical Heterogeneity Federated Data Internal Measurement Unknown Heterogeneity Privacy Parameter

October 9, 2024

September 8, 2024

DynamicFL: Federated Learning with Dynamic Communication Resource Allocation
Qi Le, Enmao Diao, Xinran Wang, Vahid Tarokh, Jie Ding, Ali Anwar
Federated Averaging Statistical Heterogeneity Federated Stochastic Communication Heterogeneity

August 18, 2024

Addressing Heterogeneity in Federated Learning: Challenges and Solutions for a Shared Production Environment
Tatjana Legler, Vinit Hegiste, Ahmed Anwar, Martin Ruskowski
Technical Challenge Solution Path Data Heterogeneity Unobserved Heterogeneity Statistical Heterogeneity Heterogeneity Challenge

July 31, 2024

Algorithms for Collaborative Machine Learning under Statistical Heterogeneity
Seok-Ju Hahn
Practical Algorithm Synthetic Data Generation Decentralized Data Collaborative Machine Learning Statistical Heterogeneity

July 23, 2024

On ADMM in Heterogeneous Federated Learning: Personalization, Robustness, and Fairness
Shengkun Zhu, Jinshan Zeng, Sheng Wang, Yuan Sun, Xiaodong Li, Yuan Yao, Zhiyong Peng
Native Robustness Procedural Fairness Heterogeneous Data Heterogeneous Federated Learning Alternating Direction Method of Multiplier Statistical Heterogeneity Superlinear Convergence Personalized Fl

June 10, 2024

Optimisation of federated learning settings under statistical heterogeneity variations
Basem Suleiman, Muhammad Johan Alibasa, Rizka Widyarini Purwanto, Lewis Jeffries, Ali Anaissi, Jacky Song
Federated Learning Optimization Process Weight Aggregation Different Datasets Statistical Heterogeneity

March 10, 2024

Analysis of Total Variation Minimization for Clustered Federated Learning
A. Jung
General Analysis Total Variation Statistical Heterogeneity Clustered Federated Learning

March 5, 2024

Federated Learning over Connected Modes
Dennis Grinwald, Philipp Wiesner, Shinichi Nakajima
Global Model Statistical Heterogeneity Linear Mode Connectivity Gradient Conflict Mode Connectivity

March 3, 2024

Quantized Hierarchical Federated Learning: A Robust Approach to Statistical Heterogeneity
Seyed Mohammad Azimi-Abarghouyi, Viktoria Fodor
Heterogeneous Data Model Aggregation Robust Framework Hierarchical Federated Learning Statistical Heterogeneity Aggregated Gradient

February 15, 2024

Investigation of Federated Learning Algorithms for Retinal Optical Coherence Tomography Image Classification with Statistical Heterogeneity
Sanskar Amgain, Prashant Shrestha, Sophia Bano, Ignacio del Valle Torres, Michael Cunniffe, Victor Hernandez, Phil Beales, Binod Bhattarai
Optical Coherence Tomography Comprehensive Investigation Statistical Heterogeneity Federated Learning Algorithm

February 7, 2024

Personalized Federated Learning for Statistical Heterogeneity
Muhammad Firdaus, Kyung-Hyune Rhee
Personalized Federated Learning Sensitive Data Data Privacy Statistical Heterogeneity Collaborative Model

October 9, 2023

Exploring Progress in Multivariate Time Series Forecasting: Comprehensive Benchmarking and Heterogeneity Analysis
Zezhi Shao, Fei Wang, Yongjun Xu, Wei Wei, Chengqing Yu, Zhao Zhang, Di Yao, Tao Sun, Guangyin Jin, Xin Cao, Gao Cong, Christian S.Jensen, Xueqi Cheng
Multivariate Time Series Much Progress Multivariate Time Series Forecasting Long Term Time Series Forecasting Spatio Temporal Forecasting Statistical Heterogeneity

September 30, 2023

Accelerating Non-IID Federated Learning via Heterogeneity-Guided Client Sampling
Huancheng Chen, Haris Vikalo
Client Selection Statistical Heterogeneity Client Sampling Federated Learning Client Based Sampling

July 28, 2023

A Practical Recipe for Federated Learning Under Statistical Heterogeneity Experimental Design
Mahdi Morafah, Weijia Wang, Bill Lin
Federated Learning Pytorch Model Data Heterogeneity Complete Recipe Statistical Heterogeneity

May 1, 2023

Personalized Federated Learning under Mixture of Distributions
Yue Wu, Shuaicheng Zhang, Wenchao Yu, Yanchi Liu, Quanquan Gu, Dawei Zhou, Haifeng Chen, Wei Cheng
Mixture Component Personalized Federated Learning Gaussian Mixture Model Covariate Shift Statistical Heterogeneity

December 14, 2022

FedSkip: Combatting Statistical Heterogeneity with Federated Skip Aggregation
Ziqing Fan, Yanfeng Wang, Jiangchao Yao, Lingjuan Lyu, Ya Zhang, Qi Tian
Aggregation Method Statistical Heterogeneity Federated Aggregation

October 7, 2022

Depersonalized Federated Learning: Tackling Statistical Heterogeneity by Alternating Stochastic Gradient Descent
Yujie Zhou, Zhidu Li, Tong Tang, Ruyan Wang
Stochastic Gradient Descent Statistical Heterogeneity Federated Learning Convergence

September 30, 2022

Rethinking Data Heterogeneity in Federated Learning: Introducing a New Notion and Standard Benchmarks
Mahdi Morafah, Saeed Vahidian, Chen Chen, Mubarak Shah, Bill Lin
Pseudo Label Data Heterogeneity Model Aggregation Statistical Heterogeneity Training Data Distribution Novel Concept Human Aligned Benchmark

Statistical Heterogeneity

Papers

Towards Robust Federated Analytics via Differentially Private Measurements of Statistical Heterogeneity

Do Contemporary CATE Models Capture Real-World Heterogeneity? Findings from a Large-Scale Benchmark

Benchmarking Data Heterogeneity Evaluation Approaches for Personalized Federated Learning

DynamicFL: Federated Learning with Dynamic Communication Resource Allocation

Addressing Heterogeneity in Federated Learning: Challenges and Solutions for a Shared Production Environment

Algorithms for Collaborative Machine Learning under Statistical Heterogeneity

On ADMM in Heterogeneous Federated Learning: Personalization, Robustness, and Fairness

Optimisation of federated learning settings under statistical heterogeneity variations

Analysis of Total Variation Minimization for Clustered Federated Learning

Federated Learning over Connected Modes

Quantized Hierarchical Federated Learning: A Robust Approach to Statistical Heterogeneity

Investigation of Federated Learning Algorithms for Retinal Optical Coherence Tomography Image Classification with Statistical Heterogeneity

Personalized Federated Learning for Statistical Heterogeneity

Exploring Progress in Multivariate Time Series Forecasting: Comprehensive Benchmarking and Heterogeneity Analysis

Accelerating Non-IID Federated Learning via Heterogeneity-Guided Client Sampling

A Practical Recipe for Federated Learning Under Statistical Heterogeneity Experimental Design

Personalized Federated Learning under Mixture of Distributions

FedSkip: Combatting Statistical Heterogeneity with Federated Skip Aggregation

Depersonalized Federated Learning: Tackling Statistical Heterogeneity by Alternating Stochastic Gradient Descent

Rethinking Data Heterogeneity in Federated Learning: Introducing a New Notion and Standard Benchmarks