Data Heterogeneity

Data heterogeneity, the variability in data distributions across different sources in federated learning, significantly hinders model accuracy and convergence. Current research focuses on mitigating this challenge through various techniques, including personalized model architectures (e.g., using adaptors or subnetworks), robust aggregation methods (e.g., weighted averaging based on client performance or data characteristics), and innovative training strategies (e.g., warmup phases, loss decomposition, and adversarial training). Addressing data heterogeneity is crucial for advancing federated learning's applicability in diverse real-world scenarios, particularly in healthcare, industrial IoT, and other domains with decentralized, privacy-sensitive data.

Papers

June 10, 2024

Decentralized Personalized Federated Learning
Salma Kharrat, Marco Canini, Samuel Horvath
Data Heterogeneity Personalized Federated Learning Personalized Learning

May 31, 2024

Non-Federated Multi-Task Split Learning for Heterogeneous Sources
Yilin Zheng, Atilla Eryilmaz
Data Heterogeneity Heterogeneous Data Split Learning Heterogeneous Computing Heterogeneous Source Federated Split Learning

May 23, 2024

A Gap in Time: The Challenge of Processing Heterogeneous IoT Data in Digitalized Buildings
Xiachong Lin, Arian Prabowo, Imran Razzak, Hao Xue, Matthew Amos, Sam Behrens, Flora D. Salim
Time Matter Challenge Task Data Heterogeneity Data Stream Residential Building Smart Building Internet of Thing Data

May 20, 2024

StatAvg: Mitigating Data Heterogeneity in Federated Learning for Intrusion Detection Systems
Pavlos S. Bouzinis, Panagiotis Radoglou-Grammatikis, Ioannis Makris, Thomas Lagkas, Vasileios Argyriou, Georgios Th. Papadopoulos, Panagiotis Sarigiannidis, George K. Karagiannidis
Federated Learning Data Heterogeneity Decentralized Learning Intrusion Detection Intrusion Detection System

May 19, 2024

The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication
Kumar Kshitij Patel, Margalit Glasgow, Ali Zindari, Lingxiao Wang, Sebastian U. Stich, Ziheng Cheng, Nirmit Joshi, Nathan Srebro
Full Potential Continuum Limit Data Heterogeneity Mini Batch Min Max Local SGD Heterogeneous Learning

May 17, 2024

Multicenter Privacy-Preserving Model Training for Deep Learning Brain Metastases Autosegmentation
Yixing Huang, Zahra Khodabakhshi, Ahmed Gomaa, Manuel Schmidt, Rainer Fietkau, Matthias Guckenberger, Nicolaus Andratschke, Christoph Bert, Stephanie Tanadini-Lang, Florian Putz
Transfer Learning Model Training Data Heterogeneity Multi Center Brain Metastasis Single Center Training

May 2, 2024

Navigating Heterogeneity and Privacy in One-Shot Federated Learning with Diffusion Models
Matias Mendieta, Guangyu Sun, Chen Chen
Diffusion Model Differential Privacy Privacy Policy Data Heterogeneity One Shot Federated Learning

April 29, 2024

April 14, 2024

MAP: Model Aggregation and Personalization in Federated Learning with Incomplete Classes
Xin-Chun Li, Shaoming Song, Yinchuan Li, Bingshuai Li, Yunfeng Shao, Yang Yang, De-Chuan Zhan
Data Heterogeneity Model Aggregation Private Model Model Personalization mAP Decentralized Framework

March 23, 2024

The Effectiveness of Local Updates for Decentralized Learning under Data Heterogeneity
Tongle Wu, Zhize Li, Ying Sun
Data Heterogeneity Decentralized Learning Communication Complexity Decentralized Optimization Local Context Gradient Correction Decentralized Gradient

March 20, 2024

Byzantine-resilient Federated Learning With Adaptivity to Data Heterogeneity
Shiyuan Zuo, Xingrun Yan, Rongfei Fan, Han Hu, Hangguan Shan, Tony Q. S. Quek
Adaptive Importance Data Heterogeneity Non Convex Loss Function Byzantine Attack Byzantine Robust Federated Learning Convex Loss Function

March 17, 2024

Federated Transfer Learning with Differential Privacy
Mengchu Li, Ye Tian, Yang Feng, Yi Yu
Differential Privacy Data Heterogeneity Privacy Constraint Federated Transfer Learning

March 12, 2024

Machine Learning Techniques for Sensor-based Human Activity Recognition with Data Heterogeneity -- A Review
Xiaozhou Ye, Kouichi Sakurai, Nirmal Nair, Kevin I-Kai Wang
Machine Learning Narrative Review Data Heterogeneity Sensor Based Human Activity Recognition Sensory Data

March 9, 2024

Towards Efficient Replay in Federated Incremental Learning
Yichen Li, Qunwei Li, Haozhao Wang, Ruixuan Li, Wenliang Zhong, Guannan Zhang
Federated Learning Catastrophic Forgetting Data Heterogeneity Efficient Generated Object Replay

March 7, 2024

HeteroSwitch: Characterizing and Taming System-Induced Data Heterogeneity in Federated Learning
Gyudong Kim, Mehdi Ghasemi, Soroush Heidari, Seungryong Kim, Young Geun Kim, Sarma Vrudhula, Carole-Jean Wu
Data Heterogeneity Heterogeneous Device Tame Function

February 25, 2024

Trustworthy Personalized Bayesian Federated Learning via Posterior Fine-Tune
Mengen Luo, Chi Xu, Ercan Engin Kuruoglu
Normalizing Flow Bayesian Neural Network Data Heterogeneity Personalized Federated Learning Personalized Model

February 22, 2024

Federated Learning on Transcriptomic Data: Model Quality and Performance Trade-Offs
Anika Hannemann, Jan Ewald, Leo Seeger, Erik Buchmann
Machine Learning Data Heterogeneity Trade Offs Federated Learning Framework Transcriptomic Data Model Quality

February 20, 2024

CCFC++: Enhancing Federated Clustering through Feature Decorrelation
Jie Yan, Jing Liu, Yi-Zi Ning, Zhong-Yuan Zhang
Contrastive Learning Data Heterogeneity Heterogeneous Data Feature Decorrelation Federated Clustering

February 19, 2024

Hierarchical Bayes Approach to Personalized Federated Unsupervised Learning
Kaan Ozkara, Bruce Huang, Ruida Zhou, Suhas Diggavi
Diffusion Model Data Heterogeneity Personalized Federated Learning Hierarchical Bayesian Personalized Interpretable

Data Heterogeneity

Papers

Decentralized Personalized Federated Learning

Non-Federated Multi-Task Split Learning for Heterogeneous Sources

A Gap in Time: The Challenge of Processing Heterogeneous IoT Data in Digitalized Buildings

StatAvg: Mitigating Data Heterogeneity in Federated Learning for Intrusion Detection Systems

The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication

Multicenter Privacy-Preserving Model Training for Deep Learning Brain Metastases Autosegmentation

Navigating Heterogeneity and Privacy in One-Shot Federated Learning with Diffusion Models

On the Impact of Data Heterogeneity in Federated Learning Environments with Application to Healthcare Networks

An Aggregation-Free Federated Learning for Tackling Data Heterogeneity

MAP: Model Aggregation and Personalization in Federated Learning with Incomplete Classes

The Effectiveness of Local Updates for Decentralized Learning under Data Heterogeneity

Byzantine-resilient Federated Learning With Adaptivity to Data Heterogeneity

Federated Transfer Learning with Differential Privacy

Machine Learning Techniques for Sensor-based Human Activity Recognition with Data Heterogeneity -- A Review

Towards Efficient Replay in Federated Incremental Learning

HeteroSwitch: Characterizing and Taming System-Induced Data Heterogeneity in Federated Learning

Trustworthy Personalized Bayesian Federated Learning via Posterior Fine-Tune

Federated Learning on Transcriptomic Data: Model Quality and Performance Trade-Offs

CCFC++: Enhancing Federated Clustering through Feature Decorrelation

Hierarchical Bayes Approach to Personalized Federated Unsupervised Learning