Data Heterogeneity

Data heterogeneity, the variability in data distributions across different sources in federated learning, significantly hinders model accuracy and convergence. Current research focuses on mitigating this challenge through various techniques, including personalized model architectures (e.g., using adaptors or subnetworks), robust aggregation methods (e.g., weighted averaging based on client performance or data characteristics), and innovative training strategies (e.g., warmup phases, loss decomposition, and adversarial training). Addressing data heterogeneity is crucial for advancing federated learning's applicability in diverse real-world scenarios, particularly in healthcare, industrial IoT, and other domains with decentralized, privacy-sensitive data.

Papers

April 14, 2024

MAP: Model Aggregation and Personalization in Federated Learning with Incomplete Classes
Xin-Chun Li, Shaoming Song, Yinchuan Li, Bingshuai Li, Yunfeng Shao, Yang Yang, De-Chuan Zhan
Data Heterogeneity Model Aggregation Private Model Model Personalization mAP Decentralized Framework

March 23, 2024

The Effectiveness of Local Updates for Decentralized Learning under Data Heterogeneity
Tongle Wu, Ying Sun
Data Heterogeneity Decentralized Learning Communication Complexity Decentralized Optimization Local Context Gradient Correction Decentralized Gradient

March 20, 2024

Byzantine-resilient Federated Learning With Adaptivity to Data Heterogeneity
Shiyuan Zuo, Xingrun Yan, Rongfei Fan, Han Hu, Hangguan Shan, Tony Q. S. Quek
Adaptive Importance Data Heterogeneity Non Convex Loss Function Byzantine Attack Byzantine Robust Federated Learning Convex Loss Function

March 17, 2024

Federated Transfer Learning with Differential Privacy
Mengchu Li, Ye Tian, Yang Feng, Yi Yu
Differential Privacy Data Heterogeneity Privacy Constraint Federated Transfer Learning

March 12, 2024

Machine Learning Techniques for Sensor-based Human Activity Recognition with Data Heterogeneity -- A Review
Xiaozhou Ye, Kouichi Sakurai, Nirmal Nair, Kevin I-Kai Wang
Machine Learning Narrative Review Data Heterogeneity Sensory Data Sensor Based Human Activity Recognition

March 9, 2024

Towards Efficient Replay in Federated Incremental Learning
Yichen Li, Qunwei Li, Haozhao Wang, Ruixuan Li, Wenliang Zhong, Guannan Zhang
Federated Learning Catastrophic Forgetting Data Heterogeneity Efficient Generated Object Replay

March 7, 2024

HeteroSwitch: Characterizing and Taming System-Induced Data Heterogeneity in Federated Learning
Gyudong Kim, Mehdi Ghasemi, Soroush Heidari, Seungryong Kim, Young Geun Kim, Sarma Vrudhula, Carole-Jean Wu
Data Heterogeneity Heterogeneous Device Tame Function

February 25, 2024

Trustworthy Personalized Bayesian Federated Learning via Posterior Fine-Tune
Mengen Luo, Chi Xu, Ercan Engin Kuruoglu
Normalizing Flow Bayesian Neural Network Data Heterogeneity Personalized Federated Learning Personalized Model

February 22, 2024

Federated Learning on Transcriptomic Data: Model Quality and Performance Trade-Offs
Anika Hannemann, Jan Ewald, Leo Seeger, Erik Buchmann
Machine Learning Data Heterogeneity Trade Offs Federated Learning Framework Transcriptomic Data Model Quality

February 20, 2024

CCFC++: Enhancing Federated Clustering through Feature Decorrelation
Jie Yan, Jing Liu, Yi-Zi Ning, Zhong-Yuan Zhang
Contrastive Learning Data Heterogeneity Heterogeneous Data Feature Decorrelation Federated Clustering

February 19, 2024

Hierarchical Bayes Approach to Personalized Federated Unsupervised Learning
Kaan Ozkara, Bruce Huang, Ruida Zhou, Suhas Diggavi
Diffusion Model Data Heterogeneity Personalized Federated Learning Hierarchical Bayesian Personalized Interpretable

February 15, 2024

Prompt-based Personalized Federated Learning for Medical Visual Question Answering
He Zhu, Ren Togo, Takahiro Ogawa, Miki Haseyama
Data Heterogeneity Personalized Federated Learning Medical Datasets Medical Visual Question Answering Heterogeneous Clinical Trainable Parameter

February 2, 2024

Federated Unlearning: a Perspective of Stability and Fairness
Jiaqi Shao, Tao Lin, Xuanyu Cao, Bing Luo
Procedural Fairness Core Stability Visual Perspective Data Heterogeneity Unlearning Framework Harmful Unlearning Federated Unlearning Local Fairness

January 4, 2024

January 2, 2024

Hierarchical Over-the-Air Federated Learning with Awareness of Interference and Data Heterogeneity
Seyed Mohammad Azimi-Abarghouyi, Viktoria Fodor
Data Heterogeneity Catastrophic Interference Social Awareness Air Computation Stochastic Geometry

December 25, 2023

Efficient Conformal Prediction under Data Heterogeneity
Vincent Plassier, Nikita Kotelevskii, Aleksandr Rubashevskii, Fedor Noskov, Maksim Velikanov, Alexander Fishkov, Samuel Horvath, Martin Takac, Eric Moulines, Maxim Panov
Uncertainty Quantification Conformal Prediction Data Heterogeneity Confidence Set Personalized Prediction

December 21, 2023

Fed-CO2: Cooperation of Online and Offline Models for Severe Data Heterogeneity in Federated Learning
Zhongyi Cai, Ye Shi, Wei Huang, Jingya Wang
Knowledge Transfer Data Heterogeneity Online Service Label Distribution Offline Model Heterogeneous Label

December 20, 2023

Fed-QSSL: A Framework for Personalized Federated Learning under Bitwidth and Data Heterogeneity
Yiyue Chen, Haris Vikalo, Chianing Wang
Self Supervised Learning New Framework Data Heterogeneity Personalized Federated Learning Centralized Machine Learning Low Bit Quantization Learning Model Re Quantization

December 18, 2023

FedCompetitors: Harmonious Collaboration in Federated Learning with Competing Participants
Shanli Tan, Hao Cheng, Xiaohu Wu, Han Yu, Tiantian He, Yew-Soon Ong, Chongjun Wang, Xiaofeng Tao
Federated Learning Data Heterogeneity Complementarity Information

Data Heterogeneity

Papers

MAP: Model Aggregation and Personalization in Federated Learning with Incomplete Classes

The Effectiveness of Local Updates for Decentralized Learning under Data Heterogeneity

Byzantine-resilient Federated Learning With Adaptivity to Data Heterogeneity

Federated Transfer Learning with Differential Privacy

Machine Learning Techniques for Sensor-based Human Activity Recognition with Data Heterogeneity -- A Review

Towards Efficient Replay in Federated Incremental Learning

HeteroSwitch: Characterizing and Taming System-Induced Data Heterogeneity in Federated Learning

Trustworthy Personalized Bayesian Federated Learning via Posterior Fine-Tune

Federated Learning on Transcriptomic Data: Model Quality and Performance Trade-Offs

CCFC++: Enhancing Federated Clustering through Feature Decorrelation

Hierarchical Bayes Approach to Personalized Federated Unsupervised Learning

Prompt-based Personalized Federated Learning for Medical Visual Question Answering

Federated Unlearning: a Perspective of Stability and Fairness

Exploring Vacant Classes in Label-Skewed Federated Learning

Federated Class-Incremental Learning with Prototype Guided Transformer

Hierarchical Over-the-Air Federated Learning with Awareness of Interference and Data Heterogeneity

Efficient Conformal Prediction under Data Heterogeneity

Fed-CO2: Cooperation of Online and Offline Models for Severe Data Heterogeneity in Federated Learning

Fed-QSSL: A Framework for Personalized Federated Learning under Bitwidth and Data Heterogeneity

FedCompetitors: Harmonious Collaboration in Federated Learning with Competing Participants