Data Heterogeneity

Data heterogeneity, the variability in data distributions across different sources in federated learning, significantly hinders model accuracy and convergence. Current research focuses on mitigating this challenge through various techniques, including personalized model architectures (e.g., using adaptors or subnetworks), robust aggregation methods (e.g., weighted averaging based on client performance or data characteristics), and innovative training strategies (e.g., warmup phases, loss decomposition, and adversarial training). Addressing data heterogeneity is crucial for advancing federated learning's applicability in diverse real-world scenarios, particularly in healthcare, industrial IoT, and other domains with decentralized, privacy-sensitive data.

Papers

November 19, 2024

Non-IID data in Federated Learning: A Systematic Review with Taxonomy, Metrics, Methods, Frameworks and Future Directions
Daniel M. Jimenez G., David Solans, Mikko Heikkila, Andrea Vitaletti, Nicolas Kourtellis, Aris Anagnostopoulos, Ioannis Chatzigiannakis
Machine Learning Model New Framework Future Direction Data Heterogeneity Heterogeneous Data Metric Library Non IID Data

November 11, 2024

Revisiting Ensembling in One-Shot Federated Learning
Youssef Allouah, Akash Dhasade, Rachid Guerraoui, Nirupam Gupta, Anne-Marie Kermarrec, Rafael Pinot, Rafael Pires, Rishi Sharma
Data Heterogeneity Cifar 10

October 29, 2024

$r$Age-$k$: Communication-Efficient Federated Learning Using Age Factor
Matin Mortaheb, Priyanka Kaswan, Sennur Ulukus
Machine Learning Model Data Heterogeneity Heterogeneous Data Communication Efficient

October 28, 2024

A Unified Solution to Diverse Heterogeneities in One-shot Federated Learning
Jun Bai, Yiliao Song, Di Wu, Atul Sajjanhar, Yong Xiang, Wei Zhou, Xiaohui Tao, Yan Li
LeArning Abstract Unified Framework Data Heterogeneity Model Heterogeneity One Shot Federated Learning

October 24, 2024

Classifier Clustering and Feature Alignment for Federated Learning under Distributed Concept Drift
Junbao Chen, Jingfeng Xue, Yong Wang, Zhenyan Liu, Lu Huang
Data Heterogeneity Concept Drift Feature Alignment

October 18, 2024

Comparative Evaluation of Clustered Federated Learning Methods
Michael Ben Ali (IRIT), Omar El-Rifai (IRIT), Imen Megdiche (IRIT, IRIT-SIG, INUC), André Peninou (IRIT, IRIT-SIG, UT2J), Olivier Teste (IRIT-SIG, IRIT, UT2J, UT)
Data Heterogeneity Federated Prompt Cooperation Comparative Evaluation

October 17, 2024

Federated scientific machine learning for approximating functions and solving differential equations with data heterogeneity
Handi Zhang, Langchen Liu, Lu Lu
Physic Informed Neural Network Partial Differential Equation Data Heterogeneity Differential Equation Function Approximation

October 9, 2024

Benchmarking Data Heterogeneity Evaluation Approaches for Personalized Federated Learning
Zhilong Li, Xiaohu Wu, Xiaoli Tang, Tiantian He, Yew-Soon Ong, Mengmeng Chen, Qiqi Liu, Qicheng Lao, Han Yu
Model Training Data Heterogeneity Personalized Federated Learning Evaluation Benchmark Statistical Heterogeneity Benchmark Framework

October 4, 2024

Collaborative and Efficient Personalization with Mixtures of Adaptors
Abdulla Jasem Almansoori, Samuel Horváth, Martin Takáč
Multi Task Learning Mixture Component Data Heterogeneity Adaptive Learning Adapter Module Federated Setting Personalization Performance

October 3, 2024

FedPeWS: Personalized Warmup via Subnetworks for Enhanced Heterogeneous Federated Learning
Nurbek Tastan, Samuel Horvath, Martin Takac, Karthik Nandakumar
Data Heterogeneity Heterogeneous Federated Learning Unobserved Heterogeneity Warmup Period

October 2, 2024

Addressing Data Heterogeneity in Federated Learning with Adaptive Normalization-Free Feature Recalibration
Vasilis Siomos, Sergio Naval-Marimont, Jonathan Passerat-Palmbach, Giacomo Tarroni
Data Heterogeneity Channel Attention Feature Calibration

September 24, 2024

Adversarial Federated Consensus Learning for Surface Defect Classification Under Data Heterogeneity in IIoT
Jixuan Cui, Jun Li, Zhen Mei, Yiyang Ni, Wen Chen, Zengxiang Li
Model Training Data Heterogeneity Surface Defect Detection Fusion Weight Industrial Internet of Thing Federated Adversarial

September 18, 2024

FedLF: Adaptive Logit Adjustment and Feature Optimization in Federated Long-Tailed Learning
Xiuhua Lu, Peng Li, Xuefeng Jiang
Class Imbalance Data Heterogeneity Cifar 10 Optimal Feature Logit Adjustment

September 11, 2024

Federated Impression for Learning with Distributed Heterogeneous Data
Atrin Arya, Sana Ayromlou, Armin Saadat, Purang Abolmaesumi, Xiaoxiao Li
Synthetic Data Catastrophic Forgetting Data Heterogeneity Heterogeneous Data Real World Clinical

September 9, 2024

September 7, 2024

Unlocking the Potential of Model Calibration in Federated Learning
Yun-Wei Chu, Dong-Jun Han, Seyyedali Hosseinalipour, Christopher Brinton
Full Potential Data Heterogeneity Model Calibration Model Accuracy

September 5, 2024

August 22, 2024

Tackling Data Heterogeneity in Federated Learning via Loss Decomposition
Shuang Zeng, Pengxin Guo, Shuai Wang, Jianbo Wang, Yuyin Zhou, Liangqiong Qu
Data Heterogeneity Group Loss Distribution Shift Problem

Data Heterogeneity

Papers

Non-IID data in Federated Learning: A Systematic Review with Taxonomy, Metrics, Methods, Frameworks and Future Directions

Revisiting Ensembling in One-Shot Federated Learning

$r$Age-$k$: Communication-Efficient Federated Learning Using Age Factor

A Unified Solution to Diverse Heterogeneities in One-shot Federated Learning

Classifier Clustering and Feature Alignment for Federated Learning under Distributed Concept Drift

Comparative Evaluation of Clustered Federated Learning Methods

Federated scientific machine learning for approximating functions and solving differential equations with data heterogeneity

Benchmarking Data Heterogeneity Evaluation Approaches for Personalized Federated Learning

Collaborative and Efficient Personalization with Mixtures of Adaptors

FedPeWS: Personalized Warmup via Subnetworks for Enhanced Heterogeneous Federated Learning

Addressing Data Heterogeneity in Federated Learning with Adaptive Normalization-Free Feature Recalibration

Adversarial Federated Consensus Learning for Surface Defect Classification Under Data Heterogeneity in IIoT

FedLF: Adaptive Logit Adjustment and Feature Optimization in Federated Long-Tailed Learning

Federated Impression for Learning with Distributed Heterogeneous Data

A System and Benchmark for LLM-based Q\&A on Heterogeneous Data

TriplePlay: Enhancing Federated Learning with CLIP for Non-IID Data and Resource Efficiency

Unlocking the Potential of Model Calibration in Federated Learning

Can We Theoretically Quantify the Impacts of Local Updates on the Generalization Performance of Federated Learning?

Efficient Multi-Task Large Model Training via Data Heterogeneity-aware Model Management

Tackling Data Heterogeneity in Federated Learning via Loss Decomposition