Non IID

Non-IID (non-independent and identically distributed) data, where data samples across different sources exhibit significant variations, poses a major challenge for machine learning, particularly in federated learning settings. Current research focuses on developing robust algorithms and model architectures, such as decentralized federated learning and mixed-effects deep learning, to mitigate the negative impacts of non-IID data on model accuracy, fairness, and generalization. These efforts aim to improve the performance and reliability of machine learning models trained on diverse and heterogeneous datasets, with significant implications for applications like healthcare and finance where data privacy and fairness are paramount.

Papers

April 20, 2024

MultiConfederated Learning: Inclusive Non-IID Data handling with Decentralized Federated Learning
Michael Duchesne, Kaiwen Zhang, Chamseddine Talhi
Transfer Learning Decentralized Federated Learning Non IID Confidential Machine Learning

March 14, 2024

Reawakening knowledge: Anticipatory recovery from catastrophic interference via structured training
Yanlai Yang, Matt Jones, Michael C. Mozer, Mengye Ren
Neural Network Training Dynamic Catastrophic Interference Unseen Entity Anticipatory Behavior Non IID

October 4, 2023

Fairness-enhancing mixed effects deep learning improves fairness on in- and out-of-distribution clustered (non-iid) data
Son Nguyen, Adam Wang, Albert Montillo
Deep Learning Procedural Fairness Mixed Effect Distribution Data Mixed Effect Model Fairness Analysis Cluster Specific Random Effect Fairness Attribute Non IID

March 19, 2023

Experimenting with Normalization Layers in Federated Learning on non-IID scenarios
Bruno Casella, Roberto Esposito, Antonio Sciarappa, Carlo Cavazzoni, Marco Aldinucci
Batch Normalization Layer Normalization Training Deep Normalization Layer Non IID

November 10, 2022

Robust Federated Learning against both Data Heterogeneity and Poisoning Attack via Aggregation Optimization
Yueqi Xie, Weizhong Zhang, Renjie Pi, Fangzhao Wu, Qifeng Chen, Xing Xie, Sunghun Kim
Poisoning Attack Data Heterogeneity Generalization Capacity Optimal Aggregation Server Side Aggregation Non IID Subspace Training

Non IID

Papers

MultiConfederated Learning: Inclusive Non-IID Data handling with Decentralized Federated Learning

Reawakening knowledge: Anticipatory recovery from catastrophic interference via structured training

Fairness-enhancing mixed effects deep learning improves fairness on in- and out-of-distribution clustered (non-iid) data

Experimenting with Normalization Layers in Federated Learning on non-IID scenarios

Robust Federated Learning against both Data Heterogeneity and Poisoning Attack via Aggregation Optimization