Non IID Data

Non-IID (non-independent and identically distributed) data, where data points across different sources exhibit varying distributions, poses a significant challenge for machine learning, particularly in federated learning settings. Current research focuses on developing robust algorithms and model architectures, such as those employing knowledge distillation, topological sample selection, and clustered aggregation, to mitigate the negative impacts of non-IID data on model accuracy and convergence. Addressing this challenge is crucial for improving the reliability and performance of machine learning models in real-world applications, where data is often inherently heterogeneous and distributed across multiple sources. This includes improving the efficiency and accuracy of federated learning for applications like person detection in IoT networks.

Papers

November 10, 2024

Mitigating covariate shift in non-colocated data with learned parameter priors
Behraj Khan, Behroz Mirza, Nouman Durrani, Tahir Syed
Many Parameter Covariate Shift Cross Validation SAM Prior Biased Training Non IID Data

October 21, 2024

Geographical Node Clustering and Grouping to Guarantee Data IIDness in Federated Learning
Minkwon Lee, Hyoil Kim, Changhee Joo
Group Setting Internet of Thing Data Non IID Data IoT Node

October 11, 2024

Evaluating Federated Kolmogorov-Arnold Networks on Non-IID Data
Arthur Mendonça Sasse, Claudio Miceli de Farias
Multi Layer Kolmogorov Arnold Network Federated Prompt Cooperation MNIST Digit Non IID Data

September 8, 2024

Some Results on Neural Network Stability, Consistency, and Convergence: Insights into Non-IID Data, High-Dimensional Settings, and Physics-Informed Neural Networks
Ronald Katende, Henry Kasumba, Godwin Kakuba, John M. Mango
Federated Learning Physic Informed Neural Network High Dimensional Strong Consistency Learning Rate Convergence Guarantee Non IID Data Stable Neural Network Uniform Stability

March 4, 2024

Mitigating Label Noise on Graph via Topological Sample Selection
Yuhao Wu, Jiangchao Yao, Xiaobo Xia, Jun Yu, Ruxin Wang, Bo Han, Tongliang Liu
Graph Drawing Label Noise Robust Learning Topological Information Topological Metric Non IID Data Informative Sampling

June 28, 2023

NIPD: A Federated Learning Person Detection Benchmark Based on Real-World Non-IID Data
Kangning Yin, Zhen Ding, Zhihua Dong, Dongsheng Chen, Jie Fu, Xinhui Ji, Guangqiang Yin, Zhiguo Wang
Non IID Data

June 26, 2023

Federated Learning on Non-iid Data via Local and Global Distillation
Xiaolin Zheng, Senci Ying, Fei Zheng, Jianwei Yin, Longfei Zheng, Chaochao Chen, Fengqin Dong
Knowledge Distillation Self Distillation FedAvg Algorithm Non IID Data Global Distillation

June 22, 2023

Concept-aware clustering for decentralized deep learning under temporal shift
Marcus Toftås, Emilie Klefbom, Edvin Listo Zec, Martin Willbo, Olof Mogren
Decentralized Learning Concept Identification Temporal Shift Concept Shift Non IID Data Decentralized Deep Learning

March 27, 2023

Neural Collapse Inspired Federated Learning with Non-iid Data
Chenxi Huang, Liang Xie, Yibo Yang, Wenxiao Wang, Binbin Lin, Deng Cai
Deep Learning Model Neural Collapse Non IID Data Global Structure Faster Convergence Speed

February 21, 2023

CADIS: Handling Cluster-skewed Non-IID Data in Federated Learning with Clustered Aggregation and Knowledge DIStilled Regularization
Nang Hung Nguyen, Duc Long Nguyen, Trong Bang Nguyen, Thanh-Hung Nguyen, Huy Hieu Pham, Truong Thao Nguyen, Phi Le Nguyen
Raw Data FedAvg Algorithm Non IID Data Skewed Non IID Two Stage Aggregation

January 8, 2023

Why Batch Normalization Damage Federated Learning on Non-IID Data?
Yanmeng Wang, Qingjiang Shi, Tsung-Hui Chang
Deep Neural Network Batch Normalization Non IID Data

November 29, 2022

UQ-ARMED: Uncertainty quantification of adversarially-regularized mixed effects deep learning for clustered non-iid data
Alex Treacher, Kevin Nguyen, Dylan Owens, Daniel Heitjan, Albert Montillo
Uncertainty Quantification Ensemble Model Ensemble Method Explainable Metric Non IID Data

October 15, 2022

Linear Scalarization for Byzantine-robust learning on non-IID data
Latifa Errami, El Houcine Bergou
Byzantine Robust Byzantine Attack Data Splitting Weighted Sum Scalarization Non IID Data

September 29, 2022

Label driven Knowledge Distillation for Federated Learning with non-IID Data
Minh-Duong Nguyen, Quoc-Viet Pham, Dinh Thai Hoang, Long Tran-Thanh, Diep N. Nguyen, Won-Joo Hwang
Knowledge Distillation Full Label Non IID Data Global Distillation

June 16, 2022

Using adversarial images to improve outcomes of federated learning for non-IID data
Anastasiya Danilenka, Maria Ganzha, Marcin Paprzycki, Jacek Mańdziuk
LeArning Abstract Imbalanced Data Adversarial Image Adversarial Input Multiple Outcome Non IID Data Skewed Non IID

January 29, 2022

Towards Fast and Accurate Federated Learning with non-IID Data for Cloud-Based IoT Applications
Tian Liu, Jiahao Ding, Ting Wang, Miao Pan, Mingsong Chen
Federated Learning Internet of Thing Device Decentralized Data Internet of Thing Network Locality Sensitive Hashing Non IID Data

November 17, 2021

Low Precision Decentralized Distributed Training over IID and non-IID Data
Sai Aparna Aketi, Sangamesh Kodge, Kaushik Roy
Training Data Communication Compression Decentralized Training Low Precision Training Non IID Data Decentralized Machine