Skewed Non IID

Skewed Non-IID data, where data across different sources are neither independent nor identically distributed and exhibit uneven distributions, poses a significant challenge in machine learning, particularly in federated learning settings. Current research focuses on mitigating the negative impacts of this data skew through techniques like adaptive weighting of client contributions, novel aggregation methods (e.g., clustered aggregation, gradient masking), and the incorporation of regularization strategies to improve model accuracy and convergence. Addressing this challenge is crucial for enhancing the reliability and performance of machine learning models trained on diverse and heterogeneous datasets, impacting various applications from reinforcement learning to genomic analysis and federated learning systems.

Papers

September 2, 2024

Supervised Pattern Recognition Involving Skewed Feature Densities
Alexandre Benatti, Luciano da F. Costa
Pattern Recognition Dissimilarity Measure Neighborhood Selection Euclidean Distance Skewed Non IID Coincidence Similarity

March 12, 2024

Symmetric Q-learning: Reducing Skewness of Bellman Error in Online Reinforcement Learning
Motoki Omura, Takayuki Osa, Yusuke Mukuta, Tatsuya Harada
Deep Reinforcement Learning Online Reinforcement Learning State of the Art Reinforcement Error Distribution Skewed Non IID Bellman Error

February 22, 2024

Comparative Analysis of Data Preprocessing Methods, Feature Selection Techniques and Machine Learning Models for Improved Classification and Regression Performance on Imbalanced Genetic Data
Arshmeet Kaur, Morteza Sarmadi
Machine Learning Model Feature Selection Data Preprocessing Class Imbalanced Data Improved Classification Skewed Non IID

January 5, 2024

Federated Learning for distribution skewed data using sample weights
Hung Nguyen, Peiyuan Wu, Morris Chang
Product Distribution Data Distribution Sample Weight Skewed Non IID

December 12, 2023

Feature Norm Regularized Federated Learning: Transforming Skewed Distributions into Global Insights
Ke Hu, WeiDong Qiu, Peng Tang
Global Model Global Context Feature Norm Skewed Non IID

February 21, 2023

CADIS: Handling Cluster-skewed Non-IID Data in Federated Learning with Clustered Aggregation and Knowledge DIStilled Regularization
Nang Hung Nguyen, Duc Long Nguyen, Trong Bang Nguyen, Thanh-Hung Nguyen, Huy Hieu Pham, Truong Thao Nguyen, Phi Le Nguyen
Raw Data FedAvg Algorithm Non IID Data Skewed Non IID Two Stage Aggregation

February 15, 2023

Multi-Task Differential Privacy Under Distribution Skew
Walid Krichene, Prateek Jain, Shuang Song, Mukund Sundararajan, Abhradeep Thakurta, Li Zhang
Differential Privacy Multi Task Learning Privacy Budget Task Distribution User Level Differential Privacy Skewed Non IID

October 23, 2022

Clustering-based Tile Embedding (CTE): A General Representation for Level Design with Skewed Tile Distributions
Mrunal Jadhav, Matthew Guzdial
Procedural Content Generation General Purpose Representation Skewed Non IID Level Design

August 4, 2022

FedDRL: Deep Reinforcement Learning-based Adaptive Aggregation for Non-IID Data in Federated Learning
Nang Hung Nguyen, Phi Le Nguyen, Duc Long Nguyen, Trung Thanh Nguyen, Thuy Dung Nguyen, Huy Hieu Pham, Truong Thao Nguyen
Deep Learning Model Federated Learning Raw Data Cifar 10 Skewed Non IID

June 16, 2022

Using adversarial images to improve outcomes of federated learning for non-IID data
Anastasiya Danilenka, Maria Ganzha, Marcin Paprzycki, Jacek Mańdziuk
LeArning Abstract Imbalanced Data Adversarial Image Adversarial Input Multiple Outcome Non IID Data Skewed Non IID

May 27, 2022

Finite mixture of skewed sub-Gaussian stable distributions
Mahdi Teimouri
Model Based Clustering Finite Mixture Skewed Non IID

January 28, 2022

Gradient Masked Averaging for Federated Learning
Irene Tenison, Sai Aravind Sreeramadas, Vaikkunth Mugunthan, Edouard Oyallon, Irina Rish, Eugene Belilovsky
Natural Gradient Distribution Generalization Gradient Update Skewed Non IID Client Gradient

Skewed Non IID

Papers

Supervised Pattern Recognition Involving Skewed Feature Densities

Symmetric Q-learning: Reducing Skewness of Bellman Error in Online Reinforcement Learning

Comparative Analysis of Data Preprocessing Methods, Feature Selection Techniques and Machine Learning Models for Improved Classification and Regression Performance on Imbalanced Genetic Data

Federated Learning for distribution skewed data using sample weights

Feature Norm Regularized Federated Learning: Transforming Skewed Distributions into Global Insights

CADIS: Handling Cluster-skewed Non-IID Data in Federated Learning with Clustered Aggregation and Knowledge DIStilled Regularization

Multi-Task Differential Privacy Under Distribution Skew

Clustering-based Tile Embedding (CTE): A General Representation for Level Design with Skewed Tile Distributions

FedDRL: Deep Reinforcement Learning-based Adaptive Aggregation for Non-IID Data in Federated Learning

Using adversarial images to improve outcomes of federated learning for non-IID data

Finite mixture of skewed sub-Gaussian stable distributions

Gradient Masked Averaging for Federated Learning