Data Balancing

Data balancing in machine learning aims to address the problem of imbalanced datasets, where some classes have significantly fewer examples than others, leading to biased and inaccurate models. Current research focuses on techniques like data augmentation (using generative models such as GANs and Stable Diffusion), data selection methods that remove specific examples driving model failures, and novel sampling strategies in federated learning to improve global model performance while respecting data privacy. These advancements are crucial for improving the fairness, robustness, and generalizability of machine learning models across diverse applications, particularly in areas like federated learning and multimodal learning where data imbalances are common.

Papers

November 12, 2024

Dual-Criterion Model Aggregation in Federated Learning: Balancing Data Quantity and Quality
Haizhou Zhang, Xianjia Yu, Tomi Westerlund
Quality Issue Data Aggregation Model Aggregation Quantity Aware Aggregation Method Data Balancing Client Data

October 10, 2024

FedEP: Tailoring Attention to Heterogeneous Data Distribution with Entropy Pooling for Decentralized Federated Learning
Chao Feng, Hongjie Guan, Alberto Huertas Celdrán, Jan von der Assen, Gérôme Bovet, Burkhard Stiller
Human Attention Heterogeneous Data Decentralized Federated Learning Data Balancing Decentralized Aggregation

September 21, 2024

Recovering Global Data Distribution Locally in Federated Learning
Ziyu Yao
Generative Model Data Balancing

June 25, 2024

Mind the Graph When Balancing Data for Fairness or Robustness
Jessica Schrouff, Alexis Bellot, Amal Rannen-Triki, Alan Malek, Isabela Albuquerque, Arthur Gretton, Alexander D'Amour, Silvia Chiappa
Native Robustness Graph Drawing Procedural Fairness Causal Graph Human Mind Robust Model Data Balancing Balanced Distribution

June 24, 2024

Data Debiasing with Datamodels (D3M): Improving Subgroup Robustness via Data Selection
Saachi Jain, Kimia Hamidieh, Kristian Georgiev, Andrew Ilyas, Marzyeh Ghassemi, Aleksander Madry
Machine Learning Model Data Selection Group Robustness Data Debiasing Data Balancing Data Model

May 13, 2024

Stable Diffusion-based Data Augmentation for Federated Learning with Non-IID Data
Mahdi Morafah, Matthias Reisser, Bill Lin, Christos Louizos
Raw Data Text to Image Model Training Data Balancing Diffusion Based Data Augmentation

March 7, 2024

CLIP the Bias: How Useful is Balancing Data in Multimodal Learning?
Ibrahim Alabdulmohsin, Xiao Wang, Andreas Steiner, Priya Goyal, Alexander D'Amour, Xiaohua Zhai
Absolute Stance Bias Multimodal Data Contrastive Language Image Multimodal Learning Data Balancing Representation Bias Moment Matching

January 17, 2024

FedLoGe: Joint Local and Generic Federated Learning under Long-tailed Data
Zikai Xiao, Zihan Chen, Liyinglan Liu, Yang Feng, Jian Wu, Wanlu Liu, Joey Tianyi Zhou, Howard Hao Yang, Zuozhu Liu
Federated Learning Personalized Federated Learning Neural Collapse Joint Framework Long Tailed Data Data Balancing Individual Classifier

December 29, 2023

LEFL: Low Entropy Client Sampling in Federated Learning
Waqwoya Abebe, Pablo Munoz, Ali Jannesari
Uniform Sampling Sampling Strategy Data Balancing Client Sampling

December 3, 2023

Neural Network Characterization and Entropy Regulated Data Balancing through Principal Component Analysis
David Yevick, Karolina Hutchison
Neural Network Variational Autoencoders Principal Component Analysis Data Balancing Principal Subspace

November 21, 2023

Federated Learning via Consensus Mechanism on Heterogeneous Data: A New Perspective on Convergence
Shu Zheng, Tiandi Ye, Xiang Li, Ming Gao
Early Stage Convergence Heterogeneous Data Convergence Guarantee New Perspective Data Balancing Consensus Mechanism

October 11, 2023

Survey on Imbalanced Data, Representation Learning and SEP Forecasting
Josias Moukpe
Timely Survey Representation Learning Imbalanced Data Data Imbalance Data Balancing Forecasting Analysis

July 13, 2023

Data Augmentation for Mathematical Objects
Tereso del Rio, Matthew England
Machine Learning Data Augmentation Machine Learning Model Symbolic Computation Data Balancing Mathematical Entity

June 30, 2023

Dataset balancing can hurt model performance
R. Channing Moore, Daniel P. W. Ellis, Eduardo Fonseca, Shawn Hershey, Aren Jansen, Manoj Plakal
Model Performance Audio Driven Audio Datasets Sound Event Classification Performance Data Balancing Rare Class

March 21, 2023

Addressing Class Variable Imbalance in Federated Semi-supervised Learning
Zehui Dong, Wenjing Liu, Siyuan Liu, Xingzhi Chen
Semi Supervised Learning Semi Supervised Class Imbalance Data Balancing

March 13, 2023

ODIN: On-demand Data Formulation to Mitigate Dataset Lock-in
SP Choi, Jihun Lee, Hyeongseok Ahn, Sanghee Jung, Bumsoo Kang
Large Language Model Training Data Zero Shot Learning Generative AI Model Data Balancing

February 13, 2023

Improving Recommendation Fairness via Data Augmentation
Lei Chen, Le Wu, Kun Zhang, Richang Hong, Defu Lian, Zhiqiang Zhang, Jun Zhou, Meng Wang
Data Augmentation Collaborative Filtering Recommendation Model Data Balancing Recommendation Fairness Fairness Aware Recommendation Fairness Optimization

February 3, 2023

Revisiting Long-tailed Image Classification: Survey and Benchmarks with New Evaluation Metrics
Chaowei Fang, Dingwen Zhang, Wen Zheng, Xue Li, Le Yang, Lechao Cheng, Junwei Han
Timely Survey New Benchmark Evaluation Metric Long Tailed Data Balancing Cifar 10 Consistent Prediction Feature Impact

December 17, 2022

Modeling Global Distribution for Federated Learning with Label Distribution Skew
Tao Sheng, Chengchao Shen, Yuan Liu, Yeyu Ou, Zhe Qu, Jianxin Wang
Generative Adversarial Network Label Distribution Data Balancing Clean Distribution

November 28, 2022

Decentralized Learning with Multi-Headed Distillation
Andrey Zhmoginov, Mark Sandler, Nolan Miller, Gus Kristiansen, Max Vladymyrov
Decentralized Learning Private Data Decentralized FL Data Balancing Communication Graph Distillation Paradigm Decentralized Knowledge Distillation