Heterogeneity Challenge

Heterogeneity, the variability in data distribution, model architectures, and computational resources across distributed systems, poses a significant challenge to the effectiveness of federated learning and other machine learning applications. Current research focuses on developing robust algorithms and model architectures, such as personalized models, improved aggregation techniques (e.g., weighted averaging based on heterogeneity measures), and prompt tuning/logit distillation, to mitigate the negative impacts of heterogeneity on model accuracy and convergence speed. Addressing heterogeneity is crucial for advancing privacy-preserving machine learning, enabling efficient training across diverse datasets in various fields like healthcare, manufacturing, and environmental monitoring, and improving the generalizability and fairness of machine learning models.

Papers

October 17, 2024

Addressing Heterogeneity and Heterophily in Graphs: A Heterogeneous Heterophilic Spectral Graph Neural Network
Kangkang Lu, Yanhua Yu, Zhiyong Huang, Jia Li, Yuling Wang, Meiyu Liang, Xiting Qin, Yimeng Ren, Tat-Seng Chua, Xidian Wang
Graph Neural Network Graph Drawing Heterogeneous Graph Heterophilic Graph Spectral Graph Neural Network Heterogeneous Setting Heterogeneity Challenge

August 27, 2024

Causal Rule Forest: Toward Interpretable and Precise Treatment Effect Estimation
Chan Hsu, Jun-Ting Wu, Yihuang Kang
Treatment Effect Conditional Average Treatment Effect Interpretable Causal Heterogeneity Challenge Causal Forest

August 18, 2024

Addressing Heterogeneity in Federated Learning: Challenges and Solutions for a Shared Production Environment
Tatjana Legler, Vinit Hegiste, Ahmed Anwar, Martin Ruskowski
Technical Challenge Solution Path Data Heterogeneity Unobserved Heterogeneity Statistical Heterogeneity Heterogeneity Challenge

August 10, 2024

CryoBench: Diverse and challenging datasets for the heterogeneity problem in cryo-EM
Minkyu Jeon, Rishwanth Raghu, Miro Astore, Geoffrey Woollard, Ryan Feathers, Alkin Kaz, Sonya M. Hanson, Pilar Cossio, Ellen D. Zhong
Molecular Conformation Cryo Electron Challenging Dataset Heterogeneity Challenge Protein Graph Structural Heterogeneity

July 22, 2024

A New Theoretical Perspective on Data Heterogeneity in Federated Optimization
Jiayi Wang, Shiqiang Wang, Rong-Rong Chen, Mingyue Ji
Data Heterogeneity Gradient Norm Federated Optimization Heterogeneity Challenge

June 25, 2024

Navigating High-Degree Heterogeneity: Federated Learning in Aerial and Space Networks
Fan Dong, Henry Leung, Steve Drew
Class Imbalance Unobserved Heterogeneity Heterogeneity Challenge Unknown Heterogeneity

May 27, 2024

FedHPL: Efficient Heterogeneous Federated Learning with Prompt Tuning and Logit Distillation
Yuting Ma, Lechao Cheng, Yaxiong Wang, Zhun Zhong, Xiaohua Xu, Meng Wang
Prompt Tuning Heterogeneous Federated Learning Logit Distillation Heterogeneity Challenge

May 6, 2024

Federated Reinforcement Learning with Constraint Heterogeneity
Hao Jin, Liangyu Zhang, Zhihua Zhang
Policy Gradient Proximal Policy Optimization Primal Dual Federated Reinforcement Learning Natural Policy Gradient Heterogeneity Challenge

April 1, 2024

Addressing Heterogeneity in Federated Load Forecasting with Personalization Layers
Shourya Bose, Yu Zhang, Kibaek Kim
Energy Consumption Load Forecasting Smart Meter Short Term Load Forecasting Heterogeneity Challenge Personalization Layer

February 2, 2024

FedShift: Tackling Dual Heterogeneity Problem of Federated Learning via Weight Shift Aggregation
Jungwon Seo, Chunming Rong, Minhoe Kim
Weight Aggregation Heterogeneity Challenge System Heterogeneity Unknown Heterogeneity Heterogeneity Environment

October 31, 2023

FedRec+: Enhancing Privacy and Addressing Heterogeneity in Federated Recommendation Systems
Lin Wang, Zhichao Wang, Xi Leng, Xiaoying Tang
Recommendation System Recommendation Performance Federated Recommendation Heterogeneity Challenge Protecting Privacy Near Optimal Virtual Rating

October 4, 2023

FedAWARE: Maximizing Gradient Diversity for Heterogeneous Federated Server-side Optimization
Dun Zeng, Zenglin Xu, Yu Pan, Qifan Wang, Xiaoying Tang
Federated Optimization Heterogeneity Challenge Diversity Optimization Server Side

July 18, 2023

Federated Learning for Computationally-Constrained Heterogeneous Devices: A Survey
Kilian Pfeiffer, Martin Rapp, Ramin Khalili, Jörg Henkel
Timely Survey Heterogeneous Federated Learning Federated Averaging Smart Device Heterogeneity Challenge

April 1, 2023

Predictive Heterogeneity: Measures and Applications
Jiashuo Liu, Jiayun Wu, Bo Li, Peng Cui
Machine Learning Model Financial Application Data Heterogeneity Unobserved Heterogeneity New Measure Heterogeneity Challenge

November 7, 2022

Closing the Gap between Client and Global Model Performance in Heterogeneous Federated Learning
Hongrui Shi, Valentin Radu, Po Yang
Heterogeneous Federated Learning Global Model Client Collaboration Client Model Heterogeneity Challenge

October 26, 2022

Addressing Heterogeneity in Federated Learning via Distributional Transformation
Haolin Yuan, Bo Hui, Yuchen Yang, Philippe Burlina, Neil Zhenqiang Gong, Yinzhi Cao
Deep Learning Model Data Heterogeneity Benchmark Datasets Heterogeneity Challenge Distribution Change

August 17, 2022

AHEAD: A Triple Attention Based Heterogeneous Graph Anomaly Detection Approach
Shujie Yang, Binchi Zhang, Shangbin Feng, Zhaoxuan Tan, Qinghua Zheng, Jun Zhou, Minnan Luo
Graph Anomaly Detection Heterogeneous Information Network Heterogeneity Challenge Triple Attention

November 22, 2021

FLIX: A Simple and Communication-Efficient Alternative to Local Methods in Federated Learning
Elnur Gasanov, Ahmed Khaled, Samuel Horváth, Peter Richtárik
Optimization Purpose Supervised Learning Communication Efficient Movie Review Heterogeneity Challenge Distance Based Regularization

Heterogeneity Challenge

Papers

Addressing Heterogeneity and Heterophily in Graphs: A Heterogeneous Heterophilic Spectral Graph Neural Network

Causal Rule Forest: Toward Interpretable and Precise Treatment Effect Estimation

Addressing Heterogeneity in Federated Learning: Challenges and Solutions for a Shared Production Environment

CryoBench: Diverse and challenging datasets for the heterogeneity problem in cryo-EM

A New Theoretical Perspective on Data Heterogeneity in Federated Optimization

Navigating High-Degree Heterogeneity: Federated Learning in Aerial and Space Networks

FedHPL: Efficient Heterogeneous Federated Learning with Prompt Tuning and Logit Distillation

Federated Reinforcement Learning with Constraint Heterogeneity

Addressing Heterogeneity in Federated Load Forecasting with Personalization Layers

FedShift: Tackling Dual Heterogeneity Problem of Federated Learning via Weight Shift Aggregation

FedRec+: Enhancing Privacy and Addressing Heterogeneity in Federated Recommendation Systems

FedAWARE: Maximizing Gradient Diversity for Heterogeneous Federated Server-side Optimization

Federated Learning for Computationally-Constrained Heterogeneous Devices: A Survey

Predictive Heterogeneity: Measures and Applications

Closing the Gap between Client and Global Model Performance in Heterogeneous Federated Learning

Addressing Heterogeneity in Federated Learning via Distributional Transformation

AHEAD: A Triple Attention Based Heterogeneous Graph Anomaly Detection Approach

FLIX: A Simple and Communication-Efficient Alternative to Local Methods in Federated Learning