Model Heterogeneity

Model heterogeneity in federated learning addresses the challenge of training a global model from diverse client devices with varying computational capabilities and model architectures. Current research focuses on techniques like submodel extraction, personalized federated learning with feature fusion, and the use of knowledge distillation and contrastive learning to mitigate the impact of differing models and data distributions. These advancements aim to improve the efficiency and accuracy of federated learning, particularly in resource-constrained environments and applications with diverse data sources, such as medical imaging and decentralized networks. The ultimate goal is to enable robust and scalable collaborative learning while preserving data privacy.

Papers

October 28, 2024

A Unified Solution to Diverse Heterogeneities in One-shot Federated Learning
Jun Bai, Yiliao Song, Di Wu, Atul Sajjanhar, Yong Xiang, Wei Zhou, Xiaohui Tao, Yan Li
LeArning Abstract Unified Framework Data Heterogeneity Model Heterogeneity One Shot Federated Learning

October 17, 2024

FedPAE: Peer-Adaptive Ensemble Learning for Asynchronous and Model-Heterogeneous Federated Learning
Brianna Mueller, W. Nick Street, Stephen Baek, Qihang Lin, Jingyi Yang, Yankun Huang
Client Model Model Heterogeneity Adaptive Ensemble Model Sharing Federated Online Adaptation

October 9, 2024

FedL2G: Learning to Guide Local Training in Heterogeneous Federated Learning
Jianqing Zhang, Yang Liu, Yang Hua, Jian Cao, Qiang Yang
Heterogeneous Federated Learning Heterogeneous Model Local Training Model Heterogeneity

July 28, 2024

FIARSE: Model-Heterogeneous Federated Learning via Importance-Aware Submodel Extraction
Feijie Wu, Xingchen Wang, Yaqing Wang, Tianci Liu, Lu Su, Jing Gao
Global Model Model Heterogeneity Submodel Extraction

June 24, 2024

Personalized federated learning based on feature fusion
Wolong Xing, Zhenkui Shi, Hongyan Peng, Xiantao Hu, Xianxian Li
LeArning Abstract Feature Fusion Personalized Subject Model Heterogeneity Gradient Amplification

May 26, 2024

Task Groupings Regularization: Data-Free Meta-Learning with Heterogeneous Pre-trained Models
Yongxian Wei, Zixuan Hu, Li Shen, Zhenyi Wang, Yu Li, Chun Yuan, Dacheng Tao
Meta Learning Heterogeneous Model Meta Model Model Heterogeneity

May 19, 2024

Overcoming Data and Model Heterogeneities in Decentralized Federated Learning via Synthetic Anchors
Chun-Yin Huang, Kartik Srinivas, Xin Zhang, Xiaoxiao Li
Knowledge Distillation Federated Learning Knowledge Transfer Decentralized Federated Learning Decentralized FL Data Challenge Model Heterogeneity

April 15, 2024

FedP3: Federated Personalized and Privacy-friendly Network Pruning under Model Heterogeneity
Kai Yi, Nidham Gazagnadou, Peter Richtárik, Lingjuan Lyu
Model Heterogeneity System Heterogeneity Personalized Federated

January 6, 2024

FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learning
Jianqing Zhang, Yang Liu, Yang Hua, Jian Cao
Contrastive Learning Heterogeneous Federated Learning Model Heterogeneity Prototype Generation Global Prototype

October 20, 2023

pFedLoRA: Model-Heterogeneous Personalized Federated Learning with LoRA Tuning
Liping Yi, Han Yu, Gang Wang, Xiaoguang Liu, Xiaoxiao Li
Federated Learning Safe LoRA Model Heterogeneity Local Heterogeneous Model

October 3, 2023

Exploring Model Learning Heterogeneity for Boosting Ensemble Robustness
Yanzhao Wu, Ka-Ho Chow, Wenqi Wei, Ling Liu
Ensemble Diversity Robust Ensemble Heterogeneous Ensemble Model Heterogeneity Deep Neural Network Ensemble

August 16, 2023

Towards Personalized Federated Learning via Heterogeneous Model Reassembly
Jiaqi Wang, Xingyi Yang, Suhan Cui, Liwei Che, Lingjuan Lyu, Dongkuan Xu, Fenglong Ma
Personalized Federated Learning Heterogeneous Model Model Reuse Model Heterogeneity

July 31, 2023

Federated Learning for Data and Model Heterogeneity in Medical Imaging
Hussain Ahmad Madni, Rao Muhammad Umer, Gian Luca Foresti
Knowledge Distillation Raw Data Medical Imaging Data Heterogeneity Model Heterogeneity Heterogeneous Label

July 20, 2023

Heterogeneous Federated Learning: State-of-the-art and Research Challenges
Mang Ye, Xiuwen Fang, Bo Du, Pong C. Yuen, Dacheng Tao
State of the Art Heterogeneous Federated Learning Research Challenge Model Heterogeneity Device Heterogeneity Communication Heterogeneity

April 4, 2023

Selective Knowledge Sharing for Privacy-Preserving Federated Distillation without A Good Teacher
Jiawei Shao, Fangzhao Wu, Jun Zhang
Good Teacher Low Temperature Distillation Federated Distillation Model Heterogeneity

April 3, 2023

FedIN: Federated Intermediate Layers Learning for Model Heterogeneity
Yun-Hin Chan, Zhihan Jiang, Jing Deng, Edith C. -H. Ngai
Federated Learning Gradient Norm Training Model Heterogeneous Model Model Heterogeneity

October 10, 2022

A Survey on Heterogeneous Federated Learning
Dashan Gao, Xin Yao, Qiang Yang
Timely Survey Heterogeneous Data Heterogeneous Federated Learning Model Heterogeneity System Heterogeneity

December 23, 2021

DENSE: Data-Free One-Shot Federated Learning
Jie Zhang, Chen Chen, Bo Li, Lingjuan Lyu, Shuang Wu, Shouhong Ding, Chunhua Shen, Chao Wu
Self Distillation Model Heterogeneity One Shot Federated Learning