Local Heterogeneous Model

Local heterogeneous models in federated learning address the challenge of training collaboratively on decentralized data with varying client resources and data distributions. Current research focuses on developing algorithms that enable efficient knowledge sharing between a global model and diverse client-specific models, often employing techniques like feature extraction sharing, mixture-of-experts, and low-rank adaptation (LoRA). These advancements aim to improve model accuracy and personalization while minimizing communication and computational overhead, thereby enhancing the practicality and scalability of federated learning across diverse environments. The resulting improvements in efficiency and accuracy have significant implications for privacy-preserving machine learning applications.

Papers

June 1, 2024

Federated Model Heterogeneous Matryoshka Representation Learning
Liping Yi, Han Yu, Chao Ren, Gang Wang, Xiaoguang Liu, Xiaoxiao Li
Federated Setting Representation Fusion Local Heterogeneous Model

April 27, 2024

pFedAFM: Adaptive Feature Mixture for Batch-Level Personalization in Heterogeneous Federated Learning
Liping Yi, Han Yu, Chao Ren, Heng Zhang, Gang Wang, Xiaoguang Liu, Xiaoxiao Li
Heterogeneous Federated Learning Scalable Personalization Federated Learning Approach Personalized Prediction Adaptive Mixture Local Heterogeneous Model

February 2, 2024

pFedMoE: Data-Level Personalization with Mixture of Experts for Model-Heterogeneous Personalized Federated Learning
Liping Yi, Han Yu, Chao Ren, Heng Zhang, Gang Wang, Xiaoguang Liu, Xiaoxiao Li
Mixture Component Expert Knowledge Local Model Scalable Personalization Local Heterogeneous Model

November 12, 2023

pFedES: Model Heterogeneous Personalized Federated Learning with Feature Extractor Sharing
Liping Yi, Han Yu, Gang Wang, Xiaoguang Liu
Privacy Preserving Local Model Feature Extractor Federated Learning Approach Local Heterogeneous Model

October 20, 2023

January 27, 2022

On the Convergence of Heterogeneous Federated Learning with Arbitrary Adaptive Online Model Pruning
Hanhan Zhou, Tian Lan, Guru Venkataramani, Wenbo Ding
Early Stage Convergence Convergence Analysis Heterogeneous Federated Learning Local Heterogeneous Model

December 27, 2021

SPIDER: Searching Personalized Neural Architecture for Federated Learning
Erum Mushtaq, Chaoyang He, Jie Ding, Salman Avestimehr
Neural Architecture Search SPIDER Max Local Heterogeneous Model

Local Heterogeneous Model

Papers

Federated Model Heterogeneous Matryoshka Representation Learning

pFedAFM: Adaptive Feature Mixture for Batch-Level Personalization in Heterogeneous Federated Learning

pFedMoE: Data-Level Personalization with Mixture of Experts for Model-Heterogeneous Personalized Federated Learning

pFedES: Model Heterogeneous Personalized Federated Learning with Feature Extractor Sharing

VFedMH: Vertical Federated Learning for Training Multiple Heterogeneous Models

pFedLoRA: Model-Heterogeneous Personalized Federated Learning with LoRA Tuning

On the Convergence of Heterogeneous Federated Learning with Arbitrary Adaptive Online Model Pruning

SPIDER: Searching Personalized Neural Architecture for Federated Learning