Heterogeneous Model

Heterogeneous model research focuses on developing and deploying machine learning models with diverse architectures, training data, and computational resources. Current efforts concentrate on efficient aggregation techniques for combining knowledge from these disparate models, including methods like likelihood composition, low-rank factorization, and knowledge distillation, often within federated learning frameworks. This work is crucial for improving the scalability, robustness, and privacy of machine learning systems, particularly in resource-constrained or distributed environments like those found in medical imaging, IoT applications, and personalized recommendation systems. The resulting advancements enhance the practical applicability of machine learning across a wider range of real-world scenarios.

Papers

April 3, 2023

FedIN: Federated Intermediate Layers Learning for Model Heterogeneity
Yun-Hin Chan, Zhihan Jiang, Jing Deng, Edith C. -H. Ngai
Federated Learning Gradient Norm Training Model Heterogeneous Model Model Heterogeneity

March 2, 2023

Distillation from Heterogeneous Models for Top-K Recommendation
SeongKu Kang, Wonbin Kweon, Dongha Lee, Jianxun Lian, Xing Xie, Hwanjo Yu
Knowledge Distillation Knowledge Transfer Mutual Distillation Heterogeneous Model Top N Recommendation Dynamic Knowledge

December 5, 2022

HierarchyFL: Heterogeneous Federated Learning via Hierarchical Self-Distillation
Jun Xia, Yi Zhang, Zhihao Yue, Ming Hu, Xian Wei, Mingsong Chen
Self Distillation Part Whole Hierarchy Heterogeneous Federated Learning Model Aggregation Heterogeneous Model Heterogeneous Artificial Intelligence

October 28, 2022

Completely Heterogeneous Federated Learning
Chang Liu, Yuwen Yang, Xun Cai, Yue Ding, Hongtao Lu
Heterogeneous Federated Learning Label Distribution Data Free Knowledge Distillation Heterogeneous Model

June 21, 2022

WrapperFL: A Model Agnostic Plug-in for Industrial Federated Learning
Xueyang Wu, Shengqi Tan, Qian Xu, Qiang Yang
Federated Learning Ensemble Learning Privacy Preserving Heterogeneous Model Generic Plugin Federated Learning Platform

May 5, 2022

Reconfigurable Heterogeneous Parallel Island Models
Lucas Ângelo da Silveira, Thaynara Arielly de Lima, Mauricio Ayala-Rincón
Particle Swarm Optimization Heterogeneous Computing Heterogeneous Model Processing in Memory Bio Inspired Algorithm

April 29, 2022

H2H: Heterogeneous Model to Heterogeneous System Mapping with Computation and Communication Awareness
Xinyi Zhang, Cong Hao, Peipei Zhou, Alex Jones, Jingtong Hu
Computation Method Heterogeneous Model Multi Accelerator Heterogeneous System

December 22, 2021

Dynamic Combination of Heterogeneous Models for Hierarchical Time Series
Xing Han, Jing Hu, Joydeep Ghosh
Forecasting Model Heterogeneous Model Item Combination Hierarchical Time Series Quantile Forecast Forecasting Pipeline

November 29, 2021

FedHM: Efficient Federated Learning for Heterogeneous Models via Low-rank Factorization
Dezhong Yao, Wanning Pan, Michael J O'Neill, Yutong Dai, Yao Wan, Hai Jin, Lichao Sun
Federated Learning Model Compression Heterogeneous Federated Learning Heterogeneous Model Low Rank Factorization