Model Aggregation

Model aggregation in federated learning aims to combine locally trained models from multiple decentralized sources into a single, improved global model without directly sharing sensitive data. Current research focuses on addressing challenges like data heterogeneity (non-IID data) and resource constraints through techniques such as weighted averaging, knowledge distillation, and meta-learning, often employing various similarity metrics to identify compatible models for aggregation. These advancements are crucial for enabling collaborative machine learning in privacy-sensitive applications across diverse settings, including healthcare, IoT, and vehicular networks, improving both model accuracy and robustness.

Papers

October 19, 2022

Latency Aware Semi-synchronous Client Selection and Model Aggregation for Wireless Federated Learning
Liangkun Yu, Xiang Sun, Rana Albelaihi, Chen Yi
Federated Learning Federated Prompt Cooperation Model Aggregation Collaborative Machine Learning Faster Convergence Speed

October 15, 2022

FedCross: Towards Accurate Federated Learning via Multi-Model Cross Aggregation
Ming Hu, Peiheng Zhou, Zhihao Yue, Zhiwei Ling, Yihao Huang, Yang Liu, Mingsong Chen
Federated Learning Model Aggregation Fusion Weight Injection Based Training

October 4, 2022

Domain Discrepancy Aware Distillation for Model Aggregation in Federated Learning
Shangchao Su, Bin Li, Xiangyang Xue
Knowledge Distillation Model Aggregation Domain Discrepancy Knowledge Aggregation

October 1, 2022

Privacy-preserving Decentralized Federated Learning over Time-varying Communication Graph
Yang Lu, Zhengxin Yu, Neeraj Suri
Decentralized Federated Learning Model Aggregation Federated Learning Framework Communication Graph Average Consensus Secret Sharing

September 30, 2022

Rethinking Data Heterogeneity in Federated Learning: Introducing a New Notion and Standard Benchmarks
Mahdi Morafah, Saeed Vahidian, Chen Chen, Mubarak Shah, Bill Lin
Pseudo Label Data Heterogeneity Model Aggregation Statistical Heterogeneity Training Data Distribution Novel Concept Human Aligned Benchmark

August 30, 2022

Online Meta-Learning for Model Update Aggregation in Federated Learning for Click-Through Rate Prediction
Xianghang Liu, Bartłomiej Twardowski, Tri Kurniawan Wijaya
Click Through Rate Prediction Model Aggregation Click Through Rate Client Heterogeneity Online Meta Learning Server Learning Rate

June 30, 2022

Cross-domain Federated Object Detection
Shangchao Su, Bin Li, Chengzhi Zhang, Mingzhao Yang, Xiangyang Xue
Object Detection Detection Model Model Aggregation Multi Party Federated Object Detection

June 28, 2022

Fundamental Limits of Communication Efficiency for Model Aggregation in Distributed Learning: A Rate-Distortion Approach
Naifu Zhang, Meixia Tao, Jia Wang, Fan Xu
Communication Efficiency Model Aggregation Distributed Learning Gradient Compression Rate Distortion Fundamental Limit Rate Distortion Function Gradient Quantization

June 27, 2022

Transfer Learning via Test-Time Neural Networks Aggregation
Bruno Casella, Alessio Barbaro Chisari, Sebastiano Battiato, Mario Valerio Giuffrida
Transfer Learning Deep Network Model Aggregation Aggregation Network Group Loss GNN Aggregation Task Specific Loss

May 24, 2022

FedEntropy: Efficient Device Grouping for Federated Learning Using Maximum Entropy Judgment
Zhiwei Ling, Zhihao Yue, Jun Xia, Ming Hu, Ting Wang, Mingsong Chen
Global Model Maximum Entropy Model Aggregation Machine Learning Paradigm

May 16, 2022

A model aggregation approach for high-dimensional large-scale optimization
Haowei Wang, Ercong Zhang, Szu Hui Ng, Giulia Pedrielli
High Dimensional Bayesian Optimization Surrogate Model Model Aggregation Uncertainty Aware Surrogate Model

April 29, 2022

Bridging Differential Privacy and Byzantine-Robustness via Model Aggregation
Heng Zhu, Qing Ling
Native Robustness Differential Privacy Model Aggregation Byzantine Robust Gradient Aggregation

April 26, 2022

One-shot Federated Learning without Server-side Training
Shangchao Su, Bin Li, Xiangyang Xue
Knowledge Distillation Model Aggregation Parameter Selection One Shot Federated Learning

April 20, 2022

Efficient Wireless Federated Learning with Partial Model Aggregation
Zhixiong Chen, Wenqiang Yi, Arumugam Nallanathan, Geoffrey Ye Li
Federated Learning Loss Function Model Aggregation Non Convex Loss Function Device Scheduling

April 7, 2022

Decentralized Event-Triggered Federated Learning with Heterogeneous Communication Thresholds
Shahryar Zehtabi, Seyyedali Hosseinalipour, Christopher G. Brinton
Decentralized FL Model Aggregation Topological Learning Consensus Graph Local Consensus Communication Heterogeneity

March 23, 2022

Contextual Model Aggregation for Fast and Robust Federated Learning in Edge Computing
Hung T. Nguyen, H. Vincent Poor, Mung Chiang
Edge Computing Model Aggregation Contextual Aggregation

March 17, 2022

Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning
Lin Zhang, Li Shen, Liang Ding, Dacheng Tao, Ling-Yu Duan
Knowledge Distillation Fine Tuning Model Aggregation Data Free Knowledge Distillation

February 17, 2022

Time-Correlated Sparsification for Efficient Over-the-Air Model Aggregation in Wireless Federated Learning
Yuxuan Sun, Sheng Zhou, Zhisheng Niu, Deniz Gündüz
Federated Learning Model Aggregation Federated Edge Learning Unstructured Sparsity Top K Sparsification Air Aggregation

January 29, 2022

Achieving Efficient Distributed Machine Learning Using a Novel Non-Linear Class of Aggregation Functions
Haizhou Du, Ryan Yang, Yijian Chen, Qiao Xiang, Andre Wibisono, Wei Huang
Machine Learning Model Aggregation Enhancing Efficiency Aggregation Function Time Varying Network Class Conditional Impression Reappearing Incremental Aggregation

December 21, 2021

On-the-fly Resource-Aware Model Aggregation for Federated Learning in Heterogeneous Edge
Hung T. Nguyen, Roberto Morabito, Kwang Taik Kim, Mung Chiang
Machine Learning Edge Computing Edge AI Model Aggregation Heterogeneous Edge Action on the Fly

Model Aggregation

Papers

Latency Aware Semi-synchronous Client Selection and Model Aggregation for Wireless Federated Learning

FedCross: Towards Accurate Federated Learning via Multi-Model Cross Aggregation

Domain Discrepancy Aware Distillation for Model Aggregation in Federated Learning

Privacy-preserving Decentralized Federated Learning over Time-varying Communication Graph

Rethinking Data Heterogeneity in Federated Learning: Introducing a New Notion and Standard Benchmarks

Online Meta-Learning for Model Update Aggregation in Federated Learning for Click-Through Rate Prediction

Cross-domain Federated Object Detection

Fundamental Limits of Communication Efficiency for Model Aggregation in Distributed Learning: A Rate-Distortion Approach

Transfer Learning via Test-Time Neural Networks Aggregation

FedEntropy: Efficient Device Grouping for Federated Learning Using Maximum Entropy Judgment

A model aggregation approach for high-dimensional large-scale optimization

Bridging Differential Privacy and Byzantine-Robustness via Model Aggregation

One-shot Federated Learning without Server-side Training

Efficient Wireless Federated Learning with Partial Model Aggregation

Decentralized Event-Triggered Federated Learning with Heterogeneous Communication Thresholds

Contextual Model Aggregation for Fast and Robust Federated Learning in Edge Computing

Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning

Time-Correlated Sparsification for Efficient Over-the-Air Model Aggregation in Wireless Federated Learning

Achieving Efficient Distributed Machine Learning Using a Novel Non-Linear Class of Aggregation Functions

On-the-fly Resource-Aware Model Aggregation for Federated Learning in Heterogeneous Edge