Federated Averaging

Federated Averaging (FedAvg) is a machine learning technique enabling collaborative model training across decentralized devices without directly sharing sensitive data. Current research focuses on addressing challenges like data heterogeneity (variations in data distributions across devices) and communication efficiency, often employing techniques such as variance reduction, adaptive learning rates, and novel aggregation methods beyond simple averaging, including generative approaches and model recombination. These advancements aim to improve the accuracy and scalability of FedAvg for diverse applications, particularly in privacy-sensitive domains like healthcare and mobile computing, where data decentralization is crucial.

Papers

February 15, 2023

FedABC: Targeting Fair Competition in Personalized Federated Learning
Dui Wang, Li Shen, Yong Luo, Han Hu, Kehua Su, Yonggang Wen, Dacheng Tao
Class Imbalance Personalized Federated Learning Federated Averaging Binary Classification Loss Personalized Interpretable

February 6, 2023

On the Convergence of Federated Averaging with Cyclic Client Participation
Yae Jee Cho, Pranay Sharma, Gauri Joshi, Zheng Xu, Satyen Kale, Tong Zhang
Early Stage Convergence Convergence Rate Federated Averaging FedAvg Converges Client Participation

February 3, 2023

Convergence Analysis of Sequential Split Learning on Heterogeneous Data
Yipeng Li, Xinchen Lyu
Convergence Analysis Heterogeneous Data Convergence Guarantee Split Learning Federated Averaging

January 30, 2023

Distributed Stochastic Optimization under a General Variance Condition
Kun Huang, Xiao Li, Shi Pu
Stochastic Gradient Stochastic Optimization Federated Averaging Stochastic Optimization Problem Distributed Stochastic

January 23, 2023

FedExP: Speeding Up Federated Averaging via Extrapolation
Divyansh Jhunjhunwala, Shiqiang Wang, Gauri Joshi
Federated Averaging Extrapolation Framework FedAvg Converges

December 16, 2022

Robust Learning Protocol for Federated Tumor Segmentation Challenge
Ambrish Rawat, Giulio Zizzo, Swanand Kadhe, Jonathan P. Epperlein, Stefano Braghin
Robust Learning Federated Averaging Dynamic Reweighting

December 14, 2022

FLAGS Framework for Comparative Analysis of Federated Learning Algorithms
Ahnaf Hannan Lodhi, Barış Akgün, Öznur Özkasap
Comparative Study Federated Averaging Decentralized Federated Learning Federated Learning Algorithm Gossip Learning

November 23, 2022

A Dynamic Weighted Federated Learning for Android Malware Classification
Ayushi Chaudhuri, Arijit Nandi, Buddhadeb Pradhan
Federated Learning Federated Averaging Android Malware

November 9, 2022

Knowledge Distillation for Federated Learning: a Practical Guide
Alessio Mora, Irene Tenison, Paolo Bellavista, Irina Rish
Knowledge Distillation Practical Guide Federated Averaging Federated Online Adaptation

November 7, 2022

HFedMS: Heterogeneous Federated Learning with Memorable Data Semantics in Industrial Metaverse
Shenglai Zeng, Zonghang Li, Hongfang Yu, Zhihao Zhang, Long Luo, Bo Li, Dusit Niyato
Privacy Preserving Heterogeneous Federated Learning Ubiquitous Semantic Metaverse Federated Averaging Edge Intelligence

November 3, 2022

Federated Hypergradient Descent
Andrew K Kan
Hyperparameter Optimization Hyperparameter Tuning Federated Averaging Federated Stochastic Federated Hyperparameter

October 31, 2022

Federated Averaging Langevin Dynamics: Toward a unified theory and new algorithms
Vincent Plassier, Alain Durmus, Eric Moulines
Bayesian Inference Federated Averaging Unified Model New Algorithm McMc Algorithm Global Posterior

September 28, 2022

FedVeca: Federated Vectorized Averaging on Non-IID Data with Adaptive Bi-directional Global Objective
Ping Luo, Jieren Cheng, Zhenhao Liu, N.Xiong, Jie Wu
Raw Data Objective Function Federated Averaging Local Gradient FedCF Method

August 16, 2022

FedMR: Fedreated Learning via Model Recombination
Ming Hu, Zhihao Yue, Zhiwei Ling, Xian Wei, Mingsong Chen
Global Model Federated Averaging Privacy Preserving Machine Learning Model Recombination

June 24, 2022

Data Leakage in Federated Averaging
Dimitar I. Dimitrov, Mislav Balunović, Nikola Konstantinov, Martin Vechev
Federated Averaging Data Leakage FedAvg Algorithm FedAvg Converges

June 22, 2022

Quantization Robust Federated Learning for Efficient Inference on Heterogeneous Devices
Kartik Gupta, Marios Fournarakis, Matthias Reisser, Christos Louizos, Markus Nagel
Efficient Inference Federated Averaging Device Learning Federated Optimization Heterogeneous Device

June 20, 2022

Mitigating Data Heterogeneity in Federated Learning with Data Augmentation
Artur Back de Luca, Guojun Zhang, Xi Chen, Yaoliang Yu
Data Augmentation Data Heterogeneity Federated Averaging Heterogeneous Domain Adaptation

June 9, 2022

On the Unreasonable Effectiveness of Federated Averaging with Heterogeneous Data
Jianyu Wang, Rudrajit Das, Gauri Joshi, Satyen Kale, Zheng Xu, Tong Zhang
Heterogeneous Data Federated Averaging FedAvg Algorithm Gradient Similarity

May 27, 2022

FedAvg with Fine Tuning: Local Updates Lead to Representation Learning
Liam Collins, Hamed Hassani, Aryan Mokhtari, Sanjay Shakkottai
Fine Tuning Representation Learning Federated Averaging FedAvg Converges Multi Task Representation Model Averaging Local Update

May 25, 2022

Scalable and Low-Latency Federated Learning with Cooperative Mobile Edge Networking
Zhenxiao Zhang, Zhidong Gao, Yuanxiong Guo, Yanmin Gong
Federated Learning Potential Scalability Federated Averaging Edge Cloud Edge Learning