Federated Averaging

Federated Averaging (FedAvg) is a machine learning technique enabling collaborative model training across decentralized devices without directly sharing sensitive data. Current research focuses on addressing challenges like data heterogeneity (variations in data distributions across devices) and communication efficiency, often employing techniques such as variance reduction, adaptive learning rates, and novel aggregation methods beyond simple averaging, including generative approaches and model recombination. These advancements aim to improve the accuracy and scalability of FedAvg for diverse applications, particularly in privacy-sensitive domains like healthcare and mobile computing, where data decentralization is crucial.

Papers

November 10, 2024

Client Contribution Normalization for Enhanced Federated Learning
Mayank Kumar Kundalwal, Anurag Saraswat, Ishan Mishra, Deepak Mishra
Heterogeneous Data Federated Averaging Client Contribution Centralized Machine Learning

November 2, 2024

Privacy-Preserving Federated Learning with Differentially Private Hyperdimensional Computing
Fardin Jalil Piran, Zhiling Chen, Mohsen Imani, Farhad Imani
Hyperdimensional Computing Federated Averaging Federated Stochastic

October 4, 2024

BN-SCAFFOLD: controlling the drift of Batch Normalization statistics in Federated Learning
Gonzalo Iñaki Quintana, Laurence Vancamberg, Vincent Jugnon, Mathilde Mougeot, Agnès Desolneux
Batch Normalization Federated Averaging Real Drift New Algorithm

October 2, 2024

Debiasing Federated Learning with Correlated Client Participation
Zhenyu Sun, Ziyang Zhang, Zheng Xu, Gauri Joshi, Pranay Sharma, Ermin Wei
Federated Averaging Cross Device Federated Learning Optimal Separation Client Participation

September 11, 2024

Riemannian Federated Learning via Averaging Gradient Stream
Zhenwei Huang, Wen Huang, Pratik Jawanpuria, Bamdev Mishra
Federated Averaging Riemannian Geometry FedAvg Converges Riemannian Stochastic

September 9, 2024

pFedGPA: Diffusion-based Generative Parameter Aggregation for Personalized Federated Learning
Jiahao Lai, Jiaqi Li, Jian Xu, Yanru Wu, Boshi Tang, Siqi Chen, Yongfeng Huang, Wenbo Ding, Yang Li
Diffusion Model Personalized Federated Learning Federated Averaging Parameter Selection

September 8, 2024

DynamicFL: Federated Learning with Dynamic Communication Resource Allocation
Qi Le, Enmao Diao, Xinran Wang, Vahid Tarokh, Jie Ding, Ali Anwar
Federated Averaging Statistical Heterogeneity Federated Stochastic Communication Heterogeneity

August 27, 2024

Bandwidth-Aware and Overlap-Weighted Compression for Communication-Efficient Federated Learning
Zichen Tang, Junlin Huang, Rudan Yan, Yuxin Wang, Zhenheng Tang, Shaohuai Shi, Amelie Chi Zhou, Xiaowen Chu
Linear Compression Compression Technique Federated Averaging

July 17, 2024

Comparing Federated Stochastic Gradient Descent and Federated Averaging for Predicting Hospital Length of Stay
Mehmet Yigit Balik
Stochastic Gradient Descent Model Training Federated Averaging Local Model Federated Stochastic

July 16, 2024

CCVA-FL: Cross-Client Variations Adaptive Federated Learning for Medical Imaging
Sunny Gupta, Amit Sethi
Medical Imaging Federated Averaging Medical Image Data Synthetic Medical Image

June 18, 2024

Training Diffusion Models with Federated Learning
Matthijs de Goede, Bart Cox, Jérémie Decouchant
Diffusion Model Denoising Diffusion Model Federated Averaging FedAvg Algorithm Diffusion Based Model

June 12, 2024

Regularizing and Aggregating Clients with Class Distribution for Personalized Federated Learning
Gyuejeong Lee, Daeyoung Choi
Personalized Federated Learning Federated Averaging Neural Network Weight Class Distribution Client Clustering

May 31, 2024

Communication-Efficient Distributed Deep Learning via Federated Dynamic Averaging
Michail Theologitis, Georgios Frangias, Georgios Anestis, Vasilis Samoladas, Antonios Deligiannakis
Deep Learning Communication Efficient Federated Averaging Synchronization Parameter Update Barrier

May 4, 2024

Understanding Server-Assisted Federated Learning in the Presence of Incomplete Client Participation
Haibo Yang, Peiwen Qiu, Prashant Khanduri, Minghong Fang, Jia Liu
Federated Learning Speech Presence Federated Averaging PAC Learnability Client Participation

April 24, 2024

Federated Learning with Only Positive Labels by Exploring Label Correlations
Xuming An, Dui Wang, Li Shen, Yong Luo, Han Hu, Bo Du, Yonggang Wen, Dacheng Tao
Multi Label Classification Federated Averaging Label Correlation

February 19, 2024

On the Byzantine-Resilience of Distillation-Based Federated Learning
Christophe Roux, Max Zimmer, Sebastian Pokutta
Knowledge Distillation Federated Averaging Federated Distillation Byzantine Resilience

February 17, 2024

Achieving Linear Speedup in Asynchronous Federated Learning with Heterogeneous Clients
Xiaolu Wang, Zijian Li, Shi Jin, Jun Zhang
Federated Averaging Asynchronous Federated Learning Heterogeneous Client Linear Speedup FedAvg Converges Stale Global Model

February 16, 2024

Enhancing Convergence in Federated Learning: A Contribution-Aware Asynchronous Approach
Changxin Xu, Yuxin Qiao, Zhanxin Zhou, Fanghao Ni, Jize Xiong
Early Stage Convergence Federated Averaging Asynchronous Federated Learning Asynchronous Approach Stale Global Model

January 24, 2024

On Principled Local Optimization Methods for Federated Learning
Honglin Yuan
Federated Averaging Federated Optimization Local Optimization Federated Stochastic

December 27, 2023

Federated Continual Learning via Knowledge Fusion: A Survey
Xin Yang, Hao Yu, Xin Gao, Hao Wang, Junbo Zhang, Tianrui Li
Timely Survey Continual LEArning Continual Learning Federated Averaging Federated Continual Learning Knowledge Fusion

Federated Averaging

Papers

Client Contribution Normalization for Enhanced Federated Learning

Privacy-Preserving Federated Learning with Differentially Private Hyperdimensional Computing

BN-SCAFFOLD: controlling the drift of Batch Normalization statistics in Federated Learning

Debiasing Federated Learning with Correlated Client Participation

Riemannian Federated Learning via Averaging Gradient Stream

pFedGPA: Diffusion-based Generative Parameter Aggregation for Personalized Federated Learning

DynamicFL: Federated Learning with Dynamic Communication Resource Allocation

Bandwidth-Aware and Overlap-Weighted Compression for Communication-Efficient Federated Learning

Comparing Federated Stochastic Gradient Descent and Federated Averaging for Predicting Hospital Length of Stay

CCVA-FL: Cross-Client Variations Adaptive Federated Learning for Medical Imaging

Training Diffusion Models with Federated Learning

Regularizing and Aggregating Clients with Class Distribution for Personalized Federated Learning

Communication-Efficient Distributed Deep Learning via Federated Dynamic Averaging

Understanding Server-Assisted Federated Learning in the Presence of Incomplete Client Participation

Federated Learning with Only Positive Labels by Exploring Label Correlations

On the Byzantine-Resilience of Distillation-Based Federated Learning

Achieving Linear Speedup in Asynchronous Federated Learning with Heterogeneous Clients

Enhancing Convergence in Federated Learning: A Contribution-Aware Asynchronous Approach

On Principled Local Optimization Methods for Federated Learning

Federated Continual Learning via Knowledge Fusion: A Survey