Federated Averaging

Federated Averaging (FedAvg) is a machine learning technique enabling collaborative model training across decentralized devices without directly sharing sensitive data. Current research focuses on addressing challenges like data heterogeneity (variations in data distributions across devices) and communication efficiency, often employing techniques such as variance reduction, adaptive learning rates, and novel aggregation methods beyond simple averaging, including generative approaches and model recombination. These advancements aim to improve the accuracy and scalability of FedAvg for diverse applications, particularly in privacy-sensitive domains like healthcare and mobile computing, where data decentralization is crucial.

Papers

December 19, 2023

On the Role of Server Momentum in Federated Learning
Jianhui Sun, Xidong Wu, Heng Huang, Aidong Zhang
Integral Role Federated Averaging Local Computation Network Momentum

November 22, 2023

FedFN: Feature Normalization for Alleviating Data Heterogeneity Problem in Federated Learning
Seongyoon Kim, Gihun Lee, Jaehoon Oh, Se-Young Yun
Data Heterogeneity Federated Averaging Feature Normalization Feature Norm

November 17, 2023

Leveraging Function Space Aggregation for Federated Learning at Scale
Nikita Dhawan, Nicole Mitchell, Zachary Charles, Zachary Garrett, Gintare Karolina Dziugaite
Fine Tuning Visual Analogue Scale Federated Averaging Training Epoch

November 3, 2023

Heterogeneous federated collaborative filtering using FAIR: Federated Averaging in Random Subspaces
Aditya Desai, Benjamin Meisburger, Zichang Liu, Anshumali Shrivastava
Recommendation System Collaborative Filtering Recommendation Model Federated Averaging Heterogeneous Setting Neural Collaborative Filtering Random Subspace

October 30, 2023

A Federated Learning Framework for Stenosis Detection
Mariachiara Di Cosmo, Giovanna Migliorelli, Matteo Francioni, Andi Mucaj, Alessandro Maolo, Alessandro Aprile, Emanuele Frontoni, Maria Chiara Fiorentino, Sara Moccia
Federated Learning Federated Averaging Stenosis Detection Coronary Angiography

October 23, 2023

B^2SFL: A Bi-level Blockchained Architecture for Secure Federated Learning-based Traffic Prediction
Hao Guo, Collin Meese, Wanxin Li, Chien-Chung Shen, Mark Nejad
Federated Learning Homomorphic Encryption Federated Averaging Privacy Preserving Machine Learning Blockchain Network Blockchain Based Framework

October 9, 2023

On the Convergence of Federated Averaging under Partial Participation for Over-parameterized Neural Networks
Xin Liu, Wei li, Dazhi Zhan, Yu Pan, Xin Ma, Yu Ding, Zhisong Pan
Neural Network Early Stage Convergence Federated Averaging Deep Linear Two Layer ReLU FedAvg Converges Partial Participation

October 4, 2023

Heterogeneous Federated Learning Using Knowledge Codistillation
Jared Lichtarge, Ehsan Amid, Shankar Kumar, Tien-Ju Yang, Rohan Anil, Rajiv Mathews
Knowledge Distillation Domain Shift Heterogeneous Federated Learning Federated Averaging Knowledge Construction Video Diffusion Distillation

September 20, 2023

Preconditioned Federated Learning
Zeyi Tao, Jindi Wu, Qun Li
Federated Averaging Adaptive Preconditioner SGD Step

September 3, 2023

A Comparative Evaluation of FedAvg and Per-FedAvg Algorithms for Dirichlet Distributed Heterogeneous Data
Hamza Reguieg, Mohammed El Hanjri, Mohamed El Kamili, Abdellatif Kobbane
Model Training Data Heterogeneity Heterogeneous Data Federated Averaging Comparative Evaluation FedAvg Algorithm FedAvg Converges Ordered Matrix Dirichlet

September 1, 2023

Leveraging Learning Metrics for Improved Federated Learning
Andre Fu
Federated Learning Stable Rank Federated Averaging Shannon Entropy Weight Aggregation Supervised Metric

August 13, 2023

Approximate and Weighted Data Reconstruction Attack in Federated Learning
Yongcun Song, Ziqi Wang, Enrique Zuazua
Loss Function Average Approximation Federated Averaging Data Reconstruction Attack

July 18, 2023

Federated Learning for Computationally-Constrained Heterogeneous Devices: A Survey
Kilian Pfeiffer, Martin Rapp, Ramin Khalili, Jörg Henkel
Timely Survey Heterogeneous Federated Learning Federated Averaging Smart Device Heterogeneity Challenge

July 16, 2023

Accelerating Distributed ML Training via Selective Synchronization
Sahil Tyagi, Martin Swany
Deep Neural Network Federated Averaging Synchronization Parameter Update Barrier DNN Training Optimal Model Asynchronous Parallel

July 12, 2023

Locally Adaptive Federated Learning
Sohom Mukherjee, Nicolas Loizou, Sebastian U. Stich
Convex Set Federated Averaging Adaptive Federated Learning Local Geometric Information

June 30, 2023

June 6, 2023

A Lightweight Method for Tackling Unknown Participation Statistics in Federated Averaging
Shiqiang Wang, Mingyue Ji
Federated Averaging Weight Aggregation Heterogeneous Participant

May 29, 2023

Global Layers: Non-IID Tabular Federated Learning
Yazan Obeidi
Multi Layer Data Heterogeneity Federated Averaging Model Personalization Tabular Learning

May 25, 2023

FAVANO: Federated AVeraging with Asynchronous NOdes
Louis Leconte, Van Minh Nguyen, Eric Moulines
Deep Neural Network Convergence Guarantee Federated Averaging Asynchronous Federated Learning Asynchronous Architecture