Global Gradient

Global gradient computation in federated learning (FL) focuses on efficiently aggregating gradients from distributed devices to train a shared model while preserving data privacy. Current research emphasizes developing algorithms that reduce communication overhead and memory requirements, often employing techniques like forward-mode auto-differentiation, approximated Hessian methods, and adaptive gradient approaches (e.g., AdaGrad, Adam) to improve convergence speed and accuracy. These advancements are crucial for enabling FL across resource-constrained devices and diverse data distributions, with significant implications for deploying large-scale machine learning models in various applications, including medical imaging and natural language processing.

Papers

October 25, 2024

Super Gradient Descent: Global Optimization requires Global Gradient
Seifeddine Achour
Gradient Descent Global Optimization Global Minimum Global Gradient

May 24, 2024

Thinking Forward: Memory-Efficient Federated Finetuning of Language Models
Kunjal Panchal, Nisarg Parikh, Sunav Choudhary, Lijun Zhang, Yuriy Brun, Hui Guan
Large Language Model Language Model Fine Tuning Back Propagation Future Perspective Mode Automatic Differentiation Peak Memory Global Gradient

March 16, 2024

FAGH: Accelerating Federated Learning with Approximated Global Hessian
Mrinmay Sen, A. K. Qin, Krishna Mohan C
Hessian Matrix Global Gradient

March 11, 2024

Adaptive Federated Learning Over the Air
Chenhao Wang, Zihan Chen, Nikolaos Pappas, Howard H. Yang, Tony Q. S. Quek, H. Vincent Poor
Adam Optimizer Convex Loss Adaptive Gradient Method Adaptive Federated Learning Full Matrix AdaGrad Global Gradient

April 16, 2022

IOP-FL: Inside-Outside Personalization for Federated Medical Image Segmentation
Meirui Jiang, Hongzheng Yang, Chen Cheng, Qi Dou
Federated Learning Local Gradient Model Personalization Federated Medical Image Segmentation Global Gradient

December 15, 2021