cs.LGFeb 21, 2024

FedADMM-InSa: An Inexact and Self-Adaptive ADMM for Federated Learning

Authors: Yongcun Song, Ziqi Wang, Enrique Zuazua

Organizations: Chair for Dynamics, Control, Machine Learning and Numerics – Alexander von Humboldt Professorship, Department of Mathematics, Friedrich-Alexander-Universität Erlangen-Nürnberg, Cauerstrasse 11, Erlangen, 91058, Germany · Chair of Computational Mathematics, Fundaci´on Deusto, Avenida de las Universidades, 24, Bilbao, 48007, Basque Country, Spain · Departamento de Matem´aticas, Universidad Aut´onoma de Madrid, Ciudad Universitaria de Cantoblanco, Madrid, 28049, Spain

Abstract

Federated learning (FL) is a promising framework for learning from distributed data while maintaining privacy. The development of efficient FL algorithms encounters various challenges, including heterogeneous data and systems, limited communication capacities, and constrained local computational resources. Recently developed FedADMM methods show great resilience to both data and system heterogeneity. However, they still suffer from performance deterioration if the hyperparameters are not carefully tuned. To address this issue, we propose an inexact and self-adaptive FedADMM algorithm, termed FedADMM-InSa. First, we design an inexactness criterion for the clients' local updates to eliminate the need for empirically setting the local training accuracy. This inexactness criterion can be assessed by each client independently based on its unique condition, thereby reducing the local computational cost and mitigating the undesirable straggle effect. The convergence of the resulting inexact ADMM is proved under the assumption of strongly convex loss functions. Additionally, we present a self-adaptive scheme that dynamically adjusts each client's penalty parameter, enhancing algorithm robustness by mitigating the need for empirical penalty parameter choices for each client. Extensive numerical experiments on both synthetic and real-world datasets are conducted. As validated by some numerical tests, our proposed algorithm can reduce the clients' local computational load significantly and also accelerate the learning process compared to the vanilla FedADMM. The source code is available at https://github.com/DCN-FAU-AvH/FedADMM-InSa.

Figures & tables

Explore similar work

CardsList
  1. A Multi-Step Inertial ADMM Algorithm for Federated Learning

    Oct 17, 2026Hao Li, Qingsong Wang, Xihong Yan +2Federated LearningAlternating Direction Method of Multipliers

  2. FedAdaVR: Adaptive Variance Reduction for Robust Federated Learning under Limited Client Participation

    Jan 29, 2026S M Ruhul Kabir Howlader, Xiao Chen, Yifei Xie +1Federated LearningHeterogeneous Federated Learning

  3. Function-Space ADMM for Decentralized Federated Learning: A Control Theoretic Perspective

    May 10, 2026Akihito Taya, Yuuki Nishiyama, Kaoru SezakiAlternating Direction Method of MultipliersNon-IID Federated Learning