cs.LGOct 17, 2026

A Multi-Step Inertial ADMM Algorithm for Federated Learning

Authors: Hao Li, Qingsong Wang, Xihong Yan, Chengcheng Yan, Deren Han

Organizations: School of Mathematics and Computational Science, Xiangtan University, Xiangtan 411105, China · Shanxi Key Laboratory for Intelligent Optimization Computing and Block-chain Technology, School of Mathematics and Statistics, Taiyuan Normal University, Jinzhong 030619, China · School of Mathematical Sciences, Beihang University, Beijing 100191, China

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Federated learning has recently emerged as an important paradigm for large-scale distributed model training, which can be naturally formulated as a class of distributed learning problems. The alternating direction method of multipliers (ADMM) is a widely used approach for solving such problems. However, its practical performance is often hindered by significant communication overhead and system heterogeneity. In this paper, we propose a multi-step inertial ADMM algorithm, called MiFedADMM, for federated learning. The proposed method leverages multiple historical iterates on the server side to construct a higher-order extrapolation after each aggregation to accelerate the convergence behavior. Under mild assumptions, we establish convergence results for the single-step variant of the proposed method and derive an O(1/K) convergence rate. Numerical experiments on both synthetic and real-world datasets demonstrate the effectiveness and improved performance of the proposed method compared with existing approaches.

Figures & tables

Figure 1: Process of FL.
Figure 2: Multi-step inertial technique.
 x^τk+1\displaystyle\hat{\mathbf{x}}^{\tau_{k+1}}=1β∑i=1Nzik,\displaystyle=\textstyle\frac{1}{\beta}\sum\limits_{i=1}^{N}\mathbf{z}^{k}_{i}, (19)
 xτk+1\displaystyle{\mathbf{x}}^{\tau_{k+1}}=x^τk+1+∑i=0s−1ai,τk+1(x^τk+1−i−x^τk+1−i−1),\displaystyle=\textstyle\hat{\mathbf{x}}^{\tau_{k+1}}+\sum\limits_{i=0}^{s-1}a_{i,\tau_{k+1}}\left(\hat{\mathbf{x}}^{\tau_{k+1}-i}-\hat{\mathbf{x}}^{\tau_{k+1}-i-1}\right), (20)
Algorithm 1 MiFedADMM
Figure 3: Objective Value vs. CR.
Figure 4: Impact of k0k_{0} .
Figure 5: Impact of ρ\rho .
Figure 6: Impact of NN .
Figure 7: Impact of larger sizes.
DatasetSourcenn (features)dd (samples)
Qsar Oral ToxicityUCI10248992
real-simLibsvm2095872309
Table 1: Description of two real-world datasets.
Figure 8: Impact of k0k_{0} .
Figure 9: Impact of ρ\rho .
Figure 10: Impact of NN .
Figure 11: Impact of larger sizes.
More in this topic

Federated Learning

Federated learning trains shared models across data holders without requiring them to pool their raw data.

See all 951 papers

Explore similar work

CardsList

  1. Function-Space ADMM for Decentralized Federated Learning: A Control Theoretic Perspective

    May 10, 2026Akihito Taya, Yuuki Nishiyama, Kaoru SezakiAlternating Direction Method of MultipliersNon-IID Federated Learning

    Institute of Industrial Science, The University of Tokyo, Meguro-ku, Tokyo 153-8505, Japan · Center for Spatial Information Science, The University of Tokyo, Kashiwa-shi, Chiba 277-8568, Japan

  2. FedZMG: Efficient Client-Side Optimization in Federated Learning

    Feb 20, 2026Fotios Zantalis, Evangelos Zervas, Grigorios KoulourasFederated LearningNon-IID Federated Learning

    TelSiP Research Laboratory, Department of Electrical and Electronic Engineering, School of Engineering, University of West Attica, Ancient Olive Grove Campus, 250 Thivon Str., GR-12241 Athens, Greece

  3. Fed-ADE: Adaptive Learning Rate for Federated Post-adaptation under Distribution Shift

    Mar 1, 2026Heewon Park, Mugon Joe, Miru Kim +2Adaptive Learning RateFederated Learning

    Department of Electrical and Computer Engineering, Sungkyunkwan University, Republic of Korea · Soongsil University, Seoul, Republic of Korea