SPADE-DFL: Communication-Efficient Decentralized Federated Learning via Derivative-Free Linearized ADMM

Mengli Wei; Mengkai Zhu; Jiawen Chen; Wenwu Yu; Duxin Che

cs.LGSep 24, 2026

SPADE-DFL: Communication-Efficient Decentralized Federated Learning via Derivative-Free Linearized ADMM

Authors: Mengli Wei, Mengkai Zhu, Jiawen Chen, Wenwu Yu, Duxin Che

Organizations: School of Mathematics, Southeast University, Nanjing 210096, China

Abstract

Reducing communication in derivative-free decentralized learning requires controlling the disagreement accumulated over multiple local updates. This paper develops SPADE-DFL, a primal--dual method that allows the number of local function-value updates between neighbor exchanges to grow with the computation budget while preserving the nonprivate convergence order. For smooth nonconvex objectives under uniform query-moment bounds, the prescribed nonprivate schedule achieves a time-averaged stationarity and consensus bound of $\mathcal{O}(T^{-1/3})$ using only $Θ(T^{2/3})$ communication rounds, where $T$ is the number of local updates per client. For private training, the accumulated data-dependent increment is isolated from the graph correction, allowing one protected state per client and round to generate all outgoing messages. We prove client-level differential privacy for the full interactive transcript and quantify the resulting optimization error over a finite horizon. Experiments on four classification tasks show that SPADE-DFL achieves higher mean test accuracy than existing decentralized learning methods.

Figures & tables

Method	Communication	Local	Single-point	Component	ADMM	Differential
Method	reduction	updates	oracle	memory	updates	privacy
Local training [ 2 , 3 , 25 , 37 , 38 ]	$\checkmark$	$\checkmark$
LT-ADMM [ 4 ]	$\checkmark$	$\checkmark$			$\checkmark$
LT-ADMM-VR [ 4 ]	$\checkmark$	$\checkmark$		$\checkmark$	$\checkmark$
LT-ADMM-DP [ 26 ]	$\checkmark$	$\checkmark$			$\checkmark$	$\checkmark$
Communication-saving methods [ 27 , 28 , 10 , 11 , 32 , 33 , 34 , 35 , 36 ]	$\checkmark$

TABLE I: Selected algorithmic features of the methods discussed in Related Work.

\displaystyle\bar{s}_{i,k}=\operatorname{clip}_{R_{k}^{\mathrm{clip}}}(s_{i,k})=s_{i,k}\min\!\left\{1,\frac{R_{k}^{\mathrm{clip}}}{\lVert s_{i,k}\rVert}\right\}.

Algorithm 1 SPADE-DFL at client

i

Fig. 1: Workflow of SPADE-DFL. In round $k$ , each client freezes its ADMM penalty, initializes the one-point estimator memory, and performs $\tau_{i}$ local derivative-free control-variate updates. The accumulated data-dependent local update is clipped and perturbed once before release, after which all neighbor messages and auxiliary-variable updates are computed from the released state.

Panel of Figure 4 — Fig. 2: Illustrative brain images related to Alzheimer’s disease and brain tumors.

Panel of Figure 5 — Fig. 3: Test accuracy on binary MNIST.

Dataset	Distribution	LT-ADMM-VR	LT-ADMM-DP	DP-SGD	DP-FedAvg	1P-DSG	1P-DSGT	SPADE-DFL	$\Delta_{\mathrm{best}}$
MNIST	IID	$85.30{\pm}7.35$	$\underline{92.22{\pm}3.97}$	$55.81{\pm}13.16$	$71.69{\pm}8.96$	$62.15{\pm}12.91$	$76.58{\pm}11.20$	$\mathbf{95.40{\pm}3.24}$	$+3.18$
	Dir. ( $\alpha=0.6$ )	$83.95{\pm}7.29$	$\underline{92.04{\pm}5.01}$	$55.81{\pm}13.16$	$71.68{\pm}9.11$	$61.81{\pm}12.81$	$75.68{\pm}11.33$	$\mathbf{95.31{\pm}2.68}$	$+3.27$
	Dir. ( $\alpha=0.3$ )	$85.07{\pm}6.97$	$\underline{91.55{\pm}4.75}$	$55.81{\pm}13.16$	$71.75{\pm}9.15$	$61.15{\pm}12.78$	$75.45{\pm}11.17$	$\mathbf{95.15{\pm}3.13}$	$+3.60$
	Pathological	$85.21{\pm}7.48$	$\underline{92.05{\pm}4.85}$	$55.81{\pm}13.16$	$71.69{\pm}9.11$	$61.98{\pm}12.97$	$75.97{\pm}11.81$	$\mathbf{95.37{\pm}3.08}$	$+3.32$
Fashion-MNIST	IID	$84.95{\pm}5.94$	$\underline{88.74{\pm}4.03}$	$57.33{\pm}13.79$	$70.67{\pm}9.98$	$58.01{\pm}12.37$	$74.43{\pm}11.36$	$\mathbf{93.28{\pm}2.21}$	$+4.54$
	Dir. ( $\alpha=0.6$ )	$84.48{\pm}6.51$	$\underline{88.42{\pm}4.68}$	$57.33{\pm}13.79$	$70.89{\pm}10.04$	$57.90{\pm}13.03$	$74.92{\pm}10.46$	$\mathbf{93.25{\pm}1.87}$	$+4.83$

TABLE II: Test accuracy at round 50 under the configurations in Section V-B .

Dataset	$N=30$	$N=40$	$N=50$	$N=75$	$N=100$
MNIST	$99.21\pm 0.15$	$99.03\pm 0.15$	$99.21\pm 0.19$	$99.21\pm 0.23$	$99.15\pm 0.11$
Fashion-MNIST	$94.22\pm 0.67$	$92.28\pm 0.98$	$92.23\pm 1.03$	$93.00\pm 0.90$	$93.55\pm 0.70$
Alzheimer’s disease	$72.16\pm 0.79$	$72.16\pm 1.43$	$72.84\pm 0.89$	$72.49\pm 1.06$	$73.00\pm 0.48$
Brain Tumor MRI	$65.46\pm 3.72$	$68.82\pm 3.56$	$65.00\pm 2.34$	$70.46\pm 3.02$	$70.96\pm 3.40$

TABLE III: Validation accuracy under varying client counts,

\varepsilon=32

Dataset	$\varepsilon=4$	$\varepsilon=8$	$\varepsilon=16$	$\varepsilon=24$	$\varepsilon=32$
MNIST	$93.32\pm 8.71$	$97.98\pm 1.79$	$99.05\pm 0.12$	$99.15\pm 0.16$	$99.15\pm 0.11$
Fashion-MNIST	$92.50\pm 2.18$	$93.38\pm 1.81$	$93.43\pm 1.01$	$93.58\pm 0.88$	$93.55\pm 0.70$
Alzheimer’s disease	$61.05\pm 3.17$	$67.26\pm 2.55$	$71.56\pm 0.74$	$72.81\pm 0.76$	$73.00\pm 0.48$
Brain Tumor MRI	$43.29\pm 9.92$	$54.00\pm 6.73$	$64.18\pm 5.50$	$69.25\pm 4.60$	$70.96\pm 3.40$

TABLE IV: Validation accuracy under varying privacy budgets,

N=100

Fig. 7: Test accuracy across five MNIST partitions.

Fig. 8: Client-level digit-7 proportions across MNIST partitions.

Fig. 9: Accuracy differences relative to LT-ADMM-VR.

Fig. 10: Communication cost under matched local-update budgets.

Explore similar work

Decentralized Federated Learning by Partial Message Exchange
Mar 2, 2026Shan Sha, Shenglong Zhou, Xin Wang +2Federated Learning
School of Mathematics and Statistics, Beijing Jiaotong University, Beijing, China · Department of Electrical and Electronic Engineering, Faculty of Engineering, Imperial College London, London, U.K.
FedA2L: Adaptive layer-wise learning rate adjustment in decentralized federated learning
Aug 10, 2026Van Truong Vo, Khoa Nguyen, Taehong KimFedavg Batch
School of Information and Communication Engineering, Chungbuk National University, Cheongju, South Korea
DPDL: Towards Differential Privacy Preservation in Decentralized Stochastic Learning on Non-IID Data
Jun 3, 2026Yunsheng Yuan, Xue Xiao, Lina Wang +1Decentralized Learning Standard Differential-Privacy
School of Computer Science and Technology, Shandong University, Qingdao 266237, China · Inspur Cloud Information Technology Co, Ltd.

Mar 2, 2026, cs.LGJ/K move · Enter open · S save

Decentralized Federated Learning by Partial Message Exchange

Shan Sha, Shenglong Zhou, Xin Wang, Lingchen Kong+1

School of Mathematics and Statistics, Beijing Jiaotong University, Beijing, China · Department of Electrical and Electronic Engineering, Faculty of Engineering, Imperial College London, London, U.K.

Federated Learning

Open paper AskPDF arXiv

Decentralized federated learning (DFL) has emerged as a transformative server-free paradigm that enables collaborative learning over large-scale heterogeneous networks. However, it continues to face fundamental challenges, including data heterogeneity, restrictive assumptions for theoretical analysis, and degraded convergence when standard communication- or privacyenhancing techniques are applied. To overcome these drawbacks, this paper develops a novel algorithm, PaME (DFL by Partial Message Exchange). The central principle is to allow only randomly selected sparse coordinates to be exchanged between two neighbor nodes. Consequently, PaME achieves substantial reductions in communication costs while still preserving a high level of privacy, without sacrificing accuracy. Moreover, grounded in rigorous analysis, the algorithm is shown to converge at a linear rate under the gradient to be locally Lipschitz continuous and the communication matrix to be doubly stochastic. These two mild assumptions not only dispense with many restrictive conditions commonly imposed by existing DFL methods but also enables PaME to effectively address data heterogeneity. Furthermore, comprehensive numerical experiments demonstrate its superior performance compared with several representative decentralized learning algorithms.

Abstract

Explore similar work

Decentralized Federated Learning by Partial Message Exchange

FedA2L: Adaptive layer-wise learning rate adjustment in decentralized federated learning

DPDL: Towards Differential Privacy Preservation in Decentralized Stochastic Learning on Non-IID Data