A Unified Framework for Fair and Personalized Decentralized Learning under Communication Constraints

Krishnendu S. Tharakan; Carlo Fischione

cs.LGAug 27, 2026

A Unified Framework for Fair and Personalized Decentralized Learning under Communication Constraints

Authors: Krishnendu S. Tharakan, Carlo Fischione

Organizations: School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, Stockholm, Sweden

Abstract

Decentralized learning systems aim to collaboratively train models across multiple clients without relying on a central coordinator. While decentralization improves scalability, privacy, and robustness, it also exacerbates three fundamental challenges: statistical heterogeneity across clients, fairness in client-level performance, and stringent communication constraints. This raises a natural question: \emph{how fair can decentralized learning be under limited communication?} We address this question by presenting a unified framework for decentralized learning under communication constraints, bringing together graph-based personalization, agnostic fairness, and compressed event-triggered communication. Specifically, we propose a new algorithm DMFL-SQ, a decentralized multi-task learning algorithm that couples personalized model training over a communication graph with an agnostic mixture fairness objective, while reducing communication through sparsification, quantization, and event-triggered synchronization. We establish convergence guarantees for general non-convex objectives and show that DMFL-SQ achieves an $\mathcal{O}(T^{-1/2})$ rate in expected squared Moreau-envelope stationarity despite sparse, quantized, and event-triggered communication. We further derive PAC-Bayes generalization guarantees for the fairness-aware mixture objective. Experiments on CIFAR-10 and the real heterogeneous MUSMET EEG dataset demonstrate that DMFL-SQ substantially reduces communication while maintaining predictive performance and improving fairness across clients. Together, our theoretical and empirical results show that personalization, fairness, and communication efficiency can be jointly achieved in decentralized learning while preserving the dominant convergence rate.

Figures & tables

Feature	[ 29 ]	[ 13 ]	[ 6 ]	[ 18 ]	[ 14 ]	[ 30 ] [ 17 ]	Proposed
Fully decentralized	✓	✓			✓		✓
Personalized			✓				✓
Fairness-aware objective				✓			✓
Compressed communication	✓	✓			✓	✓	✓
Event-triggered communication					✓		✓
Non-convex convergence guarantee	✓				✓		✓

TABLE I : Literature Review

Panel of Figure 2 — Fig. 1 : Illustration of DMFL-SQ. Each client maintains a personalized model $w_{i}$ , performs local learning at every round, and communicates only sparse-quantized event-triggered model-copy innovations. Neighboring models are softly coupled through $\mathcal{R}_{\mathcal{G}}(\mathbf{w})$ , while the active fairness component is selected through scalar max-consensus over local loss estimates.

\widehat{f}_{i}^{t}=\frac{1}{|B_{i,\mathrm{sel}}^{t}|}\sum_{z\in B_{i,\mathrm{sel}}^{t}}\ell_{i}(w_{i}^{t};z),

Algorithm 1 DMFL-SQ: Decentralized Multi-Task Fair Learning with Sparse-Quantized Event-Triggered Communication

Variant	Avg. Acc. $\uparrow$	Worst Acc. $\uparrow$	Bottom-10% Acc. $\uparrow$	Jain’s Index $\uparrow$	Bits/Client $\downarrow$
No fairness ( $\rho=0$ )	$0.7869\pm 0.0024$	$0.3743\pm 0.0241$	$0.4065\pm 0.0241$	$0.9521\pm 0.0041$	0.3 M
No graph coupling ( $\alpha=0$ )	$0.7474\pm 0.0157$	$0.3862\pm 0.0263$	$0.4206\pm 0.0207$	$0.9554\pm 0.0042$	0.2 M
No event-triggering	$0.8044\pm 0.0211$	$0.4397\pm 0.0178$	$0.4850\pm 0.0211$	$0.9748\pm 0.0046$	4.0 M
No sparsification	$0.8187\pm 0.0169$	$0.4504\pm 0.0218$	$0.4905\pm 0.0211$	$0.9737\pm 0.0043$	2.2 M
No quantization	$0.8074\pm 0.0067$	$0.4405\pm 0.0236$	$0.4820\pm 0.0210$	$0.9746\pm 0.0045$	5.3 M
DMFL-SQ	$0.8030\pm 0.0096$	$0.4810\pm 0.0225$	$0.5160\pm 0.0260$	$0.9779\pm 0.0047$	$0.6$ M

TABLE III : Component ablation of DMFL-SQ on MUSMET Dataset. Performance metrics are reported as mean

\pm

95% CI across

R=5

seeds.

Explore similar work

Sep 24, 2026cs.LG

SPADE-DFL: Communication-Efficient Decentralized Federated Learning via Derivative-Free Linearized ADMM

Reducing communication in derivative-free decentralized learning requires controlling the disagreement accumulated over multiple local updates. This paper develops SPADE-DFL, a primal--dual method that allows the number of local function-value updates between neighbor exchanges to grow with the computation budget while preserving the nonprivate convergence order. For smooth nonconvex objectives under uniform query-moment bounds, the prescribed nonprivate schedule achieves a time-averaged stationarity and consensus bound of

\mathcal{O}(T^{-1/3})

using only

Θ(T^{2/3})

communication rounds, where

T

is the number of local updates per client. For private training, the accumulated data-dependent increment is isolated from the graph correction, allowing one protected state per client and round to generate all outgoing messages. We prove client-level differential privacy for the full interactive transcript and quantify the resulting optimization error over a finite horizon. Experiments on four classification tasks show that SPADE-DFL achieves higher mean test accuracy than existing decentralized learning methods.

Mengli Wei, Mengkai Zhu, Jiawen Chen +2

School of Mathematics, Southeast University, Nanjing 210096, China

Communication-Efficient Distributed Training Black-Box Optimization Alternating Direction Method of Multipliers Decentralized Learning Differentially Private Federated Learning Derivative-Free Optimization+1

Ask PDF

Mar 2, 2026cs.LG

Decentralized Federated Learning by Partial Message Exchange

Decentralized federated learning (DFL) has emerged as a transformative server-free paradigm that enables collaborative learning over large-scale heterogeneous networks. However, it continues to face fundamental challenges, including data heterogeneity, restrictive assumptions for theoretical analysis, and degraded convergence when standard communication- or privacyenhancing techniques are applied. To overcome these drawbacks, this paper develops a novel algorithm, PaME (DFL by Partial Message Exchange). The central principle is to allow only randomly selected sparse coordinates to be exchanged between two neighbor nodes. Consequently, PaME achieves substantial reductions in communication costs while still preserving a high level of privacy, without sacrificing accuracy. Moreover, grounded in rigorous analysis, the algorithm is shown to converge at a linear rate under the gradient to be locally Lipschitz continuous and the communication matrix to be doubly stochastic. These two mild assumptions not only dispense with many restrictive conditions commonly imposed by existing DFL methods but also enables PaME to effectively address data heterogeneity. Furthermore, comprehensive numerical experiments demonstrate its superior performance compared with several representative decentralized learning algorithms.

Shan Sha, Shenglong Zhou, Xin Wang +2

School of Mathematics and Statistics, Beijing Jiaotong University, Beijing, China · Department of Electrical and Electronic Engineering, Faculty of Engineering, Imperial College London, London, U.K.

Communication-Efficient Distributed Training Federated Learning Heterogeneous Federated Learning

Ask PDF

Aug 15, 2026cs.AI

Decentralized Federated Learning for Heterogeneous Multi-Task Semantic Communication

Collaborative training in distributed semantic communication (DSC) networks typically relies on decentralized federated learning (DFL). However, pushing topology-agnostic aggregation into heterogeneous, multi-task environments creates a fundamental bottleneck: it drives negative transfer and overconsensus bias (OCB). This paper introduces a personalized DSC framework that cuts off this cross-task interference. At the node level, a policy-driven multi-path routing mechanism separates task-specific features from shared representations to preserve local fidelity. Across the network, we deploy a "communicationwhile- aggregation" protocol. It calibrates a column-stochastic consensus matrix using task affinities. This limits the system to absorbing complementary knowledge while actively blocking mismatched parameter updates. To bound the convergence, we derive a unified Lyapunov drift analysis. We reveal a strict Ushaped trade-off: deeper topological mixing reduces variance but amplifies structural OCB. Resolving this tension yields a closed-form expression for the optimal aggregation depth. We evaluate the proposed framework on NYU-v2, where the results reveal a clear trade-off between insufficient aggregation and excessive topological mixing. At the analytically derived optimal aggregation depth, our method achieves a 4.77% global relative improvement over the no-aggregation baseline and outperforms decentralized FedAvg, FedAMP, and heuristic max aggregation. We further evaluate the framework on Taskonomy and imperfect wireless links to examine the effects of network-size variation and wireless-link reliability.

Lin Yin, Tiejun Lv, Weicai Li +2

School of Information and Communication Engineering, Beijing University of Posts and Telecommunications (BUPT), Beijing 100876, China · Center for Target Cognition Information Processing Science and Technology, and the Key Laboratory of Modern Measurement and Control Technology, Ministry of Education, both at Beijing Information Science and Technology University, Beijing, China

Federated Learning Federated Learning Aggregation Semantic Communication Heterogeneous Federated Learning Multi-Task Learning

Ask PDF