Data-Topology Compatibility Enables Personalization in Decentralized Learning

Veronica Xu; Lingfei Wang; Chenlei Leng; Fengxiang He

stat.MLOct 10, 2026

Data-Topology Compatibility Enables Personalization in Decentralized Learning

Authors: Veronica Xu, Lingfei Wang, Chenlei Leng, Fengxiang He

Abstract

Decentralized learning must balance collaboration with personalization across heterogeneous agents. This paper studies when cooperation can support statistically coherent groups of personalized models. We show that data-topology compatibility enables robust orbit-wise personalization: when local objectives and communication topology have (approximately) compatible automorphisms, agents within an orbit share (approximately) equal limiting distributions, while different orbits remain distinct without explicit personalization. This creates a new intermediate regime between global consensus and independent local learning. Under exact data-topology compatibility, as a reference system, we decompose the stochastic learning dynamics into orbit-level and transverse components, derive a necessary and sufficient stability criterion for distributional synchronization, and characterize the orbit-structured stationary law. Modelling an approximate data-topology compatible system as an $\varepsilon$ -perturbed reference system, we prove the introduced errors have uniform-in-time and stationary $O(\varepsilon)$ bounds in 2-Wasserstein distance. We also prove sufficient conditions for lower stationary average excess risk than both independent local training and the best single shared parameter. Experiments on synthetic data, MNIST, and CIFAR-10 are in full agreement with the theory. The code is available at https://github.com/Veronicaaa1/orbit-personalization.

Figures & tables

Panel of Figure 1 — Figure 1: Orbit structure and robustness. (a) Reference marginals with stationary means and $\pm 2$ standard deviations. (b) Marginal $W_{2}$ under exact compatibility: within-orbit maximum and cross-orbit minimum. (c) Separation margins under joint perturbations: median (solid), sampled minimum (dashed), and 10th–90th percentiles (shaded).

Panel of Figure 2 — Figure 2: Final-checkpoint pairwise centered-logit squared distances in the Main condition, averaged over seeds. Lines mark reference-orbit boundaries; darker colors indicate closer predictions. Color scales vary across panels.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Definition
$\mathcal{G}=(\mathcal{V},\mathcal{E})$	Communication graph with node set $\mathcal{V}$ and edge set $\mathcal{E}$ .
$N,m$	Number of agents and local parameter dimension.
$\Gamma\leq\mathrm{Aut}(\mathcal{G})$	Automorphism subgroup defining the graph symmetry.
$\pi,P_{\pi}$	Graph automorphism and permutation matrix, with $P_{\pi}e_{j}=e_{\pi(j)}$ .
$\mathcal{O}(i),\mathcal{O}_{k},q,n_{k}$	Orbit of node $i$ , $k$ -th orbit, number of orbits, and size $n_{k}=\|\mathcal{O}_{k}\|$ .
$\mathcal{D}_{i},\ell,L_{i}$	Local data distribution, sample loss, and local population risk.

Appendix

Table 1: Main notation used throughout the paper.

Figure 3: Four-orbit blown-up path graph and its quotient graph. Colors indicate automorphism orbits.

Panel of Figure 4 — Figure 4: Temporal validation from exact affine moments. (a) Reference transverse errors and exponential bounds. (b) Joint-law $W_{2}/\varepsilon$ for seed $101$ at six defect levels. (c)–(d) Perturbed means and marginal distances ( $\varepsilon=0.1$ , seed $101$ ); dotted lines are stationary limits. Same-orbit means share a color.

Panel of Figure 5 — Figure 5: Stationary perturbation responses. Solid curves and shading show medians and 10th–90th percentiles over $20$ fixed directions. Dashed lines are slope-one guides, not theoretical bounds.

Figure 6: Stationary risk gains (left) and direct joint certificate

\gamma(\theta_{\varepsilon})

(right). Solid curves, shading, and dashed curves show medians, 10th–90th percentiles, and sampled minima over

20

fixed directions. These summaries are not confidence bounds or worst-case guarantees.

Panel of Figure 7 — Figure 7: Nonquadratic finite-time grouping. (a) Node-wise sample means; colors denote orbits. (b) Maximum within-orbit and minimum cross-orbit empirical $W_{2}$ . (c) Terminal violin plots at $T=30$ , with sample means and orbit separators. Empirical fluctuations include finite-sample error.

Orbit	0	1	2	3	4	5	6	7	8	9
$\mathcal{O}_{1}$	600	500	200	160	140	120	100	80	60	40
$\mathcal{O}_{2}$	60	40	600	500	200	160	140	120	100	80
$\mathcal{O}_{3}$	100	80	60	40	600	500	200	160	140	120
$\mathcal{O}_{4}$	200	160	140	120	100	80	60	40	600	500

Appendix

Table 2: Training images per class at each node in the aligned allocation. Class indices follow the dataset’s native order. Each row sums to

2{,}000

Dataset	Model	$\lambda_{\rm Main}$	$\lambda_{\rm sym}$
MNIST	MLP-LN	$0.03$	$0.0423251$
MNIST	ResNet-20-GN	$0.03$	$0.0423251$
CIFAR-10	MLP-LN	$0.10$	$0.141084$
CIFAR-10	ResNet-20-GN	$0.10$	$0.141084$

Appendix

Table 3: Communication coefficients. Fully symmetric values are rounded for display.

Dataset	Model	$\overline{D}_{\rm within}$	$\overline{D}_{\rm between}$	$\overline{R}_{\rm orbit}$
MNIST	MLP-LN	$0.2460\pm 0.0096$	$0.3092\pm 0.0073$	$0.5569\pm 0.0062$
MNIST	ResNet-20-GN	$0.2189\pm 0.0229$	$0.3236\pm 0.0180$	$0.5969\pm 0.0319$
CIFAR-10	MLP-LN	$0.2181\pm 0.0045$	$0.5094\pm 0.0098$	$0.7002\pm 0.0026$
CIFAR-10	ResNet-20-GN	$0.2689\pm 0.0054$	$0.5641\pm 0.0020$	$0.6772\pm 0.0037$

Appendix

Table 4: Main prediction geometry (mean

\pm

sample SD over three seeds). Panel A averages the five terminal checkpoints per seed, with

\overline{R}_{\rm orbit}

averaging checkpoint ratios. Panel B uses epoch

200

, with

Q

computed within each seed before aggregation.

Figure 8: Prediction-space dispersion during training. Curves show three-seed means; bands indicate

\pm

one sample SD for Main. The gray strip marks epochs

180

–

200

. Vertical scales vary across panels.

Dataset	Model	$\eta_{B}$ Main vs. NoComm	$\eta_{C}$ Main vs. Misaligned	$\eta_{D}$ FullySym vs. Main
MNIST	MLP-LN	$42.45\pm 1.01$	$47.69\pm 0.63$	$53.25\pm 0.82$
MNIST	ResNet-20-GN	$49.34\pm 3.17$	$53.00\pm 3.64$	$69.42\pm 2.46$
CIFAR-10	MLP-LN	$65.07\pm 0.57$	$65.64\pm 0.44$	$80.24\pm 0.37$
CIFAR-10	ResNet-20-GN	$56.24\pm 0.33$	$62.13\pm 0.84$	$77.29\pm 1.43$

Appendix

Table 5: Matched-seed relative reductions in terminal dispersion (mean

\pm

sample SD over three seeds, in percent), defined in Eq. equation 33 .

Figure 9: Main orbit-centroid motion over consecutive five-epoch intervals. Points show

100\rho(a,a+5)

for each seed, normalized by the smallest centroid separation at epoch

200

. The vertical axis is logarithmic.

Dataset	Model	Max. CV	Max. drift	$100\rho(175,200)$
MNIST	MLP-LN	$0.0312$	$0.0465$	$0.4494$ – $0.4652$
MNIST	ResNet-20-GN	$0.0572$	$0.1385$	$0.8536$ – $1.1952$
CIFAR-10	MLP-LN	$0.3090$	$0.6350$	$4.0586$ – $4.5155$
CIFAR-10	ResNet-20-GN	$2.6920$	$4.0136$	$6.6050$ – $9.3083$

Appendix

Table 6: Main terminal stability, in percent. CV and fitted drift are maximized over three seeds and the metrics

D_{\rm within}

D_{\rm between}

, and

R_{\rm orbit}

, using the five checkpoints at epochs

180

–

200

. The last column gives the three-seed range of normalized net centroid displacement over epochs

175

–

200

Decentralized Learning

Decentralized learning trains models through local computation and peer-to-peer information exchange rather than relying on a central coordinator.

See all 101 papers

A Unified Framework for Fair and Personalized Decentralized Learning under Communication Constraints
Aug 27, 2026Krishnendu S. Tharakan, Carlo FischioneDecentralized Learning Communication-Efficient Distributed Training
School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, Stockholm, Sweden
Population Scaling or Data Dilution? Dynamics of Local Topology Evolution in Decentralized Learning
Oct 4, 2026Yin-Kuan Liang, Yan Gao, Yang LongDistributed Optimization Decentralized Learning
Durham University · University of Cambridge
Robust Decentralized Personalized Federated Learning via Prediction-Constrained Neighborhood Collaboration
Sep 7, 2026Xiao Ma, Hong Shen, Hui Tian +2Federated Learning Aggregation Byzantine-Robust Federated Learning
Faculty of Applied Sciences, Macao Polytechnic University, Macao SAR, China

Aug 27, 2026J/K move · Enter open · S save

A Unified Framework for Fair and Personalized Decentralized Learning under Communication Constraints

Krishnendu S. Tharakan, Carlo Fischione

School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, Stockholm, Sweden

Decentralized Learning Communication-Efficient Distributed Training Multi-Task Learning

Open paper AskPDF arXiv

Decentralized learning systems aim to collaboratively train models across multiple clients without relying on a central coordinator. While decentralization improves scalability, privacy, and robustness, it also exacerbates three fundamental challenges: statistical heterogeneity across clients, fairness in client-level performance, and stringent communication constraints. This raises a natural question: how fair can decentralized learning be under limited communication? We address this question by presenting a unified framework for decentralized learning under communication constraints, bringing together graph-based personalization, agnostic fairness, and compressed event-triggered communication. Specifically, we propose a new algorithm DMFL-SQ, a decentralized multi-task learning algorithm that couples personalized model training over a communication graph with an agnostic mixture fairness objective, while reducing communication through sparsification, quantization, and event-triggered synchronization. We establish convergence guarantees for general non-convex objectives and show that DMFL-SQ achieves an

\mathcal{O}(T^{-1/2})

rate in expected squared Moreau-envelope stationarity despite sparse, quantized, and event-triggered communication. We further derive PAC-Bayes generalization guarantees for the fairness-aware mixture objective. Experiments on CIFAR-10 and the real heterogeneous MUSMET EEG dataset demonstrate that DMFL-SQ substantially reduces communication while maintaining predictive performance and improving fairness across clients. Together, our theoretical and empirical results show that personalization, fairness, and communication efficiency can be jointly achieved in decentralized learning while preserving the dominant convergence rate.

Abstract

Decentralized Learning

Explore similar work

A Unified Framework for Fair and Personalized Decentralized Learning under Communication Constraints

Population Scaling or Data Dilution? Dynamics of Local Topology Evolution in Decentralized Learning

Robust Decentralized Personalized Federated Learning via Prediction-Constrained Neighborhood Collaboration