Diversity-Aware Client Selection for Communication-Efficient Federated Learning
Authors: Max Halleryd, Sima Sinaei, Viktor Valadi, Mina Alibeigi, Henrik Abrahamsson, Salman Toor
Organizations: RISE Research Institutes of Sweden, Stockholm · KTH Royal Institute of Technology, Stockholm · Scaleout Systems, Uppsala · Zenseact, Gothenburg · Uppsala University, Uppsala
Federated learning (FL) enables collaborative model training across distributed clients without sharing raw data, making it attractive for privacy-sensitive and bandwidth-constrained applications. A practical challenge in deploying FL is reducing communication overhead while maintaining model performance under heterogeneous, non-IID data distributions. Selecting which subset of clients participates in each training round offers a principled way to address both challenges simultaneously, yet its impact on data diversity and representativeness remains underexplored. In this paper, we investigate three client selection strategies, Power-of-Choice, Fisher information, and Centered Kernel Alignment, evaluating them not only by accuracy and convergence, but through the lens of participation diversity using the Gini coefficient and KL divergence. Under non-IID conditions, restricting participation to 15% of clients per round achieves significant communication savings, and when selection promotes diverse and balanced contributions, it also improves generalization over biased alternatives. This work provides a systematic and reproducible analysis demonstrating that diversity-focused client selection renders communication efficiency and model quality complementary rather than competing objectives.
Figures & tables
Strategy
Train Acc.
Test Acc.
Baseline
95.48
94.95
CKA
79.67
76.41
Fisher
93.19
91.04
Power-of-choice
93.51
91.76
TABLE I: MNIST accuracy (%) averaged over last 15 rounds.
Fig. 1: Gini coefficient (MNIST) over 350 rounds, where 3 clients are selected in each round.
Fig. 2: KL divergence to the uniform distribution (MNIST); 3 clients selected per round.
Fig. 3: Histogram of clients selected during training for Power-of-Choice (MNIST)
Strategy
Train Acc.
Test Acc.
Baseline
97.89
59.77
CKA
87.90
51.84
Fisher
93.01
52.92
Power-of-choice
91.77
53.22
TABLE II: ImageNet50 accuracy (%) averaged over last 15 rounds.
Fig. 4: Gini coefficient (ImageNet50) over 350 rounds, where 3 clients are selected in each round.
Fig. 6: Histogram of clients selected during training for Power-of-Choice (ImageNet50)