FedCSGA: Evolutionary Client Selection with Joint Statistical and System Heterogeneity in Federated Learning
Authors: Ghani Haider, Majid Kundroo, Leo Zhang, Jinchul Choi, Taehong Kim
Organizations: School of Information and Communication Engineering, Chungbuk National University, Cheongju, Republic of Korea · School of Information and Communication Technology, Griffith University, Gold Coast, Australia · Electronics and Telecommunications Research Institute, Daejeon, Republic of Korea
Federated Learning (FL) enables collaborative training of machine learning models across distributed clients while preserving data privacy. In practical deployments, FL relies on selecting a client subset in each communication round to reduce communication cost, mitigate stragglers, and improve training efficiency under heterogeneous data and system conditions. In heterogeneous edge computing environments, both statistical heterogeneity and system heterogeneity significantly affect convergence efficiency and model performance. However, existing client selection methods often address these factors independently, overlooking their coupled and dynamic impact across communication rounds. To address this limitation, we propose FedCSGA, a Client Selection method for Federated learning based on Genetic Algorithm. FedCSGA encodes client subsets as chromosomes and evaluates these subsets through a non-linear fitness function that jointly integrates resource-aware metrics and history-aware participation weighting. This design captures subset-level interactions, mitigates persistent selection bias, and balances statistical utility with system efficiency. Extensive experiments on FMNIST, CIFAR-10, CIFAR-100, and Tiny-ImageNet demonstrate that FedCSGA consistently outperforms state-of-the-art baselines across diverse heterogeneity settings, achieving accuracy improvements of up to 7.0% and faster convergence with fewer communication rounds. These results validate the effectiveness of jointly modeling statistical and system factors to achieve robust and efficient learning in heterogeneous federated environments.
Figures & tables
Work
Strategy
Heterogeneity
Key Assumptions
Overhead
Limitations
FedAvg [ 32 ]
Random sampling
None
Uniform client availability and reliability
Very low (random sampling over clients)
Ignores statistical and system heterogeneity; slow convergence under non-IID.
AFL [ 14 ]
Loss-based prioritization
Statistical
Local training loss reflects data informativeness
Low (requires per-round loss evaluation)
Neglects system constraints; sensitive to unstable loss signals.
PowD [ 10 ]
Greedy (subset sampling and top-k loss)
Statistical
High-loss clients contribute more to global improvement
Low (selection within sampled subset)
Ignores system heterogeneity; may over-select noisy clients.
GreedyFed [ 41 ]
Marginal contribution (Shapley-based)
Statistical
Availability of unbiased server-side validation data
High (repeated validation-based contribution evaluation)
Requires proxy validation data; greedy selection with limited exploration.
FedCS [ 36 ]
Greedy scheduling (time-constrained)
System
Accurate estimation of computation and communication latency
Low (greedy selection based on resource constraints)
Requires accurate resource info; ignores statistical heterogeneity.
RL-based [ 1 ]
Policy learning
Statistical and System
Stationary environment and well-designed state/reward representation
High (policy training and state-space modeling)
Significant training overhead; scalability limited by state-space complexity.
Table 1: Comparison of Client Selection Methods in Federated Learning
Symbol
Description
w
global model parameter in Rd
F(w)
global objective function
Fi(w)
local objective function of the i -th client
Di
local dataset of the i -th client
δj
a data sample from the local dataset Di
L(w,δj)
loss function with parameter w and data sample δj
Table 2: Frequently used notations in FedCSGA
Figure 1: Overview of FedCSGA algorithm for each training round.
Algorithm
FMNIST
CIFAR-10
CIFAR-100
Tiny-Imagenet
α= 0.3
α= 0.6
α= 0.3
α= 0.6
α= 0.3
α= 0.6
α= 0.3
α= 0.6
FedAvg
79.5 ± 2.3
82.0 ± 1.8
49.3 ± 3.1
55.4 ± 1.9
26.2 ± 0.7
27.2 ± 0.8
22.6 ± 0.8
22.8 ± 0.9
FedProx
79.6 ± 1.1
81.8 ± 1.4
50.4 ± 2.8
56.1 ± 1.8
26.3 ± 0.6
27.2 ± 1.0
22.5 ± 0.6
23.3 ± 0.5
PowD
80.9 ± 1.7
82.5 ± 1.4
51.3 ± 4.3
57.2 ± 1.4
25.6 ± 0.8
26.8 ± 0.7
22.7 ± 0.5
23.2 ± 0.9
AFL
78.6 ± 2.6
82.0 ± 1.5
51.6 ± 2.9
56.8 ± 1.8
26.4 ± 0.7
27.2 ± 0.5
22.6 ± 0.4
22.7 ± 0.3
GreedyFed
80.1 ± 1.3
83.3 ± 0.5
49.2 ± 1.3
52.3 ± 1.1
18.0 ± 0.2
18.1 ± 0.7
14.1 ± 0.2
14.1 ± 0.9
Table 3: Comparison of FedCSGA and baseline accuracies on four datasets under non-IID settings ( α={0.3,0.6} ).
Figure 2: Performance comparison across FMNIST, CIFAR-10, CIFAR-100, and Tiny-Imagenet under IID and non-IID settings ( α={0.3,0.6} ).
Algorithm
FMNIST
CIFAR-10
CIFAR-100
Tiny-Imagenet
FedAvg
86.2 ± 0.1
63.7 ± 0.5
27.7 ± 0.5
23.6 ± 0.6
FedProx
86.1 ± 0.3
63.0 ± 0.4
27.5 ± 0.3
23.0 ± 0.9
PowD
86.5 ± 0.1
63.6 ± 0.3
27.7 ± 0.5
23.5 ± 0.5
AFL
86.4 ± 0.2
63.5 ± 0.2
27.6 ± 0.3
22.9 ± 0.4
GreedyFed
85.5 ± 0.3
57.3 ± 0.7
19.3 ± 0.2
17.4 ± 0.9
GreedyFL
88.2 ± 0.1
66.2 ± 0.8
28.5 ± 0.2
24.5 ± 0.53
Table 4: Comparison of FedCSGA and baseline accuracies on four datasets under IID settings.
Algorithm
Client Participation Ratio (CPR)
0.10
0.15
0.20
0.30
FedAvg
49.3 ± 3.1
52.7 ± 2.7
54.3 ± 1.7
56.0 ± 2.0
FedProx
50.4 ± 2.8
54.1 ± 1.7
54.2 ± 1.8
55.6 ± 1.3
PowD
51.3 ± 4.3
52.8 ± 2.7
55.0 ± 2.2
56.2 ± 1.7
AFL
51.6 ± 2.9
54.5 ± 2.2
54.9 ± 2.0
55.8 ± 1.8
GreedyFed
49.2 ± 1.3
51.5 ± 0.6
54.0 ± 1.0
53.8 ± 2.0
Table 5: Accuracy comparison of FedCSGA against baselines across four client participation ratios.
Federated Learning (FL) enables a distributed client-server architecture where multiple clients collaboratively train a global Machine Learning (ML) model without sharing sensitive local data. However, FL often results in lower accuracy than traditional ML algorithms due to statistical heterogeneity across clients. Prior works attempt to address this by using model updates, such as loss and bias, from client models to select participants that can improve the global model's accuracy. However, these updates neither accurately represent a client's heterogeneity nor are their selection methods deterministic. We mitigate these limitations by introducing Terraform, a novel client selection methodology that uses gradient updates and a deterministic selection algorithm to select heterogeneous clients for retraining. This bi-pronged approach allows Terraform to achieve up to 47 percent higher accuracy over prior works. We further demonstrate its efficiency through comprehensive ablation studies and training time analyses, providing strong justification for the robustness of Terraform.
Federated Learning enables collaborative model training across decentralized data sources without data transfer. Averaging-based FL is limited by the presence of non-IID data, which negatively impacts convergence speed and final model accuracy. Conventional alternatives suffer from significant inefficiency. Clients with noisy or highly heterogeneous data contribute expensive gradient computations that are either discarded or heavily down-weighted before aggregation. These reactive approaches waste computational resources, require more communication rounds and result in unnecessary privacy exposure. In this paper, we propose a proactive client selection framework that aims to find an optimal federation of clients whose combined data match utility and fairness requirements before training begins. Our method relies on mutual information computed from differentially private contingency tables to quantify the relevance of cross-feature correlations in the union dataset. We introduce a Potential Federation Loss (PFL) over the set of fixed-size federations, which balances two objectives. Maximizing collective data utility while ensuring fair cross-features correlations to prevent group unfairness. Client selection is expressed as an optimal subset search problem over the PFL objective, which we solve using simulated annealing under strong differential privacy guarantees for clients' local statistics. Experimental results on four benchmarks show faster, fairer, and more accurate models trained on optimally found federations, compared to uniform sampling, even when state-of-the-art adaptive aggregation or sampling strategies are employed.
Federated learning (FL) enables collaborative model training across distributed clients without sharing raw data, making it attractive for privacy-sensitive and bandwidth-constrained applications. A practical challenge in deploying FL is reducing communication overhead while maintaining model performance under heterogeneous, non-IID data distributions. Selecting which subset of clients participates in each training round offers a principled way to address both challenges simultaneously, yet its impact on data diversity and representativeness remains underexplored. In this paper, we investigate three client selection strategies, Power-of-Choice, Fisher information, and Centered Kernel Alignment, evaluating them not only by accuracy and convergence, but through the lens of participation diversity using the Gini coefficient and KL divergence. Under non-IID conditions, restricting participation to 15% of clients per round achieves significant communication savings, and when selection promotes diverse and balanced contributions, it also improves generalization over biased alternatives. This work provides a systematic and reproducible analysis demonstrating that diversity-focused client selection renders communication efficiency and model quality complementary rather than competing objectives.
Max Halleryd, Sima Sinaei, Viktor Valadi +3
RISE Research Institutes of Sweden, Stockholm · KTH Royal Institute of Technology, Stockholm · Scaleout Systems, Uppsala +2