FedDES: Graph-Based Dynamic Ensemble Selection for Personalized Federated Learning

Brianna Mueller; W. Nick Street

cs.LGMar 30, 2026

FedDES: Graph-Based Dynamic Ensemble Selection for Personalized Federated Learning

Authors: Brianna Mueller, W. Nick Street

Organizations: University of Iowa, Iowa City, Iowa

Abstract

Statistical heterogeneity in Federated Learning (FL) often leads to negative transfer, where a single global model fails to serve diverse client distributions. Personalized federated learning (pFL) aims to address this by tailoring models to individual clients. However, under most existing pFL approaches, clients integrate peer client contributions uniformly, which ignores the reality that not all peers are likely to be equally beneficial. Additionally, the potential for personalization at the instance level remains largely unexplored, even though the reliability of different peer models often varies across individual samples within the same client. We introduce FedDES (Federated Dynamic Ensemble Selection), a decentralized pFL framework that achieves instance-level personalization through dynamic ensemble selection. Central to our approach is a Graph Neural Network (GNN) meta-learner trained on a heterogeneous graph modeling interactions between data samples and candidate classifiers. For each test query, the GNN dynamically selects and weights peer client models, forming an ensemble of the most competent classifiers while effectively suppressing contributions from those that are irrelevant or potentially harmful for performance. Experiments on CIFAR-10 and real-world ICU healthcare data demonstrate that FedDES outperforms state-of-the-art pFL baselines in non-IID settings, offering robust protection against negative transfer.

Figures & tables

Figure 1. Overview of FedDES. Stage 1: Clients independently train heterogeneous base classifiers on their local data and exchange models via peer-to-peer communication, forming a shared classifier pool. Stage 2: Each client evaluates the full classifier pool on its local data to obtain decision-space representations $\mathbf{P}_{k}$ and meta-labels $\mathbf{Z}_{k}$ , and then constructs a heterogeneous graph in which sample nodes (circles) are linked by decision-space similarity and classifier nodes (triangles) are connected to samples based on local competence. Stage 3: A heterogeneous GATv2 meta-learner is trained to map sample embeddings, refined through message passing over the graph, to per-classifier competence scores. Inference: A new query $x_{q}$ is projected into the decision space, inserted into the graph, and processed by the trained GNN. The resulting competence scores $\sigma(\mathbf{s}_{i})$ determine which classifiers are selected (scores above 0.5) and their relative voting strength in the final weighted ensemble prediction $\hat{y}_{q}$ .

	$\alpha=1$						$\alpha=10$
	$C=3$		$C=5$		$C=7$		$C=3$		$C=5$		$C=7$
Method	Acc.	Win%	Acc.	Win%	Acc.	Win%	Acc.	Win%	Acc.	Win%	Acc.	Win%
FedDES	85.7 $\pm$ 6.0	90	74.2 $\pm$ 8.9	95	68.4 $\pm$ 7.3	95	81.9 $\pm$ 6.8	80	68.3 $\pm$ 10.5	90	60.2 $\pm$ 5.1	90
Local	83.8 $\pm$ 7.0	–	70.9 $\pm$ 10.5	–	63.7 $\pm$ 9.7	–	79.3 $\pm$ 7.2	–	64.0 $\pm$ 12.8	–	54.3 $\pm$ 7.7	–
Global	44.5 $\pm$ 16.4	0	46.7 $\pm$ 8.5	0	51.1 $\pm$ 5.8	10	49.1 $\pm$ 18.5	0	49.5 $\pm$ 7.4	20	57.0 $\pm$ 3.8	60
FML	80.4 $\pm$ 7.0	30	67.8 $\pm$ 11.8	25	60.0 $\pm$ 10.0	25	75.5 $\pm$ 8.2	20	61.1 $\pm$ 12.4	40	48.4 $\pm$ 7.6	15

Table 1. Performance comparison on CIFAR-10 across heterogeneity levels. Win rate indicates percentage of clients where a method outperforms the local baseline. Mean accuracy (%)

\pm

standard deviation computed across 20 clients.

	Shock		Mortality
Method	Mean $\pm$ Std	Win (%)	Mean $\pm$ Std	Win (%)
FedDES	65.7 $\pm$ 8.6	70	71.9 $\pm$ 7.2	86
Local	61.1 $\pm$ 7.6	-	64.4 $\pm$ 8.3	-
Global	60.1 $\pm$ 7.1	40	62.8 $\pm$ 7.2	36
LG-FedAvg	62.9 $\pm$ 9.2	64	68.1 $\pm$ 8.2	74
FedTGP	57.3 $\pm$ 9.8	30	61.1 $\pm$ 8.3	44

Table 2. Performance comparison on eICU prediction tasks. Win rate indicates percentage of hospitals where a method outperforms the local baseline. Mean balanced accuracy (%)

\pm

standard deviation computed across 50 hospitals.

Figure 2. Meta-learner selection scores vs. home client class frequency on CIFAR-10 across six heterogeneity settings. Each point represents one (classifier, target class) pair averaged across all clients. Red points indicate local classifiers; gray points indicate non-local classifiers. Dashed line shows linear trend. $\rho$ denotes Spearman correlation. Top row: $\alpha{=}1$ ; bottom row: $\alpha{=}10$ . Columns vary $C\in\{3,5,7\}$ .

	$\alpha=1$			$\alpha=10$
	$C{=}3$	$C{=}5$	$C{=}7$	$C{=}3$	$C{=}5$	$C{=}7$
Mean Ensemble Size	4.36	5.44	7.07	4.96	7.58	9.73
Mean ESS	3.71	3.73	4.12	3.96	4.38	4.71
ESS / Ensemble Size	0.85	0.68	0.58	0.80	0.58	0.48

Table 3. FedDES ensemble characteristics across heterogeneity levels on CIFAR-10. Mean ensemble size is the average number of classifiers selected (competence score

>0.5

). Effective ensemble size (ESS) measures the number of classifiers meaningfully contributing to each prediction after accounting for weight concentration. ESS/Size ratio indicates how evenly weight is distributed among selected classifiers.

Federated Learning

Federated learning trains shared models across data holders without requiring them to pool their raw data.

See all 943 papers

Few-for-Many Personalized Federated Learning
Mar 12, 2026Ping Guo, Tiantian Zhang, Xi Lin +3Federated Learning Heterogeneous Federated Learning
City University of Hong Kong · CityU Shenzhen Research Institute · Hong Kong Metropolitan University +3
Separate Aggregation of Split Network for Personalized Federated Learning
May 26, 2026Yunseok Kang, Jaeyoung SongFederated Learning Aggregation Split Federated Learning
Department of Electronics Engineering, Pusan National University, Republic of Korea
Personalized Federated Learning via Gaussian Generative Modeling
Mar 12, 2026Peng Hu, Jianwei MaFederated Learning Heterogeneous Federated Learning
P. Hu is with the School of Mathematics, Harbin Institute of Technology, Harbin, China. · J. Ma is with the Institute for Artificial Intelligence and the School of Earth and Space Sciences, Peking University, Beijing, China, and also with the Institute for Artificial Intelligence and the School of Mathematics, Harbin Institute of Technology, Harbin, China.

Mar 12, 2026J/K move · Enter open · S save

Few-for-Many Personalized Federated Learning

Ping Guo, Tiantian Zhang, Xi Lin, Xiang Li+2

City University of Hong Kong · CityU Shenzhen Research Institute · Hong Kong Metropolitan University+3

Federated Learning Heterogeneous Federated Learning Multi-Objective Optimization

Open paper AskPDF arXiv

Personalized Federated Learning (PFL) aims to train customized models for clients with highly heterogeneous data distributions while preserving data privacy. Existing approaches often rely on heuristics like clustering or model interpolation, which lack principled mechanisms for balancing heterogeneous client objectives. Serving

M

clients with distinct data distributions is inherently a multi-objective optimization problem, where achieving optimal personalization ideally requires

M

distinct models on the Pareto front. However, maintaining

M

separate models poses significant scalability challenges in federated settings with hundreds or thousands of clients. To address this challenge, we reformulate PFL as a few-for-many optimization problem that maintains only

K

shared server models (

K \ll M

) to collectively serve all

M

clients. We prove that this framework achieves near-optimal personalization: the approximation error diminishes as

K

increases and each client's model converges to each client's optimum as data grows. Building on this reformulation, we propose FedFew, a practical algorithm that jointly optimizes the

K

server models through efficient gradient-based updates. Unlike clustering-based approaches that require manual client partitioning or interpolation-based methods that demand careful hyperparameter tuning, FedFew automatically discovers the optimal model diversity through its optimization process. Experiments across vision, NLP, and real-world medical imaging datasets demonstrate that FedFew, with just 3 models, consistently outperforms other state-of-the-art approaches. Code is available at https://github.com/pgg3/FedFew.

Abstract

Federated Learning

Explore similar work

Few-for-Many Personalized Federated Learning

Separate Aggregation of Split Network for Personalized Federated Learning

Personalized Federated Learning via Gaussian Generative Modeling