Automating aggregation strategy selection in federated learning

Dian S. Y. Pang; Endrias Y. Ergetu; Eric Topham; Ahmed E. Fetit

cs.LGApr 9, 2026

Automating aggregation strategy selection in federated learning

Authors: Dian S. Y. Pang, Endrias Y. Ergetu, Eric Topham, Ahmed E. Fetit

Organizations: Department of Computing, Imperial College London, UK · OctaiPipe, London, UK

Abstract

Federated Learning enables collaborative model training without centralising data, but its effectiveness varies with the selection of the aggregation strategy. This choice is non-trivial, as performance varies widely across datasets, heterogeneity levels, and compute constraints. We present an end-to-end framework that automates, streamlines, and adapts aggregation strategy selection for federated learning. The framework operates in two modes: a single-trial mode, where large language models infer suitable strategies from user-provided or automatically detected data characteristics, and a multi-trial mode, where a lightweight genetic search efficiently explores alternatives under constrained budgets. Extensive experiments across diverse datasets show that our approach enhances robustness and generalisation under non-IID conditions while reducing the need for manual intervention. Overall, this work advances towards accessible and adaptive federated learning by automating one of its most critical design decisions, the choice of an aggregation strategy.

Figures & tables

Work	Heterogeneity Detection	Strategy Selection	Aggregation Tuning	LLM Automation	Compute-Aware
Li et al. (2021)	✓	✓(fixed mapping)
Dubey (2025)	✓	✓(fixed mapping)
Fed-Hetero (2025)	✓
Mawela et al. (2025)				✓
Shen et al. (2023)				✓
Ours	✓(lightweight)	✓(adaptive)	✓	✓	✓

Table 1: Comparison of related work on strategy selection, heterogeneity detection, and automation.

Figure 1: Framework design for the single-trial mode. Users can either provide their own description of heterogeneity (human-involved pipeline) or rely on fully automated heterogeneity detection (label skew, feature skew, and outlier detection). In both cases, the information is formatted into an LLM prompt. The LLM generates an aggregation strategy configuration, which is validated before being executed for federated training.

Figure 2: Framework design for the multi-trial mode. The first generation of candidates is randomly generated and evaluated. Top-performing configurations are then selected as parents and mutated to form new candidates, which inherit the strategy type but perturb associated parameters. Each candidate is evaluated and stored in a global archive, enabling progressive refinement of aggregation strategies while keeping the search computationally lightweight.

Dataset	Data Type	Task Type	Task Description
NASA Bearing [ 5 ]	Tabular	Fault prediction	Predicting whether a bearing is healthy or faulty from vibration sensor readings.
CIFAR-10 [ 11 ]	Images	Object recognition	Predicting the object category (e.g., car, dog, airplane) from natural images.
Wine Quality [ 6 ]	Tabular	Quality assessment	Predicting wine quality scores (1-10) based on physicochemical features.
Retinal Images [ 2 , 20 ]	Images	Disease diagnosis	Predicting whether fundus images show glaucomatous disease or healthy eyes.
Sentiment140 [ 9 ]	Text	Sentiment analysis	Predicting whether a tweet expresses positive or negative sentiment.
CartPole [ 3 ]	Reinforcement Learning environment	Control	Predicting optimal actions to balance a pole on a moving cart under varying dynamics.

Table 2: Overview of all datasets used in the experimental phases that led to the final design.

Simulated Heterogeneity (NASA Bearing)	Format-A	Format-B	Format-C
Label Skew	0.6122 $\pm$ 0.0021	0.6360 $\pm$ 0.0058	0.6094 $\pm$ 0.0043
Feature Skew	0.7002 $\pm$ 0.0183	0.7022 $\pm$ 0.0241	0.6484 $\pm$ 0.0121
Noisy Label	0.6340 $\pm$ 0.0213	0.6548 $\pm$ 0.0410	0.6007 $\pm$ 0.0171
Label Poisoning	0.5789 $\pm$ 0.0336	0.6087 $\pm$ 0.0180	0.5553 $\pm$ 0.0217
Corrupted Client	0.5626 $\pm$ 0.0149	0.6265 $\pm$ 0.0219	0.5144 $\pm$ 0.0172
IID	0.7151 $\pm$ 0.0211	0.7208 $\pm$ 0.0200	0.6751 $\pm$ 0.0254

Table 3: Summary of FL strategy selection performance across prompt formats during Development Phase 1.

Simulated Heterogeneity	JS Divergence Value	Diagnosis
Skewed (NASA Bearing, Binary)	0.1890	Skew
Skewed (CIFAR-10, Multi-label)	0.4341	Skew
IID (NASA Bearing, Binary)	0.0001	No Skew
IID (CIFAR-10, Multi-label)	0.0002	No Skew

Table 4: Intrinsic evaluation of label skew detection using JS divergence during Development Phase 2 .

Figure 5: Client weight divergence pattern by FL training rounds in outlier setting.

Figure 6: PCA centroid pariwise distances on simulated NASA Bearing datasets.

Figure 7: PCA centroid analysis on CIFAR-10 (4 nodes) dataset.

Figure 8: LLM multi-shot performance over 8 trials on five simulated datasets.

Simulated Heterogeneity	LLM Recommended Strategy	FedAvg
Label Skew	0.6360 $\pm$ 0.0058	0.6186 $\pm$ 0.0168
Label Poisoning	0.6087 $\pm$ 0.0180	0.5543 $\pm$ 0.0258
Noisy Label	0.6548 $\pm$ 0.0410	0.6393 $\pm$ 0.0500
Feature Skew	0.7022 $\pm$ 0.0241	0.6806 $\pm$ 0.0347
Corrupted Client	0.6265 $\pm$ 0.0219	0.5948 $\pm$ 0.0197
IID	0.7208 $\pm$ 0.0200	0.7253 $\pm$ 0.0261

Table 5: Downstream FL weighted accuracy (mean

\pm

std over 10 runs) comparing LLM recommendation and FedAvg.

Simulated Heterogeneity	Empirical Best	Genetic Search	Single-Shot LLM	FedAvg	Empirical Worst
Label Skew	0.6729 $\pm$ 0.0149	0.6679 $\pm$ 0.0128	0.6360 $\pm$ 0.0058	0.6186 $\pm$ 0.0168	0.6236 $\pm$ 0.0205
Label Poisoning	0.6060 $\pm$ 0.0168	0.6223 $\pm$ 0.0195	0.6087 $\pm$ 0.0180	0.5543 $\pm$ 0.0258	0.5543 $\pm$ 0.0258
Noisy Label	0.6948 $\pm$ 0.0187	0.6882 $\pm$ 0.0233	0.6548 $\pm$ 0.0410	0.6393 $\pm$ 0.0500	0.6434 $\pm$ 0.0471
Feature Skew	0.7250 $\pm$ 0.0139	0.7112 $\pm$ 0.0449	0.7022 $\pm$ 0.0241	0.6806 $\pm$ 0.0347	0.6681 $\pm$ 0.0429
Corrupted Client	0.6058 $\pm$ 0.0258	0.6120 $\pm$ 0.0198	0.6265 $\pm$ 0.0219	0.5948 $\pm$ 0.0197	0.5838 $\pm$ 0.0230

Table 6: Downstream FL weighted accuracy (mean

\pm

std over 10 runs) comparing genetic search, LLM recommendation, FedAvg, and empirical HPO bounds.

Component	Time (s)
Reference: One FL Round	1.1613
Label skew detection module	8.3163
Feature skew detection module	11.1986
Outlier detection module	54.4542
Total Time: Full heterogeneity detection process	73.9691

Table 7: Runtime of heterogeneity detection components.

Approach	Time (s)
Reference: One Naive FL Training (30 FL Rounds)	69.211
One-Trial LLM Automation	109.256
Genetic Search	251.805
Optuna (50 Trials, 30 Rounds Each)	1105.918

Table 8: Runtime analysis of different FL strategy selection approaches.

Figure 11: Performance analysis on increasing number of nodes and varying Dirichlet partitioning on CIFAR-10.

Figure 12: Heterogeneity detection runtime analysis on increasing number of nodes.

Figure 13: Performance comparison of our framework against several baselines across different datasets. Weighted accuracy is reported for all datasets, except for OpenGym CartPole which uses median average reward.

Federated Learning Aggregation

Federated learning aggregation combines client updates into shared or personalized models while balancing data heterogeneity, robustness, privacy, and communication.

See all 266 papers

A Comparative Study of Federated Learning Aggregation Strategies under Homogeneous and Heterogeneous Data Distributions
May 10, 2026Antonios Makris, Christos Dousis, Emmanouil Kritharakis +2Federated Learning Aggregation Non-IID Federated Learning
School of Electrical and Computer Engineering, National Technical University of Athens, Greece · Department of Informatics and Telematics, Harokopio University of Athens, Greece
Agentic Federated Learning: Rule-Based Client and Server Agents for Adaptive Training
Sep 28, 2026Deepthy K. Bhaskar, VP Binu, B MinimolFederated Learning Aggregation Client Selection in Federated Learning
Department of Computer Engineering, Model Engineering College, APJ Abdul Kalam Technological University, Thiruvananthapuram 695016, Kerala, India. · Department of Biomedical Engineering, Model Engineering College, APJ Abdul Kalam Technological University, Thiruvananthapuram 695016, Kerala, India.
Choose Wisely and Privately: Proactive Client Selection for Fair and Efficient Federated Learning
May 20, 2026Adda Akram Bendoukha, Heber Hwang Arcolezi, Nesrine Kaaniche +1Client Selection in Federated Learning Non-IID Federated Learning
Samovar, T´el´ecom SudParis, Institut Polytechnique de Paris, France · ´ETS Montr´eal · CEA-LIST

May 10, 2026J/K move · Enter open · S save

A Comparative Study of Federated Learning Aggregation Strategies under Homogeneous and Heterogeneous Data Distributions

Antonios Makris, Christos Dousis, Emmanouil Kritharakis, Stavros Bouras+1

School of Electrical and Computer Engineering, National Technical University of Athens, Greece · Department of Informatics and Telematics, Harokopio University of Athens, Greece

Federated Learning Aggregation Non-IID Federated Learning Federated Learning Heterogeneous Federated Learning

Open paper AskPDF arXiv

Federated Learning has emerged as a transformative paradigm for collaborative machine learning across distributed environments. However, its performance is strongly influenced by the aggregation strategy used to combine local model updates at the server, which directly affects learning performance, robustness, and system behavior. This work presents a comprehensive experimental comparison of widely used federated aggregation strategies under both homogeneous and heterogeneous data distributions. Using benchmark image classification datasets, we analyze how different aggregation mechanisms respond to varying degrees of data heterogeneity, examining their impact on centralized accuracy and loss, and system-level efficiency metrics, including aggregation, training, and communication time. The results demonstrate that aggregation strategies exhibit distinct trade-offs across datasets and data distributions, with their effectiveness varying according to dataset characteristics and operating conditions.

Abstract

Federated Learning Aggregation

Explore similar work

A Comparative Study of Federated Learning Aggregation Strategies under Homogeneous and Heterogeneous Data Distributions

Agentic Federated Learning: Rule-Based Client and Server Agents for Adaptive Training

Choose Wisely and Privately: Proactive Client Selection for Fair and Efficient Federated Learning