cs.LGFeb 24, 2026

FedVG: Gradient-Guided Aggregation for Enhanced Federated Learning

Authors: Alina Devkota, Jacob Thrasher, Donald Adjeroh, Binod Bhattarai, Prashnna K. Gyawali

Organizations: West Virginia University · University of Aberdeen · Fogsphere (Redev.AI)

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Federated Learning (FL) enables collaborative model training across multiple clients without sharing their private data. However, data heterogeneity across clients leads to client drift, which degrades the overall generalization performance of the model. This effect is further compounded by overemphasis on poorly performing clients. To address this problem, we propose FedVG, a novel gradient-based federated aggregation framework that leverages a global validation set to guide the optimization process. Such a global validation set can be established using readily available public datasets, ensuring accessibility and consistency across clients without compromising privacy. In contrast to conventional approaches that prioritize client dataset volume, FedVG assesses the generalization ability of client models by measuring the magnitude of validation gradients across layers. Specifically, we compute layerwise gradient norms to derive a client-specific score that reflects how much each client needs to adjust for improved generalization on the global validation set, thereby enabling more informed and adaptive federated aggregation. Extensive experiments on both natural and medical image benchmarking datasets, across diverse model architectures, demonstrate that FedVG consistently improves performance, particularly in highly heterogeneous settings. Moreover, FedVG is modular and can be seamlessly integrated with various state-of-the-art FL algorithms, often further improving their results. Our code is available at https://github.com/alinadevkota/FedVG.

Figures & tables

Figure 1: Comparison of federated learning methods under high data heterogeneity ( α=0.05\alpha=0.05 ). FedVG consistently achieves the highest or near-highest accuracy across all settings.
Figure 2: FedVG Framework . Locally trained client models, which converge to varying points, are sent to the server. A global validation set is used to compute their validation gradients, obtain client scores sks_{k} , and form a global model by weighted sum. Individual client updates are represented as dotted lines in the loss landscape on the right, while aggregated models are solid.
Figure 3: FL algorithm performance as α→0\alpha\rightarrow{}0 . Shaded areas indicate standard deviations. Additional results for TinyImageNet and DermaMNIST datasets on the ResNet-50 model are provided in the Appendix.
Figure 4: FL algorithm performance on ViT models as α→0\alpha\rightarrow{}0 . Shaded areas show standard deviations.
Figure 5: FedVG integration performance across CIFAR-10, OrganAMNIST, and COVID19 as α→0\alpha\rightarrow{}0 . Dashed lines denote the base methods, and solid lines their FedVG-enhanced counterparts. Shaded gray areas show the standard deviation of base methods, while colored regions reflect the variability of enhanced methods.
Figure 6: Performance of baselines and FedVG (hatched) using an external global validation set at α=0.1\alpha=0.1 and 0.050.05 .
Figure 7: FedVG performance across validation class imbalance ratios, with lower ρ\rho indicating greater class imbalance.
Figure 8: Client weights for varying norms during weight calculation, where the bold line corresponds to the balanced client.
Figure 9: Comparison of FedVG aggregation strategies at α=0.05\alpha=0.05 . Each bar shows the mean accuracy ± std over 5 runs.
Figure A.1 : Heatmaps of per-client gradient norms across network layers for two federated learning rounds. Only a subset of layers is presented for clarity.
 CIFAR-10OrganAMNISTTiny ImageNetCOVID19DermaMNIST
Number of Clients1001001002025
Join Ratio0.10.10.10.250.2
Input Channels31333
Image Size32 ×\times 3228 ×\times 2864 ×\times 64244 ×\times 224224 ×\times 224
Batch Size32321283232
OptimizerSGDSGDSGDSGDSGD
Table A.1 : Hyperparameter configurations used for different datasets in the federated learning experiments.
BaselineHyperparameterValue
FedAvgMserver momentum0.9
FedProxμ\mu0.01
Scaffoldglobal lr1.0
FedDynα\alpha0.1
max_grad_norm10
Elasticsample_ratio0.3
Table A.2 : Baseline-specific hyperparameters used in our experiments.
Figure A.2 : CIFAR-10 class distributions at different heterogeneity levels, parameterized by α\alpha in the Dirichlet distribution. Lower α\alpha indicates greater heterogeneity. Each row corresponds to a client, and colors represent different classes.
Figure A.3 : COVID19 class distributions at different heterogeneity levels, parameterized by α\alpha in the Dirichlet distribution. Lower α\alpha indicates greater heterogeneity. Each row corresponds to a client, and colors represent different classes.
Figure A.4 : FL algorithm performance for TinyImageNet and DermaMNIST datasets on the ResNet-50 model as α→0\alpha\rightarrow{}0 . Shaded areas indicate standard deviations.
Methodsα=100\alpha=100α=10\alpha=10α=1\alpha=1α=0.1\alpha=0.1α=0.05\alpha=0.05
CIFAR-10 / ResNet-18
FedAvg80.37 ± 0.8977.95 ± 0.4376.26 ± 1.6557.93 ± 1.8248.83 ± 5.53
FedAvgM71.09 ± 5.1656.64 ± 10.7551.87 ± 11.4034.18 ± 4.3825.87 ± 7.04
FedProx79.92 ± 1.5077.39 ± 1.3276.73 ± 0.6355.22 ± 5.5649.19 ± 6.55
Scaffold81.05 ± 0.1680.39 ± 0.1579.25 ± 0.2356.15 ± 2.6541.94 ± 1.91
FedDyn66.37 ± 0.3365.90 ± 0.3662.37 ± 0.2941.66 ± 1.2239.26 ± 1.99
Table A.3 : Test accuracy (%) ± standard deviation of various federated learning methods on CIFAR-10, OrganAMNIST, Tiny ImageNet, COVID19, and DermaMNIST datasets across different data heterogeneity levels (Dirichlet α\alpha ). Bold values indicate best accuracies and underlined values indicate second best accuracies.
Methodsα=100\alpha=100α=10\alpha=10α=1\alpha=1α=0.1\alpha=0.1α=0.05\alpha=0.05
COVID19 / ViT-S/16
FedAvg88.38 ± 0.7389.07 ± 0.4989.63 ± 0.4787.46 ± 0.7282.30 ± 3.63
FedAvgM88.01 ± 0.7588.77 ± 0.5089.14 ± 0.3880.06 ± 8.5851.66 ± 8.36
FedProx88.40 ± 0.5789.69 ± 0.1289.32 ± 0.5287.98 ± 1.0681.27 ± 3.57
Scaffold84.41 ± 4.0284.80 ± 4.7885.75 ± 4.3368.33 ± 15.3575.41 ± 9.81
FedDyn77.61 ± 1.6178.16 ± 1.6476.72 ± 1.5051.09 ± 8.6342.42 ± 4.94
Table A.4 : Test accuracy (%) ± standard deviation of federated learning methods on COVID19 with ViT-S/16 and DermaMNIST with ViT-B/16 models across different data heterogeneity levels (Dirichlet α\alpha ). Bold values indicate best accuracies and underlined values indicate second best accuracies.
Figure A.5 : Performance comparison of federated learning methods across datasets at high heterogeneity ( α\alpha = 0.05). The error bars denote standard deviation across five runs. At high heterogeneity, FedVG maintains both strong average performance and low variance compared to other methods.
CIFAR-10STL-10Count
airplaneairplane550
automobilecar550
birdbird550
catcat550
deerdeer550
dogdog550
Table A.5 : Class mapping from STL-10 to CIFAR-10
CIFAR-10CIFAR-100Count
airplane——
automobilebus, motorcycle531, 519
bird——
catleopard, lion, tiger520, 514, 516
deer——
dogwolf, fox520, 530
Table A.6 : Class mapping from CIFAR-100 to CIFAR-10
Methodα=100\alpha=100α=10\alpha=10α=1\alpha=1α=0.1\alpha=0.1α=0.05\alpha=0.05
CIFAR-10 / ResNet-18
FedAvg80.37 ± 0.8977.95 ± 0.4376.26 ± 1.6557.93 ± 1.8248.83 ± 5.53
FedVG80.74 ± 0.7979.65 ± 0.5275.69 ± 2.2761.06 ± 0.3453.58 ± 2.78
FedAvg + FedVG80.92 ± 0.7978.79 ± 0.5674.96 ± 2.5560.60 ± 2.2652.49 ± 3.83
FedAvgM71.09 ± 5.1656.64 ± 10.7551.87 ± 11.4034.18 ± 4.3825.87 ± 7.04
FedAvgM + FedVG77.87 ± 1.9863.05 ± 12.5539.51 ± 17.0242.63 ± 7.2533.25 ± 3.62
Table A.7 : Performance comparison of federated learning methods and their integration with FedVG on CIFAR-10, OrganAMNIST, and COVID-19 datasets using ResNet-18 and ResNet-50 models across data heterogeneity levels ( α\alpha ). Bold values indicate best accuracies and underlined values indicate second best accuracies.
Figure A.6 : Visualization of class imbalance of DvalD_{val} at various values of ρ\rho .
Figure A.7 : Visualization of sample distributions used in the norm-type ablation study. Here, client one is configured to be perfectly balanced.
Methodα=100\alpha=100α=10\alpha=10α=1\alpha=1α=0.1\alpha=0.1α=0.05\alpha=0.05
CIFAR-10 / ResNet-18
FedVG (Modelwise)80.74 ± 0.7979.65 ± 0.5275.69 ± 2.2761.06 ± 0.3453.58 ± 2.78
FedVG (Layerwise)77.87 ± 2.7973.33 ± 5.3074.03 ± 2.4159.43 ± 2.2450.25 ± 1.80
FedVG (Blockwise)79.33 ± 1.8571.15 ± 8.7971.85 ± 2.9658.06 ± 3.8750.30 ± 2.95
OrganAMNIST / ResNet-18
FedVG (Modelwise)99.41 ± 0.0899.42 ± 0.0299.12 ± 0.1094.72 ± 0.687.57 ± 1.91
Table A.8 : Comparison of FedVG variants (Modelwise, Layerwise, and Blockwise) across datasets, models, and heterogeneity levels ( α\alpha values). Bold values indicate best accuracies.
More in this topic

Federated Learning

Federated learning trains shared models across data holders without requiring them to pool their raw data.

See all 921 papers

Explore similar work

CardsList
Feb 26, 2026

FedDAG: Clustered Federated Learning via Global Data and Gradient Integration for Heterogeneous Environments

Federated Learning (FL) enables a group of clients to collaboratively train a model without sharing individual data, but its performance drops when client data are heterogeneous. Clustered FL tackles this by grouping similar clients. However, existing clustered FL approaches rely solely on either data similarity or gradient similarity; however, this results in an incomplete assessment of client similarities. Prior clustered FL approaches also restrict knowledge and representation sharing to clients within the same cluster. This prevents cluster models from benefiting from the diverse client population across clusters. To address these limitations, FedDAG introduces a clustered FL framework, FedDAG, that employs a weighted, class-wise similarity metric that integrates both data and gradient information, providing a more holistic measure of similarity during clustering. In addition, FedDAG adopts a dual-encoder architecture for cluster models, comprising a primary encoder trained on its own clients' data and a secondary encoder refined using gradients from complementary clusters. This enables cross-cluster feature transfer while preserving cluster-specific specialization. Experiments on diverse benchmarks and data heterogeneity settings show that FedDAG consistently outperforms state-of-the-art clustered FL baselines in accuracy.

Anik Pramanik, Murat Kantarcioglu, Vincent Oria +1

New Jersey Institute of Technology, USA · Virginia Tech, USA

Clustered Federated LearningFederated LearningHeterogeneous Federated LearningNon-IID Federated Learning
AskPDF
May 26, 2026

Separate Aggregation of Split Network for Personalized Federated Learning

Federated learning enables collaborative model training without sharing raw data, but its performance can degrade substantially under heterogeneous client data distributions. A single global model often cannot satisfy diverse client requirements, so personalized federated learning has therefore been explored to improve client specific performance while preserving global generalization. Existing PFL methods often face a fundamental tradeoff in which stronger global sharing can undermine local specialization, whereas stronger local adaptation can lead to overfitting under limited data, label imbalance, and missing class scenarios. In this work, we propose PGFedSplit, a personalized federated learning framework that improves both personalization and global generalization under severe client heterogeneity. PGFedSplit adopts a split architecture and performs adaptive aggregation scheduling tailored to the roles of different model components, enabling stable knowledge sharing while maintaining client specific adaptation. Each client further leverages a mixture of locally extracted representations and synthetic representations generated from server side Gaussian statistics, improving robustness under label imbalance and missing class conditions. Extensive experiments on Fashion MNIST, CIFAR 10, CIFAR 100, and Tiny ImageNet demonstrate consistent improvements over state of the art PFL methods, with stable convergence and superior personalization in highly heterogeneous settings.

Yunseok Kang, Jaeyoung Song

Department of Electronics Engineering, Pusan National University, Republic of Korea

Federated Learning AggregationSplit Federated LearningFederated LearningHeterogeneous Federated Learning
AskPDF
Feb 20, 2026

FedZMG: Efficient Client-Side Optimization in Federated Learning

Federated Learning (FL) enables distributed model training on edge devices while preserving data privacy. However, clients tend to have non-Independent and Identically Distributed (non-IID) data, which often leads to client-drift, and therefore diminishing convergence speed and model performance. While adaptive optimizers have been proposed to mitigate these effects, they frequently introduce computational complexity or communication overhead unsuitable for resource-constrained IoT environments. This paper introduces Federated Zero Mean Gradients (FedZMG), a novel, parameter-free, client-side optimization algorithm designed to tackle client-drift by structurally regularizing the optimization space. Advancing the idea of Gradient Centralization, FedZMG projects local gradients onto a zero-mean hyperplane, effectively neutralizing the "intensity" or "bias" shifts inherent in heterogeneous data distributions without requiring additional communication or hyperparameter tuning. A theoretical analysis is provided, proving that FedZMG reduces the effective gradient variance and guarantees tighter convergence bounds compared to standard FedAvg. Extensive empirical evaluations on EMNIST, CIFAR100, and Shakespeare datasets demonstrate that FedZMG achieves better convergence speed and final validation accuracy compared to the baseline FedAvg and the adaptive optimizer FedAdam, particularly in highly non-IID settings.

Fotios Zantalis, Evangelos Zervas, Grigorios Koulouras

TelSiP Research Laboratory, Department of Electrical and Electronic Engineering, School of Engineering, University of West Attica, Ancient Olive Grove Campus, 250 Thivon Str., GR-12241 Athens, Greece

Federated LearningNon-IID Federated Learning
AskPDF