cs.LGFeb 24, 2026

FedVG: Gradient-Guided Aggregation for Enhanced Federated Learning

Authors: Alina Devkota, Jacob Thrasher, Donald Adjeroh, Binod Bhattarai, Prashnna K. Gyawali

Organizations: West Virginia University · University of Aberdeen · Fogsphere (Redev.AI)

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Federated Learning (FL) enables collaborative model training across multiple clients without sharing their private data. However, data heterogeneity across clients leads to client drift, which degrades the overall generalization performance of the model. This effect is further compounded by overemphasis on poorly performing clients. To address this problem, we propose FedVG, a novel gradient-based federated aggregation framework that leverages a global validation set to guide the optimization process. Such a global validation set can be established using readily available public datasets, ensuring accessibility and consistency across clients without compromising privacy. In contrast to conventional approaches that prioritize client dataset volume, FedVG assesses the generalization ability of client models by measuring the magnitude of validation gradients across layers. Specifically, we compute layerwise gradient norms to derive a client-specific score that reflects how much each client needs to adjust for improved generalization on the global validation set, thereby enabling more informed and adaptive federated aggregation. Extensive experiments on both natural and medical image benchmarking datasets, across diverse model architectures, demonstrate that FedVG consistently improves performance, particularly in highly heterogeneous settings. Moreover, FedVG is modular and can be seamlessly integrated with various state-of-the-art FL algorithms, often further improving their results. Our code is available at https://github.com/alinadevkota/FedVG.

Figures & tables

Figure 1: Comparison of federated learning methods under high data heterogeneity ( α=0.05\alpha=0.05 ). FedVG consistently achieves the highest or near-highest accuracy across all settings.
Figure 2: FedVG Framework . Locally trained client models, which converge to varying points, are sent to the server. A global validation set is used to compute their validation gradients, obtain client scores sks_{k} , and form a global model by weighted sum. Individual client updates are represented as dotted lines in the loss landscape on the right, while aggregated models are solid.
Figure 3: FL algorithm performance as α→0\alpha\rightarrow{}0 . Shaded areas indicate standard deviations. Additional results for TinyImageNet and DermaMNIST datasets on the ResNet-50 model are provided in the Appendix.
Figure 4: FL algorithm performance on ViT models as α→0\alpha\rightarrow{}0 . Shaded areas show standard deviations.
Figure 5: FedVG integration performance across CIFAR-10, OrganAMNIST, and COVID19 as α→0\alpha\rightarrow{}0 . Dashed lines denote the base methods, and solid lines their FedVG-enhanced counterparts. Shaded gray areas show the standard deviation of base methods, while colored regions reflect the variability of enhanced methods.
Figure 6: Performance of baselines and FedVG (hatched) using an external global validation set at α=0.1\alpha=0.1 and 0.050.05 .
Figure 7: FedVG performance across validation class imbalance ratios, with lower ρ\rho indicating greater class imbalance.
Figure 8: Client weights for varying norms during weight calculation, where the bold line corresponds to the balanced client.
Figure 9: Comparison of FedVG aggregation strategies at α=0.05\alpha=0.05 . Each bar shows the mean accuracy ± std over 5 runs.
Figure A.1 : Heatmaps of per-client gradient norms across network layers for two federated learning rounds. Only a subset of layers is presented for clarity.
 CIFAR-10OrganAMNISTTiny ImageNetCOVID19DermaMNIST
Number of Clients1001001002025
Join Ratio0.10.10.10.250.2
Input Channels31333
Image Size32 ×\times 3228 ×\times 2864 ×\times 64244 ×\times 224224 ×\times 224
Batch Size32321283232
OptimizerSGDSGDSGDSGDSGD
Table A.1 : Hyperparameter configurations used for different datasets in the federated learning experiments.
BaselineHyperparameterValue
FedAvgMserver momentum0.9
FedProxμ\mu0.01
Scaffoldglobal lr1.0
FedDynα\alpha0.1
max_grad_norm10
Elasticsample_ratio0.3
Table A.2 : Baseline-specific hyperparameters used in our experiments.
Figure A.2 : CIFAR-10 class distributions at different heterogeneity levels, parameterized by α\alpha in the Dirichlet distribution. Lower α\alpha indicates greater heterogeneity. Each row corresponds to a client, and colors represent different classes.
Figure A.3 : COVID19 class distributions at different heterogeneity levels, parameterized by α\alpha in the Dirichlet distribution. Lower α\alpha indicates greater heterogeneity. Each row corresponds to a client, and colors represent different classes.
Figure A.4 : FL algorithm performance for TinyImageNet and DermaMNIST datasets on the ResNet-50 model as α→0\alpha\rightarrow{}0 . Shaded areas indicate standard deviations.
Methodsα=100\alpha=100α=10\alpha=10α=1\alpha=1α=0.1\alpha=0.1α=0.05\alpha=0.05
CIFAR-10 / ResNet-18
FedAvg80.37 ± 0.8977.95 ± 0.4376.26 ± 1.6557.93 ± 1.8248.83 ± 5.53
FedAvgM71.09 ± 5.1656.64 ± 10.7551.87 ± 11.4034.18 ± 4.3825.87 ± 7.04
FedProx79.92 ± 1.5077.39 ± 1.3276.73 ± 0.6355.22 ± 5.5649.19 ± 6.55
Scaffold81.05 ± 0.1680.39 ± 0.1579.25 ± 0.2356.15 ± 2.6541.94 ± 1.91
FedDyn66.37 ± 0.3365.90 ± 0.3662.37 ± 0.2941.66 ± 1.2239.26 ± 1.99
Table A.3 : Test accuracy (%) ± standard deviation of various federated learning methods on CIFAR-10, OrganAMNIST, Tiny ImageNet, COVID19, and DermaMNIST datasets across different data heterogeneity levels (Dirichlet α\alpha ). Bold values indicate best accuracies and underlined values indicate second best accuracies.
Methodsα=100\alpha=100α=10\alpha=10α=1\alpha=1α=0.1\alpha=0.1α=0.05\alpha=0.05
COVID19 / ViT-S/16
FedAvg88.38 ± 0.7389.07 ± 0.4989.63 ± 0.4787.46 ± 0.7282.30 ± 3.63
FedAvgM88.01 ± 0.7588.77 ± 0.5089.14 ± 0.3880.06 ± 8.5851.66 ± 8.36
FedProx88.40 ± 0.5789.69 ± 0.1289.32 ± 0.5287.98 ± 1.0681.27 ± 3.57
Scaffold84.41 ± 4.0284.80 ± 4.7885.75 ± 4.3368.33 ± 15.3575.41 ± 9.81
FedDyn77.61 ± 1.6178.16 ± 1.6476.72 ± 1.5051.09 ± 8.6342.42 ± 4.94
Table A.4 : Test accuracy (%) ± standard deviation of federated learning methods on COVID19 with ViT-S/16 and DermaMNIST with ViT-B/16 models across different data heterogeneity levels (Dirichlet α\alpha ). Bold values indicate best accuracies and underlined values indicate second best accuracies.
Figure A.5 : Performance comparison of federated learning methods across datasets at high heterogeneity ( α\alpha = 0.05). The error bars denote standard deviation across five runs. At high heterogeneity, FedVG maintains both strong average performance and low variance compared to other methods.
CIFAR-10STL-10Count
airplaneairplane550
automobilecar550
birdbird550
catcat550
deerdeer550
dogdog550
Table A.5 : Class mapping from STL-10 to CIFAR-10
CIFAR-10CIFAR-100Count
airplane——
automobilebus, motorcycle531, 519
bird——
catleopard, lion, tiger520, 514, 516
deer——
dogwolf, fox520, 530
Table A.6 : Class mapping from CIFAR-100 to CIFAR-10
Methodα=100\alpha=100α=10\alpha=10α=1\alpha=1α=0.1\alpha=0.1α=0.05\alpha=0.05
CIFAR-10 / ResNet-18
FedAvg80.37 ± 0.8977.95 ± 0.4376.26 ± 1.6557.93 ± 1.8248.83 ± 5.53
FedVG80.74 ± 0.7979.65 ± 0.5275.69 ± 2.2761.06 ± 0.3453.58 ± 2.78
FedAvg + FedVG80.92 ± 0.7978.79 ± 0.5674.96 ± 2.5560.60 ± 2.2652.49 ± 3.83
FedAvgM71.09 ± 5.1656.64 ± 10.7551.87 ± 11.4034.18 ± 4.3825.87 ± 7.04
FedAvgM + FedVG77.87 ± 1.9863.05 ± 12.5539.51 ± 17.0242.63 ± 7.2533.25 ± 3.62
Table A.7 : Performance comparison of federated learning methods and their integration with FedVG on CIFAR-10, OrganAMNIST, and COVID-19 datasets using ResNet-18 and ResNet-50 models across data heterogeneity levels ( α\alpha ). Bold values indicate best accuracies and underlined values indicate second best accuracies.
Figure A.6 : Visualization of class imbalance of DvalD_{val} at various values of ρ\rho .
Figure A.7 : Visualization of sample distributions used in the norm-type ablation study. Here, client one is configured to be perfectly balanced.
Methodα=100\alpha=100α=10\alpha=10α=1\alpha=1α=0.1\alpha=0.1α=0.05\alpha=0.05
CIFAR-10 / ResNet-18
FedVG (Modelwise)80.74 ± 0.7979.65 ± 0.5275.69 ± 2.2761.06 ± 0.3453.58 ± 2.78
FedVG (Layerwise)77.87 ± 2.7973.33 ± 5.3074.03 ± 2.4159.43 ± 2.2450.25 ± 1.80
FedVG (Blockwise)79.33 ± 1.8571.15 ± 8.7971.85 ± 2.9658.06 ± 3.8750.30 ± 2.95
OrganAMNIST / ResNet-18
FedVG (Modelwise)99.41 ± 0.0899.42 ± 0.0299.12 ± 0.1094.72 ± 0.687.57 ± 1.91
Table A.8 : Comparison of FedVG variants (Modelwise, Layerwise, and Blockwise) across datasets, models, and heterogeneity levels ( α\alpha values). Bold values indicate best accuracies.
More in this topic

Federated Learning

Federated learning trains shared models across data holders without requiring them to pool their raw data.

See all 921 papers

Explore similar work

CardsList

  1. FedDAG: Clustered Federated Learning via Global Data and Gradient Integration for Heterogeneous Environments

    Feb 26, 2026Anik Pramanik, Murat Kantarcioglu, Vincent Oria +1Clustered Federated LearningFederated Learning

    New Jersey Institute of Technology, USA · Virginia Tech, USA

  2. Separate Aggregation of Split Network for Personalized Federated Learning

    May 26, 2026Yunseok Kang, Jaeyoung SongFederated Learning AggregationSplit Federated Learning

    Department of Electronics Engineering, Pusan National University, Republic of Korea

  3. FedZMG: Efficient Client-Side Optimization in Federated Learning

    Feb 20, 2026Fotios Zantalis, Evangelos Zervas, Grigorios KoulourasFederated LearningNon-IID Federated Learning

    TelSiP Research Laboratory, Department of Electrical and Electronic Engineering, School of Engineering, University of West Attica, Ancient Olive Grove Campus, 250 Thivon Str., GR-12241 Athens, Greece