FedZMG: Efficient Client-Side Optimization in Federated Learning

Fotios Zantalis; Evangelos Zervas; Grigorios Koulouras

cs.LGFeb 20, 2026

FedZMG: Efficient Client-Side Optimization in Federated Learning

Authors: Fotios Zantalis, Evangelos Zervas, Grigorios Koulouras

Organizations: TelSiP Research Laboratory, Department of Electrical and Electronic Engineering, School of Engineering, University of West Attica, Ancient Olive Grove Campus, 250 Thivon Str., GR-12241 Athens, Greece

Abstract

Federated Learning (FL) enables distributed model training on edge devices while preserving data privacy. However, clients tend to have non-Independent and Identically Distributed (non-IID) data, which often leads to client-drift, and therefore diminishing convergence speed and model performance. While adaptive optimizers have been proposed to mitigate these effects, they frequently introduce computational complexity or communication overhead unsuitable for resource-constrained IoT environments. This paper introduces Federated Zero Mean Gradients (FedZMG), a novel, parameter-free, client-side optimization algorithm designed to tackle client-drift by structurally regularizing the optimization space. Advancing the idea of Gradient Centralization, FedZMG projects local gradients onto a zero-mean hyperplane, effectively neutralizing the "intensity" or "bias" shifts inherent in heterogeneous data distributions without requiring additional communication or hyperparameter tuning. A theoretical analysis is provided, proving that FedZMG reduces the effective gradient variance and guarantees tighter convergence bounds compared to standard FedAvg. Extensive empirical evaluations on EMNIST, CIFAR100, and Shakespeare datasets demonstrate that FedZMG achieves better convergence speed and final validation accuracy compared to the baseline FedAvg and the adaptive optimizer FedAdam, particularly in highly non-IID settings.

Figures & tables

Dataset	Volume	Label Diversity	Entropy	Gini	KL Div.
EMNIST	$236\pm 92$	$55\pm 4$	$0.84\pm 0.03$	$0.58\pm 0.03$	$0.24\pm 0.09$
CIFAR-100	$100\pm 0$	$21\pm 5$	$0.56\pm 0.07$	$0.89\pm 0.04$	$1.57\pm 0.28$
Shakespeare	$3267\pm 4030$	$49\pm 12$	$0.80\pm 0.04$	—	—

Table 1: Summary of Non-IID Metrics (Mean

\pm

Std Dev). CIFAR100 shows the highest deviation from the global distribution (KL Div), while Shakespeare has the largest vocabulary.

Client ID	Volume	Label Diversity	Entropy	KL Div.	Dom. Class. %
EMNIST (High Volume Variance)
f0698_45	374	60	0.87	0.13	10.2%
f0952_49	328	58	0.88	0.31	6.7%
f2295_67	131	49	0.82	0.30	9.9%
CIFAR100 (High Class Imbalance)
Client 54	100	29	0.68	0.98	10.0%

Table 2: Detailed metrics for representative clients. “Dom. Class %” indicates the percentage of local data belonging to the most frequent class/character.

Figure 1: Client–server ( $\eta_{c}-\eta_{s}$ ) learning rate tuning via grid search across three datasets: CIFAR100 (top), Shakespeare (middle), and EMNIST (bottom).

Panel of Figure 2 — Figure 2: Validation accuracy comparison. Top: CIFAR100; Middle: Shakespeare; Bottom: EMNIST.

Algorithm	Rounds to 20.0%	Rounds to 33.0%	Avg Acc Post-33.0%
FedZMG	95	250	39.50%
FedAvg	155	805	34.69%
FedAdam	135	420	37.68%

Table 3: Metrics comparison on CIFAR100 dataset.

Algorithm	Rounds to 20.0%	Rounds to 33.0%	Avg Acc Post-33.0%
FedZMG	15	15	55.79%
FedAvg	15	55	51.92%
FedAdam	30	45	56.62%

Table 4: Metrics comparison on Shakespeare dataset.

Algorithm	Rounds to 20.0%	Rounds to 33.0%	Avg Acc Post-33.0%
FedZMG	15	20	98.91%
FedAvg	15	20	98.43%
FedAdam	35	40	97.96%

Table 5: Metrics comparison on EMNIST dataset.

Comparison	t -Statistic	p -Value
FedZMG vs. FedAdam	2.043	$1.105\times 10^{-1}$
FedZMG vs. FedAvg	4.025	$1.579\times 10^{-2}$

Table 6: Paired

t

-test results on EMNIST dataset.

Comparison	t -Statistic	p -Value
FedZMG vs. FedAdam	1.452	$2.201\times 10^{-1}$
FedZMG vs. FedAvg	3.854	$1.824\times 10^{-2}$

Table 7: Paired

t

-test results on Shakespeare dataset.

Comparison	t -Statistic	p -Value
FedZMG vs. FedAdam	7.804	$1.455\times 10^{-3}$
FedZMG vs. FedAvg	20.716	$3.208\times 10^{-5}$

Table 8: Paired

t

-test results on CIFAR100 dataset.

Federated Learning

Federated learning trains shared models across data holders without requiring them to pool their raw data.

See all 921 papers

SpecGradFilter: A Spectral Gradient Filtering Framework for Taming Federated Heterogeneity
Jul 5, 2026Liyang Yuan, Yibo Yang, Dandan Guo +2Non-IID Federated Learning Federated Learning
School of Artificial Intelligence, Jilin University, Changchun 130012, China · King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia · School of Intelligence Science and Technology, Peking University, China
FedVG: Gradient-Guided Aggregation for Enhanced Federated Learning
Feb 24, 2026Alina Devkota, Jacob Thrasher, Donald Adjeroh +2Federated Learning Federated Learning Aggregation
West Virginia University · University of Aberdeen · Fogsphere (Redev.AI)
DriftGuard: Mitigating Asynchronous Data Drift in Federated Learning
Mar 19, 2026Yizhou Han, Di Wu, Blesson VargheseFederated Learning Federated Continual Learning
School of Computer Science, University of St Andrews, UK

Jul 5, 2026J/K move · Enter open · S save

SpecGradFilter: A Spectral Gradient Filtering Framework for Taming Federated Heterogeneity

Liyang Yuan, Yibo Yang, Dandan Guo, Peter Richtarik+1

School of Artificial Intelligence, Jilin University, Changchun 130012, China · King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia · School of Intelligence Science and Technology, Peking University, China

Non-IID Federated Learning Federated Learning Heterogeneous Federated Learning

Open paper AskPDF arXiv

Federated Learning (FL) is fundamentally challenged by statistical heterogeneity, where non-identically distributed (non-IID) data induces client drift that severely hampers global convergence. While existing approaches attempt to mitigate this drift through spatial-domain gradient correction or regularization, they overlook the intrinsic spectral structure of optimization signals. In this work, we revisit client drift from a novel frequency-domain perspective and uncover a critical Spectral Bias of Drift: inter-client gradient divergence is predominantly concentrated in low-frequency components which encode client-specific distributional shifts, while high-frequency components representing fine-grained features remain relatively consistent. Motivated by this, we propose SpecGradFilter, a unified Spectral Gradient Filtering Framework that tames heterogeneity by suppressing discordant low-frequency signals. Crucially, we demonstrate that SpecGradFilter is a generalizable principle, effective not only via precise FFT-based truncation but also through spatial approximations like Gaussian detrending. Extensive experiments on benchmarks such as CIFAR-10/100 and Tiny-ImageNet demonstrate that SpecGradFilter significantly performs better performance in highly Non-IID settings with negligible communication overhead, establishing a new paradigm for robust federated optimization.

Abstract

Federated Learning

Explore similar work

SpecGradFilter: A Spectral Gradient Filtering Framework for Taming Federated Heterogeneity

FedVG: Gradient-Guided Aggregation for Enhanced Federated Learning

DriftGuard: Mitigating Asynchronous Data Drift in Federated Learning