cs.CRAug 15, 2026

An Adaptive Gradient Clipping and Noise Injection Mechanism for Differentially Private Federated Learning

Authors: Wenjing Wei, Alla Jammine, Farid Nait-Abdesselam

Organizations: Universit´e Paris Cit´e

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Differentially private federated learning must balance privacy protection against model accuracy and training efficiency. Static gradient clipping applies a fixed threshold throughout training and across model layers, which can cause excessive clipping when the threshold is too small or unnecessarily large noise when it is too large. This paper presents DDP-SA-adaptive, an adaptive gradient clipping and noise adding mechanism for differentially private federated learning with secure aggregation. At each communication round, every client determines a separate clipping threshold for each model layer from the median of its per-sample gradient norms. The resulting layer-wise thresholds adapt to the evolving gradient distributions and calibrate the Laplace noise added before the updates are encoded and secret-shared among intermediate aggregation servers. We evaluate the proposed mechanism on a federated regression task in terms of efficiency, accuracy, privacy, convergence, clipping norm, and noise magnitude. Compared with the static DDP-SA baseline, DDP-SA-adaptive reduces the number of communication rounds by 6.81%, total training time by 19.21%, and average per-round training time by 13.33%, leading to improved training efficiency. It also reduces test loss by 98.74% and increases test R2 by 3.41%, leading to improved model accuracy. To attain R2 = 0.99, the adaptive mechanism operates with a privacy budget of approximately epsilon = 0.1, compared with epsilon = 0.4 for static DDP-SA, thus providing stronger privacy protection and achieving stronger privacy guarantees. These results demonstrate that round-wise, layer-wise adaptation can improve the privacy-accuracy-efficiency trade-off of differentially private federated learning.

Figures & tables

MetricNo-PrivateDDP-SADDP-SA-adaptive
Communication Rounds208224362270
Parameters Uploaded399
Total Time (min)112203164
Avg Time (s)6.4610.008.67
TABLE I: Efficiency Comparison under Different Defensive Mechanisms
MetricNo-PrivateDDP-SADDP-SA-adaptive
Test Loss10−1210^{-12}0.00556.9354×10−56.9354\times 10^{-5}
Test R2\mathrm{R}^{2}0.99990.96660.9996
TABLE II: Accuracy Comparison under Different Defensive Mechanisms
Fig. 1: Test R2\mathrm{R}^{2} versus privacy budget ϵ\epsilon . The red dashed line marks R2=0.99\mathrm{R}^{2}=0.99 .
Fig. 2: Train loss under different number of communication rounds.
Fig. 3: Clipping norm of the gradients of the Bob’s weights vs number of communication rounds.
Fig. 4: Noise magnitude added to the gradient of the weight w1w_{1} of Bob versus communication rounds. Shaded curves show raw noise; navy curves show EMA-smoothed trends.
Fig. 5: Noise magnitude added to the gradient of the weight w2w_{2} of Bob versus communication rounds. Shaded curves show raw noise; navy curves show EMA-smoothed trends.
More in this topic

Privacy-Preserving Federated Learning

See all 38 papers

Explore similar work

CardsList
Feb 6, 2026

An Adaptive Differentially Private Federated Learning Framework

Federated learning enables collaborative model training across distributed clients while preserving data privacy. However, in practical deployments, device heterogeneity and non-independent and identically distributed (Non-IID) data often lead to unstable and biased gradient. When differential privacy is enforced, conventional fixed gradient clipping and Gaussian noise injection may further amplify gradient perturbations, resulting in training oscillation and degraded model performance. To address these challenges, we propose an adaptive differentially private federated learning framework that explicitly targets model efficiency under heterogeneous and privacy-constrained settings. On the client side, a lightweight local dimensionality reduction module is introduced to learn reduced-dimensional intermediate representations and produce more structured gradients during backpropagation, thereby mitigating noise amplification during local optimization. On the server side, an adaptive gradient clipping strategy dynamically adjusts clipping thresholds based on historical update statistics to avoid over-clipping and noise domination. Furthermore, a constraint-aware robust aggregation mechanism is designed to suppress unreliable or noise-dominated client updates and stabilize global optimization. Extensive experiments on CIFAR-10, SVHN, and STL-10 demonstrate that the proposed method consistently improves convergence stability and classification performance under differential privacy.

Jin Wang, Hui Ma, Yajun Zhang +4

Xinjiang Key Laboratory of Intelligent Computing and Smart Applications, School of Software, Xinjiang University, Urumqi 830017, China · College of Geography and Remote Sensing Sciences, Xinjiang University, Urumqi, China · Jilin University, Changchun 130012, China

Non-IID Federated LearningDifferentially Private Federated LearningFederated LearningHeterogeneous Federated Learning
AskPDF
May 11, 2026

DP-LAC: Lightweight Adaptive Clipping for Differentially Private Federated Fine-tuning of Language Models

Federated learning (FL) enables the collaborative training of large-scale language models (LLMs) across edge devices while keeping user data on-device. However, FL still exposes sensitive information through client-provided gradients. Differentially private stochastic gradient descent (DP-SGD) mitigates this risk by clipping each client's contribution to a threshold CC and adding noise proportional to CC. Existing adaptive clipping techniques dynamically adjust CC but demand tedious hyperparameter tuning, which can erode the privacy budget. In this paper, we introduce DP-LAC, a method that first estimates an initial clipping threshold within an order of magnitude of the optimum using private histogram estimation, and then adapts this threshold during training without consuming additional privacy budget or introducing new hyperparameters. Empirical results show that DP-LAC outperforms both state-of-the-art adaptive clipping methods and vanilla DP-SGD, achieving an average accuracy gain of 6.6%6.6\%.

Haaris Mehmood, Jie Xu, Karthikeyan Saravanan +2

Samsung R&D Institute UK (SRUK) · Samsung AI Centre Cambridge

Privacy-Preserving Language ModelsDifferentially Private Stochastic Gradient DescentLLM Fine-TuningDifferentially Private Federated LearningFederated Fine-TuningFederated Learning
AskPDF
Apr 8, 2026

Scalable and Private Federated Learning Using Distributed Differential Privacy and Secure Aggregation

This article presents DDP-SA, a scalable privacy-preserving federated learning framework that jointly leverages client-side local differential privacy (LDP) and full-threshold additive secret sharing (ASS) for secure aggregation. Unlike existing methods that rely solely on differential privacy or on secure multi-party computation (MPC), DDP-SA integrates both techniques to deliver stronger end-to-end privacy guarantees while remaining computationally practical. The framework introduces a two-stage protection mechanism: clients first perturb their local gradients with calibrated Laplace noise, then decompose the noisy gradients into additive secret shares that are distributed across multiple intermediate servers. This design ensures that (i) no single compromised server or communication channel can reveal any information about individual client updates, and (ii) the parameter server reconstructs only the aggregated noisy gradient, never any client-specific contribution. Extensive experiments show that DDP-SA achieves substantially higher model accuracy than standalone LDP while providing stronger privacy protection than MPC-only approaches. The proposed framework scales linearly with the number of participants and offers a practical, privacy-preserving solution for federated learning applications with controllable computational and communication overhead.

Wenjing Wei, Farid Nait-Abdesselam, Alla Jammine

Université Paris Cité, Paris, France

Privacy-Preserving Federated LearningDifferentially Private Federated LearningFederated Learning AggregationDifferential Privacy
AskPDF