Organizations: Department of Information Engineering, The Chinese University of Hong Kong, Hong Kong · School of Science and Engineering, Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen Key Laboratory of Crowd Intelligence Empowered Low-Carbon Energy Network, and CSIJRI Joint Research Centre on Smart Energy Storage, The Chinese University of Hong Kong, Shenzhen, Guangdong, 518172, P.R. China
Differentially private federated learning (DP-FL) enables clients to collaboratively train machine learning models while preserving the privacy of their local data. However, most existing DP-FL approaches assume that all clients share a uniform privacy budget, an assumption that does not hold in real-world scenarios where privacy requirements vary widely. This privacy heterogeneity poses a significant challenge: conventional client selection strategies, which typically rely on data quantity, cannot distinguish between clients providing high-quality updates and those introducing substantial noise due to strict privacy constraints. To address this gap, we present the first systematic study of privacy-aware client selection in DP-FL. We establish a theoretical foundation by deriving a convergence analysis that quantifies the impact of privacy heterogeneity on training error. Building on this analysis, we propose a privacy-aware client selection strategy, formulated as a convex optimization problem, that adaptively adjusts selection probabilities to minimize training error. Extensive experiments on benchmark datasets demonstrate that our approach achieves up to a 10% improvement in test accuracy on CIFAR-10 compared to existing baselines under heterogeneous privacy budgets. These results highlight the importance of incorporating privacy heterogeneity into client selection for practical and effective federated learning.
Figures & tables
Fig. 1 : Illustration of differentially private federated learning.
Ref.
Client Selection Method
DP
[ 4 ]
Unbiased selection
✓
[ 5 ]
Unbiased selection
✓
[ 6 ]
Unbiased selection
✓
[ 7 ]
Unbiased selection
✓
[ 8 ]
Importance sampling based selection
✗
UCB-CS [ 9 ]
Bandit based selection
✗
TABLE I : Comparison of related works.
Symbol
Meaning
pks
Selection probability of client k
pku
Unbiased selection probability of client k
ϵk
Privacy budget of client k
σk2
Noise variance of client k
Mk
Local dataset of client k
rk
Subsampling ratio of client k
TABLE II : Summary of important symbols.
Fig. 2 : Workflow of the privacy-aware client selection.
Fig. 3 : Selection probability of 100 clients with heterogeneous dataset sizes and privacy budgets ( ϵ ). Subsampling ratio r is fixed to 0.1 for all clients.
Fig. 4 : Test accuracy on CIFAR-10.
Mechanism
Non-IID Degree
Test Accuracy
Privacy-aware
s=100 (IID)
68.71%
s=70
65.12%
s=30
68.50%
s=0
67.63%
Unbiased
s=100 (IID)
28.02%
s=70
14.69%
TABLE III : Test accuracy performance (MNIST).
Mechanism
Non-IID Degree
Test Accuracy
Privacy-aware
s=100 (IID)
53.53%
s=70
55.26%
s=30
54.80%
s=0
48.94%
Unbiased
s=100 (IID)
14.68%
s=70
16.47%
TABLE IV : Test accuracy performance (FashionMNIST).
Fig. 5 : Test accuracy versus batch size.
Fig. 6 : Test accuracy versus privacy budget distributions. "P- i - j ", "U- i - j ", and "B- i - j " refer to the proposed, unbiased and biased client selection approaches with privacy budgets following Gaussian mixture distribution 0.3N(i,0.04)+0.7N(j,1) .
Differentially private federated learning trains shared models across data-holding clients while limiting what updates reveal about individual records or users.
Federated Learning (FL) enables a distributed client-server architecture where multiple clients collaboratively train a global Machine Learning (ML) model without sharing sensitive local data. However, FL often results in lower accuracy than traditional ML algorithms due to statistical heterogeneity across clients. Prior works attempt to address this by using model updates, such as loss and bias, from client models to select participants that can improve the global model's accuracy. However, these updates neither accurately represent a client's heterogeneity nor are their selection methods deterministic. We mitigate these limitations by introducing Terraform, a novel client selection methodology that uses gradient updates and a deterministic selection algorithm to select heterogeneous clients for retraining. This bi-pronged approach allows Terraform to achieve up to 47 percent higher accuracy over prior works. We further demonstrate its efficiency through comprehensive ablation studies and training time analyses, providing strong justification for the robustness of Terraform.
Federated Learning (FL) with Differential Privacy (DP) can ensure the protection of healthcare data privacy against malicious user attacks. However, FL with the DP method faces an inherent trade-off: stronger privacy protection degrades predictive accuracy due to injected noise. On the other hand, Homomorphic Encryption (HE) enables computations on encrypted data and the aggregation of encrypted gradients without introducing DP-induced noise, thereby potentially increasing the predictive accuracy. However, it may increase the computation cost. To improve the predictive accuracy while considering the computational ability of heterogeneous clients, we propose a Privacy-Preserving Machine Learning (PPML)-Hybrid method. In the proposed PPML-Hybrid method, clients distributedly select either HE or DP based on their computational resources, so that HE clients contribute noise-free updates while DP clients reduce computational overhead. Meanwhile, clients with high computational resources can flexibly adopt HE or DP according to their privacy needs. Performance evaluation on omics datasets shows that our proposed method achieves comparable predictive accuracy while significantly reducing computation time relative to HE-only. Additionally, it outperforms DP-only methods under equivalent or stricter privacy budgets.
Yusaku Negoya, Feifei Cui, Zilong Zhang +3
Department of Computer and Network Engineering, The University of Electro-Communications, Tokyo, Japan · School of Computer Science and Technology, Hainan University, Haikou 570228, China. · Department of Electrical and Computer Engineering, University of Houston, Houston, TX, 77204, USA +1
Federated learning enables collaborative model training across distributed clients while preserving data privacy. However, in practical deployments, device heterogeneity and non-independent and identically distributed (Non-IID) data often lead to unstable and biased gradient. When differential privacy is enforced, conventional fixed gradient clipping and Gaussian noise injection may further amplify gradient perturbations, resulting in training oscillation and degraded model performance. To address these challenges, we propose an adaptive differentially private federated learning framework that explicitly targets model efficiency under heterogeneous and privacy-constrained settings. On the client side, a lightweight local dimensionality reduction module is introduced to learn reduced-dimensional intermediate representations and produce more structured gradients during backpropagation, thereby mitigating noise amplification during local optimization. On the server side, an adaptive gradient clipping strategy dynamically adjusts clipping thresholds based on historical update statistics to avoid over-clipping and noise domination. Furthermore, a constraint-aware robust aggregation mechanism is designed to suppress unreliable or noise-dominated client updates and stabilize global optimization. Extensive experiments on CIFAR-10, SVHN, and STL-10 demonstrate that the proposed method consistently improves convergence stability and classification performance under differential privacy.
Jin Wang, Hui Ma, Yajun Zhang +4
Xinjiang Key Laboratory of Intelligent Computing and Smart Applications, School of Software, Xinjiang University, Urumqi 830017, China · College of Geography and Remote Sensing Sciences, Xinjiang University, Urumqi, China · Jilin University, Changchun 130012, China