Private and Distributionally Robust Federated Multiclass Classification

Andrea P. Hernandez; Michael Ibrahim; Nagi Gebraeel

cs.LGOct 9, 2026

Private and Distributionally Robust Federated Multiclass Classification

Authors: Andrea P. Hernandez, Michael Ibrahim, Nagi Gebraeel

Organizations: H. Milton Stewart School of Industrial and Systems Engineering Georgia Institute of Technology Atlanta, GA 30332-0205, USA

Abstract

Fault diagnosis in asset-intensive industries requires multiclass models trained on noisy, sensitive data collected across geographically distributed client sites. Federated learning enables collaborative training without centralizing data, but most approaches do not provide formal privacy guarantees for individual records stored at client sites and can degrade under label noise. To address these limitations, we develop a differentially private, distributionally robust framework for federated multiclass classification that provides sample-level privacy for records stored at each client. We build on a federated Wasserstein distributionally robust multiclass objective with the Crammer-Singer loss, solved using an Alternating Direction Method of Multipliers (ADMM)-based algorithm. To protect the released client updates, our method injects calibrated Gaussian noise into the proximal linearized client updates of both the local model and the local distributionally robust variable before aggregation at the central server, and we quantify the cumulative privacy loss using the moments accountant. We provide per-round sample-level $(ε,δ)$ -differential privacy guarantees and an overall composed privacy guarantee across all training rounds, together with convergence results and an explicit privacy-utility tradeoff. Experiments on simulated industrial fault data and UCI benchmark datasets show that our method outperforms differentially private federated empirical risk minimization and stochastic gradient descent baselines in most tested settings, with the largest gains under strong privacy-induced noise.

Figures & tables

Panel of Figure 1 — Figure 1: mCCR of the proposed DP-Fed DRO method as a function of $\epsilon$ across different values of $\delta$ , compared with DP-Fed ERM, DP-Fed SGD, and non-private Fed DRO.

Model	Dermatology	Iris	Wine	Penguins	Seeds
Centralized DRO	$0.905\pm 0.070$	$0.919\pm 0.051$	$0.943\pm 0.032$	$0.996\pm 0.008$	$0.969\pm 0.023$
DP-Fed DRO	$0.505\pm 0.103$	$\mathbf{0.564\pm 0.077}$	$\mathbf{0.640\pm 0.148}$	$0.689\pm 0.142$	$\mathbf{0.733\pm 0.144}$
DP-Fed ERM	$0.527\pm 0.104$	$0.519\pm 0.146$	$0.590\pm 0.113$	$\mathbf{0.774\pm 0.113}$	$0.620\pm 0.111$
DP-Fed SGD	$\mathbf{0.599\pm 0.131}$	$0.559\pm 0.065$	$0.606\pm 0.121$	$0.637\pm 0.141$	$0.690\pm 0.145$
Non-Private Fed DRO	$0.919\pm 0.044$	$\mathbf{0.943\pm 0.042}$	$0.924\pm 0.045$	$\mathbf{0.993\pm 0.013}$	$\mathbf{0.981\pm 0.015}$
Non-Private Fed OvA DRO	$\mathbf{0.957\pm 0.017}$	$0.862\pm 0.082$	$\mathbf{0.932\pm 0.038}$	$0.992\pm 0.010$	$0.979\pm 0.014$

Table 1: Comparison of Macro-F1 scores across five multiclass benchmark datasets, namely Dermatology ( Ilter & Guvenir, 1998 ) , Iris ( Fisher, 1936 ) , Wine ( Aeberhard & Forina, 1992 ) , Palmer Penguins ( Horst et al., 2020 ) , and Seeds ( Charytanowicz et al., 2010 ) .

Panel of Figure 3 — Figure 3: Log-scale runtime to peak mCCR for the tested federated models across the client- and class-scalability settings.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Software	Version	License
Gurobi	11.0.3	Academic license
MATLAB	R2025b	Academic license
Python	3.8.3	Open source license
Scikit-Learn	0.23.1	Open source license
NumPy	1.18.5	Open source license
SciPy	1.5.0	Open source license

Appendix

Table 2: Software environment used in the experiments.

Dataset	Number of Classes	License
Dermatology ( Ilter & Guvenir, 1998 )	6	CC BY 4.0
Iris ( Fisher, 1936 )	3	CC BY 4.0
Wine ( Aeberhard & Forina, 1992 )	3	CC BY 4.0
Palmer Penguins ( Horst et al., 2020 )	3	CC0
Seeds ( Charytanowicz et al., 2010 )	3	CC BY 4.0

Appendix

Table 3: Public benchmark datasets used in the experiments.

Panel of Figure 4 — Figure 4: Additional convergence plots for the augmented objective and the consensus violation. The top row reports the running average of the augmented objective at the privatized iterates, while the bottom row reports the consensus violation. Each column corresponds to a different privacy budget $\epsilon$ .

Overlap setup	Metric	Non-Private Fed DRO	Non-Private Fed OvA DRO
Strong overlap	mCCR / Accuracy	0.8366 $\pm$ 0.0340	0.7894 $\pm$ 0.0320
	Macro-F1	0.8140 $\pm$ 0.0471	0.7559 $\pm$ 0.0456
	Macro Recall	0.8366 $\pm$ 0.0340	0.7894 $\pm$ 0.0320
	Worst-class Recall	0.4074 $\pm$ 0.1448	0.3000 $\pm$ 0.1190
	Normal Recall	0.9963 $\pm$ 0.0078	0.9963 $\pm$ 0.0078
	Fault A Recall	0.4074 $\pm$ 0.1448	0.3000 $\pm$ 0.1190

Appendix

Table 4: Performance of native multiclass DRO and OvA DRO across Fault A/Fault B overlap regimes.

Differentially Private Federated Learning

Differentially private federated learning trains shared models across data-holding clients while limiting what updates reveal about individual records or users.

See all 62 papers

Federated fairness-aware classification under differential privacy
Mar 25, 2026Gengyu Xue, Yi YuAlgorithmic Fairness Differentially Private Federated Learning
Department of Statistics, University of Warwick
Distributionally Robust Federated Learning with Multi-Source Data
Sep 17, 2026Yingzhu Liu, Zhongkui Li, Pengcheng You +1Distributionally Robust Optimization Non-IID Federated Learning
Department of Control Science and Systems Engineering, Peking University, Beijing, China. · Engineering and Technology Institute Groningen and the Jan C. Willems Center for Systems and Control, University of Groningen, The Netherlands.
An Adaptive Differentially Private Federated Learning Framework
Feb 6, 2026Jin Wang, Hui Ma, Yajun Zhang +4Non-IID Federated Learning Differentially Private Federated Learning
Xinjiang Key Laboratory of Intelligent Computing and Smart Applications, School of Software, Xinjiang University, Urumqi 830017, China · College of Geography and Remote Sensing Sciences, Xinjiang University, Urumqi, China · Jilin University, Changchun 130012, China

Mar 25, 2026J/K move · Enter open · S save

Federated fairness-aware classification under differential privacy

Gengyu Xue, Yi Yu

Department of Statistics, University of Warwick

Algorithmic Fairness Differentially Private Federated Learning Differential Privacy Group Fairness

Open paper AskPDF arXiv

Privacy and algorithmic fairness have become two central issues in modern machine learning. Although each has separately emerged as a rapidly growing research area, their joint effect remains comparatively under-explored. In this paper, we systematically study the joint impact of differential privacy and fairness on classification in a federated setting, where data are distributed across multiple servers. Targeting demographic disparity constrained classification under federated differential privacy, we propose a two-step algorithm, namely FDP-Fair. In the special case where there is only one server, we further propose a simple yet powerful algorithm, namely CDP-Fair, serving as a computationally-lightweight alternative. Under mild structural assumptions, theoretical guarantees on privacy, fairness and excess risk control are established. In particular, we disentangle the source of the private fairness-aware excess risk into a) intrinsic cost of classification, b) cost of private classification, c) non-private cost of fairness and d) private cost of fairness. Our theoretical findings are complemented by extensive numerical experiments on both synthetic and real datasets, highlighting the practicality of our designed algorithms.

Abstract

Differentially Private Federated Learning

Explore similar work

Federated fairness-aware classification under differential privacy

Distributionally Robust Federated Learning with Multi-Source Data

An Adaptive Differentially Private Federated Learning Framework