cs.LGOct 9, 2026

Private and Distributionally Robust Federated Multiclass Classification

Authors: Andrea P. Hernandez, Michael Ibrahim, Nagi Gebraeel

Organizations: H. Milton Stewart School of Industrial and Systems Engineering Georgia Institute of Technology Atlanta, GA 30332-0205, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Fault diagnosis in asset-intensive industries requires multiclass models trained on noisy, sensitive data collected across geographically distributed client sites. Federated learning enables collaborative training without centralizing data, but most approaches do not provide formal privacy guarantees for individual records stored at client sites and can degrade under label noise. To address these limitations, we develop a differentially private, distributionally robust framework for federated multiclass classification that provides sample-level privacy for records stored at each client. We build on a federated Wasserstein distributionally robust multiclass objective with the Crammer-Singer loss, solved using an Alternating Direction Method of Multipliers (ADMM)-based algorithm. To protect the released client updates, our method injects calibrated Gaussian noise into the proximal linearized client updates of both the local model and the local distributionally robust variable before aggregation at the central server, and we quantify the cumulative privacy loss using the moments accountant. We provide per-round sample-level (ε,δ)(ε,δ)-differential privacy guarantees and an overall composed privacy guarantee across all training rounds, together with convergence results and an explicit privacy-utility tradeoff. Experiments on simulated industrial fault data and UCI benchmark datasets show that our method outperforms differentially private federated empirical risk minimization and stochastic gradient descent baselines in most tested settings, with the largest gains under strong privacy-induced noise.

Figures & tables

Figure 1: mCCR of the proposed DP-Fed DRO method as a function of ϵ\epsilon across different values of δ\delta , compared with DP-Fed ERM, DP-Fed SGD, and non-private Fed DRO.
ModelDermatologyIrisWinePenguinsSeeds
Centralized DRO0.905±0.0700.905\pm 0.0700.919±0.0510.919\pm 0.0510.943±0.0320.943\pm 0.0320.996±0.0080.996\pm 0.0080.969±0.0230.969\pm 0.023
DP-Fed DRO0.505±0.1030.505\pm 0.1030.564±0.077\mathbf{0.564\pm 0.077}0.640±0.148\mathbf{0.640\pm 0.148}0.689±0.1420.689\pm 0.1420.733±0.144\mathbf{0.733\pm 0.144}
DP-Fed ERM0.527±0.1040.527\pm 0.1040.519±0.1460.519\pm 0.1460.590±0.1130.590\pm 0.1130.774±0.113\mathbf{0.774\pm 0.113}0.620±0.1110.620\pm 0.111
DP-Fed SGD0.599±0.131\mathbf{0.599\pm 0.131}0.559±0.0650.559\pm 0.0650.606±0.1210.606\pm 0.1210.637±0.1410.637\pm 0.1410.690±0.1450.690\pm 0.145
Non-Private Fed DRO0.919±0.0440.919\pm 0.0440.943±0.042\mathbf{0.943\pm 0.042}0.924±0.0450.924\pm 0.0450.993±0.013\mathbf{0.993\pm 0.013}0.981±0.015\mathbf{0.981\pm 0.015}
Non-Private Fed OvA DRO0.957±0.017\mathbf{0.957\pm 0.017}0.862±0.0820.862\pm 0.0820.932±0.038\mathbf{0.932\pm 0.038}0.992±0.0100.992\pm 0.0100.979±0.0140.979\pm 0.014
Table 1: Comparison of Macro-F1 scores across five multiclass benchmark datasets, namely Dermatology ( Ilter & Guvenir, 1998 ) , Iris ( Fisher, 1936 ) , Wine ( Aeberhard & Forina, 1992 ) , Palmer Penguins ( Horst et al., 2020 ) , and Seeds ( Charytanowicz et al., 2010 ) .
Figure 3: Log-scale runtime to peak mCCR for the tested federated models across the client- and class-scalability settings.
Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

SoftwareVersionLicense
Gurobi11.0.3Academic license
MATLABR2025bAcademic license
Python3.8.3Open source license
Scikit-Learn0.23.1Open source license
NumPy1.18.5Open source license
SciPy1.5.0Open source license
Appendix
Table 2: Software environment used in the experiments.
DatasetNumber of ClassesLicense
Dermatology ( Ilter & Guvenir, 1998 )6CC BY 4.0
Iris ( Fisher, 1936 )3CC BY 4.0
Wine ( Aeberhard & Forina, 1992 )3CC BY 4.0
Palmer Penguins ( Horst et al., 2020 )3CC0
Seeds ( Charytanowicz et al., 2010 )3CC BY 4.0
Appendix
Table 3: Public benchmark datasets used in the experiments.
Figure 4: Additional convergence plots for the augmented objective and the consensus violation. The top row reports the running average of the augmented objective at the privatized iterates, while the bottom row reports the consensus violation. Each column corresponds to a different privacy budget ϵ\epsilon .
Overlap setupMetricNon-Private Fed DRONon-Private Fed OvA DRO
Strong overlapmCCR / Accuracy0.8366 ±\pm 0.03400.7894 ±\pm 0.0320
Macro-F10.8140 ±\pm 0.04710.7559 ±\pm 0.0456
Macro Recall0.8366 ±\pm 0.03400.7894 ±\pm 0.0320
Worst-class Recall0.4074 ±\pm 0.14480.3000 ±\pm 0.1190
Normal Recall0.9963 ±\pm 0.00780.9963 ±\pm 0.0078
Fault A Recall0.4074 ±\pm 0.14480.3000 ±\pm 0.1190
Appendix
Table 4: Performance of native multiclass DRO and OvA DRO across Fault A/Fault B overlap regimes.
More in this topic

Differentially Private Federated Learning

Differentially private federated learning trains shared models across data-holding clients while limiting what updates reveal about individual records or users.

See all 62 papers

Explore similar work

CardsList
Mar 25, 2026

Federated fairness-aware classification under differential privacy

Privacy and algorithmic fairness have become two central issues in modern machine learning. Although each has separately emerged as a rapidly growing research area, their joint effect remains comparatively under-explored. In this paper, we systematically study the joint impact of differential privacy and fairness on classification in a federated setting, where data are distributed across multiple servers. Targeting demographic disparity constrained classification under federated differential privacy, we propose a two-step algorithm, namely FDP-Fair. In the special case where there is only one server, we further propose a simple yet powerful algorithm, namely CDP-Fair, serving as a computationally-lightweight alternative. Under mild structural assumptions, theoretical guarantees on privacy, fairness and excess risk control are established. In particular, we disentangle the source of the private fairness-aware excess risk into a) intrinsic cost of classification, b) cost of private classification, c) non-private cost of fairness and d) private cost of fairness. Our theoretical findings are complemented by extensive numerical experiments on both synthetic and real datasets, highlighting the practicality of our designed algorithms.

Gengyu Xue, Yi Yu

Department of Statistics, University of Warwick

Algorithmic FairnessDifferentially Private Federated LearningDifferential PrivacyGroup Fairness
AskPDF
Sep 17, 2026

Distributionally Robust Federated Learning with Multi-Source Data

Federated learning trains a shared model from private client data. In practice, data-generating distributions may differ, and the true mixture across clients is often unknown, making the underlying group distribution difficult to specify. Existing approaches address cross-client mixture uncertainty by optimizing against the worst-case mixture, yet assume accurate client-wise distribution estimates. However, these estimates can be unreliable when based on finite samples. To handle both cross-client mixture uncertainty and within-client distributional ambiguity, we construct a global ambiguity set as the union of admissible mixtures of local ambiguity sets. The construction allows client-specific ambiguity radii and admits a client-wise separable reformulation. Leveraging this structure, we establish a high-probability out-of-sample performance guarantee. We further develop a federated algorithm for a penalty-based reformulation and prove its convergence under milder regularity conditions. Simulations validate the algorithm's effectiveness.

Yingzhu Liu, Zhongkui Li, Pengcheng You +1

Department of Control Science and Systems Engineering, Peking University, Beijing, China. · Engineering and Technology Institute Groningen and the Jan C. Willems Center for Systems and Control, University of Groningen, The Netherlands.

Distributionally Robust OptimizationNon-IID Federated LearningFederated Learning
AskPDF
Feb 6, 2026

An Adaptive Differentially Private Federated Learning Framework

Federated learning enables collaborative model training across distributed clients while preserving data privacy. However, in practical deployments, device heterogeneity and non-independent and identically distributed (Non-IID) data often lead to unstable and biased gradient. When differential privacy is enforced, conventional fixed gradient clipping and Gaussian noise injection may further amplify gradient perturbations, resulting in training oscillation and degraded model performance. To address these challenges, we propose an adaptive differentially private federated learning framework that explicitly targets model efficiency under heterogeneous and privacy-constrained settings. On the client side, a lightweight local dimensionality reduction module is introduced to learn reduced-dimensional intermediate representations and produce more structured gradients during backpropagation, thereby mitigating noise amplification during local optimization. On the server side, an adaptive gradient clipping strategy dynamically adjusts clipping thresholds based on historical update statistics to avoid over-clipping and noise domination. Furthermore, a constraint-aware robust aggregation mechanism is designed to suppress unreliable or noise-dominated client updates and stabilize global optimization. Extensive experiments on CIFAR-10, SVHN, and STL-10 demonstrate that the proposed method consistently improves convergence stability and classification performance under differential privacy.

Jin Wang, Hui Ma, Yajun Zhang +4

Xinjiang Key Laboratory of Intelligent Computing and Smart Applications, School of Software, Xinjiang University, Urumqi 830017, China · College of Geography and Remote Sensing Sciences, Xinjiang University, Urumqi, China · Jilin University, Changchun 130012, China

Non-IID Federated LearningDifferentially Private Federated LearningFederated LearningHeterogeneous Federated Learning
AskPDF