Federated learning trains a shared model across clients without exchanging their data, yet the updates they transmit still leak information about individual records. Enforcing differential privacy is harder still when the model parameters lie on a Riemannian manifold. Existing methods rebuild the noise added for privacy in the geometry of each new iterate, aggregate through intrinsic maps that often lack a closed form, and prove convergence only in restricted regimes. We introduce DP-RFedProj, a record-level differentially private federated learning method for compact submanifolds that aggregates through closed-form projections and privatises gradients in a fixed ambient geometry. Each client may run any stochastic first-order optimiser, take several local steps, and participate only intermittently. An explicit record-level f-DP guarantee holds for the entire training transcript, independently of the manifold, the aggregation scheme and the local optimiser. We establish convergence of DP-RFedProj when each client runs Riemannian SGD locally and the server aggregates by projected averaging, and show that privacy enters the rate through a term scaling with the manifold's intrinsic dimension rather than the ambient one. Synthetic experiments on federated PCA over the Grassmann manifold corroborate these findings. On EEG motor-imagery classification with SPDNet, whose weights lie on the Stiefel manifold, the method outperforms a Euclidean baseline at every privacy budget considered. In both experimental settings, our accounting requires five to six times less noise than that of the closest prior method for the same budget, which translates into higher utility.
Table 1: Comparison of Riemannian federated methods. The convergence regime reports the setting in which convergence is established, as a pair (local steps, participation), with (multi-step, partial) as the widest regime. Both PriRFed and our method are optimiser-agnostic by design. The Solver column reports the optimiser under which each convergence result is proved.
Figure 1: One round t with Honest-But-Curious (HBC) server. The server draws a fixed-size active set St⊂[n] , sends xt to the selected clients (shaded), collects their submissions Δ(i) from τ private local steps and aggregates them to form the next global iterate xt+1 . The HBC server sees (xt,St,{Δ(i)}i∈St) .
Scheme
\textscAgg.Sub(xt,xt,τ(i))
\textscAgg.Out(xt,Δt)
KFAvg
Logxt(xt,τ(i))
Expxt(Δt)
RetAvg
Rxt−1(xt,τ(i))
Rxt(Δt)
ProjAvg (ours)
xt,τ(i)
Π(Δt)
RLAvg (ours)
Pxt(xt,τ(i)−xt)
Π(xt+Δt)
Table 2: Aggregation primitives for the four schemes. Agg.Sub (xt,xt,τ(i)) is the client-side submission Δ(i) and Agg.Out (xt,Δt) is the server-side global update xt+1 , where Δt=k1∑i∈StΔ(i) .
Parameter space
Metric
Privacy
Conv. rate
Sphere, Stiefel, Grassmann
induced
✓
✓
E (incl. log(Sn++) )
Euclidean
✓
×
Sn++
affine-inv.
algorithm not runnable
Table 3: Where Algorithm 1 applies. Privacy (Theorem 10 ) holds for all four schemes of Table 2 ; the convergence rate (Corollary 14 ) for ProjAvg and RLAvg with RSGD.
Clients
n=50
Clients per round
k=10
Records per client
m=5000
Local steps, batch size
τ=10 , B=500
Rounds
T=200
Privacy
ε=1 , δ=10−5
Table 4: Common settings. k and ε take these values when they are not varied.
Figure 2: Effects of manifold dimension against ambient dimension, client participation and accounting. Median and interquartile range over 10 seeds. Markers are shifted slightly where values coincide.
SPDNet
Dataset
nsubj
nchan
ncl
Native S.R. (Hz)
T
Trials/class
d
εR
BNCI2014_001
9
22
4
250
513
144
16
8⋅10−5
Weibo2014
10
60
7
200
513
80
20
2⋅10−5
Cho2017
52
64
2
512
384
100
16
6⋅10−5
Table 5: The three EEG motor-imagery datasets, with the BiMap dimension d and the ReEig floor εR of SPDNet (Section 7.2 ).
Dataset
Chance
SPDNet
EEGNet
BNCI2014_001
25.0
56.5±1.8
53.7±1.6
Cho2017
50.0
63.9±1.0
67.4±1.2
Weibo2014
14.3
49.7±1.0
44.2±1.9
Table 6: Centralized non-private test accuracy (%), mean ± standard deviation over 10 seeds, last epoch. Best per dataset in bold.
Private
Dataset
Arm
Non-private
ε=20
ε=10
ε=5
BNCI2014_001 chance 25.0
SPDNet ProjAvg
55.1±1.8
43.3±1.9
39.0±2.2
35.7±2.0
SPDNet RLAvg
55.2±1.6
43.9±1.6
39.4±1.0
35.9±1.4
SPDNet RetAvg
54.7±2.3
43.3±0.8
41.3±2.2
34.9±0.9
EEGNet
53.2±1.9
30.0±2.2
28.6±1.2
27.4±2.1
Cho2017 chance 50.0
SPDNet ProjAvg
62.9±0.8
55.6±1.5
53.9±1.3
52.0±1.2
Table 7: Federated test accuracy (%) at full client participation, mean ± standard deviation over 10 seeds, last round, δ=10−5 . Best per column within a dataset in bold.
Private
Dataset
Arm
Non-private
ε=20
ε=10
ε=5
BNCI2014_001 chance 25.0
SPDNet ProjAvg
54.7±1.6
41.5±2.0
37.8±2.1
35.0±1.3
EEGNet
52.8±0.9
28.5±2.1
27.9±2.0
25.3±1.5
Cho2017 chance 50.0
SPDNet ProjAvg
62.7±0.9
54.6±1.7
52.4±1.9
51.8±1.6
EEGNet
67.1±0.7
49.9±0.3
50.4±0.5
50.3±0.6
Weibo2014 chance 14.3
SPDNet ProjAvg
47.1±2.5
21.2±1.6
19.3±1.8
17.7±1.3
Table 8: Federated test accuracy (%) at half client participation, mean ± standard deviation over 10 seeds, last round, δ=10−5 . Learning rate, batch size and T are those of full participation. Best per column within a dataset in bold.
Figure 4: Comparison with PriRFed on SPDNet, δ=10−5 . (a) Test accuracy (%) at ε=20 over 10 seeds, chance in parentheses. PriRFed is RetAvg at the noise of its own accounting. (b) Ratio σPriRFed/σours of the noise levels against the target ε , for the mechanism of (a).
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Panel
d
p
dM
D
k
ησdp2dM/k
ProjAvg
RLAvg
(a)
40
1
39
820
10
4.57
1.24
1.23
40
2
76
820
10
8.91
2.43
2.42
40
4
144
820
10
16.9
4.18
4.16
40
8
256
820
10
30.0
7.56
7.52
(b)
26
8
144
351
10
16.9
4.02
4.01
40
4
144
820
10
16.9
4.18
4.16
Appendix
Table 9: Median cost of privacy at ε=1 over 10 seeds, in units of 10−4 .
σdp
our accounting
PriRFed
ε
ours
PriRFed
ProjAvg
RLAvg
RetAvg
η
gap
0.5
0.0452
0.247
1.5⋅10−3
1.5⋅10−3
1.5⋅10−3
0.05
3.1⋅10−2
1
0.0242
0.131
4.4⋅10−4
4.4⋅10−4
4.4⋅10−4
0.1
7.7⋅10−3
2
0.0132
0.0710
1.5⋅10−4
1.5⋅10−4
1.5⋅10−4
0.2
3.9⋅10−3
5
0.00617
0.0328
4.9⋅10−5
4.9⋅10−5
4.9⋅10−5
0.2
8.0⋅10−4
10
0.00368
0.0193
3.0⋅10−5
3.0⋅10−5
3.0⋅10−5
0.2
2.9⋅10−4
Appendix
Table 10: Noise level and median optimality gap over 10 seeds of Figure 2 (d). ProjAvg , RLAvg and RetAvg use our accounting and η=0.2 . PriRFed uses RetAvg , its round-level accounting and its tuned η .
Figure 5: EEGNet pipeline. Shapes above each arrow, with B the batch size and T the number of time samples.
Figure 6: SPDNet pipeline. Shapes above each arrow, with B the batch size and d the dimension after BiMap.
Dataset
Model
η0
B
T
BNCI2014_001
SPDNet
10−2
64
200
EEGNet
10−3
16
300
Cho2017
SPDNet
3⋅10−2
16
300
EEGNet
10−3
64
300
Weibo2014
SPDNet
10−3
32
300
EEGNet
3⋅10−3
128
300
Appendix
Table 11: Selected centralized configurations: learning rate η0 , batch size B and horizon T in epochs.
Parameters
Dataset
n
m⋆
mmax
EEGNet
SPDNet
BNCI2014_001
3
1,257
1,326
2,404
900
Cho2017
26
277
358
2,434
1,298
Weibo2014
5
762
873
3,783
2,677
Appendix
Table 13: Number of clients, smallest and largest client training sizes over the 10 seeds, and number of parameters of each model.
Figure 7: Test accuracy against communication round at ε=5 , at the horizon Tfed used in every private run and at half and twice that horizon. Mean over 10 seeds, ±1 standard deviation, smoothed over 5 rounds.
Dataset
ProjAvg
RLAvg
RetAvg
RetAvg / ProjAvg
BNCI2014_001 (3)
0.50[0.32,0.58]
0.71[0.62,0.84]
4.47[3.74,4.98]
9×
Weibo2014 (5)
0.67[0.57,0.84]
1.39[1.09,1.63]
10.8[9.77,12.0]
16×
Cho2017 (26)
1.61[1.38,2.05]
4.87[2.08,6.04]
29.3[24.0,37.8]
18×
Appendix
Table 15: Aggregation time per round (ms) of the Riemannian rules: median and interquartile range over all rounds of the 10 non-private runs, with the number of clients in parentheses. The last column is the ratio of the medians of RetAvg and ProjAvg .
Differentially private federated learning trains shared models across data-holding clients while limiting what updates reveal about individual records or users.
Federated Learning is a leading framework for training ML and AI models collaboratively across numerous user devices or databases. We study the trade-offs among estimation accuracy, privacy constraints, and communication cost for differentially private (DP) federated M estimation. The two standard methods in the literature are FedAvg, which may suffer from high federation bias, and FedSGD, which can incur high communication cost. Aimed at improving accuracy at a reduced communication cost, we propose FedHybrid, which uses FedSGD starting with an improved initialization by the FedAvg estimator. We propose FedNewton, which averages local Newton iterations to reduce bias in FedAvg, achieving an estimation accuracy comparable to FedSGD with much fewer communication rounds when the number of clients grows sufficiently slowly. We establish finite sample upper bounds on the mean-squared error rates of the DP versions of these estimators as functions of the number of clients, local sample sizes, privacy budget, and number of iterations. We further derive a minimax lower bound on the MSE of any iterative private federated procedure that provides a benchmark to assess the optimality gap of these methods. We numerically evaluate our methods for training a logistic regression and a neural network on the computer vision datasets MNIST and CIFAR-10.
Arnab Auddy, Xiangni Peng, Subhadeep Paul
Department of Statistics · The Ohio State University
Differentially private federated learning must balance privacy protection against model accuracy and training efficiency. Static gradient clipping applies a fixed threshold throughout training and across model layers, which can cause excessive clipping when the threshold is too small or unnecessarily large noise when it is too large. This paper presents DDP-SA-adaptive, an adaptive gradient clipping and noise adding mechanism for differentially private federated learning with secure aggregation. At each communication round, every client determines a separate clipping threshold for each model layer from the median of its per-sample gradient norms. The resulting layer-wise thresholds adapt to the evolving gradient distributions and calibrate the Laplace noise added before the updates are encoded and secret-shared among intermediate aggregation servers. We evaluate the proposed mechanism on a federated regression task in terms of efficiency, accuracy, privacy, convergence, clipping norm, and noise magnitude. Compared with the static DDP-SA baseline, DDP-SA-adaptive reduces the number of communication rounds by 6.81%, total training time by 19.21%, and average per-round training time by 13.33%, leading to improved training efficiency. It also reduces test loss by 98.74% and increases test R2 by 3.41%, leading to improved model accuracy. To attain R2 = 0.99, the adaptive mechanism operates with a privacy budget of approximately epsilon = 0.1, compared with epsilon = 0.4 for static DDP-SA, thus providing stronger privacy protection and achieving stronger privacy guarantees. These results demonstrate that round-wise, layer-wise adaptation can improve the privacy-accuracy-efficiency trade-off of differentially private federated learning.
We introduce two federated learning frameworks for the classical SPDnet model operating on symmetric positive definite (SPD) matrices with Stiefel-constrained parameters. Unlike standard Euclidean averaging, which violates orthogonality, our approach preserves geometric structure through two efficient aggregation strategies: ProjAvg, projecting arithmetic means onto the Stiefel manifold, and RLAvg, approximating tangent-space averaging via retractions and liftings. Both methods are computationally efficient, independent of the optimizer, and enable scalable federated learning for signal processing applications whose features are SPD matrices. Simulations on EEG motor imagery benchmarks show that FedSPDnet outperforms federated EEGnet in F1 score and robustness to federation and partial participation, while using fewer parameters per communication round.
Thibault Pautrel, Florent Bouchard, Ammar Mian +1
CentraleSup´elec, L2S, France · Universit´e Savoie Mont-Blanc, LISTIC, France