One-shot and few-round federated learning (FL) reduce communication burden by constructing a global model within one or a few communication rounds, but often achieve lower performance than conventional multi-round FL. Without frequent synchronization, client models may drift apart, causing neurons that learn similar features to occupy different positions across clients. At the same time, local data heterogeneity can cause corresponding parameters to differ in their importance to each client's predictions. Together, these challenges cause direct averaging to combine mismatched neurons and assign equal weight to parameters despite differences in their importance. We propose FedFiOT, a Fisher-weighted optimal transport framework for one-shot and few-round FL that leverages Fisher information to quantify parameter importance and guide neuron alignment and model aggregation. FedFiOT jointly optimizes neuron correspondences and global model parameters through alternating minimization of a shared Fisher-weighted objective. Theoretically, we show that FedFiOT's optimization procedure monotonically decreases the layer-wise merging objective and establish a stationarity bound that quantifies the effects of client heterogeneity and nonuniform Fisher weighting.Experiments across multiple datasets demonstrate consistent improvements over competing baselines in both one-shot and few-round federated learning settings.
Figures & tables
Figure 1: Overview of FedFiOT. At each layer, the server minimizes the joint objective Jℓ by alternating between Fisher-weighted neuron alignment and Fisher-weighted model merging. The merged layer serves as the alignment reference for the next iteration. This repeats for up to S iterations, or until no client’s alignment changes. Here, ℓ indexes the layer and [s] the iteration; wiℓ and wk,jℓ denote the weights of global neuron i and of neuron j of client k . In each permutation matrix, rows are global neurons, columns are client neurons, and orange cells mark matched pairs. wk,iℓ and fk,iℓ,λ denote the weights and damped Fisher block of the neuron of client k matched to global neuron i . For illustration, we provide a simple numerical example in Appendix G .
Dataset
α
FedAvg
OTFusion
WB Fusion
PFNM
RegMean
DENSE
Fisher Merge
FedFisher (K-FAC)
FedFiOT (Ours)
SVHN
0.2
54.4 ± 2.8
54.8 ± 0.9
60.8 ± 1.0
69.5 ± 1.3
61.6 ± 4.3
70.6 ± 0.9
50.7 ± 1.3
72.2 ± 1.8
78.9 ± 1.9
0.1
24.3 ± 2.8
31.0 ± 1.5
34.7 ± 2.3
44.1 ± 2.8
41.1 ± 5.0
48.4 ± 1.1
38.4 ± 3.2
61.6 ± 3.6
66.8 ± 0.8
CIFAR10
0.2
34.0 ± 0.8
37.0 ± 1.2
40.3 ± 1.2
46.0 ± 2.3
39.2 ± 1.7
42.5 ± 1.7
31.9 ± 6.7
48.7 ± 1.1
52.0 ± 0.9
0.1
30.3 ± 1.0
32.3 ± 2.8
29.0 ± 1.1
32.8 ± 1.3
30.9 ± 2.5
33.3 ± 1.8
25.2 ± 2.8
38.6 ± 2.1
43.3 ± 1.8
CINIC10
0.2
31.9 ± 1.1
34.4 ± 1.3
36.4 ± 1.1
39.6 ± 1.4
31.6 ± 1.5
33.2 ± 1.2
26.4 ± 4.6
40.5 ± 2.9
45.6 ± 1.4
0.1
26.6 ± 1.2
29.2 ± 0.7
25.4 ± 0.8
29.8 ± 1.2
25.6 ± 1.3
26.0 ± 0.4
25.6 ± 0.1
32.0 ± 0.5
39.6 ± 0.3
Table 1: One-shot test accuracy (%) with the number of clients K=6 fixed and the heterogeneity parameter α varied. Mean ± std over three seeds. FedFiOT uploads quantized weights and diagonal Fishers.
Figure 2: Test accuracy (%) over four rounds of FL with K=6 clients and α=0.1 .
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
α
32-bit
16-bit
Difference
SVHN
0.2
78.2 ± 2.0
78.9 ± 1.9
+0.7
0.1
62.1 ± 2.1
66.8 ± 0.8
+4.7
CIFAR-10
0.2
51.1 ± 0.9
52.0 ± 0.9
+0.9
0.1
44.6 ± 1.3
43.3 ± 1.8
−1.3
CINIC-10
0.2
45.5 ± 2.5
45.6 ± 1.4
+0.1
0.1
38.6 ± 0.4
39.6 ± 0.3
+1.0
Appendix
Table 2: One-shot test accuracy (%) of FedFiOT with full-precision (32-bit) and quantized (16-bit) uploads, K=6 clients. Mean ± std over three seeds.
Method
Accuracy
FedAvg
25.5 ± 1.7
OTFusion
27.8 ± 2.1
PFNM
20.5 ± 3.0
RegMean
26.6 ± 2.6
DENSE
26.4 ± 0.7
FisherMerge
28.0 ± 3.0
Appendix
Table 3: One-shot test accuracy (%) on CIFAR-10 with K=6 clients and α=0.05 . Mean ± std over three seeds.
Figure 3: One-shot test accuracy (%) of FedFiOT and FedBEns with six clients, mean ± sample standard deviation over three training seeds. FedFiOT returns a single model; each FedBEns variant returns a five-model ensemble. FedBEns Kron selects server checkpoints on validation data.
FedFiOT
FedBEns
Models trained per client
1
5
Upload per client
1× FedAvg
>5× FedAvg
Models stored at inference
1
5
Forward passes per prediction
1
5
Appendix
Table 4: Cost comparison between FedFiOT and FedBEns per client and at inference, relative to FedAvg.
SVHN
CIFAR-10
CINIC-10
Method
α=0.2
α=0.1
α=0.2
α=0.1
α=0.2
α=0.1
FedAvg
41.2 ± 4.3
17.6 ± 1.3
16.3 ± 2.5
18.9 ± 2.8
14.9 ± 0.8
19.6 ± 2.6
OTFusion
55.6 ± 0.5
30.2 ± 1.4
36.5 ± 1.0
30.6 ± 0.4
33.8 ± 0.5
28.4 ± 1.3
WB Fusion
59.7 ± 2.6
34.3 ± 0.3
38.6 ± 1.6
27.4 ± 1.7
35.7 ± 0.9
25.8 ± 1.1
PFNM
68.9 ± 3.1
39.3 ± 0.3
42.8 ± 2.7
33.8 ± 1.4
38.4 ± 1.5
25.4 ± 0.9
RegMean
32.0 ± 1.6
25.4 ± 2.6
20.8 ± 1.4
18.2 ± 1.1
16.7 ± 1.1
17.8 ± 1.1
Appendix
Table 5: One-shot test accuracy (%) with independently initialized clients, K=6 . Mean ± std over three seeds.
Figure 4: Fraction of hidden neurons (%) permuted by FedFiOT in each round, K=6 clients and α=0.1 . Lines show the mean over three seeds and shaded regions show ± one standard deviation.
SVHN
CIFAR-10
CINIC-10
Variant
α=0.2
α=0.1
α=0.2
α=0.1
α=0.05
α=0.2
α=0.1
(A) FedFiOT
78.9 ± 1.9
66.8 ± 0.8
52.0 ± 0.9
43.3 ± 1.8
41.4 ± 1.7
45.6 ± 1.4
39.6 ± 0.3
(B) No alignment
60.4 ± 2.7
42.4 ± 5.6
40.1 ± 4.2
33.2 ± 1.5
27.6 ± 3.9
33.1 ± 2.0
28.2 ± 1.6
(C) Euclidean matching
77.3 ± 0.2
49.9 ± 4.7
50.0 ± 1.3
34.4 ± 1.8
27.7 ± 4.1
42.2 ± 1.2
32.0 ± 0.5
(D) Uniform aggregation
61.3 ± 1.7
38.1 ± 1.0
38.2 ± 1.9
31.7 ± 1.2
21.1 ± 0.7
34.4 ± 1.0
26.2 ± 0.9
(E) WB Fusion
60.8 ± 1.0
34.7 ± 2.3
40.3 ± 1.2
29.0 ± 1.1
18.3 ± 0.7
36.4 ± 1.1
25.4 ± 0.8
Appendix
Table 6: Ablation of FedFiOT’s components on K=6 . Test accuracy (%), mean ± std over three seeds.
Merge
Alignment
w⊤
J
Average by index
identity
(5,3,7)
108
Fisher merging
identity
(2.5,3,7)
83
Average of aligned weights
(2,3,1)
(1,5,9)
12
FedFiOT
(2,3,1)
(0.5,5.5,8.5)
9
Appendix
Table 7: Merged layer and objective J for the three-neuron example. Methods without alignment use identity alignments for both clients.
Federated learning aggregation combines client updates into shared or personalized models while balancing data heterogeneity, robustness, privacy, and communication.