Federated learning (FL) facilitates the secure utilization of decentralized images, advancing applications in medical image recognition and autonomous driving. However, conventional FL faces two critical challenges in real-world deployment: ineffective knowledge fusion caused by model updates biased toward majority-class features, and prohibitive communication overhead due to frequent transmissions of high-dimensional model parameters. Inspired by the human brain's efficiency in knowledge integration, we propose a novel Generative Federated Prototype Learning (GFPL) framework to address these issues. Within this framework, a prototype generation method based on Gaussian Mixture Model (GMM) captures the statistical information of class-wise features, while a prototype aggregation strategy using Bhattacharyya distance effectively fuses semantically similar knowledge across clients. In addition, these fused prototypes are leveraged to generate pseudo-features, thereby mitigating feature distribution imbalance across clients. To further enhance feature alignment during local training, we devise a dual-classifier architecture, optimized via a hybrid loss combining Dot Regression and Cross-Entropy. Extensive experiments on benchmarks show that GFPL improves model accuracy by 3.6% under imbalanced data settings while maintaining low communication cost.
Figures & tables
Figure 1: Generative Federated Prototype Learning (GFPL):(a) the client trains feature extractor, projection layers through local data, while generating local prototypes with GMM and uploading them to the server for prototype interaction; (b) After receiving global prototypes, the client replaces local prototypes with global prototypes and further generates balanced pseudo features through GMM sampling, thereby retraining the model projection layer.
Figure 2: Visualization of features and projections under different loss functions: (a) only LDR (features); (b) LDR+LCE (features); (c) only LDR (projections); (d) LDR+LCE (projections)
Dataset
Method
(w^,s^)
Average Test Accuracy(%)
Comm. Round
Comm. Param. (×103)
wˉ=3
wˉ=4
wˉ=5
MNIST
Local
(2,2)
94.05 ± 2.93
93.35 ± 3.26
92.92 ± 3.17
0
0
FeSEM (2020)
(2,2)
95.26 ± 3.48
97.06 ± 2.72
96.31 ± 2.41
150
430
FedProx (2020)
(2,2)
96.26 ± 2.89
96.40 ± 3.33
95.65 ± 3.38
110
430
FedPer (2019)
(2,2)
95.57 ± 2.96
96.44 ± 2.62
95.55 ± 3.13
100
106
FedAvg (2017)
(2,2)
95.04 ± 6.48
94.32 ± 4.89
93.22 ± 4.39
150
430
Table 1: Comparison of Federated Learning Algorithms
Variant
CIFAR10( s^=1 )
CIFAR10( s^=20 )
FEMNIST( s^=1 )
FEMNIST( s^=20 )
CIFAR100( s^=1 )
CIFAR100( s^=20 )
GFPL w/o DCS and PFG
55.62 ± 7.15
54.23 ± 9.88
91.16 ± 5.64
90.22 ± 7.84
53.51 ± 8.75
50.31 ± 9.84
GFPL w/o DCS
64.47 ± 3.36
63.28 ± 5.77
93.25 ± 4.22
92.33 ± 6.22
62.34 ± 6.33
60.63 ± 7.23
GFPL w/o PFG
67.24 ± 2.88
66.01 ± 3.95
94.37 ± 3.43
92.74 ± 3.88
67.01 ± 4.31
66.21 ± 5.44
GFPL(Ours)
70.22 ± 1.78
68.64 ± 2.55
96.48 ± 1.52
94.82 ± 2.85
70.33 ± 1.12
68.26 ± 2.02
Table 2: The effectiveness of GFPL’s components in performance improvements of CIFAR10
Figure 3: The influence of hyperparameters on model performance: (a) λ ; (b) GMM commponent number; (c)The average of shot.
Figure 4: The influence of hyperparameters on model performance: (a) w^ ; (b)Retraining interval; (c)Initial round of prototype interaction
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
MNIST
FEMNIST
CIFAR10
CIFAR100
Projection dim
50
128
80
80
Average shot sˉ
100
92
100
100
Learning rate η
0.01
0.01
0.01
0.01
λ
2
2
2
2
Batch size B
4
4
32
32
Num_channels
1
1
3
3
Appendix
Table 3: Detailed hyperparameters of GFPL
Figure 5: Error bars of test accuracy on different datasets: (a-c) MNIST dataset with different wˉ values; (d-f) FEMNIST dataset with different wˉ values; (g-i) CIFAR10 dataset with different wˉ .
Figure 6: The influence of hyperparameters SC on average test accuracy and communication overhead
Figure 7: The influence of hyperparameters on communication overhead: (a) The number of GMM components n ; (b) The number of clients m ; (c) The average of class number for all clients.
Federated Learning (FL) aims at unburdening the training of deep models by distributing computation across multiple devices (clients) while safeguarding data privacy. On top of that, Federated Continual Learning (FCL) also accounts for data distribution evolving over time, mirroring the dynamic nature of real-world environments. While previous studies have identified Catastrophic Forgetting and Client Drift as major factors of performance degradation in FCL, we shed light on the importance of Incremental Bias and Federated Bias, which cause models to prioritize classes that are recently introduced or locally predominant, respectively. Our proposal constrains both biases to the last layer by efficiently fine-tuning a pre-trained backbone using learnable prompts, resulting in clients that produce less biased representations and more biased classifiers. Therefore, instead of solely relying on parameter aggregation, we leverage generative prototypes to effectively balance the predictions of the global model. Our proposed methodology significantly improves the current state of the art across six datasets, each including three different scenarios.
Riccardo Salami, Pietro Buzzega, Matteo Mosconi +2
AImageLab - University of Modena and Reggio Emilia, Modena, Italy
Federated learning (FL) enables collaborative learning of computer vision models, where privacy and regulatory constraints prevent centralizing data across devices or organizations. However, practical FL deployments often exhibit severe class imbalance and label skew, causing standard aggregation protocols to overfit dominant clients and degrade minority-class performance. We propose a data-free, class-wise contribution estimation and aggregation framework based on logit maximization (CELM) that does not require sharing raw data, client metadata, or auxiliary public datasets. The FL server probes client updates to obtain class-wise evidence scores and assembles a cross-client evidence matrix, which quantifies both per-class competence and class coverage. Using this matrix, we compute contribution weights that upweight clients providing strong, discriminative evidence for underrepresented classes. The resulting aggregation is stable due to simplex constraints and momentum smoothing, and it remains compatible with standard FL training pipelines. We evaluate the approach on representative vision benchmarks under controlled non-IID and pathological label splits, demonstrating that CELM-based aggregation improves robustness to imbalance and statistical heterogeneity, while yielding better performance without requiring any additional data exchange.
Asim Ukaye, Nurbek Tastan, Mubarak Abdu-Aguye +1
Mohamed bin Zayed University of Artificial Intelligence, UAE · Michigan State University, USA
Federated prototype learning (FPL), as a solution to heterogeneous federated learning (HFL), effectively alleviates the challenges of data and model heterogeneity.However, existing FPL methods fail to balance the fidelity and discriminability of the feature, and are limited by a single global prototype. In this paper, we propose FedDBP, a novel FPL method to address the above issues. On the client-side, we design a Dual-Branch feature projector that employs L2 alignment and contrastive learning simultaneously, thereby ensuring both the fidelity and discriminability of local features. On the server-side, we introduce a Personalized global prototype fusion approach that leverages Fisher information to identify the important channels of local prototypes. Extensive experiments demonstrate the superiority of FedDBP over ten existing advanced methods.
Ningzhi Gao, Siquan Huang, Leyu Shi +1
South China University of Technology, Guangzhou, China