GFPL: Generative Federated Prototype Learning for Resource-Constrained and Data-Imbalanced Vision Task

Shiwei Lu; Yuhang He; Jiashuo Li; Qiang Wang; Yihong Gong

cs.CVFeb 25, 2026

GFPL: Generative Federated Prototype Learning for Resource-Constrained and Data-Imbalanced Vision Task

Authors: Shiwei Lu, Yuhang He, Jiashuo Li, Qiang Wang, Yihong Gong

Organizations: Xi’an Jiaotong University · Air Force Engineering University

Abstract

Federated learning (FL) facilitates the secure utilization of decentralized images, advancing applications in medical image recognition and autonomous driving. However, conventional FL faces two critical challenges in real-world deployment: ineffective knowledge fusion caused by model updates biased toward majority-class features, and prohibitive communication overhead due to frequent transmissions of high-dimensional model parameters. Inspired by the human brain's efficiency in knowledge integration, we propose a novel Generative Federated Prototype Learning (GFPL) framework to address these issues. Within this framework, a prototype generation method based on Gaussian Mixture Model (GMM) captures the statistical information of class-wise features, while a prototype aggregation strategy using Bhattacharyya distance effectively fuses semantically similar knowledge across clients. In addition, these fused prototypes are leveraged to generate pseudo-features, thereby mitigating feature distribution imbalance across clients. To further enhance feature alignment during local training, we devise a dual-classifier architecture, optimized via a hybrid loss combining Dot Regression and Cross-Entropy. Extensive experiments on benchmarks show that GFPL improves model accuracy by 3.6% under imbalanced data settings while maintaining low communication cost.

Figures & tables

Figure 1: Generative Federated Prototype Learning (GFPL):(a) the client trains feature extractor, projection layers through local data, while generating local prototypes with GMM and uploading them to the server for prototype interaction; (b) After receiving global prototypes, the client replaces local prototypes with global prototypes and further generates balanced pseudo features through GMM sampling, thereby retraining the model projection layer.

Panel of Figure 2 — Figure 2: Visualization of features and projections under different loss functions: (a) only $\mathcal{L}_{DR}$ (features); (b) $\mathcal{L}_{DR}+\mathcal{L}_{CE}$ (features); (c) only $\mathcal{L}_{DR}$ (projections); (d) $\mathcal{L}_{DR}+\mathcal{L}_{CE}$ (projections)

Dataset	Method	$(\hat{w},\hat{s})$	Average Test Accuracy(%)			Comm. Round	Comm. Param. $(\times 10^{3})$
Dataset	Method	$(\hat{w},\hat{s})$	$\bar{w}=3$	$\bar{w}=4$	$\bar{w}=5$	Comm. Round	Comm. Param. $(\times 10^{3})$
MNIST	Local	(2,2)	94.05 $\pm$ 2.93	93.35 $\pm$ 3.26	92.92 $\pm$ 3.17	0	0
	FeSEM (2020)	(2,2)	95.26 $\pm$ 3.48	97.06 $\pm$ 2.72	96.31 $\pm$ 2.41	150	430
	FedProx (2020)	(2,2)	96.26 $\pm$ 2.89	96.40 $\pm$ 3.33	95.65 $\pm$ 3.38	110	430
	FedPer (2019)	(2,2)	95.57 $\pm$ 2.96	96.44 $\pm$ 2.62	95.55 $\pm$ 3.13	100	106
	FedAvg (2017)	(2,2)	95.04 $\pm$ 6.48	94.32 $\pm$ 4.89	93.22 $\pm$ 4.39	150	430

Table 1: Comparison of Federated Learning Algorithms

Variant	CIFAR10( $\hat{s}=1$ )	CIFAR10( $\hat{s}=20$ )	FEMNIST( $\hat{s}=1$ )	FEMNIST( $\hat{s}=20$ )	CIFAR100( $\hat{s}=1$ )	CIFAR100( $\hat{s}=20$ )
GFPL w/o DCS and PFG	55.62 $\pm$ 7.15	54.23 $\pm$ 9.88	91.16 $\pm$ 5.64	90.22 $\pm$ 7.84	53.51 $\pm$ 8.75	50.31 $\pm$ 9.84
GFPL w/o DCS	64.47 $\pm$ 3.36	63.28 $\pm$ 5.77	93.25 $\pm$ 4.22	92.33 $\pm$ 6.22	62.34 $\pm$ 6.33	60.63 $\pm$ 7.23
GFPL w/o PFG	67.24 $\pm$ 2.88	66.01 $\pm$ 3.95	94.37 $\pm$ 3.43	92.74 $\pm$ 3.88	67.01 $\pm$ 4.31	66.21 $\pm$ 5.44
GFPL(Ours)	70.22 $\pm$ 1.78	68.64 $\pm$ 2.55	96.48 $\pm$ 1.52	94.82 $\pm$ 2.85	70.33 $\pm$ 1.12	68.26 $\pm$ 2.02

Table 2: The effectiveness of GFPL’s components in performance improvements of CIFAR10

Panel of Figure 3 — Figure 3: The influence of hyperparameters on model performance: (a) $\lambda$ ; (b) GMM commponent number; (c)The average of shot.

Panel of Figure 4 — Figure 4: The influence of hyperparameters on model performance: (a) $\hat{w}$ ; (b)Retraining interval; (c)Initial round of prototype interaction

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

	MNIST	FEMNIST	CIFAR10	CIFAR100
Projection dim	50	128	80	80
Average shot $\bar{s}$	100	92	100	100
Learning rate $\eta$	0.01	0.01	0.01	0.01
$\lambda$	2	2	2	2
Batch size $B$	4	4	32	32
Num_channels	1	1	3	3

Appendix

Table 3: Detailed hyperparameters of GFPL

Panel of Figure 5 — Figure 5: Error bars of test accuracy on different datasets: (a-c) MNIST dataset with different $\bar{w}$ values; (d-f) FEMNIST dataset with different $\bar{w}$ values; (g-i) CIFAR10 dataset with different $\bar{w}$ .

Panel of Figure 6 — Figure 6: The influence of hyperparameters $S_{C}$ on average test accuracy and communication overhead

Panel of Figure 7 — Figure 7: The influence of hyperparameters on communication overhead: (a) The number of GMM components $n$ ; (b) The number of clients $m$ ; (c) The average of class number for all clients.

Federated Learning

Federated learning trains shared models across data holders without requiring them to pool their raw data.

See all 943 papers

Federated Class-Incremental Learning with Hierarchical Generative Prototypes
Jun 4, 2024Riccardo Salami, Pietro Buzzega, Matteo Mosconi +2Class-Incremental Learning Prototype-Based Classification
AImageLab - University of Modena and Reggio Emilia, Modena, Italy
Class-wise Contribution Estimation via Logit Maximization for Federated Learning
May 17, 2026Asim Ukaye, Nurbek Tastan, Mubarak Abdu-Aguye +1Class-Imbalanced Learning Federated Learning Aggregation
Mohamed bin Zayed University of Artificial Intelligence, UAE · Michigan State University, USA
FedDBP: Enhancing Federated Prototype Learning with Dual-Branch Features and Personalized Global Fusion
Mar 31, 2026Ningzhi Gao, Siquan Huang, Leyu Shi +1Federated Learning Heterogeneous Federated Learning
South China University of Technology, Guangzhou, China

Jun 4, 2024J/K move · Enter open · S save

Federated Class-Incremental Learning with Hierarchical Generative Prototypes

Riccardo Salami, Pietro Buzzega, Matteo Mosconi, Mattia Verasani+1

AImageLab - University of Modena and Reggio Emilia, Modena, Italy

Class-Incremental Learning Prototype-Based Classification Federated Continual Learning Federated Learning

Open paper AskPDF arXiv

Federated Learning (FL) aims at unburdening the training of deep models by distributing computation across multiple devices (clients) while safeguarding data privacy. On top of that, Federated Continual Learning (FCL) also accounts for data distribution evolving over time, mirroring the dynamic nature of real-world environments. While previous studies have identified Catastrophic Forgetting and Client Drift as major factors of performance degradation in FCL, we shed light on the importance of Incremental Bias and Federated Bias, which cause models to prioritize classes that are recently introduced or locally predominant, respectively. Our proposal constrains both biases to the last layer by efficiently fine-tuning a pre-trained backbone using learnable prompts, resulting in clients that produce less biased representations and more biased classifiers. Therefore, instead of solely relying on parameter aggregation, we leverage generative prototypes to effectively balance the predictions of the global model. Our proposed methodology significantly improves the current state of the art across six datasets, each including three different scenarios.

Abstract

Federated Learning

Explore similar work

Federated Class-Incremental Learning with Hierarchical Generative Prototypes

Class-wise Contribution Estimation via Logit Maximization for Federated Learning

FedDBP: Enhancing Federated Prototype Learning with Dual-Branch Features and Personalized Global Fusion