Personalized Federated Learning via Gaussian Generative Modeling

Peng Hu; Jianwei Ma

cs.LGMar 12, 2026

Personalized Federated Learning via Gaussian Generative Modeling

Authors: Peng Hu, Jianwei Ma

Organizations: P. Hu is with the School of Mathematics, Harbin Institute of Technology, Harbin, China. · J. Ma is with the Institute for Artificial Intelligence and the School of Earth and Space Sciences, Peking University, Beijing, China, and also with the Institute for Artificial Intelligence and the School of Mathematics, Harbin Institute of Technology, Harbin, China.

Abstract

Federated learning has emerged as a paradigm to train models collaboratively on inherently distributed client data while safeguarding privacy. In this context, personalized federated learning tackles the challenge of data heterogeneity by equipping each client with a dedicated model. A prevalent strategy decouples the model into a shared feature extractor and a personalized classifier head, where the latter actively guides the representation learning. However, previous works have focused on classifier head-guided personalization, neglecting the potential personalized characteristics in the representation distribution. Building on this insight, we propose pFedGM, a method based on Gaussian generative modeling. The approach begins by training a Gaussian generator that models client heterogeneity via weighted re-sampling. A balance between global collaboration and personalization is then struck by employing a dual objective: a shared objective that maximizes inter-class distance across clients, and a local objective that minimizes intra-class distance within them. To achieve this, we decouple the conventional Gaussian classifier into a navigator for global optimization, and a statistic extractor for capturing distributional statistics. Inspired by the Kalman gain, the algorithm then employs a dual-scale fusion framework at global and local levels to equip each client with a personalized classifier head. In this framework, we model the global representation distribution as a prior and the client-specific data as the likelihood, enabling Bayesian inference for class probability estimation. The evaluation covers a comprehensive range of scenarios: heterogeneity in class counts, environmental corruption, and multiple benchmark datasets and configurations. pFedGM achieves superior or competitive performance compared to state-of-the-art methods.

Figures & tables

Fig. 1 : t-SNE visualization of representation distribution divergence caused by client data heterogeneity. (a) Representations of two classes from two clients; (b) Colored by class; (c, d) Colored by client. Subfigure (c) displays the feature representations of class 1, while (d) displays those of class 2. Different clients exhibit distinct cluster means and covariance structures.

Fig. 2 : Overview of the generator training. (Left) Data from different clients of the same class exhibit heterogeneity, and consequently, their distributions diverge in the representation space. (Middle) The shared objective drives features of different classes to diverge along distinct directions, whereas (right) the client personalized objective prompts features of the same class to aggregate around client-specific centers.

Fig. 3 : Global-local collaborative training module. In this process, the shared and local objectives jointly optimize the generator. The navigator defines and is refined by the shared objective, while a statistics extractor captures global statistics.

Fig. 4 : Illustration of parameter decoupling. After decoupling the conventional Gaussian classifier into a navigator and covariance extractor (Covariance for short), these along with the generator are exchanged between server and clients. The navigator is utilized to generate the shared objective, and the covariance features are employed to craft the subsequent personalized classifier heads.

Fig. 5 : Navigational direction and local class prototype adaptation. (Left) The navigational direction (

\bm{\mu}_{i}

) self-adjusts based on the global representation distribution, with the objective of aligning with its own class centroid and diverging from others. (Right) The local class prototype (

\bm{\upsilon}_{i,k}

) continuously adapts toward the class mean.

Fig. 6 : Local classifier adaptation. Global information, encompassed by the navigator and covariance, is fused with local information for the joint construction of the local classifier.

Fig. 7 : The client data distribution of CIFAR-10S (all data corrupted). Different shades of the same color represent different levels of corruption, with darker shades indicating more severe corruption.

Dataset	EMNIST		CIFAR-10		CIFAR-100		TinyImageNet
Partition	Dir(0.1)	Dir(0.5)	Dir(0.1)	Dir(0.5)	Dir(0.1)	Dir(0.5)	Dir(0.1)	Dir(0.5)
Local	86.56 ( 12 )	71.00 ( 11 )	86.43 ( 13 )	58.64 ( 13 )	36.61 ( 8.9 )	15.13 ( 4.8 )	25.48 ( 7.0 )	7.07 ( 2.7 )
FedAvg	82.51 ( 13 )	83.61 ( 8.2 )	53.13 ( 11 )	61.53 ( 6.7 )	23.79 ( 5.4 )	24.59 ( 4.3 )	21.37 ( 4.2 )	19.54 ( 3.0 )
FedAvgFT	95.02 ( 5.3 )	88.73 ( 5.6 )	89.46 ( 10 )	74.13 ( 8.5 )	49.47 ( 8.5 )	30.13 ( 4.5 )	41.15 ( 6.1 )	21.41 ( 3.1 )
Ditto	94.71 ( 5.9 )	88.53 ( 5.7 )	89.60 ( 10 )	73.60 ( 8.2 )	49.54 ( 8.6 )	30.12 ( 4.9 )	42.30 ( 5.9 )	23.29 ( 3.4 )
FedBABU	91.43 ( 8.4 )	82.85 ( 8.3 )	87.63 ( 14 )	72.50 ( 9.1 )	43.37 ( 8.4 )	26.13 ( 5.2 )	39.40 ( 5.6 )	20.77 ( 3.0 )

TABLE I : Average (standard deviation) test accuracy (%) on multiple datasets.

Dataset	CIFAR-10S Dir(0.1)		CIFAR-10S Dir(0.5)		CIFAR-100S Dir(0.1)		CIFAR-100S Dir(0.5)
Corruption	50C.	100C.	50C.	100C.	50C.	100C.	50C.	100C.
Local	85.80 ( 14 )	86.52 ( 13 )	57.82 ( 13 )	56.83 ( 13 )	36.04 ( 9.8 )	35.51 ( 10 )	14.13 ( 4.7 )	14.07 ( 4.7 )
FedAvg	51.91 ( 13 )	50.34 ( 14 )	57.86 ( 10 )	55.63 ( 10 )	21.39 ( 6.9 )	19.85 ( 6.5 )	23.43 ( 5.3 )	20.99 ( 6.0 )
FedAvgFT	89.67 ( 10 )	89.63 ( 9.5 )	72.38 ( 8.2 )	71.47 ( 9.6 )	48.51 ( 8.3 )	46.99 ( 7.9 )	29.42 ( 5.4 )	27.90 ( 5.5 )
Ditto	88.67 ( 11 )	89.15 ( 9.9 )	72.01 ( 8.5 )	70.59 ( 9.2 )	48.11 ( 8.5 )	47.90 ( 8.6 )	30.22 ( 5.6 )	28.73 ( 5.9 )
FedBABU	87.00 ( 14 )	86.70 ( 13 )	69.67 ( 12 )	68.17 ( 10 )	41.05 ( 11 )	35.14 ( 10 )	24.21 ( 5.3 )	23.19 ( 5.5 )

TABLE II : Average (standard deviation) test accuracy (%) on CIFAR-10S/100S (environmental heterogeneity setting).

Fig. 8 : t-SNE visualization of the feature representations. Each subplot depicts data from different clients of a single class in the representation space, revealing that clients within the same class exhibit distinct clustering structures.

Dataset		FedAvg	FedAvgFT	FedBAU	pFedMe	LG-FedAvg	FedPAC	pFedFDA	pFedGM
Original Clients		57.64 ( 7.8 )	72.38 ( 8.4 )	69.18 ( 9.5 )	68.25 ( 9.0 )	63.25 ( 11 )	73.70 ( 8.7 )	73.96 ( 8.6 )	74.80 ( 7.3 )
New Clients	clean	59.18 ( 3.9 )	72.31 ( 9.2 )	61.87 ( 19 )	67.25 ( 12 )	63.77 ( 17 )	72.15 ( 10 )	72.94 ( 11 )	72.94 ( 13 )
	motion	49.08 ( 9.9 )	68.62 ( 9.7 )	62.31 ( 10 )	64.92 ( 8.1 )	58.00 ( 8.3 )	70.31 ( 7.1 )	68.31 ( 6.4 )	70.92 ( 5.7 )
	defocus	59.80 ( 6.0 )	74.62 ( 8.5 )	69.34 ( 4.3 )	69.34 ( 8.6 )	64.74 ( 10 )	74.28 ( 9.0 )	72.74 ( 9.5 )	75.30 ( 6.3 )
	gaussian	57.44 ( 6.9 )	69.12 ( 12 )	68.80 ( 12 )	68.16 ( 13 )	62.88 ( 14 )	73.76 ( 12 )	72.00 ( 12 )	74.72 ( 8.9 )
	shot	57.48 ( 3.5 )	67.44 ( 9.0 )	50.66 ( 20 )	66.94 ( 6.8 )	59.97 ( 9.3 )	72.43 ( 7.7 )	70.60 ( 5.4 )	72.26 ( 6.1 )

TABLE III : Evaluation of new-client generalization on CIFAR-10 Dir(0.5).

Fig. 9 : The variation of test accuracy during the training process with sampling rate

q=0.1

Dataset	EMNIST		CIFAR-10		CIFAR-100		TinyImageNet
Partition	Dir ( 0.1 )	Dir ( 0.5 )	Dir ( 0.1 )	Dir ( 0.5 )	Dir ( 0.1 )	Dir ( 0.5 )	Dir ( 0.1 )	Dir ( 0.5 )
Local	91.50 ( 8.8 )	81.71 ( 7.7 )	90.28 ( 12 )	68.19 ( 9.2 )	47.75 ( 4.7 )	25.95 ( 4.2 )	36.70 ( 4.0 )	16.42 ( 1.6 )
FedAvg	83.18 ( 14 )	85.25 ( 7.9 )	55.60 ( 13 )	68.25 ( 5.8 )	31.28 ( 6.0 )	35.15 ( 2.8 )	27.91 ( 4.3 )	29.56 ( 1.7 )
FedAvgFT	96.16 ( 4.7 )	90.79 ( 4.8 )	91.87 ( 8.3 )	79.14 ( 6.3 )	58.46 ( 3.9 )	43.44 ( 2.9 )	50.13 ( 3.1 )	36.57 ( 1.6 )
FedPAC	96.03 ( 4.5 )	90.74 ( 4.8 )	92.08 ( 9.2 )	79.32 ( 6.2 )	59.51 ( 4.5 )	44.87 ( 3.4 )	52.34 ( 2.2 )	36.21 ( 1.8 )
pFedFDA	96.43 ( 4.3 )	91.45 ( 4.6 )	91.84 ( 8.1 )	79.65 ( 6.5 )	57.10 ( 4.2 )	47.60 ( 2.8 )	47.36 ( 3.3 )	37.15 ( 1.6 )

TABLE IV : Comparison of average (standard deviation) test accuracy (%) under moderate settings.

Dataset	FedAvg	Ditto	FedPer	FedRep	FedBABU	pFedMe	LG-FedAvg	FedPAC	pFedFDA	pFedGM
CIFAR-10	64.91	96.08	61.94	67.03	62.34	102.21	62.68	113.31	164.62	81.75
CIFAR-100	65.85	104.33	68.25	68.97	63.99	114.24	61.22	167.72	172.15	89.23

TABLE V : Comparison of system runtime (min) on the CIFAR-10/100 dataset.

Ablation Strategy			EMNIST		CIFAR-10		CIFAR-100		TinyImageNet		Acc. Avg. (Improv. Rate)
NA	FA	GA	Dir(0.1)	Dir(0.5)	Dir(0.1)	Dir(0.5)	Dir(0.1)	Dir(0.5)	Dir(0.1)	Dir(0.5)	Acc. Avg. (Improv. Rate)
$\checkmark$			80.86	83.58	55.32	63.59	25.32	30.52	23.52	26.31	48.63 ( $0\%\uparrow$ )
	$\checkmark$		95.08	90.12	90.88	77.02	53.40	41.44	48.46	34.72	66.39 ( $36.52\%\uparrow$ )
		$\checkmark$	95.59	90.44	91.00	77.49	57.56	42.25	51.17	35.45	67.62 ( $1.85\%\uparrow$ )
Ablation Strategy			CIFAR-10S Dir(0.1)		CIFAR-10S Dir(0.5)		CIFAR-100S Dir(0.1)		CIFAR-100S Dir(0.5)		Acc. Avg. (Improv. Rate)
NA	FA	GA	50C.	100C.	50C.	100C.	50C.	100C.	50C.	100C.	Acc. Avg. (Improv. Rate)

TABLE VI : Ablation study on the classifier head. The ablation settings are denoted as follows: NA (No Adaptation) applies no personalized classifier head adaptation; FA (Fine-tuning Adaptation) employs only fine-tuning of the classifier head; and GA (Granular Adaptation) further incorporates a fine-grained adjustment of

\bm{b}_{i}

following the fine-tuning step.

Fig. 10 : Ablation study on the personalized objective function. NPO and WPO denote training without and with the personalized term, respectively.

Dataset	CIFAR-10S		CIFAR-100S
Partition	Dir(0.1)	Dir(0.5)	Dir(0.1)	Dir(0.5)
G.M.	89.32 ( 9.7 )	73.92 ( 8.5 )	52.46 ( 8.4 )	36.15 ( 6.4 )
G.M. - C.	89.63 ( 9.8 )	74.02 ( 8.3 )	51.65 ( 9.0 )	36.01 ( 6.6 )
G.M. + D.	89.67 ( 9.5 )	74.26 ( 8.4 )	53.72 ( 8.3 )	36.58 ( 6.6 )

TABLE VII : Ablation study on the influence of shared objective and statistic extractor. G.M. denotes training without the decoupling mechanism; G.M. - C. denotes using the decoupling mechanism but removing the statistic extractor; and G.M. + D. represents our proposed method.

Federated Learning

Federated learning trains shared models across data holders without requiring them to pool their raw data.

See all 939 papers

May 9, 2026

FedGMI: Generative Model-Driven Federated Learning for Probabilistic Mixture Inference

Federated Learning (FL) facilitates collaborative model training across decentralized clients while preserving data privacy by avoiding raw data exchange. Despite its potential, FL performance is often compromised by data heterogeneity across clients. To address this, Clustered Federated Learning (CFL) groups clients with similar data distributions to improve model performance, but constrained by intra-cluster heterogeneity. Conversely, Personalized Federated Learning (PFL) tailors models to individual clients, but usually neglects the underlying structural similarities among clients. In this work, we investigate a probabilistic mixture (PM) scenario, where each client's local data distribution is modeled as a convex combination of several shared inherent distributions. To effectively model this structure, we propose FedGMI, a framework that utilizes Variational Autoencoders (VAEs) as generative density estimators to represent these inherent distributions and infer the mixture components of clients' local data distributions. This approach enables structured personalization without sacrificing the benefits of collaborative learning. Extensive experiments demonstrate that FedGMI effectively characterizes and discriminate the inherent distributions, as well as accurately estimates mixture proportions. Furthermore, FedGMI maintains robust performance even under communication cost constraints.

Qijun Hou, Yuchen Shi, Pingyi Fan +1

Dept. of Electronic Engineering, BNRist, Tsinghua University. · Dept. of Electronic and Computer Engineering, HKUST.

Federated Learning Heterogeneous Federated Learning

Ask PDF

May 26, 2026

Separate Aggregation of Split Network for Personalized Federated Learning

Federated learning enables collaborative model training without sharing raw data, but its performance can degrade substantially under heterogeneous client data distributions. A single global model often cannot satisfy diverse client requirements, so personalized federated learning has therefore been explored to improve client specific performance while preserving global generalization. Existing PFL methods often face a fundamental tradeoff in which stronger global sharing can undermine local specialization, whereas stronger local adaptation can lead to overfitting under limited data, label imbalance, and missing class scenarios. In this work, we propose PGFedSplit, a personalized federated learning framework that improves both personalization and global generalization under severe client heterogeneity. PGFedSplit adopts a split architecture and performs adaptive aggregation scheduling tailored to the roles of different model components, enabling stable knowledge sharing while maintaining client specific adaptation. Each client further leverages a mixture of locally extracted representations and synthetic representations generated from server side Gaussian statistics, improving robustness under label imbalance and missing class conditions. Extensive experiments on Fashion MNIST, CIFAR 10, CIFAR 100, and Tiny ImageNet demonstrate consistent improvements over state of the art PFL methods, with stable convergence and superior personalization in highly heterogeneous settings.

Yunseok Kang, Jaeyoung Song

Department of Electronics Engineering, Pusan National University, Republic of Korea

Federated Learning Aggregation Split Federated Learning Federated Learning Heterogeneous Federated Learning

Ask PDF

Mar 12, 2026

Few-for-Many Personalized Federated Learning

Personalized Federated Learning (PFL) aims to train customized models for clients with highly heterogeneous data distributions while preserving data privacy. Existing approaches often rely on heuristics like clustering or model interpolation, which lack principled mechanisms for balancing heterogeneous client objectives. Serving

M

clients with distinct data distributions is inherently a multi-objective optimization problem, where achieving optimal personalization ideally requires

M

distinct models on the Pareto front. However, maintaining

M

separate models poses significant scalability challenges in federated settings with hundreds or thousands of clients. To address this challenge, we reformulate PFL as a few-for-many optimization problem that maintains only

K

shared server models (

K \ll M

) to collectively serve all

M

clients. We prove that this framework achieves near-optimal personalization: the approximation error diminishes as

K

increases and each client's model converges to each client's optimum as data grows. Building on this reformulation, we propose FedFew, a practical algorithm that jointly optimizes the

K

server models through efficient gradient-based updates. Unlike clustering-based approaches that require manual client partitioning or interpolation-based methods that demand careful hyperparameter tuning, FedFew automatically discovers the optimal model diversity through its optimization process. Experiments across vision, NLP, and real-world medical imaging datasets demonstrate that FedFew, with just 3 models, consistently outperforms other state-of-the-art approaches. Code is available at https://github.com/pgg3/FedFew.

Ping Guo, Tiantian Zhang, Xi Lin +3

City University of Hong Kong · CityU Shenzhen Research Institute · Hong Kong Metropolitan University +3

Federated Learning Heterogeneous Federated Learning Multi-Objective Optimization

Ask PDF

Abstract

Federated Learning

Explore similar work

FedGMI: Generative Model-Driven Federated Learning for Probabilistic Mixture Inference

Separate Aggregation of Split Network for Personalized Federated Learning

Few-for-Many Personalized Federated Learning