Sharpness-Aware Minimization for Generalized Embedding Learning in Federated Recommendation

Fengyuan Yu; Xiaohua Feng; Yuyuan Li; Changwang Zhang; Jun Wang; Chaochao Chen

cs.LGMar 12, 2026

Sharpness-Aware Minimization for Generalized Embedding Learning in Federated Recommendation

Authors: Fengyuan Yu, Xiaohua Feng, Yuyuan Li, Changwang Zhang, Jun Wang, Chaochao Chen

Organizations: Zhejiang University, Hangzhou, China · Hangzhou Dianzi University, Hangzhou, China · OPPO Research Institute, Shenzhen, China

Abstract

Federated recommender systems enable collaborative model training while keeping user interaction data local and sharing only essential model parameters, thereby mitigating privacy risks. However, existing methods overlook a critical issue, i.e., the stable learning of a generalized item embedding throughout the federated recommender system training process. Item embedding plays a central role in facilitating knowledge sharing across clients. Yet, under the cross-device setting, local data distributions exhibit significant heterogeneity and sparsity, exacerbating the difficulty of learning generalized embeddings. These factors make the stable learning of generalized item embeddings both indispensable for effective federated recommendation and inherently difficult to achieve. To fill this gap, we propose a new federated recommendation framework, named Federated Recommendation with Generalized Embedding Learning (FedRecGEL). We reformulate the federated recommendation problem from an item-centered perspective and cast it as a multi-task learning problem, aiming to learn generalized embeddings throughout the training procedure. Based on theoretical analysis, we employ sharpness-aware minimization to address the generalization problem, thereby stabilizing the training process and enhancing recommendation performance. Extensive experiments on four datasets demonstrate the effectiveness of FedRecGEL in significantly improving federated recommendation performance. Our code is available at https://github.com/anonymifish/FedRecGEL.

Figures & tables

Dataset	# Users (# Clients)	# Items	# Interactions	Sparsity
FilmTrust	1,227	2,059	34,889	98.62%
Lastfm-2K	1,600	12,454	185,650	99.07%
Amazon-Video	8,072	11,830	63,836	99.93%
QB-article	24,516	7,355	348,736	99.81%

Table 1. Dataset statistics.

Dataset	FilmTrust				Lastfm-2K
Methods	HR@5	NDCG@5	HR@10	NDCG@10	HR@5	NDCG@5	HR@10	NDCG@10
FedNCF	66.67 $\pm$ 0.30	53.09 $\pm$ 0.36	69.49 $\pm$ 0.17	54.20 $\pm$ 0.39	52.08 $\pm$ 0.74	38.98 $\pm$ 0.28	63.85 $\pm$ 0.49	42.82 $\pm$ 0.26
FedMF	65.23 $\pm$ 0.30	47.83 $\pm$ 0.35	69.38 $\pm$ 0.45	49.21 $\pm$ 0.40	73.10 $\pm$ 0.28	65.02 $\pm$ 0.16	78.63 $\pm$ 0.18	66.71 $\pm$ 0.23
PerFedRec	81.77 $\pm$ 1.64	66.35 $\pm$ 1.67	90.52 $\pm$ 0.91	76.95 $\pm$ 1.71	45.94 $\pm$ 1.07	37.83 $\pm$ 2.13	63.96 $\pm$ 3.70	44.24 $\pm$ 3.79
PFedRec	66.40 $\pm$ 0.33	54.30 $\pm$ 0.39	78.76 $\pm$ 0.14	64.83 $\pm$ 0.36	76.40 $\pm$ 0.45	70.59 $\pm$ 0.21	81.35 $\pm$ 0.62	72.34 $\pm$ 0.12
FedRAP	87.83 $\pm$ 0.77	73.80 $\pm$ 1.73	91.74 $\pm$ 0.10	77.67 $\pm$ 1.29	68.33 $\pm$ 0.91	64.55 $\pm$ 0.69	75.71 $\pm$ 0.31	71.53 $\pm$ 0.24

Table 2. Results of recommendation performance. Best results are typeset in bold with a colored background, e.g., 89.38

\pm

0.21 . Runner-up results use a different background color, e.g., 66.67

\pm

1.09 . We run all models 5 times and report the average results and standard deviation. Results are expressed as percentages (%).

Panel of Figure 1 — Figure 1. 3D surface plots of the post-convergence landscapes for models trained with FedRecGEL and with FedNCF.

Panel of Figure 2 — Figure 2. Effect of hyperparameter $\rho_{\mathrm{ur}}$ and $\rho_{\mathrm{co}}$ . We conduct experiments on all four datasets. We report HR@10 and NDCG@10 to represent the recommendation performance.

Dataset	FilmTrust				Lastfm-2K
Methods	HR@5	NDCG@5	HR@10	NDCG@10	HR@5	NDCG@5	HR@10	NDCG@10
FedRecGEL -w/o- non-share	89.02 $\pm$ 0.14	79.54 $\pm$ 2.17	91.63 $\pm$ 0.23	77.29 $\pm$ 2.20	71.88 $\pm$ 0.21	69.88 $\pm$ 0.34	70.19 $\pm$ 0.37	72.88 $\pm$ 2.18
FedRecGEL -w/o- shared	89.05 $\pm$ 0.17	81.23 $\pm$ 0.37	91.58 $\pm$ 0.20	81.72 $\pm$ 0.86	70.69 $\pm$ 0.31	69.09 $\pm$ 0.22	79.50 $\pm$ 0.12	72.26 $\pm$ 0.28
FedRecGEL	89.38 $\pm$ 0.21	81.33 $\pm$ 0.32	91.66 $\pm$ 0.10	82.01 $\pm$ 0.23	73.92 $\pm$ 0.22	70.79 $\pm$ 0.16	80.69 $\pm$ 0.25	73.56 $\pm$ 0.14
Dataset	Amazon-Video				QB-article
Methods	HR@5	NDCG@5	HR@10	NDCG@10	HR@5	NDCG@5	HR@10	NDCG@10

Table 3. Ablation study on non-shared and shared parts in SAM training. We compare (i) FedRecGEL -w/o- non-shared , which disables SAM on non-shared parts (user embeddings), keeping it on shared parts; (ii) FedRecGEL -w/o- shared , which disables SAM on shared parts (item embeddings and parameters), keeping it on non-shared parts; and (iii) FedRecGEL , the full model with SAM applied to both. Results are expressed as percentages (%).

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 3 — Figure 3. The contour map visualization of the post-convergence landscapes for models trained with FedRecGEL and with FedNCF across all four datasets.

Methods	Magnitude	FilmTrust	Lastfm-2K	Amazon-Video	QB-article
FedNCF	10	$\sim$ 2e19	0.6504	0.6211	0.4268
	1	$\sim$ 2e19	0.7506	0.6128	0.4179
	1e-1	0.2200	0.7662	0.6210	0.4037
	1e-2	0.2207	0.7678	0.6219	0.4047
	1e-3	0.2445	0.7679	0.6220	0.4051
	1e-4	0.2206	0.7679	0.6220	0.4051

Appendix

Table 4. Average loss values under different perturbation magnitudes for FedNCF and FedRecGEL across all datasets.

Recommender Systems

Recommender systems predict and rank items for people using interaction histories, content, context, and feedback.

See all 502 papers

Learning to Collaborate via Structures: Cluster-Guided Item Alignment for Federated Recommendation
Feb 25, 2026Yuchun Tu, Zhiwei Li, Bingli Sun +2Recommender Systems Federated Learning
Beihang University, Beijing, China · University of Technology Sydney, Sydney, Australia · University of Edinburgh, Edinburgh, UK
Personalized and Multi-View Representation for Federated Cold-Start Recommendation
Aug 28, 2026Jaehyung Lim, Wonbin Kweon, Woojoo Kim +3Recommender Systems Federated Learning
Pohang University of Science and Technology Pohang, Republic of Korea · University of Illinois Urbana-Champaign Champaign, IL, USA
FastPFRec: A Fast Personalized Federated Recommendation with Secure Sharing
Mar 18, 2026Zhenxing Yan, Jidong Yuan, Yongqi Sun +2Recommender Systems Federated Learning
Key Laboratory of Big Data & Artificial Intelligence in Transportation (Beijing Jiaotong University), Ministry of Education, Beijing, 100044, China · School of Computer Science and Technology, Beijing Jiaotong University, Beijing, 100044, China

Feb 25, 2026J/K move · Enter open · S save

Learning to Collaborate via Structures: Cluster-Guided Item Alignment for Federated Recommendation

Yuchun Tu, Zhiwei Li, Bingli Sun, Yixuan Li+1

Beihang University, Beijing, China · University of Technology Sydney, Sydney, Australia · University of Edinburgh, Edinburgh, UK

Recommender Systems Federated Learning Communication-Efficient Distributed Training

Open paper AskPDF arXiv

Federated recommendation facilitates collaborative model training across distributed clients while keeping sensitive user interaction data local. Conventional approaches typically rely on synchronizing high-dimensional item representations between the server and clients. This paradigm implicitly assumes that precise geometric alignment of embedding coordinates is necessary for collaboration across clients. We posit that establishing relative semantic relationships among items is more effective than enforcing shared representations. Specifically, global semantic relations serve as structural constraints for items. Within these constraints, the framework allows item representations to vary locally on each client, which flexibility enables the model to capture fine-grained user personalization while maintaining global consistency. To this end, we propose Cluster-Guided FedRec framework (CGFedRec), a framework that transforms uploaded embeddings into compact cluster labels. In this framework, the server functions as a global structure discoverer to learn item clusters and distributes only the resulting labels. This mechanism explicitly cuts off the downstream transmission of item embeddings, relieving clients from maintaining global shared item embeddings. Consequently, CGFedRec achieves the effective injection of global collaborative signals into local item representations without transmitting full embeddings. Extensive experiments demonstrate that our approach significantly improves communication efficiency while maintaining superior recommendation accuracy across multiple datasets.

Abstract

Recommender Systems

Explore similar work

Learning to Collaborate via Structures: Cluster-Guided Item Alignment for Federated Recommendation

Personalized and Multi-View Representation for Federated Cold-Start Recommendation

FastPFRec: A Fast Personalized Federated Recommendation with Secure Sharing