Representation-Aligned Multi-Scale Personalization for Federated Learning

Wenfei Liang; Wee Peng Tay

cs.LGApr 13, 2026

Representation-Aligned Multi-Scale Personalization for Federated Learning

Authors: Wenfei Liang, Wee Peng Tay

Organizations: Department of Electrical and Electronic Engineering, Nanyang Technological University, Singapore.

Abstract

In federated learning (FL), accommodating clients with diverse resource constraints remains a significant challenge. A widely adopted approach is to use a shared full-size model, from which each client extracts a submodel aligned with its computational budget. However, regardless of the specific scoring strategy, these methods rely on the same global backbone, limiting both structural diversity and representational adaptation across clients. This paper presents FRAMP, a unified framework for personalized and resource-adaptive federated learning. Instead of relying on a fixed global model, FRAMP generates client-specific models from compact client descriptors, enabling fine-grained adaptation to both data characteristics and computational budgets. Each client trains a tailored lightweight submodel and aligns its learned representation with others to maintain global semantic consistency. Extensive experiments on vision and graph benchmarks demonstrate that FRAMP enhances generalization and adaptivity across a wide range of client settings.

Figures & tables

Figure 1: Cumulative distribution of preserved parameters across submodels of different sizes. Each curve represents the cumulative proportion of parameters selected (mask=1).

Figure 2: Client accuracy distribution of submodels across different sizes. Each box represents the variation in accuracy among clients.

Figure 3: Overview of the FRAMP framework. The server generates personalized full models using HN and extracts submodels via dynamic masking for each client. Clients train these submodels on local data and align class-level prototypes with server-aggregated global prototypes to ensure semantic consistency. Updates are used to refine the HN and global prototypes.

Method	CIFAR-10						CIFAR-100						ogbn-arxiv
Method	Local	1/64	1/16	1/4	1.0	Union	Local	1/64	1/16	1/4	1.0	Union	Local	Union
HeteroFL	68.88	60.24	69.32	72.18	73.76	66.05	31.75	27.24	29.80	33.52	36.44	30.67	31.53	31.26
FedRolex	67.18	54.60	64.96	70.08	79.08	65.98	31.67	21.00	30.84	36.44	38.40	29.89	28.26	27.98
ScaleFL	72.10	69.04	71.64	70.08	77.64	67.37	39.69	36.16	40.48	42.56	39.56	37.56	41.53	37.77
FIARSE	77.04	73.12	77.20	77.24	82.04	73.75	41.76	39.12	43.24	43.72	40.96	38.63	46.18	41.53
FRAMP	79.11	78.20	78.56	77.40	82.28	75.65	42.95	43.00	44.08	43.72	41.00	40.26	48.00	44.34

Table 1: Test accuracy of submodels across four sizes on CIFAR-10, CIFAR-100, and ogbn-arxiv.

Figure 4: Histograms on CIFAR-10 showing client counts at different test accuracy levels under four submodel sizes for each baseline. The curves depict the accuracy distribution for each model size.

Panel of Figure 5 — Figure 5: (a) Test accuracy of submodels with different sizes for all baselines, evaluated on the union test set of CIFAR-10. (b) Cumulative coverage of parameter masks for different submodel sizes in FRAMP, showing a more uniform and efficient utilization of full-model parameters across submodels, compared to \cref fig:lim_mask_overlap. (c) t-SNE visualization of local prototypes (dots, one per class per client) and global prototypes (stars) in FRAMP on CIFAR-10. Each color represents a class.

Figure 6: Test accuracy of submodels (across four model sizes) on CIFAR-10 and CIFAR-100, evaluated under three training settings: (1) training with all clients; (2) randomly excluding 20% of clients from each model size group; (3) excluding only clients with the smallest model size ( $\gamma=1/64$ ).

Method	Local	1/64	1/16	1/4	1.0	Union
FRAMP	79.11	78.20	78.56	77.40	82.28	75.65
w/o RA (a)	74.49	75.16	74.12	72.44	76.24	72.40
w/o Per (b)	75.21	70.68	74.40	75.44	80.32	71.17
w/o Both (c)	73.97	68.68	74.40	74.72	78.08	70.84
Layerwise (d)	67.79	63.00	69.04	67.52	71.60	65.46
MLP (e)	67.12	62.92	66.96	65.68	72.92	64.12

Table 2: Ablation study on CIFAR-10.

Method	HeteroFL	FedRolex	ScaleFL	FIARSE	FRAMP
Rnd 1	24.74	54.05	25.45	26.89	37.96
Rnd 2+	8.01	8.09	8.35	7.67	7.98

Table 3: Server training time per communication round on CIFAR-10 (in milliseconds).

	No Noise	GN $a=0.01$	GN $a=0.1$	GN $a=0.5$	Random Rotation
Local	42.95	42.40	41.45	41.25	41.73
Union	40.26	40.26	40.04	38.41	39.02

Table 4: Test accuracy under Gaussian noise and random rotation on CIFAR-100.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

	CIFAR-10	CIFAR-100	ogbn-arxiv
Local Epochs	100	100	120
Batch Size	32	32	-
Communication Rounds	800	800	200
Learning rate	{0.1, 0.5}	{0.1, 0.4}	{0.1}
HN Learning rate	0.12	0.08	0.1
$\lambda$	0.7	0.7	0.2

Appendix

Table 5: Hyperparameter Settings

Method	CIFAR-100
Method	Local	0.04	0.16	0.36	0.64	1.0	Union
HeteroFL	35.01	30.15	33.60	36.10	37.75	37.45	32.60
FedRolex	38.51	28.40	37.75	41.10	42.10	43.19	36.09
ScaleFL	42.43	42.90	44.75	42.49	42.05	39.95	40.21
FIARSE	45.94	44.35	45.65	47.80	47.00	44.90	42.61
FRAMP	46.71	46.75	46.65	47.85	47.35	44.95	43.67

Appendix

Table 6: Test accuracy of submodels across five submodel sizes.

Method	CIFAR-100
Method	Local	1/64	1/16	1/4	1.0	Union
HeteroFL	22.24	17.28	21.96	22.16	27.56	21.48
FedRolex	12.63	3.08	5.84	16.12	25.48	11.67
ScaleFL	29.14	28.68	32.04	28.64	27.20	27.71
FIARSE	29.29	25.68	31.84	29.24	30.40	29.25
FRAMP	30.92	31.32	33.12	28.72	30.52	30.10

Appendix

Table 7: Test accuracy on CIFAR-100 with stronger data heterogeneity (

\alpha=0.05

Method	CIFAR-100
Method	Local	1/64	1/16	1/4	1.0	Union
HeteroFL	25.80	20.68	23.92	29.36	29.24	23.79
FedRolex	16.06	2.56	8.12	23.32	30.24	15.04
ScaleFL	33.74	31.36	35.08	37.76	30.76	31.61
FIARSE	34.64	29.92	36.04	40.08	32.52	34.10
FRAMP	36.57	36.56	38.16	38.96	32.60	35.40

Appendix

Table 8: Test accuracy on CIFAR-100 with stronger data heterogeneity (

\alpha=0.1

Method	CIFAR-100
Method	Local	1/64	1/16	1/4	1.0	Union
HeteroFL	30.73	25.64	28.36	33.16	35.76	29.18
FedRolex	19.06	3.28	10.52	27.28	35.16	18.81
ScaleFL	40.64	35.48	43.16	43.20	40.72	37.30
FIARSE	41.13	37.88	43.08	43.80	39.76	38.91
FRAMP	42.22	40.96	43.20	43.88	40.84	40.17

Appendix

Table 9: Test accuracy on CIFAR-100 with milder data heterogeneity (

\alpha=0.5

Method	CIFAR-100
Method	Local	1/64	1/16	1/4	1.0	Union
HeteroFL	31.16	28.60	28.80	31.40	35.84	29.69
FedRolex	19.06	3.12	11.76	26.80	34.56	18.80
ScaleFL	42.05	39.96	43.28	43.44	41.52	38.44
FIARSE	40.31	38.20	41.44	41.24	40.36	37.71
FRAMP	43.25	44.84	44.32	42.72	41.12	40.53

Appendix

Table 10: Test accuracy on CIFAR-100 with milder data heterogeneity (

\alpha=0.7

Method	CIFAR-100
Method	Local	1/64	1/16	1/4	1.0	Union
HeteroFL	31.87	26.88	31.40	33.36	35.84	29.11
FedRolex	20.55	2.68	12.24	30.16	37.12	19.31
ScaleFL	44.37	41.80	44.76	46.52	44.40	41.15
FIARSE	42.38	40.12	43.56	44.52	41.32	40.63
FRAMP	46.20	45.88	48.32	45.84	44.76	43.06

Appendix

Table 11: Test accuracy on CIFAR-100 with IID split.

Method	CIFAR-100
Method	Local	1/64	1/16	1/4	1.0	Union
HeteroFL	32.23	28.32	31.52	33.96	35.12	30.30
FedRolex	33.00	21.36	34.12	36.72	39.80	31.33
ScaleFL	39.57	37.92	39.60	41.84	38.92	37.63
FIARSE	42.27	40.32	43.28	43.52	41.96	38.97
FRAMP	42.72	41.80	44.80	43.28	41.00	39.59

Appendix

Table 12: Test accuracy of submodels across four submodel sizes with 20% participation rate.

Figure 7: Test accuracy across different model sizes (Local, 1/64, 1/16, 1/4, 1.0, Union) under varying values of the alignment weight $\lambda$ .

Method	CIFAR-100
Method	Local	1/64	1/16	1/4	1.0	Union
NoNoise	42.95	43.00	44.08	43.72	41.00	40.26
GN $a=0.01$	42.40	43.36	44.64	41.80	39.80	40.26
GN $a=0.1$	41.45	43.20	43.04	41.12	38.44	40.04
GN $a=0.5$	41.25	43.04	43.32	41.00	37.64	38.41
Random Rotation	41.73	43.52	44.24	40.84	38.32	39.02

Appendix

Table 13: Test accuracy under prototype perturbations on CIFAR-100.

Figure 8: Client accuracy distribution of submodels across different sizes. Each box represents the variation in accuracy among clients.

Federated Learning

Federated learning trains shared models across data holders without requiring them to pool their raw data.

See all 943 papers

Few-for-Many Personalized Federated Learning
Mar 12, 2026Ping Guo, Tiantian Zhang, Xi Lin +3Federated Learning Heterogeneous Federated Learning
City University of Hong Kong · CityU Shenzhen Research Institute · Hong Kong Metropolitan University +3
Separate Aggregation of Split Network for Personalized Federated Learning
May 26, 2026Yunseok Kang, Jaeyoung SongFederated Learning Aggregation Split Federated Learning
Department of Electronics Engineering, Pusan National University, Republic of Korea
FedLAFP: Low-Rank Aggregation Meets Full-Rank Personalization in Federated Fine-Tuning
Sep 29, 2026Mengjun Yi, Huaian Gu, Yinghao Ai +2Low-Rank Adaptation Federated Fine-Tuning
State Key Laboratory for Novel Software Technology · School of Artificial Intelligence · School of Computer Science +1

Mar 12, 2026J/K move · Enter open · S save

Few-for-Many Personalized Federated Learning

Ping Guo, Tiantian Zhang, Xi Lin, Xiang Li+2

City University of Hong Kong · CityU Shenzhen Research Institute · Hong Kong Metropolitan University+3

Federated Learning Heterogeneous Federated Learning Multi-Objective Optimization

Open paper AskPDF arXiv

Personalized Federated Learning (PFL) aims to train customized models for clients with highly heterogeneous data distributions while preserving data privacy. Existing approaches often rely on heuristics like clustering or model interpolation, which lack principled mechanisms for balancing heterogeneous client objectives. Serving

M

clients with distinct data distributions is inherently a multi-objective optimization problem, where achieving optimal personalization ideally requires

M

distinct models on the Pareto front. However, maintaining

M

separate models poses significant scalability challenges in federated settings with hundreds or thousands of clients. To address this challenge, we reformulate PFL as a few-for-many optimization problem that maintains only

K

shared server models (

K \ll M

) to collectively serve all

M

clients. We prove that this framework achieves near-optimal personalization: the approximation error diminishes as

K

increases and each client's model converges to each client's optimum as data grows. Building on this reformulation, we propose FedFew, a practical algorithm that jointly optimizes the

K

server models through efficient gradient-based updates. Unlike clustering-based approaches that require manual client partitioning or interpolation-based methods that demand careful hyperparameter tuning, FedFew automatically discovers the optimal model diversity through its optimization process. Experiments across vision, NLP, and real-world medical imaging datasets demonstrate that FedFew, with just 3 models, consistently outperforms other state-of-the-art approaches. Code is available at https://github.com/pgg3/FedFew.

Abstract

Federated Learning

Explore similar work

Few-for-Many Personalized Federated Learning

Separate Aggregation of Split Network for Personalized Federated Learning

FedLAFP: Low-Rank Aggregation Meets Full-Rank Personalization in Federated Fine-Tuning