Federated Learning (FL) faces major challenges in real-world deployments due to statistical heterogeneity across clients and system heterogeneity arising from resource-constrained devices. While clustering-based approaches mitigate statistical heterogeneity and pruning techniques improve memory and communication efficiency, these strategies are typically studied in isolation. We propose CA-AFP, a unified framework that jointly addresses both challenges by performing cluster-specific model pruning. In CA-AFP, clients are first grouped into clusters, and a separate model for each cluster is adaptively pruned during training. The framework introduces two key innovations: (1) a cluster-aware importance scoring mechanism that combines weight magnitude, intra-cluster coherence, and gradient consistency to identify parameters for pruning, and (2) an iterative pruning schedule that progressively removes parameters while enabling model self-healing through weight regrowth. We evaluate CA-AFP on two widely used human activity recognition benchmarks, UCI HAR and WISDM, under natural user-based federated partitions. Experimental results demonstrate that CA-AFP achieves a favorable balance between predictive accuracy, inter-client fairness, and communication efficiency. Compared to pruning-based baselines, CA-AFP consistently improves accuracy and lower performance disparity across clients with limited fine-tuning, while requiring substantially less communication than dense clustering-based methods. It also shows robustness to different Non-IID levels of data. Finally, ablation studies analyze the impact of clustering, pruning schedules and scoring mechanism offering practical insights into the design of efficient and adaptive FL systems.
Figures & tables
Figure 1. The overall workflow of CA-AFP.
Heterogeneity Type
Metric
WISDM
UCI HAR
Quantity Skew
Coef. of Variation ( CV=μσ×100% )
30.4%
10.2%
Sample Count ( μ±σ )
297±90
343±35
Label Skew
Clients with Missing Classes
17 / 36
0 / 30
Missing Rate
47.2%
0.0%
Feature Skew
Mean Inter-Client CV
6.5%
0.6%
Table 1. Heterogeneity analysis of WISDM and UCI-HAR.
WISDM
UCI-HAR
Method
Sparsity
FT Epochs
Avg Acc ( μ )
Fairness ( σ )
Comm
Avg Acc ( μ )
Fairness ( σ )
Comm
( ↑ )
( ↓ )
(MB)
( ↑ )
( ↓ )
(MB)
Dense Baseline
FedCHAR ( Li et al., 2023b )
0%
3
96.64%
8.39%
483
96.87%
6.70%
347
0%
25
97.16%
8.32%
483
97.48%
6.19%
347
ClusterFL ( Ouyang et al., 2023 )
0%
3
96.18%
7.02%
474
95.80%
6.87%
340
Table 2. Performance comparison on the WISDM and UCI-HAR datasets with 3 and 25 fine-tuning (FT) epochs.
Scenario
Dataset
α
β
γ
Avg Acc (%)
Fairness (%)
Standard
UCI HAR
0.25
0.25
0.50
91.19
9.81
WISDM
0.25
0.25
0.50
91.57
10.51
Drift
UCI HAR
0.25
0.25
0.50
66.54
32.70
WISDM
0.00
0.00
1.00
57.63
33.12
Noisy Clients
UCI HAR
0.50
0.25
0.25
74.42
19.03
WISDM
0.25
0.25
0.50
74.36
20.72
Table 3. Best “importance score” configurations.
Figure 2. Effect of “Start Sparsity” and “Rounds” on accuracy in WISDM.
Figure 3. Impact of Local Fine-Tuning on accuracy in WISDM.
Figure 4. Performance across three Non-IID levels.
Weights
UCI HAR
WISDM
Scenario
α
β
γ
Acc
Fairness
Score
Acc
Fairness
Score
Drift
1.00
0.00
0.00
0.6860
0.3526
1.9453
0.6129
0.3820
1.6047
Drift
0.00
1.00
0.00
0.6339
0.3148
2.0135
0.5915
0.3456
1.7116
Drift
0.00
0.00
1.00
0.6475
0.3264
1.9840
0.5763
0.3312
1.7399
Drift
0.50
0.25
0.25
0.6939
0.3452
2.0102
0.5774
0.3396
1.7001
Drift
0.25
0.50
0.25
0.6853
0.3546
1.9323
0.5974
0.3671
1.6273
Table 4. Combined ablation study of importance score weights (α,β,γ) on UCI HAR and WISDM datasets. Note: "Extreme Non-IID" is excluded as it was not evaluated on WISDM.
Method
Parameter
Value
Common Settings
Rounds phase wise ( P1,P2,P3 )
0, 0, 50
Local Epochs ( E )
3
Batch Size ( B )
32
Optimizer
Adam ( lr=1e−3 )
Fine-Tuning Rounds ( P4 )
3
CA-AFP (Ours)
Target Sparsity
Adaptive (Start 70%)
Table 5. Hyperparameter Configuration for UCI HAR Experiments
Method
Parameter
Value
Common Settings
Rounds phase wise ( P1,P2,P3 )
0, 0, 50
Local Epochs ( E )
3
Batch Size ( B )
32
Optimizer
Adam ( lr=1e−3 )
Fine-Tuning Rounds ( P4 )
25
CA-AFP (Ours)
Target Sparsity
Adaptive (Start 70%)
Table 6. Configuration for WISDM Experiments
Ablation Study
P1
P2
P3
P4
Start
(Warmup)
(Adapt)
(Prune)
(Fine-Tune)
Sparsity
Study A: Start Sparsity
0
0
40
3
Varied
(Varying Sstart )
( 0.0−0.9 )
Study B: Post-Cluster
0
Varied
40
3
0.5
(Varying P2 )
{0,5,…,20}
(50%)
Study C: Initial Warmup
Varied
0
40
3
0.5
Table 7. Phase Configuration for CA-AFP Ablation Studies (WISDM)
Component
Parameter
Value / Description
Scoring Weights
Grid Search
(1.0,0.0,0.0) – Pure Magnitude
( α,β,γ )
(7 Combinations)
(0.0,1.0,0.0) – Pure Coherence
(0.0,0.0,1.0) – Pure Consistency
(0.50,0.25,0.25) – Magnitude Dominant
(0.25,0.50,0.25) – Coherence Dominant
(0.25,0.25,0.50) – Consistency Dominant
Table 8. Configuration for Importance Score Ablation Study
Federated learning on heterogeneous edge devices requires personalized compression while preserving aggregation compatibility and stable convergence. We present Curvature-Aware Heterogeneous Federated Pruning (CA-HFP), a practical framework that enables each client perform structured, device-specific pruning guided by a curvature-informed significance score, and subsequently maps its compact submodel back into a common global parameter space via a lightweight reconstruction. We derive a convergence bound for federated optimization with multiple local SGD steps that explicitly accounts for local computation, data heterogeneity, and pruning-induced perturbations; from which a principled loss-based pruning criterion is derived. Extensive experiments on FMNIST, CIFAR-10, and CIFAR-100 using VGG and ResNet architectures under varying degrees of data heterogeneity demonstrate that CA-HFP preserves model accuracy while significantly reducing per-client computation and communication costs, outperforming standard federated training and existing pruning-based baselines.
Federated Learning (FL) enables collaborative model training while preserving data privacy, but its practical deployment is hampered by system and statistical heterogeneity. While federated network pruning offers a path to mitigate these issues, existing methods face a critical dilemma: server-side pruning lacks personalization, whereas client-side pruning is computationally prohibitive for resource-constrained devices. Furthermore, the pruning process itself induces significant parametric divergence among heterogeneous submodels, destabilizing training and hindering global convergence. To address these challenges, we propose SubFLOT, a novel framework for server-side personalized federated pruning. SubFLOT introduces an Optimal Transport-enhanced Pruning (OTP) module that treats historical client models as proxies for local data distributions, formulating the pruning task as a Wasserstein distance minimization problem to generate customized submodels without accessing raw data. Concurrently, to counteract parametric divergence, our Scaling-based Adaptive Regularization (SAR) module adaptively penalizes a submodel's deviation from the global model, with the penalty's strength scaled by the client's pruning rate. Comprehensive experiments demonstrate that SubFLOT consistently and substantially outperforms state-of-the-art methods, underscoring its potential for deploying efficient and personalized models on resource-constrained edge devices.
Zheng Jiang, Nan He, Yiming Chen +1
Tsinghua University · Beijing University of Technology · Key Laboratory of Pervasive Computing, Ministry of Education
In federated learning (FL), accommodating clients with diverse resource constraints remains a significant challenge. A widely adopted approach is to use a shared full-size model, from which each client extracts a submodel aligned with its computational budget. However, regardless of the specific scoring strategy, these methods rely on the same global backbone, limiting both structural diversity and representational adaptation across clients. This paper presents FRAMP, a unified framework for personalized and resource-adaptive federated learning. Instead of relying on a fixed global model, FRAMP generates client-specific models from compact client descriptors, enabling fine-grained adaptation to both data characteristics and computational budgets. Each client trains a tailored lightweight submodel and aligns its learned representation with others to maintain global semantic consistency. Extensive experiments on vision and graph benchmarks demonstrate that FRAMP enhances generalization and adaptivity across a wide range of client settings.
Wenfei Liang, Wee Peng Tay
Department of Electrical and Electronic Engineering, Nanyang Technological University, Singapore.