Organizations: School of Computer Science and Technology, University of Science and Technology of China, Hefei, Anhui, China, 230027 · Suzhou Institute for Advanced Research, University of Science and Technology of China, Suzhou, Jiangsu, China, 215123
Split Federated Learning (SFL) has emerged as a promising paradigm for training large-scale deep models on resource-constrained edge devices. Although asynchronous SFL alleviates the straggler bottleneck caused by system heterogeneity, it inevitably introduces two coupled dimensions of inconsistency: horizontal inconsistency across clients and vertical inconsistency between clients and the server. These inconsistencies, further exacerbated by statistical heterogeneity, result in biased activation representations, model divergence, and severe client drift. To address these challenges, we propose CoASFL, an inconsistency-aware asynchronous SFL framework that incorporates a dynamic activation merging mechanism and a consistency-aware model aggregation mechanism. Specifically, CoASFL constructs more consistent activation sequences from heterogeneous clients to alleviate horizontal inconsistency, and dynamically adjusts aggregation weights according to client participation frequencies and progress gaps to mitigate vertical inconsistency. By jointly coordinating activation-level and model-level consistency, CoASFL effectively suppresses inconsistency propagation during asynchronous training. Extensive experiments on an 80-device heterogeneous edge testbed demonstrate that CoASFL consistently outperforms state-of-the-art SFL approaches, accelerating training by 1.61× to 3.74× and improving final model accuracy by up to 3.26% under heterogeneous scenarios.
Figures & tables
Fig. 1: Impact of horizontal inconsistency.
Fig. 2: Impact of vertical inconsistency.
Fig. 3: Overview of CoASFL
AI Performance
GPU Type
Jetson TX2
6 TOPs
256-core Pascal
Jetson NX
21 TOPs
384-core Volta
Jetson AGX
32 TOPs
512-core Volta
CPU Type
RAM
Jetson TX2
Denver 2 and ARM 4
8 GB LPDDR4
Jetson NX
6-core Carmel ARM 8
8 GB LPDDR4x
TABLE I: Device technical specifications.
Fig. 4: Test accuracy on the three IID datasets
Fig. 5: Test accuracy on the three non-IID datasets
Fig. 6: Traffic consumption to achieve target accuracies
Fig. 7: Average waiting time on the three datasets
Fig. 8: Test accuracy under different non-IID levels
Fig. 9: Effects of DAM and CMA
TABLE II: Training Performance of Various Buffer Capacity
Federated learning (FL) enables collaborative model training across distributed clients without sharing raw data, yet its scalability is limited by synchronization overhead. Asynchronous federated learning (AFL) alleviates this issue by allowing clients to communicate independently, thereby improving wall-clock efficiency in large-scale, hardware-heterogeneous environments. However, asynchrony introduces updates computed on outdated global models (staleness) that can destabilize optimization and hinder convergence. We propose FedRevive, an AFL framework that revives stale updates through data-free knowledge distillation (DFKD). FedRevive integrates parameter-space aggregation with a server-side DFKD process that transfers knowledge from stale client updates to the current global model without access to data. A meta-learned generator synthesizes pseudo-samples for multi-teacher distillation. A hybrid aggregation scheme combining raw with DFKD updates effectively mitigates staleness while retaining AFL scalability. Experiments on vision and text benchmarks show that FedRevive achieves faster training by up to 29.7% and higher final accuracy by up to 11.4 percentage points than baselines.
Baris Askin, Holger R. Roth, Zhenyu Sun +3
Carnegie Mellon University · NVIDIA · Northwestern University
Federated learning (FL) enables multiple clients to collaboratively train a machine learning model without sharing their raw data. However, the limited computation resources of the clients may result in a high delay and energy consumption on training. In this paper, we propose an adaptive split federated learning (ASFL) framework over wireless networks. ASFL exploits the computation resources of the central server to train part of the model and enables adaptive model splitting as well as resource allocation during training. To optimize the learning performance (i.e., convergence rate) and efficiency (i.e., delay and energy consumption) of ASFL, we theoretically analyze the convergence rate and formulate a joint learning performance and resource allocation optimization problem. Solving this problem is challenging due to the long-term delay and energy consumption constraints as well as the coupling of the model splitting and resource allocation decisions. We propose an online optimization enhanced block coordinate descent (OOE-BCD) algorithm to solve the problem iteratively. Experimental results show that when compared with five baseline schemes, our proposed ASFL framework converges faster and reduces the total delay and energy consumption by up to 75% and 80%, respectively.
Chuiyang Meng, Ming Tang, Vincent W. S. Wong
Department of Electrical and Computer Engineering, The University of British Columbia, Vancouver, Canada · Department of Computer Science and Engineering, Southern University of Science and Technology, China
Split Federated Learning (SFL) enables distributed model training by splitting networks between the server and clients. However, under client heterogeneity, the conventional static split strategy may be suboptimal because clients can differ in data distributions, adaptation dynamics, and representation learning progress, making a single split point insufficient to accommodate client-specific training states. In this paper, we propose FedSGA, a Sufficiency-Guided Adaptive split Federated learning framework that addresses this question through client-specific shallow sufficiency estimation. First, we introduce a client-specific adaptation channel based on private prompt tokens, which tracks local adaptation dynamics separately from the shared backbone and provides a lightweight signal for detecting whether client adaptation remains active. To further avoid repeated online probing over multiple candidate depths, we design a shallow sufficiency estimator that combines cross-client semantic alignment, temporal interface stability, and prompt-state variation to estimate whether the shallowest split is already sufficient. Finally, we introduce a split-compatible interface harmonization module that projects activations from different split depths into a shared semantic space, improving the comparability of heterogeneous client interfaces before server-side prediction. Extensive experiments on multiple heterogeneous benchmarks demonstrate the effectiveness of FedSGA in improving model performance compared with state-of-the-art methods while reducing unnecessary client-side computation.
Wenhao Yuan, Chenchen Lin, Wentao Hu +4
The Hong Kong Polytechnic University Hong Kong, SAR, China