cs.DCAug 16, 2026

When Is Shallow Enough? Adaptive Split Federated Learning with Client-Specific Sufficiency Estimation

Authors: Wenhao Yuan, Chenchen Lin, Wentao Hu, Jian Chen, Jinfeng Xu, Shujie Li, Edith Cheuk Han Ngai

Organizations: The Hong Kong Polytechnic University Hong Kong, SAR, China

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

\textit{Split Federated Learning} (SFL) enables distributed model training by splitting networks between the server and clients. However, under client heterogeneity, the conventional static split strategy may be suboptimal because clients can differ in data distributions, adaptation dynamics, and representation learning progress, making a single split point insufficient to accommodate client-specific training states. In this paper, we propose \textsc{FedSGA}, a \textbf{S}ufficiency-\textbf{G}uided \textbf{A}daptive split \textbf{Fed}erated learning framework that addresses this question through client-specific shallow sufficiency estimation. First, we introduce a client-specific adaptation channel based on private prompt tokens, which tracks local adaptation dynamics separately from the shared backbone and provides a lightweight signal for detecting whether client adaptation remains active. To further avoid repeated online probing over multiple candidate depths, we design a shallow sufficiency estimator that combines cross-client semantic alignment, temporal interface stability, and prompt-state variation to estimate whether the shallowest split is already sufficient. Finally, we introduce a split-compatible interface harmonization module that projects activations from different split depths into a shared semantic space, improving the comparability of heterogeneous client interfaces before server-side prediction. Extensive experiments on multiple heterogeneous benchmarks demonstrate the effectiveness of \textsc{FedSGA} in improving model performance compared with state-of-the-art methods while reducing unnecessary client-side computation.

Figures & tables

Figure 1. Overview illustration of our proposed FedSGA method. framework
DatasetsTraining SizeTest SizeClassImage Size
CIFAR-1050,00010,000103 ×\times 32 ×\times 32
CIFAR-10050,00010,0001003 ×\times 32 ×\times 32
Tiny-ImageNet100,00010,0002003 ×\times 64 ×\times 64
DomainNet586,57534,0003453 ×\times 224 ×\times 224
Table 1. Basic Information of Datasets
MethodCIFAR-10CIFAR-100Tiny-ImageNetDomainNet
IIDNon-IIDIIDNon-IIDIIDNon-IIDIIDNon-IID
FedAvg64.79 ±0.14\pm\scriptstyle{0.14}48.01 ±2.73\pm\scriptstyle{2.73}43.01 ±0.19\pm\scriptstyle{0.19}35.23 ±0.37\pm\scriptstyle{0.37}26.43 ±0.22\pm\scriptstyle{0.22}19.76 ±0.67\pm\scriptstyle{0.67}38.90 ±0.43\pm\scriptstyle{0.43}27.92 ±1.54\pm\scriptstyle{1.54}
FedProx65.24 ±0.18\pm\scriptstyle{0.18}51.32 ±2.11\pm\scriptstyle{2.11}44.02 ±0.21\pm\scriptstyle{0.21}36.84 ±0.49\pm\scriptstyle{0.49}27.11 ±0.24\pm\scriptstyle{0.24}20.53 ±0.61\pm\scriptstyle{0.61}39.11 ±0.57\pm\scriptstyle{0.57}28.74 ±1.36\pm\scriptstyle{1.36}
FedBABU67.88 ±0.24\pm\scriptstyle{0.24}57.84 ±1.76\pm\scriptstyle{1.76}47.23 ±0.28\pm\scriptstyle{0.28}40.18 ±0.57\pm\scriptstyle{0.57}29.84 ±0.31\pm\scriptstyle{0.31}23.96 ±0.74\pm\scriptstyle{0.74}41.76 ±0.34\pm\scriptstyle{0.34}32.45 ±1.12\pm\scriptstyle{1.12}
FedGH68.57 ±0.21\pm\scriptstyle{0.21}59.21 ±1.44\pm\scriptstyle{1.44}48.32 ±0.25\pm\scriptstyle{0.25}41.36 ±0.46\pm\scriptstyle{0.46}30.62 ±0.27\pm\scriptstyle{0.27}24.88 ±0.69\pm\scriptstyle{0.69}42.38 ±0.39\pm\scriptstyle{0.39}33.27 ±0.98\pm\scriptstyle{0.98}
FedAS70.45 ±0.17\pm\scriptstyle{0.17}61.47 ±1.28\pm\scriptstyle{1.28}50.67 ±0.45\pm\scriptstyle{0.45}43.92 ±0.41\pm\scriptstyle{0.41}32.72 ±0.25\pm\scriptstyle{0.25}27.15 ±0.58\pm\scriptstyle{0.58}44.59 ±0.17\pm\scriptstyle{0.17}35.64 ±0.86\pm\scriptstyle{0.86}
Table 2. Accuracy comparison of FedSGA and other benchmark methods on ResNet-18 backbone. The best accuracy is in bold .
MethodCIFAR-10CIFAR-100Tiny-ImageNetDomainNet
IIDNon-IIDIIDNon-IIDIIDNon-IIDIIDNon-IID
FedAvg72.84 ±0.12\pm\scriptstyle{0.12}56.31 ±2.14\pm\scriptstyle{2.14}49.72 ±0.18\pm\scriptstyle{0.18}40.96 ±0.41\pm\scriptstyle{0.41}32.88 ±0.26\pm\scriptstyle{0.26}24.37 ±0.58\pm\scriptstyle{0.58}44.62 ±0.37\pm\scriptstyle{0.37}32.85 ±1.42\pm\scriptstyle{1.42}
FedProx73.56 ±0.15\pm\scriptstyle{0.15}59.48 ±1.83\pm\scriptstyle{1.83}50.83 ±0.21\pm\scriptstyle{0.21}42.37 ±0.46\pm\scriptstyle{0.46}33.71 ±0.22\pm\scriptstyle{0.22}25.68 ±0.54\pm\scriptstyle{0.54}45.21 ±0.42\pm\scriptstyle{0.42}34.16 ±1.27\pm\scriptstyle{1.27}
FedBABU76.42 ±0.19\pm\scriptstyle{0.19}64.37 ±1.44\pm\scriptstyle{1.44}54.86 ±0.24\pm\scriptstyle{0.24}46.83 ±0.51\pm\scriptstyle{0.51}37.12 ±0.28\pm\scriptstyle{0.28}30.41 ±0.63\pm\scriptstyle{0.63}48.93 ±0.31\pm\scriptstyle{0.31}38.85 ±1.04\pm\scriptstyle{1.04}
FedGH77.28 ±0.17\pm\scriptstyle{0.17}65.91 ±1.26\pm\scriptstyle{1.26}55.73 ±0.23\pm\scriptstyle{0.23}48.15 ±0.43\pm\scriptstyle{0.43}38.26 ±0.24\pm\scriptstyle{0.24}31.74 ±0.57\pm\scriptstyle{0.57}49.84 ±0.28\pm\scriptstyle{0.28}40.12 ±0.92\pm\scriptstyle{0.92}
FedAS79.35 ±0.14\pm\scriptstyle{0.14}68.42 ±1.03\pm\scriptstyle{1.03}58.96 ±0.31\pm\scriptstyle{0.31}50.93 ±0.36\pm\scriptstyle{0.36}41.08 ±0.21\pm\scriptstyle{0.21}35.46 ±0.44\pm\scriptstyle{0.44}52.77 ±0.16\pm\scriptstyle{0.16}43.56 ±0.73\pm\scriptstyle{0.73}
Table 3. Accuracy comparison of FedSGA and other benchmark methods on ViT-B/16 backbone. The best accuracy is in bold .
Table 4. Efficacy of each module on various datasets.
Figure 2. Impact of hyperparameters on model performance on the CIFAR-100 dataset.
Figure 3. Impact of hyperparameters on model performance on the DomainNet dataset.
Table 5. Computational cost analysis under different backbone architectures.
Figure 4. Comparison of different split strategies on the CIFAR-100 and DomainNet datasets.

Explore similar work

CardsList
May 26, 2026cs.LG

Separate Aggregation of Split Network for Personalized Federated Learning

Federated learning enables collaborative model training without sharing raw data, but its performance can degrade substantially under heterogeneous client data distributions. A single global model often cannot satisfy diverse client requirements, so personalized federated learning has therefore been explored to improve client specific performance while preserving global generalization. Existing PFL methods often face a fundamental tradeoff in which stronger global sharing can undermine local specialization, whereas stronger local adaptation can lead to overfitting under limited data, label imbalance, and missing class scenarios. In this work, we propose PGFedSplit, a personalized federated learning framework that improves both personalization and global generalization under severe client heterogeneity. PGFedSplit adopts a split architecture and performs adaptive aggregation scheduling tailored to the roles of different model components, enabling stable knowledge sharing while maintaining client specific adaptation. Each client further leverages a mixture of locally extracted representations and synthetic representations generated from server side Gaussian statistics, improving robustness under label imbalance and missing class conditions. Extensive experiments on Fashion MNIST, CIFAR 10, CIFAR 100, and Tiny ImageNet demonstrate consistent improvements over state of the art PFL methods, with stable convergence and superior personalization in highly heterogeneous settings.

Yunseok Kang, Jaeyoung Song

Department of Electronics Engineering, Pusan National University, Republic of Korea

Federated Learning AggregationSplit Federated LearningFederated LearningHeterogeneous Federated Learning
AskPDF
Sep 28, 2026cs.AI

CoeF-SFL: Preserving Collaborative Server-Client Learning with Enhanced Communication Efficiency

Split Federated Learning (SFL) enables resource-constrained clients to participate in collaborative training, but vanilla SFL exchanges smashed data and gradients at every batch, which incurs significant communication overhead. Recent methods reduce this overhead with an auxiliary network at the client-side cut layer. However, we identify that this approach makes the client optimize a local objective that differs from the end-to-end objective, which fundamentally limits the collaborative training between the client and the server. We propose Compensated Feedback based SFL (CoeF-SFL), a communication-efficient framework that retains the end-to-end objective without any auxiliary network. In CoeF-SFL, the client and the server exchange the smashed data and the gradients once per round and reuse them during local training. Since this reuse makes the gradients stale on the client side, we compensate them with a curvature-based correction in the activation space and develop two variants. CoeF-D approximates the Hessian with a diagonal gradient outer product, while CoeF-J exploits the tractable Jacobian-based Hessian of a surrogate loss that upper-bounds the true loss. We provide the theoretical background of each method, characterizing its compensation. Across vision and language tasks, model capacities, cut layers, and data distributions, CoeF-SFL significantly outperforms auxiliary-network-based methods under the same communication frequency, and the improvement is most substantial on vision tasks. Code is available at https://anonymous.4open.science/r/CoeF-SFL-2686/README.md

Junwoo Bae, Jin-Hyun Ahn

Myongji University

Communication-Efficient Distributed TrainingSplit Federated Learning
AskPDF
May 17, 2026cs.LG

BESplit: Bias-Compensated Split Federated Learning with Evidential Aggregation

Split Federated Learning (SFL) enables privacy-preserving collaborative training by partitioning models between clients and a server. However, under non-IID data distributions, SFL often suffers from biased optimization and unstable convergence, while existing solutions largely adapt techniques from conventional federated learning. In this work, we observe that the split architecture of SFL inherently alters how client information is represented and coordinated, opening opportunities for bias compensation beyond parameter-level aggregation. Based on this insight, we propose BESplit, an architecture-aware framework that exploits the intrinsic structure of SFL to mitigate non-IID effects. First, to prevent biased local data from dominating global updates, we introduce Evidential Aggregation (EA) to perform fine-grained reweighting of client contributions based on evidential uncertainty. Second, to further reduce distributional skew, we develop Bias-Compensated Collaboration (BCC) to align split-layer representations by pairing complementary clients. Finally, Dual-Teacher Distillation (DTD) is incorporated to synchronize knowledge between decoupled client and server models, enabling independent local inference. Extensive experiments on five benchmark datasets demonstrate that BESplit consistently outperforms state-of-the-art methods in accuracy, convergence stability, and computational efficiency under diverse non-IID settings.

Yuhan Xie, Chen Lyu, Jingrong Huang

Shanghai University of Finance and Economics, China · MoE Key Laboratory of Interdisciplinary Research of Computation and Economics, China.

Federated Learning AggregationSplit Federated LearningNon-IID Federated LearningFederated Learning
AskPDF