CoeF-SFL: Preserving Collaborative Server-Client Learning with Enhanced Communication Efficiency

Junwoo Bae; Jin-Hyun Ahn

cs.AISep 28, 2026

CoeF-SFL: Preserving Collaborative Server-Client Learning with Enhanced Communication Efficiency

Authors: Junwoo Bae, Jin-Hyun Ahn

Organizations: Myongji University

Abstract

Split Federated Learning (SFL) enables resource-constrained clients to participate in collaborative training, but vanilla SFL exchanges smashed data and gradients at every batch, which incurs significant communication overhead. Recent methods reduce this overhead with an auxiliary network at the client-side cut layer. However, we identify that this approach makes the client optimize a local objective that differs from the end-to-end objective, which fundamentally limits the collaborative training between the client and the server. We propose Compensated Feedback based SFL (CoeF-SFL), a communication-efficient framework that retains the end-to-end objective without any auxiliary network. In CoeF-SFL, the client and the server exchange the smashed data and the gradients once per round and reuse them during local training. Since this reuse makes the gradients stale on the client side, we compensate them with a curvature-based correction in the activation space and develop two variants. CoeF-D approximates the Hessian with a diagonal gradient outer product, while CoeF-J exploits the tractable Jacobian-based Hessian of a surrogate loss that upper-bounds the true loss. We provide the theoretical background of each method, characterizing its compensation. Across vision and language tasks, model capacities, cut layers, and data distributions, CoeF-SFL significantly outperforms auxiliary-network-based methods under the same communication frequency, and the improvement is most substantial on vision tasks. Code is available at https://anonymous.4open.science/r/CoeF-SFL-2686/README.md

Figures & tables

Method	CIFAR-100	Food-101	Tiny-ImageNet
Vanilla SFL	$85.0\,\mbox{$ \pm $\,0.3}$	$83.8\,\mbox{$ \pm $\,0.1}$	$78.6\,\mbox{$ \pm $\,0.2}$
AccSFL	$65.6\,\mbox{$ \pm $\,1.2}$	$67.5\,\mbox{$ \pm $\,0.1}$	$47.4\,\mbox{$ \pm $\,0.2}$
Gradient reuse (w/o comp.)	$78.4\,\mbox{$ \pm $\,0.4}$	$75.3\,\mbox{$ \pm $\,2.1}$	$72.8\,\mbox{$ \pm $\,0.2}$
Gradient reuse (w/ perfect comp.)	$84.5\,\mbox{$ \pm $\,0.3}$	$83.3\,\mbox{$ \pm $\,0.0}$	$78.3\,\mbox{$ \pm $\,0.3}$

Table 1: Preliminary results of vanilla SFL ( Thapa et al., 2022 ) , AccSFL ( Han et al., 2022 ) , two reference methods, reported as test accuracy (%) and EM/F1 (%) on vision and language tasks with ViT-Tiny and DistilRoBERTa, respectively. Gradient reuse (w/o comp.) is the naive baseline, and Gradient reuse (w/ perfect comp.) is the oracle. For all methods except vanilla SFL, each round, i.e., the interval between two consecutive server feedbacks, corresponds to one local epoch. The other settings, including the SFL setting, local training, and hyperparameter tuning, are described in Section 5 .

Figure 1: Comparison of SFL frameworks. Vanilla SFL optimizes the end-to-end objective $\mathcal{L}$ through server-side feedback at every batch, which incurs heavy communication. Auxiliary-network-based SFL communicates once per round, but the client optimizes an auxiliary objective $\mathcal{L}_{\mathrm{aux}}$ that differs from $\mathcal{L}$ . CoeF-SFL communicates once per round and preserves the end-to-end objective, compensating the received gradients according to the displacement $\delta$ of activation.

Model	Method	CIFAR-100	Food-101	Tiny-ImageNet
ViT-Tiny	Vanilla SFL	85.0 $\pm$ 0.3	83.8 $\pm$ 0.1	78.6 $\pm$ 0.2
	Gradient reuse (w/ perfect comp.)	84.5 $\pm$ 0.3	83.3 $\pm$ 0.0	78.3 $\pm$ 0.3
	Gradient reuse (w/o comp.)	78.4 $\pm$ 0.4	75.3 $\pm$ 2.1	72.8 $\pm$ 0.2
	AccSFL	65.6 $\pm$ 1.2	67.5 $\pm$ 0.1	47.4 $\pm$ 0.2
	FSL-SAGE	66.9 $\pm$ 7.2	60.5 $\pm$ 0.9	50.9 $\pm$ 9.0
	GAS	75.2 $\pm$ 0.9	73.8 $\pm$ 0.6	71.7 $\pm$ 0.2

Table 2: Comparison results: test accuracy (%) on vision and language tasks, and EM and F1 (%) on SQuAD v2.0. – denotes that the method is not applicable to the task.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Method	cut 1	cut 2	cut 4	cut 6	cut 8
Vanilla SFL	85.1 $\pm$ 0.1	84.9 $\pm$ 0.1	85.0 $\pm$ 0.3	84.9 $\pm$ 0.3	84.8 $\pm$ 0.3
Gradient reuse (w/ perfect comp.)	84.8 $\pm$ 0.1	84.8 $\pm$ 0.1	84.5 $\pm$ 0.3	84.3 $\pm$ 0.2	83.9 $\pm$ 0.2
Gradient reuse (w/o comp.)	84.5 $\pm$ 0.3	84.3 $\pm$ 0.2	78.4 $\pm$ 0.4	78.5 $\pm$ 0.3	77.7 $\pm$ 0.3
AccSFL	71.3 $\pm$ 1.4	62.0 $\pm$ 1.3	65.6 $\pm$ 1.2	71.4 $\pm$ 0.4	76.0 $\pm$ 0.2
FSL-SAGE	47.0 $\pm$ 17.2	55.2 $\pm$ 7.7	66.9 $\pm$ 7.2	69.6 $\pm$ 5.7	73.7 $\pm$ 4.1
GAS	82.9 $\pm$ 0.2	83.1 $\pm$ 0.3	75.2 $\pm$ 0.9	75.6 $\pm$ 0.6	76.1 $\pm$ 0.7

Appendix

Table 3: Effect of the cut layer on ViT-Tiny/CIFAR-100 (IID): test accuracy (%). Cut

k

places the first

k

of the twelve transformer blocks on the client, and the last row reports the ratio of trainable parameters, excluding the embedding layer, between the client-side and server-side models.

Method	cut 1	cut 2	cut 3	cut 4
Vanilla SFL	69.6 $\pm$ 0.2	69.8 $\pm$ 0.2	69.7 $\pm$ 0.3	69.7 $\pm$ 0.3
Gradient reuse (w/ perfect comp.)	69.6 $\pm$ 0.2	69.3 $\pm$ 0.2	69.2 $\pm$ 0.3	69.2 $\pm$ 0.2
Gradient reuse (w/o comp.)	69.5 $\pm$ 0.0	69.3 $\pm$ 0.0	68.8 $\pm$ 0.2	67.7 $\pm$ 0.1
AccSFL	68.2 $\pm$ 0.3	66.6 $\pm$ 0.3	66.8 $\pm$ 0.3	67.8 $\pm$ 0.3
FSL-SAGE	59.9 $\pm$ 0.7	62.0 $\pm$ 0.8	64.8 $\pm$ 0.4	66.5 $\pm$ 0.0
GAS	68.4 $\pm$ 0.5	68.0 $\pm$ 0.5	66.9 $\pm$ 0.2	66.6 $\pm$ 0.7

Appendix

Table 4: Effect of the cut layer on DistilRoBERTa/20 Newsgroups (IID): test accuracy (%). Cut

k

places the first

k

of the six transformer blocks on the client, and the last row reports the ratio of trainable parameters, excluding the embedding layer, between the client-side and server-side models.

Model	Method	CIFAR-100	Food-101	Tiny-ImageNet
ViT-Tiny	Vanilla SFL	70.9 $\pm$ 1.2	63.9 $\pm$ 3.1	59.1 $\pm$ 3.3
	Gradient reuse (with perfect comp.)	66.9 $\pm$ 0.5	55.6 $\pm$ 1.5	54.7 $\pm$ 2.9
	Gradient reuse (w/o comp.)	1.0 $\pm$ 0.1	1.0 $\pm$ 0.0	0.5 $\pm$ 0.0
	AccSFL	28.2 $\pm$ 1.3	32.5 $\pm$ 4.3	18.8 $\pm$ 2.1
	FSL-SAGE	33.6 $\pm$ 2.0	30.3 $\pm$ 5.5	20.0 $\pm$ 2.3
	GAS	46.0 $\pm$ 10.2	1.0 $\pm$ 0.0	0.6 $\pm$ 0.1

Appendix

Table 5: ViT-Tiny/vision tasks (non-IID): test accuracy (%).

Model	Method	20 Newsgroups	MNLI-m	MNLI-mm
DistilRoBERTa	Vanilla SFL	67.2 $\pm$ 0.6	80.4 $\pm$ 0.5	80.9 $\pm$ 0.7
	Gradient reuse (with perfect comp.)	64.9 $\pm$ 1.3	74.6 $\pm$ 4.2	74.5 $\pm$ 4.8
	Gradient reuse (w/o comp.)	62.5 $\pm$ 1.7	33.3 $\pm$ 1.5	33.3 $\pm$ 1.4
	AccSFL	61.1 $\pm$ 1.5	71.4 $\pm$ 1.7	71.9 $\pm$ 2.2
	FSL-SAGE	53.2 $\pm$ 1.6	73.0 $\pm$ 1.3	73.4 $\pm$ 1.3
	GAS	64.2 $\pm$ 0.9	37.9 $\pm$ 6.2	38.3 $\pm$ 7.0

Appendix

Table 6: DistilRoBERTa/language tasks (non-IID): test accuracy (%).

Symbol	Meaning
$N$	samples per client
$B$	mini-batches per client and round
$d$	dimension of the smashed data of one sample
$\mathrm{f}^{c}$ , $\mathrm{b}^{c}$	client forward / backward pass
$\mathrm{f}^{s}$ , $\mathrm{b}^{s}$	server forward / full backward pass
$\mathrm{b}^{s}_{\mathrm{in}}$	server backward pass to the smashed data

Appendix

Table 7: Notation for further analysis.

Method	Server computation (GFLOPs)	Computation complexity	Comm. cost (MB)	Comm. volume
Gradient reuse (w/o comp.)	7.23	$2(\mathrm{f}^{s}+\mathrm{b}^{s})$	144.29	$O(Nd)$
GGN	41.45	$(k+1)(\mathrm{f}^{s}{+}\mathrm{b}^{s})$	721.45	$O((k+1)Nd)$
Hutchinson diagonal	64.69	$\mathrm{f}^{s}+\mathrm{b}^{s}+2m_{d}(\mathrm{f}^{s}+\mathrm{b}^{s}_{\mathrm{in}})$	288.58	$O(2Nd)$
CoeF-D	7.23	$2(\mathrm{f}^{s}{+}\mathrm{b}^{s})$	144.29	$O(Nd)$

Appendix

Table 8: Gradient reuse with different curvature approximations on ViT-Tiny/CIFAR-100. The top table reports test accuracy (%), and the bottom table reports the server computation per sample and the communication cost at cut 4. The values are measured in our setting from Section 5

Method	Exch.	Transfers	Volume	Up (MB)	Down (MB)
Vanilla SFL	$O(B)$	64	$O(Nd)$	144.29	144.29
Gradient reuse (w/ perfect comp.)	$O(B)$	65	$O(Nd)$	288.58	144.29
Gradient reuse (w/o comp.)	$O(1)$	2	$O(Nd)$	144.29	144.29
AccSFL	$O(1)$	1	$O(Nd)$	144.29	0.00
MU-SplitFed	$O(1)$	2	$O(Nd)$	432.87	0.000122
FSL-SAGE	$O(1)$	1 / 2	$O(Nd)$	144.29	0.00 / 5.32

Appendix

Table 9: Communication per client and round (top) and computation per sample (bottom) on ViT-Tiny/CIFAR-100 at cut 4. Transfers count uplink and downlink messages separately. For FSL-SAGE, entries with a slash give a round without alignment (left) and an alignment round (right, every

l=10

rounds), in which the server recomputes the gradients of

N_{\mathrm{buf}}=64

stored samples and trains the auxiliary network on them for

E=40

epochs, with

\mathrm{a}

counting the double backward of the alignment loss. The values are measured in our setting from Section 5 .

	Aux. params		IID			Non-IID
Method	#	% client	CIFAR-100	Food-101	Tiny-ImageNet	CIFAR-100	Food-101	Tiny-ImageNet
FC + LoRA ( $r{=}2$ )	0.7K	1.2	63.6 $\pm$ 1.2	64.9 $\pm$ 0.4	48.8 $\pm$ 0.7	36.6 $\pm$ 0.6	37.0 $\pm$ 4.5	24.1 $\pm$ 2.7
FC + LoRA ( $r{=}4$ )	1.3K	2.3	63.8 $\pm$ 1.0	65.2 $\pm$ 0.5	49.4 $\pm$ 1.1	37.1 $\pm$ 0.8	37.5 $\pm$ 4.6	24.5 $\pm$ 2.2
FC (AccSFL)	19.3K	34.9	65.6 $\pm$ 1.2	67.5 $\pm$ 0.1	47.4 $\pm$ 0.2	28.2 $\pm$ 1.3	32.5 $\pm$ 4.3	18.8 $\pm$ 2.1
MLP, 1 hidden	28.2K	51.0	78.8 $\pm$ 0.2	77.5 $\pm$ 0.2	70.7 $\pm$ 0.2	54.7 $\pm$ 1.9	51.5 $\pm$ 3.8	42.0 $\pm$ 2.8
MLP, 4 hidden	56.2K	101.6	81.5 $\pm$ 0.4	79.9 $\pm$ 0.2	74.4 $\pm$ 0.1	62.7 $\pm$ 1.9	56.9 $\pm$ 3.2	51.4 $\pm$ 3.2

Appendix

Table 10: Auxiliary-network size sweep for AccSFL on ViT-Tiny at cut 4: server-side test accuracy (%). Aux. params gives the trainable parameters of the auxiliary network and their ratio to the trainable client-side parameters (55.3K under LoRA). FC is the auxiliary network of AccSFL in the other tables, and FC + LoRA freezes the FC weight and trains rank-

r

adapters and the bias. MLP stacks the given number of hidden layers of width

d/2=96

with GELU. FSL-SAGE uses its own auxiliary network, which consists of three server-side blocks followed by a layer norm and an FC head.

	Aux. params
Method	#	% client	IID	Non-IID
FC + LoRA ( $r{=}2$ )	1.6K	1.4	65.3 $\pm$ 0.6	59.7 $\pm$ 1.9
FC + LoRA ( $r{=}4$ )	3.2K	2.9	65.4 $\pm$ 0.5	59.8 $\pm$ 1.7
FC (AccSFL)	15.4K	13.9	66.6 $\pm$ 0.3	61.1 $\pm$ 1.5
MLP, 1 hidden	303.0K	274.0	68.1 $\pm$ 0.1	64.0 $\pm$ 1.1
MLP, 4 hidden	746.5K	675.0	67.9 $\pm$ 0.0	64.0 $\pm$ 0.6

Appendix

Table 11: Auxiliary-network size sweep on DistilRoBERTa/20 Newsgroups at cut 2: server-side test accuracy (%). Columns as in Table 10 , where the client-side model has 110.6K trainable parameters (LoRA) and MLP uses hidden width

d/2=384

. FSL-SAGE uses its own auxiliary network, which consists of two of the four server-side blocks followed by a layer norm and an FC head, with 14.3M parameters.

Explore similar work

May 17, 2026cs.LG

BESplit: Bias-Compensated Split Federated Learning with Evidential Aggregation

Split Federated Learning (SFL) enables privacy-preserving collaborative training by partitioning models between clients and a server. However, under non-IID data distributions, SFL often suffers from biased optimization and unstable convergence, while existing solutions largely adapt techniques from conventional federated learning. In this work, we observe that the split architecture of SFL inherently alters how client information is represented and coordinated, opening opportunities for bias compensation beyond parameter-level aggregation. Based on this insight, we propose BESplit, an architecture-aware framework that exploits the intrinsic structure of SFL to mitigate non-IID effects. First, to prevent biased local data from dominating global updates, we introduce Evidential Aggregation (EA) to perform fine-grained reweighting of client contributions based on evidential uncertainty. Second, to further reduce distributional skew, we develop Bias-Compensated Collaboration (BCC) to align split-layer representations by pairing complementary clients. Finally, Dual-Teacher Distillation (DTD) is incorporated to synchronize knowledge between decoupled client and server models, enabling independent local inference. Extensive experiments on five benchmark datasets demonstrate that BESplit consistently outperforms state-of-the-art methods in accuracy, convergence stability, and computational efficiency under diverse non-IID settings.

Yuhan Xie, Chen Lyu, Jingrong Huang

Shanghai University of Finance and Economics, China · MoE Key Laboratory of Interdisciplinary Research of Computation and Economics, China.

Federated Learning Balanced Learning Distributions Evidential Uncertainty Collaboration

Ask PDF

Nov 6, 2023cs.LG

Convergence Analysis of Sequential Federated Learning on Heterogeneous Data

There are two categories of methods in Federated Learning (FL) for joint training across multiple clients: (i) parallel FL (PFL), where clients train models in a parallel manner; and (ii) sequential FL (SFL), where clients train models in a sequential manner. In contrast to that of PFL, the convergence theory of SFL on heterogeneous data is still lacking. In this paper, we establish the convergence guarantees of SFL for strongly/general/non-convex objectives on heterogeneous data. The convergence guarantees of SFL are better than that of PFL on heterogeneous data with both full and partial client participation. Experimental results validate the counterintuitive analysis result that SFL outperforms PFL on extremely heterogeneous data in cross-device settings.

Yipeng Li, Xinchen Lyu

National Engineering Research Center for Mobile Network Technologies Beijing University of Posts and Telecommunications Beijing, 100876, China

Federated Learning Convergence Global-Local Fusion Framework Sequential

Ask PDF

Mar 16, 2026cs.LG

HO-SFL: Hybrid-Order Split Federated Learning with Backprop-Free Clients and Dimension-Free Aggregation

Fine-tuning large models on edge devices is severely hindered by the memory-intensive backpropagation (BP) in standard frameworks like federated learning and split learning. While substituting BP with zeroth-order optimization can significantly reduce memory footprints, it typically suffers from prohibitively degraded convergence speed. To resolve this dilemma, we propose Hybrid-Order Split Federated Learning (HO-SFL). By reformulating the split learning process within a Lagrangian framework, HO-SFL decouples the optimization landscape: The server performs precise first-order updates (i.e., BP), whereas clients conduct memory-efficient zeroth-order optimization. This hybrid design not only eliminates the need for client-side BP but also enables dimension-free model aggregation, drastically lowering communication costs. Crucially, we provide a theoretical convergence analysis, demonstrating that HO-SFL mitigates the dimension-dependent convergence slowdown of zeroth-order optimization, achieving a convergence rate comparable to first-order methods. Extensive experiments on tasks across vision and language modalities validate that HO-SFL achieves convergence speeds comparable to first-order baselines while significantly reducing communication costs and client memory footprints.

Qiyuan Chen, Xian Wu, Yi Wang +1

Department of Electrical and Computer Engineering, The University of Hong Kong, Hong Kong SAR, China.

Federated Learning Zeroth-Order Optimization Convergence Backpropagation

Ask PDF