Unsupervised Head Network Personalization for Split Computing via Federated Learning

Yutaro Horikawa; Takayuki Nishio

cs.LGOct 20, 2026

Unsupervised Head Network Personalization for Split Computing via Federated Learning

Authors: Yutaro Horikawa, Takayuki Nishio

Organizations: School of Engineering, Institute of Science Tokyo, Tokyo 152-8550, Japan

Abstract

Split Computing (SC) is a distributed inference method designed for balancing computational load and reducing latency by partitioning a neural network into head and tail networks and deploying them across both a mobile device and a server, respectively. Due to the limited bandwidth and computational power of mobile devices, the head network is often replaced with a neural network that requires lower computational and communication resources. However, it is challenging to prepare a sufficient quantity and quality of labeled data on the mobile device side for training the replacement head network. Furthermore, the data prepared by mobile users often contains sensitive privacy information, making data aggregation difficult. In this paper, we propose a data-efficient head network training method for SC. This method utilizes federated learning and knowledge distillation to train the head network using a small amount of non-IID and unlabeled data from mobile devices, ensuring that the training datasets are not exposed. Moreover, by introducing personalization, we enhance the accuracy to levels comparable to or exceeding those of existing methods. The experimental evaluations demonstrate that, despite data constraints imposed only on our method, it achieves accuracy comparable to the baselines. Furthermore, it significantly reduces the communication requirements by 95% on mobile devices, with only a 3-5 percentage point drop in accuracy from the original model.

Figures & tables

				Data constraints
Research	Metrics	Loss	Unlabeled	Decentralized	Non-IID
[ 9 ]	A, D, L	CE	–	–	–
[ 17 ]	A, D, L	CE	–	–	–
[ 11 ]	A, C, D, L	KD	$\checkmark$	–	–
[ 12 ]	A, C, D, L	KD	$\checkmark$	–	–
[ 13 ]	A, D	reconst.	$\checkmark$	–	–

TABLE I : Summary of related works on head network training. A: model accuracy, C: computation load, D: amount of data transmitted, L: latency, CE: cross-entropy, KD: knowledge distillation, reconst.: reconstruction loss

Panel of Figure 2 — Fig. 1 : Head Network Distillation [ 11 ] : (a) teacher and student models extracted from the original pre-trained model, and (b) the training procedure minimizing the error between the intermediate outputs.

Fig. 3 : Proposed method: Federated Head Network Personalization

Fig. 4 : Architecture of the original and bottleneck injected models. Training uses only the teacher and student models.

Parameters	Values for CIFAR-10	Values for CIFAR-100
Sum of clients’ data	1000 to 20000	4000 to 20000
Number of pre-training epochs	200
Number of clients to choose from in one round of FL step in FedHNP $C$	10
Number of client epochs within one round of FL in FedHNP $E_{\mathrm{FL}}$	$\lfloor 2\times 10^{5}/\mathrm{sum\ of\ clients^{\prime}\ data}\rfloor$
Number of rounds of FL for FedHNP $R$	20	100
Number of epochs of personalization in FedHNP $E_{p}$	10	20

TABLE II : Training parameters

		Is the client data label needed?	Is the labeled data for pre-train needed?	Is communication during training needed?	Is the privacy of the client data maintained?
Method1	FedHNP (Proposed)	No	Yes	Yes	Yes
Method2	Centralized HND	No	Yes	No	No
Method3	Individual HND	No	Yes	No	Yes

TABLE III : Compared methods

Methods	Split point	Computation load (FLOPs)		Communication traffic
Methods	Split point	Client	Server	Data shape	Data volume (8bit)	Data volume (32bit)
Cloud computing using original model	-	0	$1.40\times 10^{8}$	(3, 32, 32)	3.1 KB	12.3 KB
Split computing using original model	2	$1.77\times 10^{6}$	$1.38\times 10^{8}$	(64, 16, 16)	16.4 KB	65.5 KB
(two split points are listed)	8	$5.42\times 10^{7}$	$8.60\times 10^{7}$	(128, 8, 8)	8.20 KB	32.8 KB
Split computing using replacement model	3	$2.02\times 10^{6}$	$1.04\times 10^{8}$	(12, 9, 9)	1.0 KB	3.9 KB

TABLE IV : Computation and Communication Cost Reduction via Split Computing vs. Cloud Computing.

Fig. 5 : Comparison of training curves with CIFAR-10, 20000 client training samples. For FedHNP, we plot the stage 1 accuracy for each of the ten epochs and the accuracy after stage 2 in the “personalized” position.

Panel of Figure 10 — Fig. 6 : CIFAR-10 / CIFAR-100 test accuracy in different total number of clients data

Panel of Figure 11 — Fig. 7 : Stage 2 training curve with CIFAR-10, 20000 data. Personalizing for 100 epochs.

Panel of Figure 12 — Fig. 8 : Test accuracy before / after personalization in FedHNP. The triangle represents the mean, and the line in the box represents the median.

Federated Learning

Federated learning trains shared models across data holders without requiring them to pool their raw data.

See all 943 papers

Lightweight User-Personalization Method for Closed Split Computing
Mar 16, 2026Yuya Okada, Takayuki NishioEdge Computing Parameter-Efficient Fine-Tuning
School of Engineering, Institute of Science Tokyo, Tokyo 152-8550, Japan
When Is Shallow Enough? Adaptive Split Federated Learning with Client-Specific Sufficiency Estimation
Aug 16, 2026Wenhao Yuan, Chenchen Lin, Wentao Hu +4Split Federated Learning Heterogeneous Federated Learning
The Hong Kong Polytechnic University Hong Kong, SAR, China
Separate Aggregation of Split Network for Personalized Federated Learning
May 26, 2026Yunseok Kang, Jaeyoung SongFederated Learning Aggregation Split Federated Learning
Department of Electronics Engineering, Pusan National University, Republic of Korea

Mar 16, 2026J/K move · Enter open · S save

Lightweight User-Personalization Method for Closed Split Computing

Yuya Okada, Takayuki Nishio

School of Engineering, Institute of Science Tokyo, Tokyo 152-8550, Japan

Edge Computing Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

Split Computing enables collaborative inference between edge devices and the cloud by partitioning a deep neural network into an edge-side head and a server-side tail, reducing latency and limiting exposure of raw input data. However, inference performance often degrades in practical deployments due to user-specific data distribution shifts, unreliable communication, and privacy-oriented perturbations, especially in closed environments where model architectures and parameters are inaccessible. To address this challenge, we propose SALT (Split-Adaptive Lightweight Tuning), a lightweight adaptation framework for closed Split Computing systems. SALT introduces a compact client-side adapter that refines intermediate representations produced by a frozen head network, enabling effective model adaptation without modifying the head or tail networks or increasing communication overhead. By modifying only the training conditions, SALT supports multiple adaptation objectives, including user personalization, communication robustness, and privacy-aware inference. Experiments using ResNet-18 on CIFAR-10 and CIFAR-100 show that SALT achieves higher accuracy than conventional retraining and fine-tuning while significantly reducing training cost. On CIFAR-10, SALT improves personalized accuracy from 88.1% to 93.8% while reducing training latency by more than 60%. SALT also maintains over 90% accuracy under 75% packet loss and preserves high accuracy (about 88% at sigma = 1.0) under noise injection. These results demonstrate that SALT provides an efficient and practical adaptation framework for real-world Split Computing systems.

Abstract

Federated Learning

Explore similar work

Lightweight User-Personalization Method for Closed Split Computing

When Is Shallow Enough? Adaptive Split Federated Learning with Client-Specific Sufficiency Estimation

Separate Aggregation of Split Network for Personalized Federated Learning