Concurrent Split Learning Through Stable Client Clustering

Mohammad Kohankhaki; Valentin Rentschler; Anke Schmeink

cs.DCSep 24, 2026

Concurrent Split Learning Through Stable Client Clustering

Authors: Mohammad Kohankhaki, Valentin Rentschler, Anke Schmeink

Organizations: Chair of Information Theory and Data Analytics, RWTH Aachen University, Aachen, Germany

Abstract

Training with a fixed global batch limits how many distributed clients can provide examples in any one step. We examine a way to use additional server workers without increasing the batch processed by an individual workload. Global Clustered Parallel Split Learning (GCPSL) assigns clients to fixed clusters, executes a Parallel Split Learning with Global Sampling (GPSL) workload for each cluster concurrently, and periodically fuses the client and server model segments. In simulations with 256 logical clients, dividing the population across more workloads improves direct data participation, while smaller clusters can incur an accuracy cost. A four-H100 implementation of label-aware GCPSL reaches 85% CIFAR-10 validation accuracy in $6.13 \pm 0.15$ minutes over three matched runs, versus $19.09 \pm 0.45$ minutes when the same workloads are serialized. Within the four-GPU allocation, size-balanced and random fixed affiliations reach the target in similar mean times (5.70 and 5.66 minutes); size balancing increases direct participation by 3.25 percentage points. These measurements characterize a trade-off among execution concurrency, assignment information, participation, and accuracy for stable-client split learning.

Figures & tables

Fig. 2: Execution organization. GPSL uses one global workload with batch size $B$ , so only clients represented in that batch supply data. GCPSL assigns clients to stable clusters and runs one GPSL workload with batch size $B$ in each cluster. The cluster workloads execute concurrently, and their split-model replicas are fused at periodic barriers. Clients that supply no data in a round still receive the synchronized client-model update.

Method	Assignment	$N$	Acc. (%) $\uparrow$	$R_{85}$ $\downarrow$	$R_{88}$ $\downarrow$	Inactivity (%) $\downarrow$
GPSL	global	–	89.40 $\pm$ 0.24	6.78 $\pm$ 0.45	14.86 $\pm$ 2.07	78.23 $\pm$ 0.03
GCPSL	label-aware	2	91.36 $\pm$ 0.10	4.44 $\pm$ 0.22	8.36 $\pm$ 0.83	61.69 $\pm$ 0.07
		4	90.17 $\pm$ 0.44	3.85 $\pm$ 0.14	6.45 $\pm$ 0.64	39.30 $\pm$ 0.16
		8	88.93 $\pm$ 0.15	3.40 $\pm$ 0.10	6.76 $\pm$ 0.81	17.32 $\pm$ 0.15
		16	86.64 $\pm$ 0.47	3.61 $\pm$ 0.02	–	4.35 $\pm$ 0.21
	size-balanced	2	90.91 $\pm$ 0.88	4.69 $\pm$ 0.39	7.95 $\pm$ 0.45	61.69 $\pm$ 0.07

TABLE I: Cluster-count study.

Fixed-assignment comparison
Method	GPUs	$t_{85}$ (min) $\downarrow$	GPU-min $\downarrow$	Acc. (%) $\uparrow$	Best-val (%) $\uparrow$	Part. (%) $\uparrow$	Payload (TB) $\downarrow$
Random	4	5.66 $\pm$ 0.18	22.62	89.69	90.07	56.96	1.461
Size-balanced	4	5.70 $\pm$ 0.50	22.81	89.32	89.52	60.21	1.483
Label-aware	4	6.13 $\pm$ 0.15	24.51	89.53	89.73	59.43	1.528
Other reference baselines
Single-worker	1	10.88 $\pm$ 0.58	10.88	89.59	89.83	21.70	1.654

TABLE II: Real-execution results. Values summarize three matched runs.

Execution	Fusion interval	Acc. (%) $\uparrow$	$R_{85}$ $\downarrow$	$R_{88}$ $\downarrow$
Concurrent	one epoch	88.93	3.40	6.76
Sequential	one epoch	89.06	26.19	50.02
Concurrent	100 steps	87.95	3.20	8.95
Concurrent	none	20.59	–	–

TABLE III: Fusion and payload controls.

Method	$B$	$N$	$R_{50}$	$R_{60}$	Acc. (%)
GPSL	128	–	4.95	15.38	59.70
Label-aware GCPSL	128	4	3.22	7.34	61.25
Size-balanced GCPSL	128	4	3.59	9.11 *	59.43
GPSL	512	–	0.95	1.80	64.36

TABLE IV: CIFAR-100 boundary.

Explore similar work

Importance-Aware Feature Sparsification for Wireless Split Learning
Sep 30, 2026Bumjun Kim, Yoon Huh, Wan ChoiSparsity Balanced Learning
Department of Electrical and Computer Engineering, Seoul National University (SNU), and the Institute of New Media and Communications, SNU, Seoul 08826, Korea
Separate Aggregation of Split Network for Personalized Federated Learning
May 26, 2026Yunseok Kang, Jaeyoung SongFederated Learning Heterogeneity
Department of Electronics Engineering, Pusan National University, Republic of Korea
Fed-BAC: Federated Bandit-Guided Additive Clustering in Hierarchical Federated Learning
May 12, 2026Satwat Bashir, Tasos Dagiuklas, Muddesar IqbalHeterogeneous Federated Learning Federated Learning
Computer Science and Informatics · London South Bank University · London, UK +1

Sep 30, 2026, cs.LGJ/K move · Enter open · S save

Importance-Aware Feature Sparsification for Wireless Split Learning

Bumjun Kim, Yoon Huh, Wan Choi

Department of Electrical and Computer Engineering, Seoul National University (SNU), and the Institute of New Media and Communications, SNU, Seoul 08826, Korea

Sparsity Balanced Learning Importance Top-K Bottlenecks

Open paper AskPDF arXiv

Wireless split learning (SL) reduces on-device computation by offloading upper layers to a server, yet transmitting high-dimensional intermediate features at each iteration remains a major communication bottleneck. Existing methods select features at the client side using task-agnostic criteria such as magnitude, statistics, or clustering, which increases client-side processing and often degrades accuracy under non-independent and identically distributed (non-i.i.d.) client data. We propose importance-aware class-balanced sparsification (ICS), a lightweight approach in which the server ranks feature channels using Grad-CAM-based scores obtained from the true-class logit during backpropagation. The per-class scores are aggregated into a class-balanced, label-agnostic importance vector that mitigates head-class bias under label skew, and each client reuses this vector in the next round to retain the top-

N

feature channels, incurring no additional client-side forward or backward passes. We further derive a non-asymptotic convergence bound that isolates the sparsification-induced error and characterizes how the sparsification ratio and mini-batch size jointly affect convergence under a fixed communication budget, and we analyze the communication and computational overhead of ICS against representative baselines. Beyond sequential CNN-based SL, we extend ICS to parallel split learning and to transformer-based models. Experiments show that ICS consistently outperforms the baselines, with larger gains under severe non-i.i.d. partitions.

Abstract

Explore similar work

Importance-Aware Feature Sparsification for Wireless Split Learning

Separate Aggregation of Split Network for Personalized Federated Learning

Fed-BAC: Federated Bandit-Guided Additive Clustering in Hierarchical Federated Learning