Exploiting Adaptive Channel Pruning for Communication-Efficient Split Learning

Jialei Tan; Zheng Lin; Xiangming Cai; Ruoxi Zhu; Zihan Fang; Pingping Chen; Wei Ni

cs.LGMar 10, 2026

Exploiting Adaptive Channel Pruning for Communication-Efficient Split Learning

Authors: Jialei Tan, Zheng Lin, Xiangming Cai, Ruoxi Zhu, Zihan Fang, Pingping Chen, Wei Ni

Organizations: School of Physics and Information Engineering, Fuzhou University, Fuzhou 350108, China · Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong · School of Electrical Engineering and Telecommunications, University of New South Wales Sydney, Sydney, NSW 2052, Australia · Hong Kong JC STEM Lab of Smart City and the Department of Computer Science, City University of Hong Kong, Kowloon, Hong Kong SAR, China ·

Abstract

Split learning (SL) transfers most of the training workload to the server, which alleviates computational burden on client devices. However, the transmission of intermediate feature representations, referred to as smashed data, incurs significant communication overhead, particularly when a large number of client devices are involved. To address this challenge, we propose an adaptive channel pruning-aided SL (ACP-SL) scheme. In ACP-SL, a label-aware channel importance scoring (LCIS) module is designed to generate channel importance scores, distinguishing important channels from less important ones. Based on these scores, an adaptive channel pruning (ACP) module is developed to prune less important channels, thereby compressing the corresponding smashed data and reducing the communication overhead. Experimental results show that ACP-SL consistently outperforms benchmark schemes in test accuracy. Furthermore, it reaches a target test accuracy in fewer training rounds, thereby reducing communication overhead.

Figures & tables

Fig. 2: Effect of channel importance on the training loss of SL.

Fig. 3: Effects of the instantaneous channel importance score

S_{i,\rm{Inst}}^{t}

, the historical channel importance score

{S}_{i,\rm{Hist}}^{t}

, and the combined channel importance score

{S}_{i,\rm{Comb}}^{t}

on the training loss and test accuracy of SL.

Fig. 4: Test accuracy on CIFAR-10 and Fashion-MNIST under IID and non-IID settings.

Fig. 5: Ablation experiment results of the proposed LCIS on CIFAR-10 and Fashion-MNIST under the non-IID setting.

Fig. 6: Ablation experiment results of the proposed ACP on CIFAR-10 and Fashion-MNIST under the non-IID setting.

Distributed Training

Distributed training coordinates computation across multiple devices or machines to train models that exceed single-device limits.

See all 127 papers

SL-FAC: A Communication-Efficient Split Learning Framework with Frequency-Aware Compression
Apr 8, 2026Zehang Lin, Miao Yang, Haihan Zhu +9Activation Compression Communication-Efficient Distributed Training
School of Computer and Information Engineering, Xiamen University of Technology, Xiamen, 361000, China · School of Artificial Intelligence, Hebei University of Technology · Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong +5
LightSplit: Practical Privacy-Preserving Split Learning via Orthogonal Projections
May 13, 2026Mert Cihangiroglu, Alessandro Pegoraro, Phillip Rieger +2Communication-Efficient Distributed Training Privacy-Preserving ML
University of Pavia · 2Technical University of Darmstadt
Importance-Aware Feature Sparsification for Wireless Split Learning
Sep 30, 2026Bumjun Kim, Yoon Huh, Wan ChoiCommunication-Efficient Distributed Training Split Federated Learning
Department of Electrical and Computer Engineering, Seoul National University (SNU), and the Institute of New Media and Communications, SNU, Seoul 08826, Korea

Apr 8, 2026J/K move · Enter open · S save

SL-FAC: A Communication-Efficient Split Learning Framework with Frequency-Aware Compression

Zehang Lin, Miao Yang, Haihan Zhu, Zheng Lin+8

School of Computer and Information Engineering, Xiamen University of Technology, Xiamen, 361000, China · School of Artificial Intelligence, Hebei University of Technology · Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong+5

Activation Compression Communication-Efficient Distributed Training Distributed Training Activation Quantization Neural Network Compression

Open paper AskPDF arXiv

The growing complexity of neural networks hinders the deployment of distributed machine learning on resource-constrained devices. Split learning (SL) offers a promising solution by partitioning the large model and offloading the primary training workload from edge devices to an edge server. However, the increasing number of participating devices and model complexity leads to significant communication overhead from the transmission of smashed data (e.g., activations and gradients), which constitutes a critical bottleneck for SL. To tackle this challenge, we propose SL-FAC, a communication-efficient SL framework comprising two key components: adaptive frequency decomposition (AFD) and frequency-based quantization compression (FQC). AFD first transforms the smashed data into the frequency domain and decomposes it into spectral components with distinct information. FQC then applies customized quantization bit widths to each component based on its spectral energy distribution. This collaborative approach enables SL-FAC to achieve significant communication reduction while strategically preserving the information most crucial for model convergence. Extensive experiments confirm the superior performance of SL-FAC for improving the training efficiency.

Abstract

Distributed Training

Explore similar work

SL-FAC: A Communication-Efficient Split Learning Framework with Frequency-Aware Compression

LightSplit: Practical Privacy-Preserving Split Learning via Orthogonal Projections

Importance-Aware Feature Sparsification for Wireless Split Learning