Organizations: School of Computer and Information Engineering, Xiamen University of Technology, Xiamen, 361000, China · School of Artificial Intelligence, Hebei University of Technology · Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong · Center of Research for Cyber Security and Network (CSNET), Faculty of Computer Science and Information Technology, Universiti Malaya, 50603 Kuala Lumpur, Malaysia · Department of Electrical Engineering, Chalmers University of Technology, 41296 Gothenburg, Sweden · Institute for Imaging, Data and Communications, School of Engineering, The University of Edinburgh, EH9 3JL Edinburgh, U.K. · Department of Computer Science, City University of Hong Kong, Kowloon, Hong Kong SAR, China · School of Engineering, Edith Cowan University, Perth, WA 6027, Australia
The growing complexity of neural networks hinders the deployment of distributed machine learning on resource-constrained devices. Split learning (SL) offers a promising solution by partitioning the large model and offloading the primary training workload from edge devices to an edge server. However, the increasing number of participating devices and model complexity leads to significant communication overhead from the transmission of smashed data (e.g., activations and gradients), which constitutes a critical bottleneck for SL. To tackle this challenge, we propose SL-FAC, a communication-efficient SL framework comprising two key components: adaptive frequency decomposition (AFD) and frequency-based quantization compression (FQC). AFD first transforms the smashed data into the frequency domain and decomposes it into spectral components with distinct information. FQC then applies customized quantization bit widths to each component based on its spectral energy distribution. This collaborative approach enables SL-FAC to achieve significant communication reduction while strategically preserving the information most crucial for model convergence. Extensive experiments confirm the superior performance of SL-FAC for improving the training efficiency.
Figures & tables
Fig. 1: The architecture of the proposed SL-FAC framework. (a) AFD for transforming the smashed data into the frequency domain and decoupling it into components corresponding to distinct information types; (b) FQC for customizing quantization bit widths for each component. Here, Lf- and Hf-features (gradients) denote low and high frequency features (gradients), while Le- and He-feature represent low and high energy features, respectively.
Fig. 2: The training performance of SL-FAC on the MNIST and HAM10000 datasets (from top to bottom), under IID (left) and non-IID (right) settings.
Fig. 3: The impact of energy threshold θ on model performance using the MNIST dataset under IID and non-IID settings.
Fig. 4: The ablation experiments for AFD and FQC (from top to bottom) on the MNIST dataset under IID (left) and non-IID (right) settings.
School of Physics and Information Engineering, Fuzhou University, Fuzhou 350108, China · Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong · School of Electrical Engineering and Telecommunications, University of New South Wales Sydney, Sydney, NSW 2052, Australia +3
Department of Electrical and Computer Engineering, The University of Hong Kong, Hong Kong SAR, China · Department of Electrical Engineering, California Institute of Technology, Pasadena, CA, USA