cs.LGNov 10, 2026

PSL-RGHA: Parallel Split Learning with Reference Gradient Half-Space Alignment for Mitigating Gradient Divergence

Authors: Han Li, Zheng Lin, Miao Yang, Zehang Lin, Jing Yang, Zihan Fang, Qiyang Zhang, Zhaolu Kang, 

Organizations: Xiamen University of Technology Xiamen, Fujian, China · SnT, University of Luxembourg Luxembourg · Universiti Malaya Kuala Lumpur, Malaysia · City University of Hong Kong Hong Kong, China · 

Abstract

The escalating complexity of deep neural networks hinders the deployment of federated learning (FL) on resource-constrained devices, especially for multimedia applications such as on-device image classification and speech recognition. Parallel split learning (PSL) has emerged as a promising solution by offloading a substantial portion of the computational workload to the server through layer-wise model partitioning, making it particularly suitable for resource-limited visual and speech intelligence scenarios. However, the absence of client-side aggregation renders PSL sus ceptible to severe training divergence and potential convergence failure under heterogeneous data distributions. To address this challenge, we propose PSL-RGHA, a gradient-aligned PSL framework that employs reference gradient half-space alignment mechanism to mitigate training divergence among client devices. PSL-RGHA comprises two novel components: reference gradient progressive construction (RGPC) and gradient half-space alignment (GHA). Specifically, RGPC evaluates the directional consistency of client gradients and progressively synthesizes a robust reference gradient via Weiszfeld updates, effectively approximating the global descent direction. Then, GHA constrains client gradients within the reference half-space defined by RGPC to mitigate gradient conflicts and rectify local updates. Extensive experiments demonstrate that our PSL-RGHA framework outperforms existing baselines in various settings.

Figures & tables

Explore similar work

CardsList
  1. GAPSL: A Gradient-Aligned Parallel Split Learning over Data-Heterogeneous Edge Computing Systems

    Mar 19, 2026Zheng Lin, Ons Aouedi, Zihan Fang +4Split Federated LearningGradient Alignment

  2. Personalized Federated Learning for Gradient Alignment

    May 4, 2026Dongwon Kim, Gyuejeong LeeFederated Learning AggregationPAC-Bayesian Theory