ParaBlock++: Communication-Computation Parallel Blockwise Hierarchical Federated Learning

Kai Wang; Chee Wei Tan

cs.LGOct 31, 2026

ParaBlock++: Communication-Computation Parallel Blockwise Hierarchical Federated Learning

Authors: Kai Wang, Chee Wei Tan

Organizations: College of Computing and Data Science, Nanyang Technological University, Singapore 639798

Abstract

In the emerging 6G paradigm, where AI-driven services operate over decentralized and resource-constrained infrastructures, the convergence of distributed learning algorithms remains a significant challenge. This paper introduces $\mathrm{ParaBlock}$ ++, a low Earth orbit (LEO) satellite-assisted hierarchical federated learning (HFL) framework, enabling resource-limited terrestrial clients to collaboratively engage in federated learning (FL) via ground-satellite links (GSLs) supported by LEO satellite megaconstellations. Within this architecture, LEO satellites serve multiple roles: as edge servers aggregating and broadcasting models, as clients accelerating training, and as high-throughput relays managing inter-satellite routing through inter-satellite links (ISLs). Departing from conventional synchronous Jacobian-type updates, $\mathrm{ParaBlock}$ ++ employs asynchronous BCD-type updates to enhance convergence speed. We provide theoretical convergence guarantees for convex, strongly convex, and non-convex objectives, and formulate optimization problems to determine optimal cluster resource allocation and model partitioning for minimizing global training latency. Experimental evaluations demonstrate that $\mathrm{ParaBlock}$ ++ achieves faster initial convergence and comparable final performance for deep neural networks (DNNs) such as ResNet-18, offering promising directions for federated learning across geographically remote areas enabled by 6G technologies, but much less global rounds and communication cost.

Figures & tables

	$\displaystyle T^{\text{total}}_{i}=\begin{cases}T\cdot N_{i}+\max_{j\in\mathcal{C}_{i}}\big\{\tau^{\text{agg}}_{C,j}\big\},&\text{ (case$ a $)$ \qquad\qquad\qquad\qquad\qquad\qquad\qquad $}\\ \max_{j\in C_{i}}\big\{\max\big(T\cdot\big\lfloor\dfrac{T^{\text{cluster}}_{C,j^{\prime}}}{T}\big\rfloor,T^{\text{cluster}}_{C,j}\big)+\tau^{\text{agg}}_{C,j}\big\},&\text{ (case$ b $)$ \qquad\qquad\qquad\qquad\qquad\qquad\qquad $}\\ T\cdot\big(\big\lfloor\dfrac{T^{\text{cluster}}_{C,j^{\prime}}}{T}\big\rfloor+1\big)+\max_{j\in C_{i}}\big\{\tau^{\text{agg}}_{C,j}\big\},&\text{ (case$ c $)$ \qquad\qquad\qquad\qquad\qquad\qquad\qquad $}\end{cases}\qquad$		(14)
	$\displaystyle\begin{cases}\text{case$ a $:}&T^{\text{cluster}}_{C,j^{\prime}}\leq T\cdot N_{i}\\ \text{case$ b $:}&T^{\text{cluster}}_{C,j^{\prime}}>T\cdot N_{i}\text{ and }\max_{j\in C_{i}}\big\{\max\big(T\cdot\big\lfloor\dfrac{T^{\text{cluster}}_{C,j^{\prime}}}{T}\big\rfloor,T^{\text{cluster}}_{C,j}\big)\!+\!\tau^{\text{agg}}_{C,j}\big\}\!\leq\!T\cdot\big(\big\lfloor\dfrac{T^{\text{cluster}}_{C,j^{\prime}}}{T}\big\rfloor\!+\!1\big)\\ \text{case$ c $:}&T^{\text{cluster}}_{C,j^{\prime}}>T\cdot N_{i}\text{ and }\max_{j\in C_{i}}\big\{\max\big(T\cdot\big\lfloor\dfrac{T^{\text{cluster}}_{C,j^{\prime}}}{T}\big\rfloor,T^{\text{cluster}}_{C,j}\big)\!+\!\tau^{\text{agg}}_{C,j}\big\}\!>\!T\cdot\big(\big\lfloor\dfrac{T^{\text{cluster}}_{C,j^{\prime}}}{T}\big\rfloor\!+\!1\big)\end{cases}$		(15)

Table 1

Fig. 1: An illustration of the submodel transmission scheme. Solid lines denote the transmission stage and dotted lines denote the propagation stage. For example, cluster 2 needs

S^{1}_{1}

to train

S^{2}_{2}

t=2

and hence

S^{1}_{1}

must be received before

t=2

, corresponding to the transmission stage. Although cluster 3 also needs

S^{1}_{1}

to train

S^{3}_{3}

t=3

, it is time-insensitive and can be received during training at

t=2

, corresponding to the propagation stage.

Parameter	Value	Parameter	Value	Parameter	Value
$f_{S,i}$	$1$ GHz	$m_{S,i}\ ^{\dagger}$	$3\times 10^{7}$ cycles/sample	$P_{S,i}$	$10$ W
$f_{C,j}$	$3\times 10^{8}$ Hz	$m_{C,j}\ ^{\dagger}$	$3\times 10^{7}$ cycles/sample	$P_{C,j}$	$0.3$ W
$T$	$360$ sec	$\xi$	$2$	$N_{0}$	$3.98\times 10^{-21}$ W/Hz
$\kappa$	$10^{-28}$ Joules $\cdot$ seconds 2 /cycles 3	$B_{i}$	$10$ MHz	$-$	$-$

TABLE I: Simulation Settings for the LEO Megaconstellation

Fig. 2: Comparison for LR with default hyperparameters: balanced partition,

T\!=\!50

E\!=\!5

H\!=\!10

\#\text{clusters}\!=\!4

\#\text{clients}\_\text{per}\_\text{cluster}\!=\!2

\alpha_{\text{inter}}\!=\!10.0

\alpha_{\text{intra}}\!=\!3.0

\alpha^{t}\!=\!0.01/(t+1)

and batch size

b\!=\!32

. Upper left: varying

E

and

H

, upper right: varying

\#\text{clients}\_\text{per}\_\text{cluster}

, lower left: varying

\alpha_{\text{inter}}

and

\alpha_{\text{intra}}

, lower right: varying partition balance and

\alpha^{t}

Fig. 3: Comparison for Crammer-Singer SVM with the same default parameters and subplots as for LR.

Fig. 4: Comparison for ResNet-18 with the same default parameters and subplots as for LR.

Model	Avg. Comp. Time	Avg. Comm. Time
$\mathrm{ParaBlock}$ ++	$\sum_{i\in\mathcal{I}}\bar{T}^{\text{comp}}_{i}$	$T^{\text{stsp}}$
HIST/FedBCD	$\sum_{i\in\mathcal{I}}\bar{T}^{\text{comp}}_{i}$	$\max_{i\in\mathcal{I}}T^{\text{dst}}_{i}$

TABLE II: Computation-Communication Time Per Global Round

Fig. 5:

\mathrm{ParaBlock}

++ for ResNet-18 with

T=50

E=20

H=20

under i.i.d. and non-i.i.d., respectively.

Block Coordinate Descent

See all 8 papers

Oct 31, 2026

Hierarchical Asynchronous Federated Learning over Space-Air-Ground Integrated Networks

With the emerging 6G paradigm where AI services operate over decentralized and resource-constrained edge infrastructure, performance of distributed learning algorithms remains a significant challenge. This paper proposes HAFL, a Low Earth Orbit (LEO) satellite-assisted hierarchical asynchronous Federated Learning (FL) framework, enabling resource-limited terrestrial clients to collaborate via Ground-Satellite Links (GSLs). Within this framework, LEO satellites serve multiple roles: as edge servers aggregating and broadcasting models, as clients accelerating training, and as high-throughput relays managing inter-satellite routing through Inter-Satellite Links (ISLs). Unlike conventional synchronous Jacobian-type updates, HAFL employs asynchronous Gauss-Seidel-type updates to enhance convergence speed. We provide theoretical convergences for convex, strongly convex, and nonconvex objectives, respectively, and formulate optimization problems to jointly determine optimal hyperparameters to minimize global training latency. Evaluations demonstrate that HAFL achieves faster initial convergence and comparable final performance for benchmark DNNs and datasets, offering promising FL systems across remote areas with 6G technologies, but much less global rounds and communication cost.

Kai Wang, Chee Wei Tan

College of Computing and Data Science, Nanyang Technological University, Singapore 639798

Federated Learning Hierarchical Federated Learning Asynchronous Federated Learning

Ask PDF

Aug 4, 2026

FedRings: A Scalable and Topology-Aware Federated Learning Framework for LEO Satellite Constellations

Federated learning over low Earth orbit (LEO) satellite networks is limited by frequent link changes, short contact times, and a highly dynamic topology, making centralized or synchronized training inefficient and hard to scale. To address this, we propose FedRings, a decentralized framework that organizes satellites into ring-based communication structures. It uses a spatio-temporal routing strategy with link-aware communication scheduling to align model exchange with actual visibility windows and time-varying connectivity patterns in LEO. Model updates are propagated along the ring using adaptive sparse incremental aggregation, which reduces communication overhead by progressively combining and compressing updates. To handle communication interruptions, a historical compensation mechanism maintains training continuity. By combining topology-aware routing, communication scheduling, and efficient aggregation, FedRings enables stable and efficient learning in dynamic LEO networks while reducing communication cost, and experiments show it consistently outperforms existing methods in realistic settings.

Ziwu Liu, Inês Pinto Gouveia, Rehana Yasmin +2

CEMSE Division, King Abdullah University of Science and Technology (KAUST) Thuwal 23955-6900, Kingdom of Saudi Arabia

Communication-Efficient Distributed Training Decentralized Learning Wireless Communications LEO Satellite Networks Federated Learning

Ask PDF

Aug 10, 2026

FedOrbit: Adaptive Personalized Federated Learning for Non-IID LEO Satellite Constellations

Federated learning (FL) in Low Earth Orbit (LEO) satellite constellations is affected by non-IID data and irregular ground-station visibility, both driven by orbital geometry. Global aggregation performs poorly when orbit-level class distributions are disjoint, while strong personalisation can be excessive when these distributions overlap. We present FedOrbit, which combines continuous orbit-level training over inter-satellite links, class-aware hierarchical aggregation, quality-weighted feature aggregation with return-rate dampening, and adaptive feature decomposition based on inter-orbit class similarity. Across three remote-sensing benchmarks and two non-IID partitions, FedOrbit achieves the highest accuracy in five of six settings and is within

0.9

percentage points of the best result in the sixth. The gains over the strongest baseline reach

16.1

percentage points under Dirichlet partitioning and

8.6

under pathological partitioning, with the smallest per-orbit accuracy spread in five of six settings.

Satwat Bashir, Tasos Dagiuklas, Muddesar Iqbal

School of Engineering London South Bank University, London, UK

Hierarchical Federated Learning Federated Learning Aggregation Non-IID Federated Learning LEO Satellite Networks Federated Learning

Ask PDF

Abstract

Block Coordinate Descent

Explore similar work

Hierarchical Asynchronous Federated Learning over Space-Air-Ground Integrated Networks

FedRings: A Scalable and Topology-Aware Federated Learning Framework for LEO Satellite Constellations

FedOrbit: Adaptive Personalized Federated Learning for Non-IID LEO Satellite Constellations