DeFRiS: Silo-Cooperative IoT Applications Scheduling via Decentralized Federated Reinforcement Learning
Authors: Zhiyu Wang, Mohammad Goudarzi, Mingming Gong, Rajkumar Buyya
Organizations: The Quantum Cloud Computing and Distributed Systems (qCLOUDS) Laboratory, School of Computing and Information Systems, The University of Melbourne, Australia · The Faculty of Information Technology, Monash University, Australia · School of Mathematics and Statistics, The University of Melbourne, Australia
Next-generation IoT applications increasingly span across autonomous administrative entities, necessitating silo-cooperative scheduling to leverage diverse computational resources while preserving data privacy. However, realizing efficient cooperation faces significant challenges arising from infrastructure heterogeneity, Non-IID workload shifts, and the inherent risks of adversarial environments. Existing approaches, relying predominantly on centralized coordination or independent learning, fail to address the incompatibility of state-action spaces across heterogeneous silos and lack robustness against malicious attacks. This paper proposes DeFRiS, a Decentralized Federated Reinforcement Learning framework for robust and scalable Silo-cooperative IoT application scheduling. DeFRiS integrates three synergistic innovations: (i) an action-space-agnostic policy utilizing candidate resource scoring to enable seamless knowledge transfer across heterogeneous silos; (ii) a silo-optimized local learning mechanism combining Generalized Advantage Estimation (GAE) with clipped policy updates to resolve sparse delayed reward challenges; and (iii) a Dual-Track Non-IID robust decentralized aggregation protocol leveraging gradient fingerprints for similarity-aware knowledge transfer and anomaly detection, and gradient tracking for optimization momentum. Extensive experiments on a distributed testbed with 20 heterogeneous silos and realistic IoT workloads demonstrate that DeFRiS significantly outperforms state-of-the-art baselines, reducing average response time by 6.4% and energy consumption by 7.2%, while lowering tail latency risk (CVaR0.95) by 10.4% and achieving near-zero deadline violations. Furthermore, DeFRiS achieves over 3 times better performance retention as the system scales and over 8 times better stability in adversarial environments compared to the best-performing baseline.
Figures & tables
Work
System & Application Properties
Problem Formulation
Approach Properties
Evaluation
Task Number
Task Dependency
Multi- Silo
Time
Energy
Multi- Objective
QoS
Learning Paradigm
Algorithm
Coordination Architecture
Action Space
Sparse Reward Handling
Non-IID Handling
Aggregation
Robustness
Edge
Cloud
IoT Apps
Hetero- geneity
[ 18 ]
Multiple
✓
×
✓
×
×
×
Independent
DQN
N/A
Fixed
×
N/A
N/A
×
○
○
○
✓
[ 19 ]
Multiple
✓
×
✓
✓
✓
✓
DQN
N/A
Fixed
×
N/A
N/A
×
○
○
◐
×
[ 20 ]
Multiple
✓
×
✓
×
×
×
PPO
N/A
Fixed
✓
N/A
N/A
×
○
○
◐
✓
[ 21 ]
Multiple
✓
×
✓
×
✓
×
PPO
N/A
Fixed
✓
N/A
N/A
×
●
●
●
✓
[ 22 ]
Multiple
✓
×
✓
×
✓
✓
D3QN
N/A
Fixed
×
N/A
N/A
×
○
○
●
✓
TABLE I : Qualitative Comparison of IoT Application Scheduling Approaches
Fig. 1 : The distributed IoT system model illustrating autonomous silos with heterogeneous resources connected for cooperative learning.
Fig. 2 : The overview of the DeFRiS framework. It consists of three synergistic components: (1) Action-Space-Agnostic Policy enables parameter sharing across heterogeneous silos via candidate scoring; (2) Silo-Optimized Local Learning ensures stable convergence under sparse rewards using GAE and clipped updates; (3) Dual-Track Non-IID Robust Decentralized Aggregation facilitates robust and similarity-aware knowledge transfer through gradient fingerprints and tracking.
Parameter
Value
Objective Function
CVaR risk level ( α )
0.95
CVaR weight ( β )
0.3
Response time weight ( λRT(i) )
0.5
Energy consumption weight ( λEnergy(i) )
0.5
Local Training
TABLE II : Hyperparameter Configuration
Fig. 3 : Convergence performance comparison across 100 training iterations.
Figure 6
Fig. 6 : Scalability evaluation showing average weighted cost across different numbers of silos.
Fig. 7 : Robustness evaluation in adversarial environments.
Federated split learning (FSL) enables collaborative training across bandwidth-constrained IoT devices, but repeated activation and gradient exchange creates a communication bot-tleneck. Prior work optimises either activation compression or synchronisation frequency in isolation. This paper presents an FSL framework for IoT rainfall prediction that jointly regulates activation compression and the synchronisation interval \r{ho} via a latency driven scheduler on a server with per client EMA smoothing. The system is evaluated on hourly ERA5 data from 11 weather stations through a 17 scenario simulation matrix and a four scenario Raspberry Pi deployment over a real wide-area link. The simulation matrix validates scheduler switching across low, high, and mixed latency profiles, while the Pi deployment validates the high latency endpoint selected by the same policy. AUPRC varies only slightly across configurations (0.6381-0.6484 in simulation; within 0.011 on Pi), indicating that aggressive quantisation and sparser aggregation do not materially degrade predictive quality in this setting. On Pi, the selected endpoint (int8 with rho=3) achieves an 87% reduction in activation upload payload and a 54% reduction in synchronisation traffic relative to the float32 baseline, while reducing runtime jitter from +/-688 s to +/-10 s.
Wenjie Ding, Yi Sin Lin, Jiale Liu +8
School of Computing · Newcastle University · Newcastle upon Tyne, United Kingdom
With the rapid development of the Internet of Things (IoT), federated learning (FL) has gained increasing attention for its privacy-preserving use of distributed data. However, conventional two-tier FL architectures are poorly suited to the hierarchical and heterogeneous nature of real-world IoT systems. Hierarchical Federated Learning (HFL) introduces multi-layer aggregation to better match IoT environments, but still suffers from communication inefficiencies and performance limitations caused by large data transfers, non-IID data distributions, and uneven device participation. These challenges hinder the realization of low-latency and high-accuracy training in practical IoT deployments. To address these limitations, we propose H-FedSN for practical IoT environments. H-FedSN leverages a binary mask mechanism with shared and personalized layers to reduce communication overhead by creating a sparse network without altering original weights. To tackle data heterogeneity and imbalanced device distribution, H-FedSN incorporates personalized layers for local data adaptation and employs Bayesian aggregation with cumulative Beta distribution updates at edge and cloud levels, effectively balancing contributions from diverse client groups. Experiments on three real-world IoT datasets and MNIST under non-IID conditions show that H-FedSN reduces communication costs by up to 477 times compared to baseline methods while maintaining high accuracy, making it well-suited for hierarchical FL in IoT deployments.
In recent years, the Internet of Things (IoT) paradigm has been shifting toward batteryless, energy-harvesting architectures. Sustaining reliable operation in these systems requires intelligent management of highly volatile stored energy. As edge applications grow in complexity, traditional energy-aware schedulers struggle with unpredictable workloads due to their reliance on static execution thresholds or pre-measured, hardware-specific task profiles. To overcome this, we propose two novel, hardware-agnostic dynamic scheduling strategies treating applications as a "black box," requiring no prior energy information: a model-free Reinforcement Learning (RL) agent and an on-the-fly Approximated Prediction (AP) method. We evaluate these methods against an adaptive task rate approach (AsTAR) and optimized static thresholds using a custom-built, physically accurate simulation framework driven by real-world solar data and dynamic LoRa transmission profiles. Rather than claiming universal superiority, our analysis exposes the distinct operational trade-offs of each method: the AP approach delivers lightweight, near-oracle task throughput; the RL agent provides tunable survival-execution balancing; and AsTAR excels at execution pacing across long energy gaps. Finally, we demonstrate that while these advanced strategies provide critical resilience for severely constrained systems with small capacitors, devices with larger energy buffers can efficiently rely on simpler, less computationally expensive static policies.