DeFRiS: Silo-Cooperative IoT Applications Scheduling via Decentralized Federated Reinforcement Learning
Organizations: The Quantum Cloud Computing and Distributed Systems (qCLOUDS) Laboratory, School of Computing and Information Systems, The University of Melbourne, Australia · The Faculty of Information Technology, Monash University, Australia · School of Mathematics and Statistics, The University of Melbourne, Australia
Abstract
Next-generation IoT applications increasingly span across autonomous administrative entities, necessitating silo-cooperative scheduling to leverage diverse computational resources while preserving data privacy. However, realizing efficient cooperation faces significant challenges arising from infrastructure heterogeneity, Non-IID workload shifts, and the inherent risks of adversarial environments. Existing approaches, relying predominantly on centralized coordination or independent learning, fail to address the incompatibility of state-action spaces across heterogeneous silos and lack robustness against malicious attacks. This paper proposes DeFRiS, a Decentralized Federated Reinforcement Learning framework for robust and scalable Silo-cooperative IoT application scheduling. DeFRiS integrates three synergistic innovations: (i) an action-space-agnostic policy utilizing candidate resource scoring to enable seamless knowledge transfer across heterogeneous silos; (ii) a silo-optimized local learning mechanism combining Generalized Advantage Estimation (GAE) with clipped policy updates to resolve sparse delayed reward challenges; and (iii) a Dual-Track Non-IID robust decentralized aggregation protocol leveraging gradient fingerprints for similarity-aware knowledge transfer and anomaly detection, and gradient tracking for optimization momentum. Extensive experiments on a distributed testbed with 20 heterogeneous silos and realistic IoT workloads demonstrate that DeFRiS significantly outperforms state-of-the-art baselines, reducing average response time by 6.4% and energy consumption by 7.2%, while lowering tail latency risk (CVaR) by 10.4% and achieving near-zero deadline violations. Furthermore, DeFRiS achieves over 3 times better performance retention as the system scales and over 8 times better stability in adversarial environments compared to the best-performing baseline.
Figures & tables
| Work | System & Application Properties | Problem Formulation | Approach Properties | Evaluation | |||||||||||||||
| Task Number | Task Dependency | Multi- Silo | Time | Energy | Multi- Objective | QoS | Learning Paradigm | Algorithm | Coordination Architecture | Action Space | Sparse Reward Handling | Non-IID Handling | Aggregation | Robustness | Edge | Cloud | IoT Apps | Hetero- geneity | |
| [ 18 ] | Multiple | ✓ | ✓ | Independent | DQN | N/A | Fixed | N/A | N/A | ○ | ○ | ○ | ✓ | ||||||
| [ 19 ] | Multiple | ✓ | ✓ | ✓ | ✓ | ✓ | DQN | N/A | Fixed | N/A | N/A | ○ | ○ | ◐ | |||||
| [ 20 ] | Multiple | ✓ | ✓ | PPO | N/A | Fixed | ✓ | N/A | N/A | ○ | ○ | ◐ | ✓ | ||||||
| [ 21 ] | Multiple | ✓ | ✓ | ✓ | PPO | N/A | Fixed | ✓ | N/A | N/A | ● | ● | ● | ✓ | |||||
| [ 22 ] | Multiple | ✓ | ✓ | ✓ | ✓ | D3QN | N/A | Fixed | N/A | N/A | ○ | ○ | ● | ✓ | |||||
| Parameter | Value |
|---|---|
| Objective Function | |
| CVaR risk level ( ) | 0.95 |
| CVaR weight ( ) | 0.3 |
| Response time weight ( ) | 0.5 |
| Energy consumption weight ( ) | 0.5 |
| Local Training | |
Federated RL
Federated reinforcement learning lets distributed agents learn policies collaboratively while keeping their local data and experience private.
See all 33 papers