SLA-Aware Task Offloading in the IoT Edge-Cloud Continuum: Class-Weighted Deep Reinforcement Learning Against an Exact Optimum
Organizations: Deggendorf Institute of Technology (DIT), Cham, Germany · Technologietransferzentrum Kitzingen (TTZ-KT), Technische Hochschule Würzburg-Schweinfurt (THWS), Kitzingen, Germany · independent researcher
Abstract
Deep reinforcement learning (RL) is often used to decide where Internet of Things (IoT) tasks with different latency limits run in the edge-cloud continuum, yet some methods give every missed task the same penalty and few compare their agents with an optimal policy. This benchmark study builds four offloading systems small enough to solve exactly by dynamic programming and measures reward and learner against the optimum. A bonus for a met limit makes each miss cost its penalty plus the bonus, which shifts the class weights of published per-class rewards. A class-blind penalty stays 65% to 118% above the optimal cost even with a perfect learner, while a corrected per-class penalty comes within 0.13% of the optimum. With a per-class reward, tuned proximal policy optimization (PPO) agents come within 9% to 11% of the optimum in the smallest system, and the corrected one beats a tuned reservation rule in 9 of 10 seeds. In larger systems the agents stay 41% to 76% above the optimum, and a tuned rule beats their mean. Training ten times longer does not close this gap, and a PPO-Lagrangian agent with reachable targets does no better than the per-class reward.
Figures & tables
| , , | nodes, classes, steps | , | share, limit of class |
|---|---|---|---|
| , | queue, service prob. | cloud meets | |
| , | time, wait per task | 1 if task misses | |
| , | background, load | , , | bonus, penalty, cloud cost |
| , | state, action | , | flat penalty, scale |
| share of episode left | , , | weight, miss rate, cost | |
| discount factor | , | adaptive penalty, step |
| Nominal | Hetero3 | Hetero3-3C | Large4-3C | |
|---|---|---|---|---|
| , | 2, 2 | 3, 2 | 3, 3 | 4, 3 |
| 30,30 | 20,30,40 | 20,30,40 | 20,25,30,40 | |
| 15,15 | 10,15,20 | 10,15,20 | 10,12,15,20 | |
| .6,.6 | .7,.6,.5 | .7,.6,.5 | .7,.6,.6,.5 | |
| 0.2 | 0.8 | 0.8 | 1.4 | |
| Arrivals | fixed | random | random | random |
| System | Optimum | W | C | F | W | C | F |
|---|---|---|---|---|---|---|---|
| Nominal | 0.2614 | 1.0 | 0.0 | 95.2 | 3.3 | 0.0 | 95.2 |
| Hetero3 | 0.0899 | 1.1 | 0.1 | 64.6 | 1.7 | 0.8 | 115.5 |
| Hetero3-3C | 0.0839 | 10.4 | 0.0 | 116.2 | 10.5 | 0.1 | 169.7 |
| Large4-3C | 0.0777 | 4.9 | 0.1 | 118.2 | 5.5 | 4.1 | 174.0 |
| Policy | (%) | (%) | All (%) | (ms) | (ms) | Edge (%) | SLA cost | Gap | Beats |
|---|---|---|---|---|---|---|---|---|---|
| JSQ | 51.8 | 14.4 | 33.1 | 61.5 | 61.5 | 100.0 | 2.735 | 946% | |
| SLA rule | 11.7 | 0.0 | 5.8 | 45.4 | 39.7 | 94.2 | 0.583 | 123% | |
| Reservation rule | 1.3 | 24.1 | 12.7 | 34.4 | 49.3 | 87.3 | 0.304 | 16% | |
| A2C flat | 13.5 | 0.2 | 6.9 | 46.0 | 42.2 | 94.1 | 0.678 [0.608, 0.763] | 160% | 0/10 |
| A2C weighted | 18.1 | 3.9 | 11.0 | 45.1 | 46.9 | 93.7 | 0.946 [0.678, 1.247] | 262% | 0/10 |
| DQN flat | 11.0 | 0.1 | 5.6 | 44.7 | 43.5 | 94.5 | 0.550 [0.540, 0.563] | 110% | 0/10 |
| Variant | SLA cost | Gap | (%) | ||
|---|---|---|---|---|---|
| full | 0.001 | 0.360 [0.339, 0.382] | 38% | 5.6 | 11.0, 4.9 |
| full | 0.01 | 0.383 [0.357, 0.417] | 47% | 6.3 | 28.8, 15.8 |
| tiers only | 0.01 | 0.414 [0.393, 0.440] | 58% | 7.1 | 37.8, 24.1 |
| weighted | 0.291 [0.278, 0.307] | 11% | 2.9 | 10, 2 | |
| flat | 0.566 [0.556, 0.577] | 117% | 11.3 | 6, 6 |
| Policy | SLA cost | Above | (%) | Beats |
|---|---|---|---|---|
| Hetero3 , optimum 0.0899 | ||||
| SLA rule | 0.527 | 486% | 10.5, 0.0 | |
| Reservation | 0.141 | 57% | 1.3, 7.6 | |
| Node-set | 0.117 | 31% | 0.6, 8.9 | |
| PPO flat | 0.213 [0.171, 0.262] | 137% | 3.5, 3.8 | 1/10 |
| PPO weighted | 0.130 [0.120, 0.144] | 45% | 1.2, 7.0 | 3/10 |