HQARRF: Hierarchical Q-learning and Force-aware Routing for Multi-Charger Scheduling in Wireless Rechargeable Sensor Networks

Liang-Ching Tao; Pi-Chung Wang

cs.NISep 12, 2026

HQARRF: Hierarchical Q-learning and Force-aware Routing for Multi-Charger Scheduling in Wireless Rechargeable Sensor Networks

Authors: Liang-Ching Tao, Pi-Chung Wang

Organizations: Department of Computer Science and Engineering, National Chung Hsing University, 145 Xingda Rd., South District, Taichung 402, Taiwan

Abstract

Multi-charger scheduling in wireless rechargeable sensor networks must weigh sensor death risk, charger energy, travel cost, return-to-base feasibility and inter-charger coordination at once, and schedulers driven by local urgency alone duplicate service and leave whole regions unattended. We present HQARRF, a two-level scheduler. Below, an interpretable ARR-F score ranks candidate clusters through an attraction term for local urgency, a repulsion term against charger crowding and a force bonus from nearby critical sensors. Above, adaptive zones compress regional state into a deadline-based risk estimate, and a gated Q-learning controller decides only whether to redirect service to a high-risk, under-served zone. Over 27 parameter points HQARRF attains the highest mean survival rate at 26, improving survival by 20.7 percentage points over the mean of five baselines and 9.2 over the strongest baseline at each point. An ablation isolates the upper level: its gain tracks how often the controller fires.

Figures & tables

Figure 1: A deployment at

t=0

: sensors, the candidate clusters produced by minimum circle cover with their docking points, the base station, and the five chargers with the clusters they have currently selected. Scheduling decisions are made over docking points rather than over individual sensors.

Field side $L$	1000–3000 m ( 1000 )	Charger capacity $E_{m}^{max}$	$10^{4}$ J
Sensors $\|V\|$	300–500 ( 250 ) †	Charger speed $v_{m}$	1–10 m/s ( 5 )
Base station	field centre	Movement power $P_{m}^{move}$	0.1 W
Sensor battery $E^{max}$	100–400 J ( 150 )	Charging radius $R_{a}$	30 m
Consumption $\rho_{i}$	0.5–1.5 mJ/s ( 1.0 )	Charging power	10 W
Sensing range	40 m	Charging efficiency	0.9
Comm. range	80 m	Chargers $M$	5 (2–7 in § 5.4 )

Table 1: Simulation parameters. Swept values are given as ranges with the default, used when another parameter is swept, in bold.

Figure 2: Survival rate versus sensor consumption rate. Whiskers are

\pm 1

standard error over the five runs at each point. At 0.5 mJ/s the leading methods lie within each other’s error bars; as workload rises HQARRF degrades gracefully while the baselines fall away.

Figure 3: Survival rate versus network size. HQARRF holds near 80% while every other method decays. ARR-F is HQARRF with its upper level removed and the routing score unchanged; it decays by 15.8 points, placing the flatness in the upper level rather than the routing score.

Figure 4: (a) The four variants on the survival–travel plane, averaged over the 27 parameter points; whiskers are

\pm 1

standard error across those points , so they show how much the scenarios differ, not run-to-run noise. The survival ordering holds at all 27 points individually. (b) For each point, how often the zone controller engaged against the survival it recovered relative to ARR-F. The relationship is monotone within the battery and consumption sweeps and near-monotone across charger speed; the two sweeps that change the field itself sit at the high end of both axes.

Figure 5: Survival rate versus fleet size, at 300 sensors. Every method saturates by four chargers without the methods converging.

Internet of Things

The Internet of Things (IoT) connects networked sensors, devices, and systems for sensing, communication, and automated action.

See all 105 papers

Deep Adaptive Rate Allocation in Volatile Heterogeneous Wireless Networks
Mar 21, 2026Gregorio Maglione, Veselin Rakocevic, Markus Amend +1Wireless Resource Allocation
Department of Engineering, City St George’s, University of London, London EC1V 0HB, United Kingdom · Deutsche Telekom, Darmstadt, Germany
Toward Proactive RF Charging Scheduling: Generative AI for Decision Support
Jun 9, 2026Amirhossein Azarbahram, Osmel M. Rosabal, David Ernesto Ruiz-Guirola +3Wireless Resource Allocation Decision-Making under Uncertainty
Centre for Wireless Communications (CWC), University of Oulu, Finland · School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, Canada · Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong
Beyond Freshness and Semantics: A Coupon-Collector Framework for Effective Status Updates
Mar 27, 2026Youssef Ahmed, Arnob Ghosh, Chih-Chun Wang +1Age of Information Wireless Resource Allocation
Department of ECE, The Ohio State University, Columbus, OH, USA · Department of Electrical and Computer Engineering, New Jersey Institute of Technology, Newark, NJ, USA · Elmore Family School of ECE, Purdue University, West Lafayette, IN, USA +1

Mar 21, 2026J/K move · Enter open · S save

Deep Adaptive Rate Allocation in Volatile Heterogeneous Wireless Networks

Gregorio Maglione, Veselin Rakocevic, Markus Amend, Touraj Soleymani

Department of Engineering, City St George’s, University of London, London EC1V 0HB, United Kingdom · Deutsche Telekom, Darmstadt, Germany

Wireless Resource Allocation

Open paper AskPDF arXiv

Modern multi-access 5G+ networks provide mobile terminals with additional capacity, improving network stability and performance. However, in highly mobile environments such as vehicular networks, supporting multi-access connectivity remains challenging. The rapid fluctuations of wireless link quality often outpace the responsiveness of existing multipath schedulers and transport-layer protocols. This paper addresses this challenge by integrating Transformer-based path state forecasting with a new multipath splitting scheduler called Deep Adaptive Rate Allocation (DARA). The proposed scheduler employs a deep reinforcement learning engine to dynamically compute optimal congestion window fractions on available paths, determining data allocation among them. A six-component normalised reward function with weight-mediated conflict resolution drives a DQN policy that eliminates the observation-reaction lag inherent in reactive schedulers. Performance evaluation uses a Mininet-based Multipath Datagram Congestion Control Protocol testbed with traces from mobile users in vehicular environments. Experimental results demonstrate that DARA achieves better file transfer time reductions compared to learning-based schedulers under moderate-volatility traces. For buffered video streaming, resolution improvements are maintained across all tested conditions. Under controlled burst scenarios with sub-second buffer constraints, DARA achieves substantial rebuffering improvements whilst state-of-the-art schedulers exhibit near-continuous stalling.

Abstract

Internet of Things

Explore similar work

Deep Adaptive Rate Allocation in Volatile Heterogeneous Wireless Networks

Toward Proactive RF Charging Scheduling: Generative AI for Decision Support

Beyond Freshness and Semantics: A Coupon-Collector Framework for Effective Status Updates