Organizations: Lab. of Robotics, Informatics and Complex Systems, National Engineering School of Tunis, University of Tunis El Manar, Tunis, Tunisia · Faculty of Information Technology, University of Tripoli, Tripoli, Libya · Lab. of Robotics, Informaticsand Complex Systems, National Engineering School of Tunis, University of Tunis El Manar, Tunis, Tunisia · Lab. of Robotics, Informatics and Complex Systems, National Engineering School of Tunis, University of Tunis El Manar,2026 Tunis, Tunisia · Computer Engineering Dept., University of Tripoli26 Tripoli, Libya · Laboratory of Novel Technologies, University of Picardie Jules Verne Amiens, France[cs.RO]
Exploration in sparse-reward long-horizon tasks poses significant challenges for reinforcement learning. To address these challenges, we propose a two-level Hierarchical Reinforcement Learning (HRL) framework. The first level handles high-level strategic planning, while the low-level uses the continuous-control Soft Actor-Critic (SAC) algorithm, and they utilize entropy-regularized policy optimization. The proposed framework was trained and evaluated using the Search-and-Rescue-2 (SAR-2) dataset. HRL-SAC effectively addresses sparse-reward long-horizon search problems characterized by delayed rewards and continuous control, and its outperforming the flat SAC baseline reinforcement learning in terms of success rates, coverage efficiency, and convergence. These findings indicate that hierarchical entropy-regularized policies are a promising solution to tackle long-horizon sparse-reward reinforcement learning tasks.