Hilbert-Augmented Reinforcement Learning for Scalable Multi-Robot Coverage and Exploration

Hierarchical multi-robot exploration decouples frontier allocation from local navigation, making systems brittle in dynamic environments. Without awareness of execution difficulty, allocators may send robots toward frontiers whose routes are congested or unsafe, causing bottlenecks, oscillatory replanning, and redundant coverage. We propose VORL-EXPLORE, a hybrid learning and planning framework that makes exploration execution-aware through execution fidelity, an estimate of local navigability and execution reliability. This fidelity score is used to penalize risky or crowded frontiers in a fidelity-coupled Voronoi allocation objective with inter-robot repulsion, and to arbitrate between global $A^*$ guidance and a reactive reinforcement learning policy during navigation. As a result, robots follow efficient global plans in reliable regions and rely more on safer local reactions in crowded or changing areas. The fidelity model is recalibrated using self-supervised pseudo-labels derived from recent progress and safety outcomes. Quantitative experiments on randomized dynamic grids and an ablation under severe traffic show higher success rates, shorter exploration lengths, and lower inter-robot coverage overlap. A Gazebo factory scenario is included as a deployment illustration. The source code is available at https://github.com/21ning/VORL-EXPLORE.

Multi-robot exploration under communication and mobility constraints requires balancing information gain with reachability. We present a distributed semantic exploration framework that combines Dirichlet mapping, budget-constrained trajectory selection, and Resource-Aware distributed Greedy (RAG) coordination. Each robot clusters frontiers, computes terrain- and heading-aware least-cost paths with a single budget-limited Dijkstra search, and selects feasible paths by marginal information gain, represented by the expected reduction in Dirichlet entropy. Each robot's per-round selection is exact, the team's joint per-round information gain is at least half the centralized optimum, and the bound degrades gracefully under limited communication. Ten-run gridworld ablations show improved coverage, semantic mIoU, and map certainty with lower entropy and traversal cost per unit distance than shortest-distance frontier selection. Budget-sensitivity experiments show that an intermediate budget best balances path reach and receding-horizon execution. In photorealistic full-system simulation against five coordination rules run in their native published forms, the proposed method attains the highest mean coverage and semantic accuracy while being the only one that respects the per-trajectory mobility budget.

Coordinating multiple autonomous agents to explore and serve spatially heterogeneous demand requires jointly learning unknown spatial patterns and planning trajectories that maximize task performance. Pure model-based approaches provide structured uncertainty estimates but lack adaptive policy learning, while deep reinforcement learning often suffers from poor sample efficiency when spatial priors are absent. This paper presents a hybrid belief-reinforcement learning (HBRL) framework to address this gap. In the first phase, agents construct spatial beliefs using a Log-Gaussian Cox Process (LGCP) and execute information-driven trajectories guided by a Pathwise Mutual Information (PathMI) planner with multi-step lookahead. In the second phase, trajectory control is transferred to a Soft Actor-Critic (SAC) agent, warm-started through dual-channel knowledge transfer: i) belief state transfer supplies spatial uncertainty, and ii) replay buffer seeding provides demonstration trajectories generated during LGCP exploration. A variance-normalized overlap penalty enables coordinated coverage through shared belief state, permitting cooperative sensing in high-uncertainty regions while discouraging redundant coverage in well-explored areas. The framework is instantiated in two scenarios: multi-Unmanned Aerial Vehicle (UAV) wireless service provisioning, and mobility-on-demand sensing over measured New York City taxi demand. In the wireless scenario it achieves 10.8% higher cumulative reward and takes 38% fewer episodes to reach the reward level of the baselines, with ablation studies confirming that dual-channel transfer outperforms either channel alone; the taxi scenario yields consistent gains with all algorithmic components unchanged.