Cross domain Persistent Monitoring for Hybrid Aerial Underwater Vehicles

Ricardo B. Grando; Victor A. Kich; Alisson H. Kolling; Junior C. D. Jesus; Rodrigo S. Guerra; Paulo L. J. Drews-Jr

cs.ROFeb 23, 2026

Cross domain Persistent Monitoring for Hybrid Aerial Underwater Vehicles

Authors: Ricardo B. Grando, Victor A. Kich, Alisson H. Kolling, Junior C. D. Jesus, Rodrigo S. Guerra, Paulo L. J. Drews-Jr

Abstract

Hybrid Unmanned Aerial Underwater Vehicles (HUAUVs) have emerged as platforms capable of operating in both aerial and underwater environments, enabling applications such as inspection, mapping, search, and rescue in challenging scenarios. However, the development of novel methodologies poses significant challenges due to the distinct dynamics and constraints of the air and water domains. In this work, we present persistent monitoring tasks for HUAUVs by combining Deep Reinforcement Learning (DRL) and Transfer Learning to enable cross-domain adaptability. Our approach employs a shared DRL architecture trained on Lidar sensor data (on air) and Sonar data (underwater), demonstrating the feasibility of a unified policy for both environments. We further show that the methodology presents promising results, taking into account the uncertainty of the environment and the dynamics of multiple mobile targets. The proposed framework lays the groundwork for scalable autonomous persistent monitoring solutions based on DRL for hybrid aerial-underwater vehicles.

Figures & tables

Panel of Figure 1 — Figure 1: Simulated HUAUV persistent monitoring environments. (a) Aerial domain, Env 1 (obstacle-free). (b) Aerial domain, Env 2 (four cylindrical obstacles). (c)–(d) Example LiDAR scans in Env 1 and Env 2, respectively. Underwater scenarios use the same layouts but with sonar range measurements.

Model	$t_{air}$ (s)	$t_{water}$ (s)	$\tau_{1air}$	$\tau_{2air}$	$\tau_{3air}$	$\tau_{1und}$	$\tau_{2und}$	$\tau_{3und}$
DSAC (Env1)	$35.40$ $\pm$ $30.61$	97.95 $\pm$ 83.62	9.91	9.66	9.08	27.75	27.45	23.84
BUG2 (Env1)	36.90 $\pm$ 16.98	$127.43$ $\pm$ $56.85$	$10.32$	$10.20$	$12.51$	$35.54$	$36.69$	$42.12$
DSAC (Env2)	49.34 $\pm$ 49.45	133.67 $\pm$ 142.21	12.98	10.41	13.39	26.62	32.87	27.72
BUG2 (Env2)	$93.63$ $\pm$ $119.55$	$155.73$ $\pm$ $151.44$	$30.06$	$37.82$	$26.09$	$36.92$	$45.20$	$35.27$

Table I: Average Time and Uncertainty.

Panel of Figure 2 — Figure 2: Evolution of target uncertainties $\sigma_{i,t}$ during evaluation for Env 1 and Env 2 in the aerial and underwater domains. The vertical axis shows the uncertainty, and the horizontal axis is the simulation time steps. DSAC reduces the peaks and frequency of high-uncertainty intervals compared to the Bug2 baseline.

Robotics

Robotics studies the design and control of machines that perceive, move, and act in physical environments.

See all 751 papers

Towards End to End Motion Planning and Execution for Autonomous Underwater Vehicles Using Reinforcement Learning
Jun 7, 2026Elisei Shafer, Oren GalHierarchical RL Visuomotor Policy Learning
Department of Marine Technologies, University of Haifa, Israel
Contextual Multi-Task Reinforcement Learning for Autonomous Reef Monitoring
Apr 14, 2026Melvin Laux, Yi-Ling Liu, Rina Alo +4RL Control Underwater Robotics
Robotics Research Group, University of Bremen, Bremen, Germany · Robotics Innovation Center, German Research Center for Artificial Intelligence, Bremen, Germany
Custom PX4 firmware for autonomous hybrid aerial-marine missions
Sep 17, 2026Andrea Capuozzo, Fabio Ruggiero, Vincenzo LippielloAerial Robotics Robot Navigation
PRISMA Lab, Department of Electrical Engineering and Information Technology, University of Naples Federico II, Via Claudio 21, Naples, 80125, Italy

Jun 7, 2026J/K move · Enter open · S save

Towards End to End Motion Planning and Execution for Autonomous Underwater Vehicles Using Reinforcement Learning

Elisei Shafer, Oren Gal

Department of Marine Technologies, University of Haifa, Israel

Hierarchical RL Visuomotor Policy Learning Underwater Robotics Autonomous Underwater Navigation Robust RL

Open paper AskPDF arXiv

Autonomous Underwater Vehicles (AUVs) traditionally rely on complex, heavily engineered pipelines for perception, path planning, and motion control. This paper explores the feasibility of an end-to-end Deep Reinforcement Learning (DRL) approach that maps raw sensor data directly to thruster commands, reducing manual engineering. We propose a hierarchical reinforcement learning (HRL) architecture splitting the problem into two Markov Decision Processes. A High-Level (HL) policy operating at 2Hz processes raw

84 \times 84

pixel monocular camera frames, stacked

100 \times 100

pixel forward-looking imaging sonar, and proprioceptive data to generate spatial subgoals. Simultaneously, a Low-Level (LL) policy operating at 10Hz converts these subgoals into thruster commands. The HL policy is trained using Reinforcement Learning from Prior Demonstrations (RLPD) within a modified Sample-Efficient Robotic Reinforcement Learning (SERL) framework, while the LL policy utilizes Soft Actor-Critic (SAC) combined with Hindsight Experience Replay (HER). Evaluated in the high-fidelity HoloOcean simulator, our method demonstrates successful obstacle avoidance, achieving trajectory lengths closely approximating (within 4% to 6% of) an

\text{RRT}^*

planning baseline. Furthermore, the learned policy exhibits strong robustness to simulated sensor noise and decreased visibility. While the system navigates familiar geometries effectively, experiments reveal generalization limitations when encountering unvisited areas with novel obstacle shapes. Ultimately, this work demonstrates the promise of sample-efficient, end-to-end DRL for underwater navigation using minimal computational hardware.

Abstract

Robotics

Explore similar work

Towards End to End Motion Planning and Execution for Autonomous Underwater Vehicles Using Reinforcement Learning

Contextual Multi-Task Reinforcement Learning for Autonomous Reef Monitoring

Custom PX4 firmware for autonomous hybrid aerial-marine missions