SHRUMS: Sensor Hallucination for Real-time Underwater Motion Planning with a Compact 3D Sonar

Susheel Vadakkekuruppath; Lauritz R. Fosso; Michail Chatzispyrou; Herman B. Amundsen; Sveinung J. Ohrem; Jason M. O'Kane; Marios Xanthidis

cs.ROOct 21, 2025

SHRUMS: Sensor Hallucination for Real-time Underwater Motion Planning with a Compact 3D Sonar

Authors: Susheel Vadakkekuruppath, Lauritz R. Fosso, Michail Chatzispyrou, Herman B. Amundsen, Sveinung J. Ohrem, Jason M. O'Kane, Marios Xanthidis

Organizations: Susheel Vadakkekuruppath and Jason M. O’Kane are with the Department of Computer Science and Engineering, Texas A&M University, USA · Herman B. Amundsen and Marios Xanthidis are with the Aquaculture Robotics and Automation, SINTEF Ocean

Abstract

Autonomous navigation in 3D is a fundamental problem for autonomy. Despite major advancements in terrestrial and aerial settings due to improved range sensors including LiDAR, compact sensors with similar capabilities for underwater robots have only recently become available, in the form of 3D sonars. This paper introduces a novel underwater 3D navigation pipeline, called SHRUMS (Sensor Hallucination for Robust Underwater Motion planning with 3D Sonar). To the best of the authors' knowledge, SHRUMS is the first underwater autonomous navigation stack to integrate a 3D sonar and demonstrate safe operation in complex 3D environments without reliance on visual input, thereby remaining independent of visibility conditions. To accommodate the intricacies of the novel sensor data stream while achieving real-time locally optimal performance, SHRUMS introduces the concept of hallucinating sensor measurements from non-existent sensors with convenient arbitrary parameters, tailored to application specific requirements. The proposed concepts are validated in simulations informed with real 3D sonar sensor data, in-pool clean-water trials, and field deployments in real conditions with significantly degraded visibility conditions.

Figures & tables

Panel of Figure 1 — Fig. 1 : (a) A submerged container as seen in a video from an underwater robot’s camera. These extremely poor visibility conditions severely limit vision-based methodologies. (b) A range image from the Sonar 3D-15, showing significantly clearer information about the container’s geometry. (c) An example range image produced using sensor hallucination within SHRUMS (the proposed pipeline), enhancing the characteristics of the original sensing capacity in terms of resolution, field of view, and sensing completeness. (d) A 3D path produced by SHRUMS to navigate the observed map.

Fig. 2 : The SHRUMS pipeline with its main processing stages, including the integration of a 3D Sonar, local mapping, sensor hallucination, mesh reconstruction and convex decomposition, path optimization, and, finally, the path follower informing the robot’s controls.

Panel of Figure 3 — Fig. 3 : Intricacies of the 3D sonar’s sensing output. (a) A point cloud from the 3D sonar, though there is incomplete coverage of some surfaces, such as the door at the left side. (b) Local consistency is exploited by KISS-ICP to provide the entire scene.

Panel of Figure 4 — Fig. 4 : In (a), the sonar output is shown observing pillars from a deployment under a dock, while (b) shows the useless view from the robot’s camera and (c) the local map produced by KISS-ICP. Simulated range images are shown in (d) before and (e) after applying a median filter. For higher-resolution images, dense range images may require preprocessing for smooth mesh reconstruction.

Panel of Figure 5 — Fig. 5 : Qualitative results for the Container, Car, and Pillars environment (rows)). Planning instance for each at the C1 configuration showing the convex polytopes representation and the path with red arrows in (a,c,e), while the yellow the trajectory of the robot for the C3 configuration in (b,d,f).

	SHRUMS
Map	Sensor Config.	Sensor Simulation	Mesh Recon.	Convex Decomp.	Path Opt.
	C1	0.0022	0.0025	0.7701	0.6808
Container	C2	0.0029	0.0446	0.8008	0.7268
	C3	0.0035	0.0299	0.7503	0.4035
	C1	0.0062	0.0143	0.9606	0.1930
Car	C2	0.0051	0.0337	1.0471	0.2079

TABLE I : Average times of different stages of SHRUMS (s)

Panel of Figure 7 — Fig. 6 : Qualitative results from E1, E2, and E3 deployments (rows). Planning instances during operation shown in (a,c,e), while a representative visual inputs in (b,d,f).

Explore similar work

uScenes: A Multimodal RGB and 3D Sonar Dataset for Underwater Robot Perception
Aug 28, 2026Trung Tien Dong, Zhenqi Wu, Aditya Penumarti +4Underwater Robotics 3D Scene Understanding
Embodied Robotics and Automation Lab, University of South Florida, Tampa, FL 33610, USA · Mechanical and Aerospace Engineering Department, University of Florida, Gainesville, FL 32611, USA · Naval Architecture and Ocean Engineering at the Seoul National University, Seoul 08826, South Korea
Towards Realistic 3D Sonar Simulation
Jun 4, 2026Youssef Attia, Davide Costa, Francesco Wanderlingh +2Robotics Simulation Underwater Robotics
University of Genova, Italy · University of Padova, Italy · Interuniversity Research Center on Integrated Systems for the Marine Environment, Italy +1
Towards Versatile Opti-Acoustic Sensor Fusion and Volumetric Mapping
Mar 15, 2026Ivana Collado-Gonzalez, John McConnell, Brendan EnglotUnderwater 3D Reconstruction Underwater Robotics
I. Collado-Gonzalez, and B. Englot are with Stevens Inst. of Technology, Hoboken, NJ, USA · J. McConnell is with the U.S. Naval Academy, Annapolis, MD, USA

Aug 28, 2026J/K move · Enter open · S save

uScenes: A Multimodal RGB and 3D Sonar Dataset for Underwater Robot Perception

Trung Tien Dong, Zhenqi Wu, Aditya Penumarti, Zi-Hao Zhang+3

Embodied Robotics and Automation Lab, University of South Florida, Tampa, FL 33610, USA · Mechanical and Aerospace Engineering Department, University of Florida, Gainesville, FL 32611, USA · Naval Architecture and Ocean Engineering at the Seoul National University, Seoul 08826, South Korea

Underwater Robotics 3D Scene Understanding Multimodal Sensor Fusion

Open paper AskPDF arXiv

Robust perception is essential for the deployment of autonomous underwater robots. However, optical cameras become unreliable under poor illumination and backscatter. Forward looking (2D) acoustic sensors remain effective under these conditions, but they measure range and bearing while leaving elevation unresolved, creating an ambiguity that prevents individual sonar returns from being localized in three dimensional (3D) space. This complicates the sensor use for 3D scene understanding and precise object detection. We introduce uScenes, a multimodal underwater dataset containing synchronized 3D multibeam sonar point clouds and RGB imagery. The dataset contains 110 scenes and 95,834 synchronized observation, representing 277.6 minutes of data collected across multiple field sessions. uScenes establishes a foundation for underwater sensor fusion, cross modal representation learning and 3D scene understanding. Code and datasets are given at https://github.com/era-research-lab/uScenes.

Abstract

Explore similar work

uScenes: A Multimodal RGB and 3D Sonar Dataset for Underwater Robot Perception

Towards Realistic 3D Sonar Simulation

Towards Versatile Opti-Acoustic Sensor Fusion and Volumetric Mapping