Towards Versatile Opti-Acoustic Sensor Fusion and Volumetric Mapping

Ivana Collado-Gonzalez; John McConnell; Brendan Englot

cs.ROMar 15, 2026

Towards Versatile Opti-Acoustic Sensor Fusion and Volumetric Mapping

Authors: Ivana Collado-Gonzalez, John McConnell, Brendan Englot

Organizations: I. Collado-Gonzalez, and B. Englot are with Stevens Inst. of Technology, Hoboken, NJ, USA · J. McConnell is with the U.S. Naval Academy, Annapolis, MD, USA

Abstract

Accurate 3D volumetric mapping is critical for autonomous underwater vehicles operating in obstacle-rich environments. Vision-based perception provides high-resolution data but fails in turbid conditions, while sonar is robust to lighting and turbidity but suffers from low resolution and elevation ambiguity. This paper presents a volumetric mapping framework that fuses a stereo sonar pair with a monocular camera to enable safe navigation under varying visibility conditions. Overlapping sonar fields of view resolve elevation ambiguity, producing fully defined 3D point clouds at each time step. The framework identifies regions of interest in camera images, associates them with corresponding sonar returns, and combines sonar range with camera-derived elevation cues to generate additional 3D points. Each 3D point is assigned a confidence value reflecting its reliability. These confidence-weighted points are fused using a Gaussian Process Volumetric Mapping framework that prioritizes the most reliable measurements. Experimental comparisons with other opti-acoustic and sonar-based approaches, along with field tests in a marina environment, demonstrate the method's effectiveness in capturing complex geometries and preserving critical information for robot navigation in both clear and turbid conditions. Our code is open-source to support community adoption.

Figures & tables

Panel of Figure 1 — Fig. 1 : Overview. (a) Our BlueROV2 platform; (b) an illustration of the fields of view of its camera and sonars, with the gray volume representing the camera and the blue/red volumes indicating the horizontal/vertical sonars respectively; (c) an illustration of our proposed volumetric mapping pipeline.

Fig. 2 : Forward looking imaging sonar model. The point $\mathbf{P}_{s}$ can be represented by $[r,\theta,\phi]^{T}$ in a spherical coordinate frame. The range $r$ and the bearing angle $\theta$ of $\mathbf{P}_{s}$ are measured, while the elevation angle $\phi$ is not captured in the resulting 2D sonar image. The 3D transformation $\mathbf{T}_{s\to c}$ from sonar to camera frame is also depicted.

Fig. 3 : Testing environments and representative sonar images. Environments include: Tank Single Disk (left column), Tank Double Disk (middle column), and Marina Pier (right column). The top row shows an example image of each environment, the middle row shows an example horizontal sonar image, and the bottom row shows an example vertical sonar image.

Method	MAE (cm) ↓	RMSE (cm) ↓	SD (cm) ↓	Precision (%) ↑	Inlier Voxels ↑
Octo SS RGB	5.9844	0.8106	0.5468	34.5989	1212
GP SS RGB	6.6030	0.8816	0.5841	35.7711	450
GPC SS RGB	1.1712	0.1579	0.1059	88.1844	306
GP S RGB	6.0007	0.8305	0.5741	38.8336	273
GP SS	1.3590	0.1819	0.1209	81.5789	31

TABLE I : Performance of volumetric mapping methods over the single disk structure in tank setting - proposed method is highlighted in gray. Best results per metric are shown in bold.

Fig. 4 : Tank single disk mapping results. Each column in the image shows the voxel map result for a different method. From left to right the order is: Octo SS RGB, GP SS RGB, GPC SS RGB (proposed approach highlighted in blue), GP S RGB, and GP SS. Voxel colors depict height.

Method	MAE (cm) ↓	SD (cm) ↓	RMSE (cm) ↓	Precision (%) ↑	Inlier Voxels ↑
Octo SS RGB	2.6064	0.3141	0.4081	64.1379	1302
GP SS RGB	2.5962	0.3338	0.4229	68.8243	521
GPC SS RGB	1.6219	0.1651	0.2315	81.5825	598
GP S RGB	1.3885	0.1306	0.1906	79.5385	517
GP SS	2.3853	0.3511	0.4244	68.4685	76

TABLE II : Performance of volumetric mapping methods over the double disk structures in tank setting - proposed method is highlighted in gray. Best results per metric are shown in bold.

Fig. 5 : Tank double disk mapping results. Each column in the image shows the voxel map result for a different method. From left to right the order is: Octo SS RGB, GP SS RGB, GPC SS RGB (proposed approach highlighted in blue), GP S RGB, and GP SS. Voxel colors depict height.

Fig. 6 : Marina Pier mapping results. Each row in the image shows voxel map results for a different method. From top to bottom the order is: Octo SS RGB, GP SS RGB, GPC SS RGB (proposed approach highlighted in blue), GP S RGB, and GP SS. Voxel colors depict height. Pink circles emphasize the presence of two small metal pipes in front of the wooden pilings.

Method	Tank Single Disk	Tank Double Disk	Marina Pier
Octo SS RGB	0.1495 $\pm$ 0.0838	0.1168 $\pm$ 0.0414	0.2797 $\pm$ 0.3474
GP SS RGB	0.0551 $\pm$ 0.0634	0.0699 $\pm$ 0.0363	0.1040 $\pm$ 0.0365
GPC SS RGB	0.0555 $\pm$ 0.0197	0.0712 $\pm$ 0.0311	0.0970 $\pm$ 0.0324

TABLE III : Mapping time (in seconds) with standard deviation. The proposed approach is highlighted in gray. Best-performing results are shown in bold.

Explore similar work

Sonar-MASt3R: Real-Time Opti-Acoustic Fusion in Turbid, Unstructured Environments
Mar 13, 2026Amy Phung, Richard CamilliUnderwater 3D Reconstruction 3D Reconstruction
Applied Ocean Physics and Engineering, Woods Hole Oceanographic Institution, Deep Submergence Laboratory, Woods Hole, MA, USA · Massachusetts Institute of Technology, Cambridge, MA, USA
ISOPoT: Imaging Sonar Odometry by Point Tracking
Jun 22, 2026Jaša Samec, Vid Rijavec, Marko Peljhan +4Point Tracking Relative Pose Estimation
Faculty of Computer and Information Science, University of Ljubljana, Večna pot 113, 1000 Ljubljana, Slovenia · OZON Research Group, TIMTEC, Goriska cesta 6c, Vipava, 5271, Slovenia · MAT Systemics, University of California, Santa Barbara, 3309 Phelps Hall, Santa Barbara, 93106-6065, California, United States of America +1
Odometry-Aided Real-Time Mapping for Underwater Robots Using Forward-Looking Sonar
Sep 21, 2026Siyuan Du, Kanzhong Yao, Youdong Wang +5Underwater Imaging Underwater Robotics
Institute of Artificial Intelligence (TeleAI), China Telecom, China. · Tongji University, Shanghai, China.

Mar 13, 2026J/K move · Enter open · S save

Sonar-MASt3R: Real-Time Opti-Acoustic Fusion in Turbid, Unstructured Environments

Amy Phung, Richard Camilli

Applied Ocean Physics and Engineering, Woods Hole Oceanographic Institution, Deep Submergence Laboratory, Woods Hole, MA, USA · Massachusetts Institute of Technology, Cambridge, MA, USA

Underwater 3D Reconstruction 3D Reconstruction Multimodal Sensor Fusion Underwater Imaging

Open paper AskPDF arXiv

Underwater intervention is an important capability in several marine domains, with numerous industrial, scientific, and defense applications. However, existing perception systems used during intervention operations rely on data from optical cameras, which limits capabilities in poor visibility or lighting conditions. Prior work has examined opti-acoustic fusion methods, which use sonar data to resolve the depth ambiguity of the camera data while using camera data to resolve the elevation angle ambiguity of the sonar data. However, existing methods cannot achieve dense 3D reconstructions in real-time, and few studies have reported results from applying these methods in a turbid environment. In this work, we propose the opti-acoustic fusion method Sonar-MASt3R, which uses MASt3R to extract dense correspondences from optical camera data in real-time and pairs it with geometric cues from an acoustic 3D reconstruction to ensure robustness in turbid conditions. Experimental results using data recorded from an opti-acoustic eye-in-hand configuration across turbidity values ranging from <0.5 to >12 NTU highlight this method's improved robustness to turbidity relative to baseline methods.

Abstract

Explore similar work

Sonar-MASt3R: Real-Time Opti-Acoustic Fusion in Turbid, Unstructured Environments

ISOPoT: Imaging Sonar Odometry by Point Tracking

Odometry-Aided Real-Time Mapping for Underwater Robots Using Forward-Looking Sonar