SonarVoxNet: Diver Detection in 3D Bounding Box using 3D Sonar

Eugene Park; Jiwon Lee; Seyoung Kan; Trung Dong; Xiaomin Lin; Jane Shin

cs.ROOct 1, 2026

SonarVoxNet: Diver Detection in 3D Bounding Box using 3D Sonar

Authors: Eugene Park, Jiwon Lee, Seyoung Kan, Trung Dong, Xiaomin Lin, Jane Shin

Abstract

Autonomous underwater vehicles (AUVs) assisting human divers must continuously track not only the diver's 3D position but also their full-body orientation. However, vision-based perception is unreliable underwater, and forward-looking sonar -- despite being widely used -- discards the elevation information needed for orientation estimation, posing a fundamental limitation. Recently commercialized 3D sonar preserves elevation but produces sparse, noisy returns, and existing detectors are built for dense LiDAR data and for targets that remain upright and rotate only about the yaw axis (e.g., vehicles, pedestrians), making them unable to represent a freely pitching and rolling diver. To address this gap, we present two contributions. First, SonarVoxNet adapts a voxel-based encoder and an anchor-free center-based detection head to 3D sonar data, replacing the conventional yaw-only rotation representation with a continuous 6D rotation parameterization to predict full 9-DoF oriented bounding boxes -- to our knowledge, the first 3D sonar diver detector to do so. Second, Diver3D is the first public 3D sonar dataset with full 3D orientation labels for divers in diverse, non-upright poses, collected at a natural cave-diving site. Through controlled ablations over the backbone and detection head, we show that the dominant factor behind accurate 3D sonar-based diver detection is the transition from yaw-only rotation to full-SO(3) rotation. This transition substantially improves detection accuracy and reduces orientation error. These results demonstrate that full-body diver orientation is recoverable from 3D sonar alone, laying the groundwork for future work on diver pose estimation and diver-robot interaction.

Figures & tables

Fig. 1: 3D Bounding Box Prediction for Underwater Diver Detection from SonarVoxNet using 3D Sonar Sensor

Method	Backbone	Head	Rotation
VoxelNet [ 6 ]	Dense voxel	Anchor-based	Yaw only
SECOND [ 19 ]	Sparse voxel	Anchor-based	Yaw only
PointPillars [ 21 ]	Pillar (2D voxel)	Anchor-based	Yaw only
CenterPoint [ 7 ]	Sparse voxel	Anchor-free (center)	Yaw only
VoxelNeXt [ 20 ]	Fully sparse voxel	Anchor-free (center)	Yaw only
SonarVoxNet (ours)	Sparse voxel	Anchor-free (center)	Full-SO(3)

TABLE I: Architecture comparison to existing related works.

Fig. 2: The architecture of the presented SonarVoxNet

Method	AP 3D @0.35	AP 3D @0.50	SDS
VoxelNet [ 6 ]	0.126	0.016	0.323
SECOND [ 19 ]	0.315	0.016	0.436
PointPillars [ 21 ]	0.190	0.018	0.370
CenterPoint [ 7 ]	0.324	0.019	0.433
VoxelNeXt [ 20 ]	0.374	0.019	0.463
SonarVoxNet(no aug)	0.515	0.233	0.574

TABLE II: Performance comparison of SonarVoxNet and other 3D detectors on the Diver3D test set

		AP 3D $\uparrow$				AP BEV $\uparrow$	TP-IoU $\uparrow$	ATE $\downarrow$	ASE $\downarrow$	AOE $\downarrow$	AOE 3D $\downarrow$	SDS $\uparrow$
Backbone	Head	@0.30	@0.35	@0.40	@0.50	@0.35	(3D)	(m)		( ∘ )	( ∘ )
	Anchor + yaw	0.346	0.242	0.137	0.022	0.394	0.430	0.233	0.332	33.0	65.1	0.406
Dense	Center + yaw	0.549	0.328	0.139	0.020	0.680	0.420	0.214	0.350	35.1	70.8	0.444
	Center + 6D rotation	0.625	0.538	0.427	0.187	0.663	0.492	0.219	0.347	37.5	51.3	0.576
	Anchor + yaw	0.314	0.218	0.117	0.016	0.354	0.431	0.246	0.336	34.7	64.8	0.391
Sparse	Center + yaw	0.521	0.319	0.144	0.016	0.623	0.422	0.224	0.328	31.5	66.8	0.448

TABLE III: Backbone

\times

Head ablation on the Diver3D test set without augmentation.

Continuous
Representation	AP 3D @0.35	AP 3D @0.5	AOE 3D ( ∘ )
6D (ours)	$0.515$ $\pm$ .030	$\mathbf{0.233}$ $\pm$ .016	$48.5$ $\pm$ 0.7
9D + SVD	$\mathbf{0.551}$ $\pm$ .011	$0.216$ $\pm$ .007	$50.4$ $\pm$ 1.1
Discontinuous
Quaternion	$0.521$ $\pm$ .028	$0.193$ $\pm$ .031	$55.9$ $\pm$ 1.0
Axis–angle	$0.504$ $\pm$ .025	$0.207$ $\pm$ .007	$\mathbf{45.3}$ $\pm$ 2.1

TABLE IV: Rotation representation. Underline : adopted. No representation leads on all three metrics.

Fig. 3: Diver Detection Performance per Number of Divers

Training	AP 3D @0.35	AP 3D @0.5	AOE 3D ( ∘ )	SDS
No augmentation	$0.515$	$0.233$	$48.5$	$0.574$
Constant target 4	$0.664$	$0.326$	$46.5$	$0.651$
Curriculum, zero floor	$0.676$	$0.344$	$\mathbf{44.6}$	$0.662$
Curriculum, floor 1 (ours)	$\mathbf{0.703}$	$\mathbf{0.359}$	$46.2$	$\mathbf{0.673}$

TABLE V: Effect of augmentation on the Diver3D test set.

Metric	Detector	+ Refinement	$\Delta(\%)$
AP 3D @0.30	$0.754$	$\mathbf{0.808}$	$+7.16$
AP 3D @0.35	$0.703$	$\mathbf{0.774}$	$+10.10$
AP 3D @0.40	$0.614$	$\mathbf{0.709}$	$+15.47$
AP 3D @0.50	$0.359$	$\mathbf{0.467}$	$+30.08$
mAP 3D	$0.690$	$\mathbf{0.764}$	$+10.58$
ATE (m) $\downarrow$	$0.183$	$\mathbf{0.169}$	$-7.65$

TABLE VI: Causal geometry refinement on the Diver3D test set.

Explore similar work

Jun 22, 2026cs.RO

ISOPoT: Imaging Sonar Odometry by Point Tracking

Reliable navigation in underwater environments remains a key challenge in marine robotics. In such scenarios, forward-looking sonars are a natural choice for long-range perception, offering wide coverage even in turbid, low-visibility conditions. However, sonar images are inherently noisy, contain artifacts, and lack rich semantic structure, causing standard computer vision methods for keypoint detection and matching to perform poorly. In this paper, we introduce ISOPoT, an imaging sonar odometry method based on modern point tracking techniques. We propose a sonar odometry pipeline that uses multi-frame point tracks as its primary correspondence representation, augmented with lightweight optimizations to improve robustness. We evaluated the proposed method on the Aracati 2017 dataset, as well as on an internal sonar dataset collected in real-world underwater environments. Our results show that ISOPoT outperforms previous state-of-the-art methods consistently in both sonar-only scenarios and in multi-sensor settings.

Jaša Samec, Vid Rijavec, Marko Peljhan +4

Faculty of Computer and Information Science, University of Ljubljana, Večna pot 113, 1000 Ljubljana, Slovenia · OZON Research Group, TIMTEC, Goriska cesta 6c, Vipava, 5271, Slovenia · MAT Systemics, University of California, Santa Barbara, 3309 Phelps Hall, Santa Barbara, 93106-6065, California, United States of America +1

Forward-Looking Sonar Inertial Navigation Systems Keypoint Detection

Ask PDF

May 31, 2026cs.RO

A Sonar-Visual Dataset for Cross-Modal Underwater Robot Perception

Underwater robots typically use both cameras and sonar for perception to leverage the rich semantic details of vision and the robust range measurements of acoustics. However, learning to map between these modalities via cross-modal prediction remains underexplored due to limited sonar-visual paired datasets. We present SOVIS, a sonar-visual dataset for cross-modal underwater perception. SOVIS comprises over 76,000 paired frames collected across 17 dives at six sites in the Trondheimfjord, supported by an end-to-end pipeline that cleans and synchronizes the cross-modal sensor data. We also introduce an interactive annotation tool designed to accelerate the labeling process for this paired data. Finally, we demonstrate a proof-of-concept cross-modal fish detection task using a small subset of labeled data, achieving a 7x improvement in mAP@0.10 over a monocular camera baseline. SOVIS serves as the first step toward advancing cross-modal underwater perception research, enabling research directions such as dense sonar prediction from monocular images.

Weitung Chen, Phil Tinn, Per Gunnar Auran +2

Massachusetts Institute of Technology (MIT), Cambridge, MA, USA · SINTEF, Trondheim, Norway · Department of Marine Technology, Norwegian University of Science and Technology (NTNU), Trondheim, Norway

Underwater Acoustic Target Recognition Forward-Looking Sonar Underwater Robots Multi-Modal Sensing Cross-Modal Perception+1

Ask PDF

May 24, 2026cs.CV

MambaDSF: Multi-Scale SSM with Dilated Feature Fusion for Sonar Small Target Detection

Sonar imaging is the primary modality for underwater target detection, yet small targets remain difficult to detect due to insufficient pixel coverage, low acoustic contrast, and scale ambiguity across imaging ranges. CNN-based detectors extract local features efficiently but cannot suppress noise-induced false alarms without global acoustic context. Transformer-based methods capture long-range dependencies at quadratic computational cost. Existing Mamba-based vision models offer efficient linear-cost scanning but lack multi-scale semantic alignment across pyramid levels, multi-receptive-field fusion, and small-target-aware training supervision needed for reliable sonar detection. This letter proposes Mamba Dilated-Scale Fusion (MambaDSF), a hybrid framework addressing these limitations through three contributions: a Mamba Enhanced Feature Pyramid (MambaEFP) backbone that jointly captures local echo cues and global acoustic context at linear complexity; a Dilate Fusion Mamba (DFMamba) encoder that enforces multi-scale feature alignment across pyramid levels; and Scale-Adaptive Weighted IoU (SA-WIoU) and Cross-Scale Coherence (CSC) losses that stabilize small-target training. MambaDSF achieves 91.5% mAP50 on the UATD forward-looking sonar benchmark with 28.7 million parameters, surpassing all compared detectors. On a small-target subset the gain reached +2.2 percentage points, and cross-domain evaluation on FLS and MD-FLS confirms the generalization of the proposed architecture. The codes are publicly available at https://github.com/IDontKnowAAA/MambaDSF.

Hui Lin, Jiayi Li, Jing Wang +1

School of Information Science and Engineering, Ocean University of China, Qingdao 266100, China · School of Information and Communication Technology, Griffith University, Nathan, QLD 4111, Australia

Forward-Looking Sonar Feature Pyramid State Space Models

Ask PDF