One Model, Two Worlds: Bidirectional Sonar-Optical Translation

Shengji Jin; Trung Tien Dong; Ahmed Lamidi; Chen Chen; Xiaomin Lin; Yi Sheng

cs.CVSep 5, 2026

One Model, Two Worlds: Bidirectional Sonar-Optical Translation

Authors: Shengji Jin, Trung Tien Dong, Ahmed Lamidi, Chen Chen, Xiaomin Lin, Yi Sheng

Organizations: University of South Florida · University of Central Florida

Abstract

Translating between imaging sonar and optical cameras is valuable for underwater perception, but supporting both directions with separate models duplicates storage and computation. A unified bidirectional model is therefore attractive, yet existing approaches largely treat the two directions symmetrically despite their fundamentally different image-formation physics. We argue that sharing a generative model does not require sharing the physics. We introduce the Direction-Asymmetric Realism Bridge (DARB), which retains a shared diffusion-bridge trunk while routing direction-specific physical priors through asymmetric pathways: range-aware modulation for sonar-to-optical translation and polar ray-dependent processing for optical-to-sonar translation. We further show that symmetry in training is also costly: applying a common realism schedule reduces sonar-to-optical PSNR by 2.60 dB. Our Adaptive Realism Supervision (ARS) instead determines when, where, and how strongly perceptual supervision is applied from reconstruction quality and gradient balance. Together, DARB and ARS enable one bidirectional model to match the sonar-to-optical specialist within 0.11 dB PSNR, outperform the optical-to-sonar specialist by 0.70 FID, and surpass two independently trained BBDMs on seven of eight metrics.

Figures & tables

Figure 1: Overview of DARB. A single diffusion bridge serves both directions, each conditioned on a prior read out of its own source image. Sonar $\to$ RGB (orange): the first-return contour gives a per-beam range field, applied by UAM. RGB $\to$ Sonar (green): the fan-shaped support mask drives SPADE, and RCASM propagates occlusion along each bearing in polar coordinates.

Figure 2: DARB. A single trunk serves both directions. The direction scalar $d$ enters through its own zero-initialised AdaGN branch and selects which physical operators the forward pass is routed through: UAM for the optical target, SPADE and RCASM for the acoustic one.

Method	#Models	Ep./dir.	Sonar $\rightarrow$ RGB				RGB $\rightarrow$ Sonar
Method	#Models	Ep./dir.	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$
Two models, one trained per direction
pix2pix [ 19 ]	2	100	23.36	0.856	0.183	70.22	23.24	0.583	0.215	106.21
MUNIT [ 17 ]	2	100	22.39	0.848	0.281	126.49	18.28	0.397	0.491	160.33
OT-CFM [ 38 ]	2	100	30.14	0.929	0.071	29.68	31.14	0.735	0.061	28.98
BBDM [ 24 ]	2	100	30.61	0.936	0.047	14.04	32.15	0.762	0.050	17.29

Table 1: Bidirectional sonar and optical translation on DeeperSense , 1181 held-out pairs. Methods reported in one direction only are in the supplement. Ep./dir. is the budget each direction receives, so an alternating model is listed at half its raw epoch count. Bold marks the best single-model result and underline the best result overall.

Figure 3: Both directions on one held-out pair. Column groups separate methods that need one trained model per direction from those that serve both with one. Per-image LPIPS is inset. Sonar renderings share a colour map and scale.

Sonar $\rightarrow$ RGB
Configuration	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$
Dedicated model
Bridge only	30.61	0.936	0.047	14.04
$+$ UAM prior	32.45	0.946	0.042	18.47
$\hookrightarrow$ realism, always on	31.48	0.938	0.036	11.13
$\hookrightarrow$ realism, gated in $t$	31.01	0.935	0.038	11.70

Table 2: Ablation, read separately in each direction. 1181 held-out pairs, 100 epochs per direction. Upper block: one model per direction, the three

\hookrightarrow

rows being alternative realism schedules on top of the physical prior. Lower block: one model serving both directions with both priors routed by direction, rows cumulative, the last being DARB, the same model in both panels. The schedules differ in weight as well as in timing, so the comparison is between whole schedules rather than single knobs.

Image-to-Image Translation

Image-to-image translation maps images between visual domains or modalities while preserving relevant content, structure, and semantics.

See all 115 papers

SonarLLM: A Native Sonar--Optical Multimodal Large Language Model for Underwater Perception
Aug 25, 2026Cong Su, longxuan ma, Ling Dong +4Multimodal Sensor Fusion Underwater Imaging
Faculty of Information Engineering and Automation, Kunming University of Science and Technology Kunming, China · Yunnan Key Laboratory of Artificial Intelligence Kunming, China
Adapting Vision Foundation Models to Acoustics for Pose-Free 3D Sonar Reconstruction
Sep 5, 2026Kevin Zhang, Jingxi Chen, Mohamad Qadri +5Underwater 3D Reconstruction 3D Reconstruction
University of Maryland, College Park · Carnegie Mellon University · University of Rhode Island +1
Sonar-MASt3R: Real-Time Opti-Acoustic Fusion in Turbid, Unstructured Environments
Mar 13, 2026Amy Phung, Richard CamilliUnderwater 3D Reconstruction 3D Reconstruction
Applied Ocean Physics and Engineering, Woods Hole Oceanographic Institution, Deep Submergence Laboratory, Woods Hole, MA, USA · Massachusetts Institute of Technology, Cambridge, MA, USA

Aug 25, 2026J/K move · Enter open · S save

SonarLLM: A Native Sonar--Optical Multimodal Large Language Model for Underwater Perception

Cong Su, longxuan ma, Ling Dong, Guofeng Tang+3

Faculty of Information Engineering and Automation, Kunming University of Science and Technology Kunming, China · Yunnan Key Laboratory of Artificial Intelligence Kunming, China

Multimodal Sensor Fusion Underwater Imaging

Open paper AskPDF arXiv

Reliable underwater perception requires complementary sensing under variable visibility. Optical cameras capture appearance and semantics but degrade rapidly with turbidity, whereas imaging sonar preserves geometry while exhibiting distinct range-azimuth structure and acoustic artifacts. Existing MLLMs, built primarily on optical encoders, are therefore ill-suited to model sonar or adaptively exploit sonar-optical complementarity. We propose SonarLLM, a sonar-optical MLLM that treats sonar as a native perceptual modality. It combines a sonar-specific encoder, modality-specific physics-aware feature enhancement, and reliability-aware hierarchical fusion to align acoustic structure with optical semantics and dynamically adjust their contributions as sensing quality changes. We also introduce SonarBench, a paired benchmark that spans four tasks: recognition, counting, visual question answering, and captioning; and, across the benchmark, three input settings: sonar-only, optical-only, and fusion. By fixing the scene and sonar observation while varying optical degradation, SonarBench enables controlled measurement of cross-modal complementarity. SonarLLM achieves 72.0% macro accuracy across sonar-only recognition, counting, and VQA, outperforming the strongest baseline by 34.4 percentage points, and 68.7% under fusion, exceeding the best baseline by 25.1 points. For recognition and counting, the fusion-over-optical gain grows from 6.0 to 36.0 points as turbidity increases, indicating the increasing complementary value of sonar under controlled optical degradation. Together, these results show that robust heterogeneous perception depends not only on adding sonar, but on representing and weighting it according to its sensing characteristics.

Abstract

Image-to-Image Translation

Explore similar work

SonarLLM: A Native Sonar--Optical Multimodal Large Language Model for Underwater Perception

Adapting Vision Foundation Models to Acoustics for Pose-Free 3D Sonar Reconstruction

Sonar-MASt3R: Real-Time Opti-Acoustic Fusion in Turbid, Unstructured Environments