cs.CVNov 4, 2026

Navigational Safety of Marine Surface Vehicles in Restricted Visibility: From Intelligent Vision to Vision-Centered Multi-Sensor Fusion

Authors: Ryan Wen Liu, Fengchuan Song, Tianxiang Wang, Guobei Peng, Youngrong Kim, Xian Zhong, Hooi-Siang Kang, Maohan Liang, 

Organizations: Sanya Science and Education Innovation Park of Wuhan University of Technology, Sanya 572025, China · School of Navigation, Wuhan University of Technology, Wuhan 430063, China · School of Marine Science and Technology, Wuhan University of Technology, Sanya 572025, China · State Key Laboratory of Maritime Technology and Safety, Wuhan 430063, China · 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Marine surface vehicles (MSVs) have attracted considerable attention for their commercial, scientific, and military applications. Owing to their favorable performance-to-cost ratio, shipboard cameras are widely used to detect and recognize potentially hazardous situations across diverse operational scenarios. However, visual perception is inevitably degraded by restricted conditions, including irregular motion of the imaging platform, adverse weather, and long imaging distances. To improve situational awareness and navigation capabilities, complementary sensors, such as the automatic identification system (AIS), radar, and light detection and ranging (LiDAR), etc., have become essential shipboard equipment. Multi-sensor fusion can provide richer perceptual information and help ensure navigational safety of MSVs in restricted visibility. This article reviews recent progress from intelligent vision to vision-centered multi-sensor fusion for newcomers and researchers in intelligent ship navigation and intelligent waterborne transportation system (IWTS). After introducing the generation of quality-degraded images in restricted visibility, we will review intelligent vision methods for four key issues, i.e., shipboard video stabilization, visual perception enhancement, object detection and tracking, and navigable area segmentation. We then examine vision-centered multi-sensor fusion methods, including infrared-visible image fusion, radar-vision fusion, LiDAR-vision fusion, AIS-vision fusion, and other multi-sensor fusion scenarios. Open-source datasets and safety-critical applications are also summarized and discussed. Finally, we highlight open challenges and future research directions. The collected maritime datasets are available at https://github.com/MAPGroup/Datasets and will be regularly updated.

Figures & tables

Figure 1: Fundamental architecture of a typical MSV. Visual illustration of an MSV equipped with multiple sensors, a ground station, GNC (guidance, navigation and control) system, communication system, propulsion and power system. Please refer to Ref. ( Liu et al., 2016 ) for more details on the typical elements of MSV, i.e. , sensors ( a.k.a. , sensing data collection equipment), hull and auxiliary structural elements, propulsion and power system, GNC system, communication system, ground station, etc .
Figure 2: Organizational structure of our work. The image generation, intelligent vision, and vision-centered multi-sensor fusion methods, as well as the maritime datasets and safety-critical practical applications are firstly discussed. We then analyze the open challenges and future research directions in this work.
AbbreviationFull termAbbreviationFull term
AIArtificial intelligenceCVComputer vision
AISAutomatic identification systemMMSIMaritime mobile service identity
ASVsAutonomous surface vehiclesUSVsUnmanned surface vehicles
MSVsMarine surface vehiclesMASSMaritime autonomous surface ship
IMUInertial measurement unitVHFVery-high-frequency
UHDUltra-high definitionmmWaveMillimeter-wave
Table 2: Abbreviations used in this article.
Figure 3: Overview of visually degraded image generation methods. Existing methods are mainly divided into two categories, i.e. , physical modeling and data-driven methods.
Figure 4: Overview of intelligent vision for marine surface vehicles. The intelligent vision pipeline integrates shipboard video stabilization, visual perception enhancement (including hazy, low-light, and multi-scene enhancement), object detection and tracking, and navigable area segmentation, providing robust situational awareness for MSVs in restricted visibility.
Figure 5: Illustrations of specific-purpose , general-purpose , and multi-scene image enhancement . Each specific-purpose model is tailored to handle a particular type of degradation ( e.g. , haze, snow, rain, and low-lightness). The general-purpose model is proposed to tackle a variety of image enhancement tasks using a shared or scalable network ( Cui et al., 2026 ) . In contrast, the multi-scene framework is capable of tackling various degradations using one single model ( a.k.a. , all-in-one).
MethodYearModalityFramework
Complex-scene-aware methods
RetinaMFANet ( Ma et al., 2022 )2022VISRetinaNet
Weather-OD ( Chen et al., 2024 )2024VISMultiple detectors
MDD-ShipNet ( Wang et al., 2024d )2024VISCNN
FREGNet ( Tian et al., 2024 )2024VISCNN-GCN
CFDBNet ( Tian and Meng, 2025 )2025VISDual-branch CNN
Table 3: Representative maritime object detection methods.
Figure 6: Representative YOLO- and DETR-based frameworks for maritime object detection. The upper branch illustrates multi-scale feature extraction and aggregation followed by YOLO detection heads. The lower branch illustrates a DETR-based architecture with a hybrid encoder, uncertainty-minimal query selection, and a transformer decoder for object classification and localization.
MethodYearTracking paradigmBase framework
Tracking-by-detection methods
DeepSORT ( Wojke et al., 2017 )2017Motion-appearance associationSORT and deep Re-ID
YOLOv4-UKF ( Hong et al., 2022 )2022Detection and motionYOLOv4 and UKF
ORB-and-size tracker ( Park et al., 2022 )2022Spatial-feature associationIoU and ORB
MaritimeMOT ( Shao et al., 2024 )2024Maritime target associationYOLOv7-based MOT
Sea-IoUTracker ( Guo et al., 2024 )2024Buffered-IoU associationSORT
Table 4: Representative maritime object tracking methods.
Figure 7: Schematic overview of 4D mmWave radar-vision fusion. Vision and 4D mmWave radar provide complementary semantic, geometric, and kinematic information. Cross-modal alignment establishes temporal, spatial, and regional correspondences between the two modalities, followed by radar-vision feature fusion using simple, attention-based, graph-based, or asymmetric fusion strategies. The fused features support downstream maritime perception tasks, including unified perception, multi-object tracking, and language-guided perception.
Figure 8: Representative LiDAR–camera fusion paradigms for maritime 3D object detection. (a) Image detections define viewing frustums for selecting LiDAR points before 3D detection. (b) Image and LiDAR features are aligned and fused in a bird’s-eye-view representation. (c) Independently generated image detections and LiDAR proposals are associated across modalities and jointly refined to produce 3D bounding boxes.
Figure 9: Flowchart of AIS and vision data fusion. The AIS and vision data fusion contributes to an augmented reality (AR) navigation system for intelligent surface vehicles. It improves situational awareness and decision-making capacities for safe maritime navigation under complex conditions.
Figure 10: Framework of vision-centered multi-sensor fusion. In particular, gyro denotes the gyrocompass device. MCGA, DAWF/TDSF, and VARF, respectively, denote the AIS–radar data association, AIS–radar data fusion, and vision–AIS–radar data fusion methods. The selection strategy seselects the appropriate fusion method based on available observation sources. Please refer to Ref. ( Liu et al., 2026 ) for more details on vision-centered multi-sensor fusion and its application in situational awareness for safe navigation.
Figure 11: Overview of vessel behavior prediction for navigational safety in restricted visibility. Behavior prediction involves estimating the future movements, trajectories, speeds, and destinations of vessels of interest. It is able to enhance maritime safety, strengthen maritime security, optimize port and traffic management, etc. In current literature, vessel behavior prediction mainly includes motion and trajectory prediction, route and destination prediction, maneuvering intention prediction, and interaction-aware behavior prediction in this work.
Figure 12: Conceptual framework for intelligent collision avoidance in restricted visibility. Fused observations, behavior forecasts, and operating context support encounter evaluation and collision-risk assessment. Rule- and knowledge-based, model-based, and learning-based methods can be combined to generate avoidance decisions under safety, COLREGs, and maneuverability constraints. Guidance and motion control execute the resulting maneuvers, while updated observations close the feedback loop.
Figure 13: Examples of vision-centered navigation systems. From top-left to bottom-right: (a) Furuno Envision AR-based navigation system; (b) marine situational awareness system developed by Orca AI Ltd.; (c) Hyundai intelligent navigation assistant system (HiNAS) with assisted navigation (top) and berthing/unberthing assistance (bottom); and (d) Marautec i-EYE with several practical capabilities.
Figure 14: Overview of the open challenges and future research directions for navigational safety in restricted visibility. The five central topics, i.e. , datasets and benchmarks, robust imaging in severely restricted visibility, object detection in complex conditions, multi-sensor fusion in dynamic environments, and scene understanding and reasoning, are organized from their key challenges to promising future research directions.
Figure 15: Emerging imaging technologies for enhanced visibility. Illustrations of two advanced imaging technologies, i.e. , (a) HADAR and (b) Tesla’s AI photon count reconstruction, which can improve MSV perception in dense-haze scenarios.
Figure 16: Conceptual framework for maritime scene understanding and reasoning under restricted visibility. Multimodal observations support scene understanding, maritime reasoning, and verified decision support, with dashed arrows indicating evidence refinement and uncertainty feedback.
More in this topic

Maritime Navigation

See all 22 papers

Explore similar work

CardsList

  1. NavEYE: Vision-Centered Multi-Sensor Fusion-Based Situational Awareness System for Intelligent Surface Vehicles

    Jul 4, 2026Ryan Wen Liu, Junxiong Liang, Haoyu Wang +1Camera-Radar FusionMultimodal Sensor Fusion

    Sanya Science and Education Innovation Park, Wuhan University of Technology, Sanya, 572000, Hainan, China · School of Navigation, Wuhan University of Technology, Wuhan, 430063, Hubei, China · State Key Laboratory of Maritime Technology and Safety, Wuhan, 430063, Hubei, China

  2. Unified Restoration-Perception Learning: Maritime Infrared-Visible Image Fusion and Segmentation

    Mar 30, 2026Weichao Cai, Weiliang Huang, Biao Xue +3Infrared-Visible Image FusionSemantic Segmentation

    Laboratory of Underwater Acoustic Communication and Marine Information Technology, Ministry of Education, Xiamen University, Xiamen, Fujian 361005, China. · PAMI Research Group, Department of Computer and Information Science, University of Macau, Macau, China · School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University, Shenzhen 518107, China

  3. Multi-Modal Graph Neural Network-Inspired Vessel Identification and Tracking for UAV-Based Maritime Surveillance

    Oct 9, 2026Chenjie Zhao, Jiangtian Nie, Jiawen Kang +2Maritime SurveillanceMulti-Object Tracking

    Sanya Science and Education Innovation Park of Wuhan University of Technology, Sanya 572025, China · School of Navigation, Wuhan University of Technology, Wuhan 430063, China · School of Natural and Computing Sciences, University of Aberdeen, Aberdeen AB24 3UE, U.K. +2