USIS-PGM: Photometric Gaussian Mixtures for Underwater Salient Instance Segmentation

Lin Hong; Xiangtong Yao; Mürüvvet Bozkurt; Xin Wang; Fumin Zhang

cs.CVMar 14, 2026

USIS-PGM: Photometric Gaussian Mixtures for Underwater Salient Instance Segmentation

Authors: Lin Hong, Xiangtong Yao, Mürüvvet Bozkurt, Xin Wang, Fumin Zhang

Organizations: Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology, Hong Kong, China · School of CIT, Chair i6, Technical University of Munich, Munich, Germany · School of CIT, Technical University of Munich, Munich, Germany · School of Robotics and Advanced Manufacture, Harbin Institute of Technology (Shenzhen), Shenzhen, China

Abstract

Underwater salient instance segmentation (USIS) is crucial for marine robotic systems, as it enables both underwater salient object detection and instance-level mask prediction for visual scene understanding. Compared with its terrestrial counterpart, USIS is more challenging due to the underwater image degradation. To address this issue, this paper proposes USIS-PGM, a single-stage framework for USIS. Specifically, the encoder enhances boundary cues through a frequency-aware module and performs content-adaptive feature reweighting via a dynamic weighting module. The decoder incorporates a Transformer-based instance activation module to better distinguish salient instances. In addition, USIS-PGM employs multi-scale Gaussian heatmaps generated from ground-truth masks through Photometric Gaussian Mixture (PGM) to supervise intermediate decoder features, thereby improving salient instance localization and producing more structurally coherent mask predictions. Experimental results demonstrate the superiority and practical applicability of the proposed USIS-PGM model.

Figures & tables

Fig. 1: Overall framework of the proposed USIS-PGM model, consisting of an encoder, a decoder, and the PGM-based multi-scale supervision strategy. The encoder integrates FAN and DWM to enhance boundary-aware representation and content-adaptive feature reweighting, while the decoder employs a Transformer-based IAM for accurate salient instance localization and mask prediction.

Fig. 2: Visualization of Gaussian heatmaps generated from ground-truth mask based on $G(I;\lambda)$ , where $\lambda=\{1,5,10,20\}$ .

Fig. 3: Qualitative evaluation of the proposed USIS-PGM model with 12 benchmark methods on the USIS16K dataset.

Methods	$mAP(\uparrow)$	$AP_{50}(\uparrow)$	$AP_{75}(\uparrow)$	$AP_{S}(\uparrow)$	$AP_{M}(\uparrow)$	$AP_{L}(\uparrow)$
Mask R-CNN [ 15 ]	.736	.900	.817	.400	.604	.745
Mask Scoring R-CNN [ 16 ]	.716	.885	.798	.450	.515	.726
YOLACT [ 17 ]	.754	.904	.826	.450	.545	.760
SOLO [ 18 ]	.595	.763	.659	.119	.327	.606
SOLOV2 [ 19 ]	.717	.859	.771	.450	.482	.726
PointRend [ 20 ]	.763	.894	.827	.500	.659	.774

TABLE I: Quantitative evaluation results of the USIS-PGM model.

Fig. 4: PR curve performance of the proposed USIS-PGM model and 12 benchmark methods on four representative object categories, i.e., (a) Diver, (b) Swimmer, (c) Plastic cup, and (d) Sea chest grating.

Methods	$mAP(\uparrow)$	$AP_{50}(\uparrow)$	$AP_{75}(\uparrow)$	$AP_{S}(\uparrow)$	$AP_{M}(\uparrow)$	$AP_{L}(\uparrow)$
USIS-PGM (-PGM)	.798	.889	.859	.540	.549	.809
USIS-PGM (ours)	.816	.912	.882	.568	.670	.814

TABLE II: Ablation study of the PGM-based multi-scale supervision.

Fig. 5: 3D reconstruction results of underwater objects, including (a) sacrificial anodes and (b) sea snails, obtained using USIS-PGM and COLMAP [ 26 ] .

Image Segmentation

Image segmentation assigns pixels or voxels to objects and regions, supporting measurement, scene understanding, and decisions across image domains.

See all 399 papers

CoRef-DETR: Decoupled Global Context and Selective Boundary-Category Refinement for Underwater Instance Segmentation
Jun 6, 2026M. Fazri Nizar, Julian Supardi, Muhammad Naufal RachmatullahUnderwater Imaging Instance Segmentation
Department of Informatics Engineering, Universitas Sriwijaya, Indonesia
Learning Dynamic Structural Specialization for Underwater Salient Object Detection
May 15, 2026Lin Hong, Chenhui Wang, Linan Deng +7Underwater Imaging Salient Object Detection
School of Automation, Southeast University, Nanjing, China.
GeoPrompt: Geometry-Conditioned Prompt Learning for Underwater Open-Vocabulary Instance Segmentation
Oct 25, 2026Shanshan Han, Yuxi Wang, Can Qin +4Instance Segmentation Prompt Learning
Ocean University of China, Qingdao, China · Nanjing University of Aeronautics and Astronautics, Nanjing, China

Jun 6, 2026J/K move · Enter open · S save

CoRef-DETR: Decoupled Global Context and Selective Boundary-Category Refinement for Underwater Instance Segmentation

M. Fazri Nizar, Julian Supardi, Muhammad Naufal Rachmatullah

Department of Informatics Engineering, Universitas Sriwijaya, Indonesia

Underwater Imaging Instance Segmentation Detection Transformer

Open paper AskPDF arXiv

Underwater instance segmentation supports marine surveys, aquaculture, and underwater robotics. Haze, colour casts, and low contrast blur object boundaries, and recent benchmarks distinguish up to 158 fine-grained categories. The strongest published methods mostly have 219 to 632 million parameters. This paper proposes CoRef-DETR, a compact detection transformer that targets three error sources of a real-time detector: context, boundaries, and categories. First, a decoupled global context (DGC) detector, with the base detector's 33.4 M parameters, applies global self-attention to a 384-pixel token grid but supervises and decodes masks in a 576-pixel frame. Second, a posterior-conditioned boundary refiner with 47.7 k parameters corrects mask logits on predicted boundaries, and a quality score on the corrected masks re-ranks the detections. Third, a crop-probe category branch on frozen DINOv2 features revises the categories of at most three masks per image, where a rule fixed on validation data chooses soft fusion or a confident-disagreement gate. On five protocols of UIIS, UIIS10K, USIS10K, and USIS16K, CoRef-DETR achieves state-of-the-art mask average precision (AP), exceeding the strongest published results by 2.3 to 5.2 points, and outperforms three reproduced recent methods on the same test images (on USIS16K, with its category branch). In half precision with TensorRT, it processes an image in a median of 34 to 40 ms, JPEG decoding included, on a T4 GPU, 1.4 to 11 times faster than these methods. Ablations attribute most of DGC's gain to the larger frame; an error analysis locates the remaining error in crowds of small objects.

Abstract

Image Segmentation

Explore similar work

CoRef-DETR: Decoupled Global Context and Selective Boundary-Category Refinement for Underwater Instance Segmentation

Learning Dynamic Structural Specialization for Underwater Salient Object Detection

GeoPrompt: Geometry-Conditioned Prompt Learning for Underwater Open-Vocabulary Instance Segmentation