cs.CVAug 25, 2026

Weakly Supervised Seafloor Segmentation for Seagrass Habitat Mapping in Side-Scan Sonar Imagery

Authors: Hayat Rajani, Nuno Gracias, Rafael Garcia

Organizations: Computer Vision and Robotics Research Institute (ViCOROB) of the University of Girona, Spain

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Seagrass meadows are crucial blue-carbon habitats, and mapping their extent is a prerequisite for coastal management and carbon inventory. Optical satellite sensors cover large areas but cannot reach deep or turbid water, whereas side-scan sonar (SSS) images the seabed at high resolution and at any depth. Interpreting SSS, however, still relies on dense manual annotation, which is slow and costly. We address this by adapting a weakly supervised semantic segmentation framework to SSS benthic habitat mapping, so that pixel-level maps are learned from image-level labels alone. The framework couples a ViT-based encoder-decoder with a classification branch, extracts class activation maps, and refines them into pseudo-labels with a dense conditional random field that we tune for the noise and weak boundaries of acoustic imagery. It follows an iterative self-training scheme, together with a sampling strategy to cope with the strong class imbalance of the data. We also study the effect of different loss functions on segmentation quality, finding Lovász-Softmax loss the most effective. On a held-out transect, the refined pseudo-labels reached an mIoU of 89.3% against the ground truth, and the segmentation branch, trained without any pixel-level labels, reached 87.6%. Self-supervised pretraining on unlabelled SSS added a further 3% in mean intersection-over-union. Field trials further demonstrate the generalizability of the trained model. These results show that accurate and label-efficient benthic habitat mapping from side-scan sonar is feasible at the scale needed for coast-wide seagrass monitoring.

Figures & tables

Fig. 1: The overall weakly-supervised segmentation framework. (a) denotes the encoder, (b) denotes the classification sub-network, and (c) denotes the decoder.
Fig. 2: Examples of patches generated from SSS waterfalls and the corresponding image-level ground truth.
Fig. 3: Evolution of pseudo-masks during the course of training.
Fig. 4: Visualization of predictions on SSS images from the test set.
Fig. 5: (Continued) Visualization of predictions on SSS images from the test set.
Fig. 6: Comparison of results between fully- and weakly-supervised models during field trials.
More in this topic

Remote Sensing Image Segmentation

Remote sensing image segmentation labels land cover and objects in satellite, aerial, radar, and multispectral imagery.

See all 220 papers

Explore similar work

CardsList
Oct 6, 2025

BenthiCat: An opti-acoustic dataset for advancing benthic classification and habitat mapping

Benthic habitat mapping is fundamental for understanding marine ecosystems, guiding conservation efforts, and supporting sustainable resource management. Yet, the scarcity of large, annotated datasets limits the development and benchmarking of machine learning models in this domain. This paper introduces a thorough multi-modal dataset, comprising about a million side-scan sonar (SSS) tiles collected along the coast of Catalonia (Spain), complemented by bathymetric maps and a set of co-registered optical images from targeted surveys using an autonomous underwater vehicle (AUV). Approximately 36000 of the SSS tiles have been manually annotated with segmentation masks to enable supervised fine-tuning of classification models. All the raw sensor data, together with mosaics, are also released to support further exploration and algorithm development. To address challenges in multi-sensor data fusion for AUVs, we spatially associate optical images with corresponding SSS tiles, facilitating self-supervised, cross-modal representation learning. Accompanying open-source preprocessing and annotation tools are provided to enhance accessibility and encourage research. This resource aims to establish a standardized benchmark for underwater habitat mapping, promoting advancements in autonomous seafloor classification and multi-sensor integration.

Hayat Rajani, Valerio Franchi, Borja Martinez-Clavel Valles +3

Computer Vision and Robotics Research Institute, University of Girona, Spain · Tecnoambiente SLU, Spain

Underwater Imaging
AskPDF
Aug 18, 2026

Leveraging existing sparse point annotations for benthic imagery dense segmentation

The health of marine ecosystems is a critical indicator of global environmental change, yet the physical constraints of underwater observation and the intrinsic challenges of processing marine imagery severely limit the scalability of systematic monitoring. While recent visual foundation models such as the Segment Anything Model (SAM) series show great promise, they still struggle with the fine-grained recognition required in these complex scenarios and still require expert supervision. Our work addresses this gap by bridging state-of-the-art foundation models with existing sparse supervision. Because historical benthic surveys are typically annotated with only a few sparse expert points per image, we utilize these legacy point-labels as visual prompts for SAM2. Our primary contribution is a novel mechanism to automatically identify which of these points are suitable, and which are actively harmful, when used for propagation. By filtering out unreliable points, we extract high-quality pseudo-ground-truth masks capable of training more accurate, fine-grained semantic segmentation models. We demonstrate the effectiveness of our approach on public benthic data and introduce a new, challenging benchmark featuring real-world sparse expert annotations, paving the way for scalable ecological analysis.

Cesar Borja, Breck A. McCollum, Jarret E. Byrnes +2

DIIS - I3A, Universidad de Zaragoza, Zaragoza, Spain · Berklee College of Music, Boston, USA · University of Massachusetts: Boston, Boston, USA +1

Image SegmentationSemantic SegmentationWeakly Supervised Semantic SegmentationPromptable Image SegmentationPseudo-Labeling
AskPDF
Aug 24, 2026

Geometry-Driven Opti-Acoustic Co-Registration and View-Invariant Reflectivity Mapping for Side-Scan Sonar

Side-Scan Sonar (SSS) is a primary modality for large-scale underwater mapping, yet automated perception and cross-modal alignment are severely bottlenecked by acoustic complexities such as speckle noise, shadows, and extreme viewpoint dependencies. Traditional handcrafted descriptors and modern deep learning matchers fail to bridge the physical domain gap between optical and acoustic imagery without 3D geometric constraints. To overcome these limitations, we propose a novel geometry-driven framework for pixel-level opti-acoustic co-registration and view-invariant reflectivity mapping. Our method utilizes Structure-from-Motion (SfM) to reconstruct a dense 3D seafloor mesh, acting as a geometric anchor between the visual and acoustic domains. We introduce a First Bottom Return (FBR) extraction algorithm to dynamically correct non-linear altitude drift caused by uncalibrated SfM reconstruction. Furthermore, we apply an inverse Lambertian model and a dual-Gaussian weighting function to isolate the intrinsic seabed reflectivity, effectively neutralizing slant-range propagation loss and geometric view-dependence. By deterministically associating these isolated acoustic properties with optical pixels, our pipeline generates highly accurate, strictly co-registered multi-modal datasets. This automated, physics-guided approach eliminates the need for manual annotation and paves the way for advanced self-supervised learning in benthic habitat mapping.

Taqi Hamoda, Nuno Gracias

Computer Vision and Robotics Research Institute (ViCOROB) of the University of Girona, Spain

Cross-Modal Image RegistrationUnderwater Imaging
AskPDF