Apr 20, 2026, cs.CVJ/K move · Enter open · S save
Masaharu Miyazaki, Yurie Otake, Koichi Ito, Wataru Makino+2
Graduate School of Information Sciences, Tohoku University 6-6-05, Aramaki Aza Aoba, Sendai, 9808579, Japan. · The Center for Ecological Research, Kyoto University, 2–509–3, Hirano, Otsu-shi, Shiga-ken, 5202113, Japan. · Graduate School of Life Sciences, Tohoku University, 6–3, Aramaki Aza Aoba, Aoba-ku, Sendai-shi, 9808578, Japan.
Plankton monitoring is essential for assessing aquatic ecosystems but is limited by the labor-intensive nature of manual microscopic analysis. Automating the segmentation of plankton from crowded images is crucial, however, it faces two major challenges: (i) the scarcity of pixel-level annotated datasets and (ii) the difficulty of distinguishing plankton from debris and overlapping individuals using conventional CNN-based methods. To address these issues, we propose PlankFormer, a novel framework for plankton instance segmentation. First, to overcome the data shortage, we introduce a method to generate labeled Pseudo Community Images (PCI) by synthesizing individual plankton images onto diverse backgrounds, including those created by generative models. Second, we propose a segmentation model utilizing a Vision Transformer (ViT) backbone with a Mask2Former decoder. To robustly capture the global structural features of plankton against occlusion and debris, we employ a Masked Autoencoder (MAE) for self-supervised pre-training on unlabeled individual images. Experimental results on real-world datasets demonstrate that our method significantly outperforms conventional methods, such as Mask R-CNN, particularly in challenging environments with high debris density. We demonstrate that our synthetic training strategy and MAE-based architecture enable high-precision segmentation with requiring less manual annotations for individual plankton images.