Unsupervised Image Segmentation

Ask about this topic

Momentum

4 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 11

All topics
CardsList
Sep 30, 2026cs.CV

Seeing as Humans Do: Learning from Motion to Segment Anything Without Supervision

The Segment Anything Model (SAM) relies heavily on massive manual annotations, creating a fundamental bottleneck for model scaling. While unsupervised methods attempt to learn object concepts from motion, they typically overfit to moving entities, lacking both multi-granularity understanding and the ability to generalize to static objects. To overcome this, we introduce Motion-Grounded Segment Anything (MoSA), a highly scalable unsupervised framework that learns a transferable objectness prior from unlabeled videos. MoSA operates in three progressive stages: (1) automatically generating multi-granularity motion pseudo-labels from large-scale video data; (2) training a Perceptual Grouping Model (PGM) via contrastive learning to internalize a generalized, appearance-driven concept of objects; and (3) transferring this learned prior into a prompt-guided architecture for segment-anything-style inference on images. Extensive zero-shot evaluations across seven challenging benchmarks (e.g., COCO and ADE20K) demonstrate that MoSA significantly outperforms existing unsupervised methods. Notably, despite using zero manual annotations, MoSA achieves segmentation performance comparable to the fully supervised SAM. Our findings reveal that harnessing large-scale unlabeled motion is a feasible and highly scalable alternative to annotation-driven segment-anything pipelines.

Weijian Jian, Xiaoyue Zhang, Bin Xiao +5

AI Research · Independent Researcher · University of Ottawa +1

Image SegmentationSelf-Supervised LearningPromptable Image SegmentationUnsupervised Image Segmentation
AskPDF
Sep 29, 2026cs.CV

ProGuT: Label-Efficient Panoptic Segmentation for Forest Scenes

Panoptic segmentation in forest environments is bottlenecked not by semantic quality but by instance separation; existing unsupervised panoptic approaches produce usable stuff maps but near-zero thing quality. Depth or flow-based instance discovery methods needs sensors that are not always available. We present ProGuT (Prototype Guided Training), which produces panoptic pseudo-labels without per-image training masks, needing only unlabeled images and one-time cluster-to-class mapping. ProGuT clusters CLIP patch features, then recovers trunk instances through multiscale geometric prior that falsifies non-trunk structures via structure-tensor. This is cheap compared to depth, flow or class-supervision methods to create pseudo labels. These are then used for downstream tasks which we evaluate against other unsupervised baselines. ProGuT achieves a Panoptic Quality (PQ) of 65.2 on Our-forest dataset (2.6x improvement over the initial pseudo-label quality) and reaches 65.9 mIoU on Freiburg Forest, outperforming unsupervised baselines like PiCIE (45.3 IoU) and STEGO(57.6IoU). Additionally, ProGuT outperforms existing unsupervised methods for class-agnostic trunk instance benchmark.

Pankaj Deoli, Karsten Berns

Robotics Research Lab, RPTU Kaiserslautern-Landau Gottlieb-Daimler-Str. 48, 67663 Kaiserslautern, Germany

Image SegmentationPanoptic SegmentationInstance SegmentationPseudo-LabelingUnsupervised Image Segmentation
AskPDF
Sep 24, 2026cs.CV

Can Frozen Hyperspherical Features Guide the Selection of Pseudo Masks?

Foundation segmenters such as SAM return several plausible masks for an unlabeled image, and a student trained on the wrong one inherits its errors. Choosing among them means querying a second large model or fitting a quality head to annotated masks. We show that a candidate can be judged by what it does to a frozen self-supervised backbone's features. Normalized DINOv2 patch features lie on a hypersphere, and a candidate mask splits that sphere in two. Based on this reading, we introduce SphereTrust, which scores each candidate by three properties of the split, the angular contrast between the two sides, the coverage of the foreground's appearance modes, and contact with the image frame, one for each of three common ways a mask fails, and ranks a pool in 0.55 s per image from the frozen features alone. On eight SAM and SAM3 candidate pools spanning camouflaged, salient, and dichotomous segmentation and camouflage under low light, SphereTrust exceeds the strongest evaluated external baseline on six pools by 1.7 to 9.3 percentage points in mean selected Dice. These comparisons include published selection rules and explicitly labeled adaptations of DSS and UCOD-MKD. On the two prompted camouflage pools, its mean selected Dice is within 0.1 percentage points of the candidate-derived DSS adaptation, with a lower catastrophic-error rate. Which cue carries the signal depends on the candidate pool. The same sphere also supports training. The leading candidates enter as a candidate set with their scores as priors, prototypes reorder them, and a cross-fitted second round completes the labels, raising weighted F by 4.5, 2.3, and 5.5 points over fixed-label training on the three MLLM anchor pools, with students competitive with published unsupervised methods on nineteen test sets.

Xinge Guo, Fengyang Xiao, Dingming Zhang +6

Duke University · Carnegie Mellon University · University of Alabama at Birmingham

Promptable Image SegmentationPseudo-LabelingUnsupervised Image Segmentation
AskPDF
Sep 23, 2026cs.CV

ICM: Intra-class Mixing for Domain Adaptation in Adverse Weather

Unsupervised domain adaptation (UDA) for semantic segmentation remains challenging under adverse weather conditions because severe appearance changes enlarge the domain gap and degrade the reliability of pseudo labels in the target domain. To address this problem, we propose an Intra-Class Mixing Consistency (ICM) framework that enforces prediction consistency between an intra-class mixed image and its original counterpart. Unlike previous mixing-based consistency methods that combine regions across different images or domains and may introduce unrealistic semantic inconsistencies, ICM performs mixing within the same image and semantic class, preserving realistic semantic layout for consistency regularization. With ICM, we establish a new state-of-the-art performance for clear-to-adverse-weather unsupervised domain adaptation (UDA) in semantic segmentation. On the Cityscapes →\rightarrow ACDC benchmark, our method achieves 75.7% mIoU, outperforming the previous state of the art by +1.9 pp, demonstrating its effectiveness in mitigating class confusion under challenging environmental conditions. The code is provided in the supplementary material.

Boying Li, Chang Liu, Britta Ayano Wilde +4

Unsupervised Domain AdaptationDomain AdaptationConsistency RegularizationSemantic SegmentationUnsupervised Image Segmentation
AskPDF
Aug 3, 2026cs.CV

PixelUp: Zero-Shot Semantic Feature Upsampling for Fine-Grained Vision Tasks

Self-supervised Vision Foundation Models (VFMs) have become essential backbones for downstream tasks due to their strong and transferable visual representations. However, their patch-token-level features are often too coarse for dense prediction tasks such as semantic segmentation and depth estimation when accurate fine-grained predictions are required. Feature upsampling methods have been developed to recover pixel-level detail but still face limitations. Learnable upsamplers are often designed for a specific encoders and must be retrained for different encoders. Image-guided methods that use shallow pixel encoders often introduce textural artifacts and lack the semantic guidance needed for accurate downstream predictions. We introduce PixelUp, a zero-shot VFM-agnostic upsampler achieving semantic awareness through a coarse-to-fine chain of windowed cross-attention architecture guided by multi-scale semantic features. We demonstrate that PixelUp outperforms both VFM-specific and VFM-agnostic upsamplers, achieving state-of-the-art performance on dense prediction tasks with an average improvement of +1.2 mIoU on semantic segmentation and +0.25 δ1δ_1, on NYUv2 depth estimation across VFMs. PixelUp further improves training-free open-vocabulary and unsupervised semantic segmentation by an average of +1.3 mIoU and +0.5 mIoU, respectively. Code available at https://pixelup-project.vercel.app/

Deepank Singh, Anurag Nihal, Vedhus Hoskere

University of Houston

Depth EstimationVision Foundation Model AdaptationSemantic SegmentationUnsupervised Image Segmentation
AskPDF
Jul 14, 2026cs.CV

UMSS: Towards Unsupervised Multi-modal Semantic Segmentation

Multimodal semantic segmentation (MSS) is essential for robust perception in complex environments, yet its potential remains largely untapped because of the prohibitive cost of human annotations. While unsupervised semantic segmentation (USS) has achieved strong results on a single RGB modality, its naive extension to multimodal data is often hindered by fusion degradation. This occurs because, without explicit supervision, existing frameworks struggle to reconcile the heterogeneous structural patterns captured by different sensors and therefore fail to effectively exploit their complementary information. In this paper, we make the first attempt to address the novel problem of Unsupervised Multimodal Semantic Segmentation (UMSS), aiming to effectively exploit complementary sensor information in a fully label free setting. To this end, we propose UniM2 (Unified Multimodal), a novel framework built on DINOv3 that transforms conventional fusion methods into consistent performance gains. Our key idea is to learn a unified latent space driven by Cross Modal Correspondence Synergy (CMCS) to extract intrinsic shared semantic cues, bypassing the need for label guided adaptive fusion. To mitigate inherent intermodal conflicts, we introduce a Cross Modal Harmonizer (CMH) that designates RGB as a stable reference, effectively suppressing inconsistent relational supervision while guiding the model to exploit complementary structural features. Extensive experimental results on NYU Depth v2 and MFNet show that UniM2 improves mIoU by 6.4% and 9.8%, respectively, demonstrating clear advantages over existing frameworks for UMSS.

Haitian Zhang, Thai Duy Nguyen, Xiangyuan Wang +2

EmPACT Lab, School of EEE, Nanyang Technological University, Singapore · The University of Hong Kong, Hong Kong SAR, China

Cross-Modal Representation LearningSemantic SegmentationUnsupervised Image SegmentationMultimodal Sensor Fusion
AskPDF
Jun 18, 2026cs.CV

Speeding up the annotation process in semantic segmentation industrial applications

Current machine learning models commonly require large and well-annotated datasets. However, the annotation process often becomes a bottleneck, with increased complexity leading to higher chances of human errors. Within this context, our goal in this paper is to leverage unsupervised algorithms to improve data annotation efficiency for complex semantic segmentation problems in industrial materials science. Previous research has quantified labeling time and others explored unsupervised methods. However, to the best of our knowledge, this is the first study to quantify how much unsupervised algorithms accelerate the labeling process. We aim to validate the extent to which this laborious process can be accelerated, focusing on semantic segmentation tasks that involve annotating each pixel of high-resolution images, such as the microstructure characterization challenge in materials science. Specifically, we demonstrate that by using unsupervised computer vision algorithms, the time required for the labeling process can be reduced from 170 hours to 37 hours, achieving an approximate reduction of 78%. The dataset we work with includes large images of dimensions 1280x959 and 960x703, which further increases the complexity of the annotation task. Despite these challenges, we create and share the largest public steel microstructure segmentation dataset to date, available under MIT License with permanent DOI, contributing a fully annotated, high-resolution dataset to the field. Additionally, this is the first work to compare the labeling time from scratch (a common approach in previous studies) to the labeling time when using these unsupervised algorithms as a pre-annotation step. Furthermore, we provide a Deep Learning model trained on this dataset, validated by field experts, and deployed in an industrial setting, serving as an initial benchmark for this public dataset.

Marta Fernandez-Moreno, Margarita Guerrero, Rosalia Rementeria +2

Department of Computer Science and Artificial Intelligence, Andalusian Research Institute in Data Science and Computational Intelligence, DaSCI, University of Granada, 18071, Granada, Spain · Ingénica STS, Av. Pablo Iglesias 38, Bj, 33205, Gijón, Spain · ArcelorMittal Global R&D, SLab—Steel Labs, Calle Marineros 4, 33490, Avilés, Spain +1

Human-in-the-Loop AnnotationSemantic SegmentationUnsupervised Image Segmentation
AskPDF
May 28, 2026cs.CV

Unsupervised Semantic Segmentation Facilitates Model Understanding

Self-supervised learning (SSL) has produced a diverse landscape of vision transformers (ViTs) whose pretrained representations support a wide range of downstream tasks. Towards a better understanding of these models, a body of work has assessed the mechanics of their self-attention as well as the types of information captured across their representations, revealing, for example, stark differences between models trained with contrastive learning (CL) and masked image modeling (MIM). However, the total of these advances on model understanding has to date not yet fully permeated a larger community, where, e.g., insights that are specific to CL models are still at times generalized to MIM models. To make model understanding straightforward and intuitive for a broad community, we propose a simple and easily interpretable visualization protocol. Our protocol is based on visualizing unsupervised semantic segmentation results, yet by no means do we focus on top segmentation performance. Instead, our protocol allows us to easily convey model behavior that consistently emerges across images. Benchmarked on a diverse set of SSL models across layers and representations, our protocol allows us to gain novel insights into distinct positional biases and scaling behaviors, including, e.g., strong boundary artifacts in DINOv3-Large model tokens. These novel insights come on top of more easily conveying a range of previous findings. Our protocol further allows us to clearly visually convey and distinguish between positional effects and the closely related but distinct locality bias, the latter being much more extensively studied in the literature so far. Our protocol is publicly available, serving to catalyze further model understanding for a broad community.

Xiaoyan Yu, Lisa Mais, Jannik Franzen +4

Max-Delbrück-Center (MDC), Berlin, Germany · Helmholtz Imaging · Humboldt-Universität zu Berlin, Berlin, Germany +2

Transformer InterpretabilityVision TransformerUnsupervised Image SegmentationSelf-Supervised Visual Representation Learning
AskPDF
May 20, 2026eess.IV

Local-sensitive connectivity filter (ls-cf): A post-processing unsupervised improvement of the frangi, hessian and vesselness filters for multimodal vessel segmentation

A retinal vessel analysis is a procedure that can be used as an assessment of risks to the eye. This work proposes an unsupervised multimodal approach that improves the response of the Frangi filter, enabling automatic vessel segmentation. We propose a filter that computes pixel-level vessel continuity while introducing a local tolerance heuristic to fill in vessel discontinuities produced by the Frangi response. This proposal, called the local-sensitive connectivity filter (LS-CF), is compared against a naive connectivity filter to the baseline thresholded Frangi filter response and to the naive connectivity filter response in combination with the morphological closing and to the current approaches in the literature. The proposal was able to achieve competitive results in a variety of multimodal datasets. It was robust enough to outperform all the state-of-the-art approaches in the literature for the OSIRIX angiographic dataset in terms of accuracy and 4 out of 5 works in the case of the IOSTAR dataset while also outperforming several works in the case of the DRIVE and STARE datasets and 6 out of 10 in the CHASE-DB dataset. For the CHASE-DB, it also outperformed all the state-of-the-art unsupervised methods.

Erick O Rodrigues, Lucas O Rodrigues, João HP Machado +5

Department of Academic Informatics (DAINF), Universidade Tecnologica Federal do Parana (UTFPR), Pato Branco 85503-390, PR, Brazil · Graduate Program of Sciences Applied to Health Products, Universidade Federal Fluminense (UFF), Niteroi 24241000, RJ, Brazil · Institute of Technological Sciences (ICT), Universidade Federal de Itajuba (UNIFEI), Ttabira 35903-087, MG, Brazil +1

Retinal Vessel SegmentationMedical Image SegmentationUnsupervised Image Segmentation
AskPDF
Apr 21, 2026cs.CV

Deep Sprite-based Image Models: An Analysis

While foundation models drive steady progress in image segmentation and diffusion algorithms compose always more realistic images, the seemingly simple problem of identifying recurrent patterns in a collection of images remains very much open. In this paper, we focus on sprite-based image decomposition models, which have shown some promise for clustering and image decomposition and are appealing because of their high interpretability. These models come in different flavors, need to be tailored to specific datasets, and struggle to scale to images with many objects. We dive into the details of their design, identify their core components, and perform an extensive analysis on clustering benchmarks. We leverage this analysis to propose a deep sprite-based image decomposition method that performs on par with state-of-the-art unsupervised class-aware image segmentation methods on the standard CLEVR benchmark, scales linearly with the number of objects, identifies explicitly object categories, and fully models images in an easily interpretable way.

Zeynep Sonat Baltacı, Romain Loiseau, Mathieu Aubry

LIGM, CNRS, Univ Gustave Eiffel, ENPC, Institut Polytechnique de Paris, France

ClusteringObject-Centric Representation LearningImage ClusteringUnsupervised Image Segmentation
AskPDF
Feb 23, 2026cs.CV

Segmenting Low-Contrast XCTs of Concrete: An Unsupervised Approach

X-Ray Computed Tomography (XCT) is a compelling tool in experimental mechanics, capable of non-destructively extracting information pertaining to the internal morphology of materials. For materials with random heterogeneous morphology such as concrete, such information is of particular relevance since it allows for studies of morphology-related behaviour and for predictive modelling. Nevertheless, XCT images require semantic segmentation for practical usage. Here, concrete poses a unique challenge due to the similar X-ray attenuation coefficients of aggregates and mortar, which result in low contrast between the two phases in the ensuing XCT images. As such, purely intensity-dependent semantic segmentation tools remain unfeasible. While vision transformers (ViTs) and convolutional neural networks (CNNs) are proven techniques for semantic segmentation in such challenging cases, they typically require labelled training data, which is often unavailable for concrete or resource-intensive to obtain, thereby limiting their relevance. To address this challenge, a self-annotation technique is presented here that leverages superpixel algorithms to identify perceptually similar local regions in an image and relates them to the global context by utilizing the receptive field of a CNN-based model. This enables the model to learn a global-local relationship in the images and facilitates the identification of semantically similar structures. When evaluated against manually annotated ground truth on out-of-distribution data, the proposed methodology consistently outperformed direct greyscale thresholding across all pertinent metrics, demonstrating improved discernibility between aggregates and mortar, and providing the most favourable balance of sensitivity and precision for aggregate-phase identification.

Kaustav Das, Gaston Rauchs, Jan Sykora +1

Department of Mechanics, Faculty of Civil Engineering, Czech Technical University in Prague, Thákurova 7, 166 29 Prague 6, Czech Republic · Structural Composites Unit, Luxembourg Institute of Science and Technology (LIST), 5 avenue des Hauts-Fourneaux L-4362 Esch-sur-Alzette, Luxembourg

Image SegmentationComputed TomographySemantic SegmentationUnsupervised Image Segmentation
AskPDF