Superpixel Segmentation

Local anomaly detection is the task of identifying spatially confined anomalous or out-of-distribution (OOD) regions within an otherwise in-distribution (ID) image. Most existing methods are designed for a specific context, with a strong focus on industrial defect detection, where they achieve excellent performance on well-established benchmarks. However, we observe that these methods can exhibit significant performance drops in other contexts, especially when the inlier distribution is multimodal. To account for this inlier complexity, we propose LACToS, a principled local OOD detection method based on statistical typicality and made tractable through two innovations: (i) grouping pixels into superpixels and (ii) using a computationally efficient proxy to estimate typicality. We evaluate LACToS across three diverse applications spanning medical imaging, human faces, and X-ray security screening. LACToS performs more robustly than prior methods across these benchmarks. The code is released here: https://github.com/voilalab/LACToS.

Optic disc (OD) segmentation is essential for diagnosing ophthalmic pathologies from retinal fundus images. However, prevailing deep learning approaches operate as opaque black boxes, lacking the inference-stage mathematical traceability--a critical requirement for algorithmic auditing and failure analysis in clinical workflows. This paper presents a fully algorithmically traceable and trainable segmentation pipeline that jointly combines superpixel decomposition, hybrid brightness-proximity superpixel scoring, morphological regularization, iterative GrabCut refinement, and elliptical shape fitting. The hyperparameter optimization is formulated as an objective function and solved via Bayesian optimization to eliminate manual parameter tuning. A quantitative evaluation on the Drishti-GS dataset demonstrates that our method achieves a Dice coefficient of 0.9536, matching state-of-the-art performance. By maintaining explicit mathematical transparency across all processing stages, our framework offers a deterministic, traceable alternative to black-box architectures for medical review and debugging.

Quadratic unconstrained binary optimization (QUBO) has emerged as a powerful framework for medical computing problems. Binary decision variables naturally represent clinical choices, making QUBO formulations well-suited for quantum annealing hardware. However, a fundamental scalability challenge limits practical deployment: problem size grows rapidly with input dimensionality, creating computational bottlenecks that restrict applications to simplified scenarios. This paper addresses this challenge through hierarchical problem reduction, as demonstrated in medical image segmentation, where pixel-level QUBO formulations create over 65,000 variables for a 256x256 image, forcing existing approaches to downsample to 42x42 resolution and discard 97% of pixel information. A superpixel-based QUBO framework is proposed using simple linear iterative clustering (SLIC) to group pixels into perceptually meaningful regions, then formulate segmentation as QUBO over a region adjacency graph (RAG) combining min-cut and smoothness objectives. Validation on INbreast mammography breast cancer images demonstrates a 4.2% improvement in segmentation quality (mean IoU 0.76 vs 0.73) with 33 computational speedup (0.67s vs 21.97s) and a 97.3% reduction in problem size (1764 to 48 variables), all achieved while processing full-resolution images rather than downsampled versions. The reduced problem size also fits well within current quantum annealer connectivity limits, removing the embedding overhead that has historically blocked direct deployment of pixel-level QUBO segmentation on quantum hardware.

An unsupervised framework for hyperspectral image (HSI) clustering is proposed that incorporates masked deep representation learning with diffusion-based clustering, extending the Spatially-Regularized Superpixel-based Diffusion Learning ($S^2DL$) algorithm. Initially, a denoised latent representation of the original HSI is learned via an unsupervised masked autoencoder (UMAE) model with a Vision Transformer backbone. The UMAE takes spatial context and long-range spectral correlations into account and incorporates an efficient pretraining process via masking that utilizes only a small subset of training pixels. In the next stage, the entropy rate superpixel (ERS) algorithm is used to segment the image into superpixels, and a spatially regularized diffusion graph is constructed using Euclidean and diffusion distances within the compressed latent space instead of the HSI space. The proposed algorithm, Deep Spatially-Regularized Superpixel-based Diffusion Learning ($DS^2DL$), leverages more faithful diffusion distances and subsequent diffusion graph construction that better reflect the intrinsic geometry of the underlying data manifold, improving labeling accuracy and clustering quality. Experiments on Botswana and KSC datasets demonstrate the efficacy of $DS^2DL$.

Superpixels offer a compact image representation by grouping pixels into coherent regions. Recent methods have reached a plateau in terms of segmentation accuracy by generating noisy superpixel shapes. Moreover, most existing approaches produce a single fixed-scale partition that limits their use in vision pipelines that would benefit multi-scale representations. In this work, we introduce H-SPAM (Hierarchical Superpixel Anything Model), a unified framework for generating accurate, regular, and perfectly nested hierarchical superpixels. Starting from a fine partition, guided by deep features and external object priors, H-SPAM constructs the hierarchy through a two-phase region merging process that first preserves object consistency and then allows controlled inter-object grouping. The hierarchy can also be modulated using visual attention maps or user input to preserve important regions longer in the hierarchy. Experiments on standard benchmarks show that H-SPAM strongly outperforms existing hierarchical methods in both accuracy and regularity, while performing on par with most recent state-of-the-art non-hierarchical methods. Code and pretrained models are available: https://github.com/waldo-j/hspam.

Superpixels provide a compact region-based representation that preserves object boundaries and local structures, and have therefore been widely used in a variety of vision tasks to reduce computational cost. However, most existing superpixel algorithms produce irregularly shaped regions, which are not well aligned with regular operators such as convolutions. Consequently, superpixels are often treated as an offline preprocessing step, limiting parallel implementation and hindering end-to-end optimization within deep learning pipelines. Motivated by the adaptive representation and coverage property of granular-ball computing, we develop a square superpixel generation approach. Specifically, we approximate superpixels using multi-scale square blocks to avoid the computational and implementation difficulties induced by irregular shapes, enabling efficient parallel processing and learnable feature extraction. For each block, a purity score is computed based on pixel-intensity similarity, and high-quality blocks are selected accordingly. The resulting square superpixels can be readily integrated as graph nodes in graph neural networks (GNNs) or as tokens in Vision Transformers (ViTs), facilitating multi-scale information aggregation and structured visual representation. Experimental results on downstream tasks demonstrate consistent performance improvements, validating the effectiveness of the proposed method.

Conventional cameras generate a lot of data that can be challenging to process in resource-constrained applications. Usually, cameras generate data streams on the order of the number of pixels in the image. However, most of this captured data is redundant for many downstream computer vision algorithms. We propose a novel camera design, which we call SuperCam, that adaptively processes captured data by performing superpixel segmentation on the fly. We show that SuperCam performs better than current state-of-the-art superpixel algorithms under memory-constrained situations. We also compare how well SuperCam performs when the compressed data is used for downstream computer vision tasks. Our results demonstrate that the proposed design provides superior output for image segmentation, object detection, and monocular depth estimation in situations where the available memory on the camera is limited. We posit that superpixel segmentation will play a crucial role as more computer vision inference models are deployed in edge devices. SuperCam would allow computer vision engineers to design more efficient systems for these applications.

X-Ray Computed Tomography (XCT) is a compelling tool in experimental mechanics, capable of non-destructively extracting information pertaining to the internal morphology of materials. For materials with random heterogeneous morphology such as concrete, such information is of particular relevance since it allows for studies of morphology-related behaviour and for predictive modelling. Nevertheless, XCT images require semantic segmentation for practical usage. Here, concrete poses a unique challenge due to the similar X-ray attenuation coefficients of aggregates and mortar, which result in low contrast between the two phases in the ensuing XCT images. As such, purely intensity-dependent semantic segmentation tools remain unfeasible. While vision transformers (ViTs) and convolutional neural networks (CNNs) are proven techniques for semantic segmentation in such challenging cases, they typically require labelled training data, which is often unavailable for concrete or resource-intensive to obtain, thereby limiting their relevance. To address this challenge, a self-annotation technique is presented here that leverages superpixel algorithms to identify perceptually similar local regions in an image and relates them to the global context by utilizing the receptive field of a CNN-based model. This enables the model to learn a global-local relationship in the images and facilitates the identification of semantically similar structures. When evaluated against manually annotated ground truth on out-of-distribution data, the proposed methodology consistently outperformed direct greyscale thresholding across all pertinent metrics, demonstrating improved discernibility between aggregates and mortar, and providing the most favourable balance of sensitivity and precision for aggregate-phase identification.

We present an efficient method for image segmentation in the presence of strong intensity inhomogeneities. The approach combines two levels of clustering: pixels are first grouped into superpixels using Power-SLIC, which admits a discrete optimal transport (OT) interpretation, and these superpixels are subsequently merged using squared 2-Wasserstein distances between their empirical distributions. We formulate the merging process as an adjacency-constrained hierarchical optimization problem with a regularized objective that accounts for the heterogeneities of previously formed regions. We establish NP-completeness of the associated global optimization problem, even for two final regions and two-bin histograms. For the proposed greedy algorithm, we prove exact recovery under suitable histogram separation conditions and derive an additive approximation bound for the regularized energy. Numerical experiments demonstrate competitive segmentation accuracy on images exhibiting substantial intensity inhomogeneity, while retaining computational efficiency.