From Proper Scoring Rules and Edge Biases to Mumford-Shah Models: Robust Unsupervised Hyperspectral Image Segmentation

Fabian Conrad; Benjamin Berkels

cs.CVOct 20, 2026

From Proper Scoring Rules and Edge Biases to Mumford-Shah Models: Robust Unsupervised Hyperspectral Image Segmentation

Authors: Fabian Conrad, Benjamin Berkels

Organizations: IGPM, RWTH Aachen University, Germany

Abstract

Hyperspectral images provide a rich representation of the underlying spectrum for each pixel, allowing for a pixel-wise classification/segmentation into different classes. As the availability of labeled hyperspectral images is limited and the acquisition of labeled training data is time-consuming, unsupervised segmentation methods are important in hyperspectral image segmentation. Spectral variability, mixed pixels, noise as well as high correlation between neighboring spectral bands make this task challenging. We present a feature-agnostic, unsupervised hyperspectral segmentation framework based on the piecewise-constant Mumford-Shah functional. Our framework turns a strictly proper scoring rule and a family of parametric probability distributions into an indicator function for the Mumford-Shah functional, and we naturally generalize some previous Mumford-Shah based models for image segmentation this way. Our focus will be on heavy-tailed elliptical distributions, the negative log-likelihood score and the score based on the beta-divergence to deal with the heavy outliers in hyperspectral imaging. Using the interplay between the calculus of variations and statistics, we also propose a robustness step to estimate the statistical parameters of these heavy-tailed elliptical distributions. In addition, an edge bias, which contains prior boundary information, is incorporated into both the data fidelity term and the perimeter term of the Mumford-Shah functional. Numerical experiments show that, to the best of our knowledge, our method achieves the highest overall accuracy among fully unsupervised methods on three of the four standard benchmarks (Indian Pines, Kennedy Space Center, Pavia University and Salinas) evaluated on the full scenes. It also improves upon our previous Mumford-Shah based model.

Figures & tables

Name	Density	Range for $\tau$
Gaussian	$\frac{1}{(2\pi)^{\frac{N}{2}}}\det\Sigma^{-\frac{1}{2}}e^{-\frac{1}{2}\\|y-\mu\\|^{2}_{\Sigma}}$	–
Student- $t$	$\frac{\Gamma(\frac{N+\tau}{2})}{\Gamma(\frac{\tau}{2})(\pi\tau)^{\frac{N}{2}}}\det\Sigma^{-\frac{1}{2}}\bigg(1+\frac{1}{\tau}\\|y-\mu\\|^{2}_{\Sigma}\bigg)^{-\frac{\tau+N}{2}}$	$(1,\infty)$
Power exponential	$\frac{\tau\Gamma(\frac{N}{2})}{\pi^{\frac{N}{2}}\Gamma(\frac{N}{2\tau})2^{\frac{N}{2\tau}}}\det\Sigma^{-\frac{1}{2}}e^{-\frac{1}{2}\\|y-\mu\\|^{2\tau}_{\Sigma}}$	$(0,1)$
Log-normal	$\frac{1}{(2\pi)^{\frac{N}{2}}}\det\Sigma^{-\frac{1}{2}}\frac{1}{\prod_{n=1}^{N}y_{n}}e^{-\frac{1}{2}\\|\ln y-\mu\\|^{2}_{\Sigma}}$	–

Table I: Probability densities

Distribution	$\psi$	Indicator function
Gaussian	$\frac{t^{\beta+1}}{\beta+1}$	$\begin{array}[]{l}(2\pi)^{-\frac{\beta N}{2}}\det\Sigma^{-\frac{\beta}{2}}\\ \bigg(-e^{-\frac{\beta}{2}\\|y-\mu\\|_{\Sigma}^{2}}+\beta(1+\beta)^{-\frac{N+2}{2}}\bigg)\end{array}$
Gaussian	$t\ln t$	$\frac{N}{2}\ln(2\pi)+\frac{1}{2}\ln\det\Sigma+\frac{1}{2}\\|y-\mu\\|_{\Sigma}^{2}$
Student- $t$	$t\ln t$	$\begin{array}[]{l}\ln\Gamma(\frac{\tau}{2})-\ln\Gamma(\frac{\tau+N}{2})+\frac{N}{2}\ln(\tau\pi)\\ +\frac{1}{2}\ln\det\Sigma+\frac{\tau+N}{2}\ln\big(1+\frac{\\|y-\mu\\|_{\Sigma}^{2}}{\tau}\big)\end{array}$
Power exponential	$t\ln t$	$\begin{array}[]{l}-\ln\tau-\ln\Gamma(\frac{N}{2})+\frac{N}{2\tau}\ln 2+\frac{N}{2}\ln\pi\\ +\ln\Gamma(\frac{N}{2\tau})+\frac{1}{2}\\|y-\mu\\|_{\Sigma}^{2\tau}+\frac{1}{2}\ln\det\Sigma\end{array}$
Log-normal	$t\ln t$	$\begin{array}[]{l}\frac{N}{2}\ln(2\pi)+\frac{1}{2}\ln\det\Sigma\\ +\langle\ln y,1\rangle+\frac{1}{2}\\|\ln y-\mu\\|_{\Sigma}^{2}\end{array}$

Table II: Indicator functions

Figure 1: Histogram for the true 5th segment of a hyperspectral dataset (Indian Pines) introduced in Section III-A .

Figure 2: Visual comparison of the Di Zenzo weight (left) and graph metric weight (right) on the Indian Pines dataset introduced in Section III-A .

Figure 3: Flowchart of our segmentation framework.

Step	Complexity	Additional quantities	Typical calls per full segmentation
Preprocessing & MNF	$\mathcal{O}(L^{3}+L^{2}HW)$	–	$1$
Computing the metric	$\mathcal{O}(HWN)$	–	$1$
$\mathcal{R}$ -step	$\mathcal{O}(m_{\text{FP}}kHWN^{2})$	Number of fixed-point iterations $m_{\text{FP}}$	$20$
PDHG	$\mathcal{O}(m_{\text{PDHG}}HWk\ln k)$	Number of PDHG iterations $m_{\text{PDHG}}$	$20$

Table III: Computational complexity of each step of Algorithm 2

Name	Dimensions $(H,W,L)$	Sensor	Classes in GT
Indian Pines	$(145,145,200)$	AVIRIS	$17$
Kennedy Space Center	$(512,614,176)$	AVIRIS	$13$
Pavia University	(610, 340, 103)	ROSIS	$9$
Salinas	$(512,217,204)$	AVIRIS	$17$

Table IV: HSI dataset overview

Image	$N$	$Q_{\Theta}$	$\kappa$	$q^{*}$	$\lambda$	$\mathcal{R}$ -step	Branching
Indian Pines	8	PE	$2.5$	$87.5\%$	0.1604	MCD	✓
Kennedy Space Center	5	PE	$1$	$70\%$	0.1285	SMD	✓
Pavia University	$5$	ST	$2.5$	$65\%$	0.0753	MCD	✓
Salinas	$7$	PE	1	$85\%$	1.6468	–	✓

Table V: Overview of the parameters (NLL)

Image	$N$	$Q_{\Theta}$	$\kappa$	$q^{*}$	$\lambda$	$\mathcal{R}$ -step	Branching
Indian Pines	8	$\mathcal{N}(\mu,\Sigma)$	$2.5$	$87.5\%$	0.1810	–	–
Kennedy Space Center	5	$\mathcal{N}(\mu,\Sigma)$	$1$	$70\%$	0.0037	–	–
Pavia University	$5$	$\mathcal{N}(\mu,\Sigma)$	$2.5$	$65\%$	0.0662	–	–
Salinas	$7$	$\mathcal{N}(\mu,\Sigma)$	1	$85\%$	0.02552	–	–

Table VI: Overview of the parameters (

\beta

-score)

	Indian Pines	Kennedy Space Center	Pavia University	Salinas
$k$ -means++	$50.2\pm 0.0\%$	$65.5\pm 0.0\%$	$51.2\pm 0.1\%$	$73.1\pm 0.0\%$
SSC	$42.9\%$	–	$\mathbf{68.2\%}$	$81.8\%$
NLTV	$42.4\%$	$41.5\%$	$44.0\%$	–
$\text{S}^{2}$ DL	$64.7\%$	–	–	$87.4\%$
AMS	$67.1\pm 3.4\%$	$66.9\pm 0.0\%$	$56.7\pm 1.7\%$	$82.8\pm 7.4\%$
Ours (NLL)	$\mathbf{70.9\pm 0.7\%}$	$\mathbf{67.5\pm 0.0\%}$	$62.8\pm 0.0\%$	$\mathbf{89.8\pm 0.5\%}$

Table VII: OA score of different segmentations

	Indian Pines	Kennedy Space Center	Pavia University	Salinas
$k$ -means++	$46.7\pm 0.4\%$	$44.4\pm 0.0\%$	$58.4\pm 0.1\%$	$65.1\pm 0.0\%$
$\text{S}^{2}$ DL	$59.1\%$	–	–	$72.9\%$
AMS	$60.7\pm 3.0\%$	$45.5\pm 0.1\%$	$61.6\pm 0.7\%$	$73.0\pm 4.5\%$
Ours (NLL)	$\mathbf{63.1\pm 0.7\%}$	$\mathbf{45.9\pm 0.0\%}$	$\mathbf{66.4\pm 0.2\%}$	$\mathbf{74.6\pm 0.6\%}$
Ours ( $\beta$ -score)	$57.5\pm 0.4\%$	$\mathbf{45.9\pm 0.0\%}$	$60.3\pm 0.0\%$	$68.6\pm 0.0\%$

Table VIII: AA score of different segmentations

Panel of Figure 4 — Figure 4: Ground truth and segmentations of IP for a fixed seed.

Panel of Figure 5 — Figure 5: Ground truth and segmentations of KSC for a fixed seed.

Panel of Figure 6 — Figure 6: Ground truth and segmentations of PU for a fixed seed.

Panel of Figure 7 — Figure 7: Ground truth and segmentations of Salinas for a fixed seed.

Robust Statistics

See all 17 papers

Mar 31, 2026

Polyhedral Unmixing: Bridging Semantic Segmentation with Hyperspectral Unmixing via Polyhedral-Cone Partitioning

Semantic segmentation and hyperspectral unmixing are two central problems in spectral image analysis. The former assigns each pixel a discrete label corresponding to its material class, whereas the latter estimates pure material spectra, called endmembers, and, for each pixel, a vector representing material abundances in the observed scene. Despite their complementarity, these two problems are usually addressed independently. This paper aims to bridge these two lines of work by formally showing that, under the linear mixing model, pixel classification by dominant materials induces polyhedral-cone regions in the spectral space. We leverage this fundamental property to propose a direct segmentation-to-unmixing pipeline that performs blind hyperspectral unmixing from any semantic segmentation by constructing a polyhedral-cone partition of the space that best fits the labeled pixels. Signed distances from pixels to the estimated regions are then computed, linearly transformed via a change of basis in the distance space, and projected onto the probability simplex, yielding an initial abundance estimate. This estimate is used to extract endmembers and recover final abundances via matrix pseudo-inversion. Because the segmentation method can be freely chosen, the user gains explicit control over the unmixing process, while the rest of the pipeline remains essentially deterministic and lightweight. Beyond improving interpretability, experiments on three real datasets demonstrate the effectiveness of the proposed approach when associated with appropriate clustering algorithms, and show consistent improvements over recent deep and non-deep state-of-the-art methods. The code is available at: https://github.com/antoine-bottenmuller/polyhedral-unmixing

Antoine Bottenmuller, Etienne Decencière, Petr Dokládal

CMM, PSL, STIM · Mines Paris, PSL University, Centre for Mathematical Morphology (CMM), 77300 Fontainebleau, France

Hyperspectral Unmixing Spectral Unmixing Hyperspectral Imaging

Ask PDF

Sep 30, 2026

A PyTorch Library for Hyperspectral Image Models: Technical Report

Hyperspectral remote sensing has advanced across diverse deep learning paradigms, including spectral spatial CNNs, Vision Transformers, Mamba, graph neural networks, Kolmogorov Arnold networks, and self supervised masked autoencoding. Yet progress remains hindered by fragmented repositories, incompatible tensor conventions, and non standardized evaluation. Hyperspectral Image Models addresses these challenges through a modular framework unifying 55 representative models across six paradigms with a common registry, automatic 4D/5D tensor adaptation, and standardized constructors. It integrates 24 benchmark scenes from Airborne, Spaceborne, UAV, and Mars CRISM sensors, with caching, label remapping, PCA, explicit band selection or raw spectra, optional spatial max pooling, and arbitrary PxP patch extraction. To prevent inflated accuracy from overlapping windows, it supports class balanced random partitioning and spatially disjoint regional blocking with Chebyshev guard bands that eliminate train test pixel overlap. Experiments use a single config with deterministic seeds and complete provenance, generating LaTeX benchmark tables and classification maps. Across 1,320 model scene evaluations and 6,600 seeded runs, scene difficulty dominates architecture, with mean accuracy ranging from 96.40% on Botswana to 56.70% on Houston 2018, versus a 15 point spread across paradigm means. No paradigm universally dominates, while sub 1 M parameter models can match architectures two orders of magnitude larger. Code is publicly available at https://github.com/Tanishq251/Hyperspectral-Image-Models.

Tanishq Rachamalla, Aryan Das, Srishti Kaushik +1

Department of Information Technology Siddhartha Academy of Higher Education Vijayawada, Andhra Pradesh 521108, India · Department of Computer Science and Engineering Vellore Institute of Technology Bhopal, Madhya Pradesh 466114, India · Department of Computer and Information Sciences Indira Gandhi National Open University New Delhi 110068, India +1

Remote Sensing Benchmarks Hyperspectral Image Classification

Ask PDF

Apr 14, 2026

Deep Spatially-Regularized and Superpixel-Based Diffusion Learning for Unsupervised Hyperspectral Image Clustering

An unsupervised framework for hyperspectral image (HSI) clustering is proposed that incorporates masked deep representation learning with diffusion-based clustering, extending the Spatially-Regularized Superpixel-based Diffusion Learning (

S^2DL

) algorithm. Initially, a denoised latent representation of the original HSI is learned via an unsupervised masked autoencoder (UMAE) model with a Vision Transformer backbone. The UMAE takes spatial context and long-range spectral correlations into account and incorporates an efficient pretraining process via masking that utilizes only a small subset of training pixels. In the next stage, the entropy rate superpixel (ERS) algorithm is used to segment the image into superpixels, and a spatially regularized diffusion graph is constructed using Euclidean and diffusion distances within the compressed latent space instead of the HSI space. The proposed algorithm, Deep Spatially-Regularized Superpixel-based Diffusion Learning (

DS^2DL

), leverages more faithful diffusion distances and subsequent diffusion graph construction that better reflect the intrinsic geometry of the underlying data manifold, improving labeling accuracy and clustering quality. Experiments on Botswana and KSC datasets demonstrate the efficacy of

DS^2DL

Vutichart Buranasiri, James M. Murphy

Department of Mathematics, Tufts University, Medford, MA 02155, USA

Superpixel Segmentation Clustering Image Clustering Hyperspectral Imaging

Ask PDF

Abstract

Robust Statistics

Explore similar work

Polyhedral Unmixing: Bridging Semantic Segmentation with Hyperspectral Unmixing via Polyhedral-Cone Partitioning

A PyTorch Library for Hyperspectral Image Models: Technical Report

Deep Spatially-Regularized and Superpixel-Based Diffusion Learning for Unsupervised Hyperspectral Image Clustering