Signal separation problems are common in science. A prominent example of this occurs during the use of diffraction or spectroscopy to identify the individual components of a mixture by measuring it. In the simplest case, the measured signal is a linear combination of basis patterns corresponding to the constituent parts. The unmixing problem is to infer all or some of these basis patterns and abundances of components from measurements of distinct mixtures. One of the core challenges of this task is the variation of the basis from mixture to mixture due to noise and the exact physics of the measurement process. This is usually addressed with tailored model-based and parametric methods that are then limited in use to specific application domains by the nature of the assumptions made. We propose a novel geometric approach to unmixing problems which views the generation of data during measurement through a metric space lens, thereby shifting the focus from parametrised models to a general relationship between basis transformations and the corresponding geometry. We take advantage of the optimal transport distances to capture commonly occurring basis variations, and use minimisation of in-class variance of candidate solutions to drive the optimisation. We pay special attention to the one-dimensional case due to its practical importance and availability of efficient distance and transport map routines. The effectiveness of our approach is demonstrated on a range of unmixing tasks using random Gaussian mixture models, simulated powder X-ray diffraction, and laboratory hyperspectral imaging datasets.
Figures & tables
Figure 1: A mixing model operating in the space of pattern P , incorporating independent variation of basis patterns prior to combination. Left: Basis patterns bi . Middle: The varied basis copies bij that correspond to the mixture cj . Right: The resulting mixture cj . Transitions from left to middle and from middle to right are provided through the variation ϕ and combination κ functions respectively (illustrated in red and blue respectively).
Figure 2: A distance between two distributions μ and ν can often be written as a sum of distances over parts of μ and ν that it takes to correspond. Here, DTW is dynamic time warping. Take μ to be an initial pattern (blue) and ν a transformed (orange) counterpart, with corresponding points (red markers) linked by dotted lines. In such a sum, change in one part is measured independently of change in another part, as if a point along μ had moved along its dotted line to become the corresponding point in ν .
Figure 3: While a Monge map seeks a one to one correspondence, with a Kantorovich plan, mass can be broken up and sent to multiple locations. This splitting of mass can be necessary in the discrete setting, as a larger “stick” (Dirac) may need to be broken into parts and contribute towards multiple others for a mass preserving matching.
Figure 4: Overlap in Monge maps T and Kantorovich plans π . The density correponding to bivariate distributions π are represented here through heatmaps, it is only noteworthy here where the density is zero (white) and where it is not. Nonzero density at (x,y) signifies transport from x to y , these (x,y) are in the “support” of the Kantorovich plan, supp(π∗) . While a Kantorovich plan allows for mass splitting if necessary, it is generally more economical to avoid splitting if possible. Therefore if possible supp(π∗)⊆graph(T∗) . Here, the pattern ν was generated by shifting and changing the breadth of the peaks in μ .
Figure 5: Distances after offsetting an original density f (a-c, solid blue), producing shifted counterparts (a-c, dashed) for various shifts s , the markers in panels (d) and (e) are colored according to corresponding values of s . Panels (d) and (e) track a pointwise distance (Euclidean) and an OT distance (Wasserstein) respectively, between the original and a shifted copy, as a function of shift. Note, since a pointwise distance only compares features that align vertically (see Fig. 2 ), we obtain local minima whenever non-corresponding peaks happen to align (such as in the yellow case). Note also the plateau in (d) when no features align (green case), where the gradient is near 0 , and provides little direction in optimisation (even after perturbation).
Figure 6: Geometric views of mixing under three kinds of basis variation (see Fig. 1 ) and linear combination, with the action of the variation ϕ and combination κ functions denoted with red and blue arrows respectively. Dark grey represents possible variations b of basis pattern b , lighter grey - possible combinations, and lightest - the whole space of patterns; the simplex Sd−1={x∈R+d:∥x∥1=1} or an arbitrary pattern space P .
Figure 7: The displacement interpolation ( OT barycenter construction) from three perspectives (one per column). The barycenter captures aspects of multiple patterns by striving to be ‘in-between’ them, attempting to be similar to both. With an appropriate distance, this can combine information in a meaningful way. The barycenter β will be more μ -like or ν -like according to the weighting coefficient λ . Ordinarily a transport map associates some mass at position x in μ to mass at position y in ν (see Fig. 3 ). With this association, we can consider continuously moving this mass from its location within μ to its location within ν , along the shortest path between x and y . With λ=0 , all mass elements are at their source position and constitute μ . With λ=1 , all mass elements are at their target position and constitute ν . Mass elements in intermediate positions, constitute an intermediate pattern β . In latter columns, a segment of one colour in μ corresponds to that of the same color in β and in ν . The blue curve in the Kantorovich column illustrates the Monge map from μ to β .
Figure 8: The segmentation problem as it emerges from the variation cost in the Wasserstein context (a), minimising with respect to multiple coupled plans, and an equivalent, single transport plan/distance evaluation simplification (b). Note, in the former case we inherently discover the abundances a when optimising for unnormalised bj , but in the latter case we must evaluate the distance/plan for each choice of a . However, since we can construct exact plans for the ordinary 1D optimal transport problem easily, now we only have to optimise for a .
Problem
Amount of basis known
Primary goal
Separation (non-blind)
All
For each mixture, determine parts likely arising from each basis
Table 1: Table of variations of the unmixing problem. The anomaly detection relaxation can be optionally made from any of the first three variants, it is treated separately from partially blind unmixing since a sought additional component may be too statistically insignificant to estimate (though may be indirectly detected).
Figure 9: The North-west corner algorithm ( Algorithm 5 ) and the monotonic trace produced, corresponding to monotonic parameterisations i(k) and j(k) , or in the continuous case x(t) and y(t) . As the name suggests, we start in the north-west corner of the transport plan, and at each step k , compare how much mass is left at each marginal index i and j , transfer the most mutually available, and increment the saturated index/indices ( i , j , or both). Note, the horizontal stepping of the Kantorovich plan in the middle panels of Fig. 7 , how a Monge function cannot do this, and how a function pair (x(t),y(t)) can (see Lemma 1 ), as one can linger at the same position for some time, e.g. i(k)=3 for k∈[3,6] .
Figure 10: Quantisation of various continuous measures μ with density ρμ (one per column), into a discrete measure μˉ of various numbers of Diracs Nδ (one per row). The colored region surrounding a dashed stick denotes the continuous area that is condensed into that stick, with taller sticks (larger weight) having absorbed larger areas. With lower quantisation cost (higher Nδ ), we of course have better proxies for estimating the Wasserstein (p=2) distance between different μ s.
Figure 11: Approximation of d(μ,ν) through d(μˉ,νˉ) where μˉ,νˉ are within balls of radius τ (solid black boundary), where d(μˉ,νˉ)→d(μ,ν) as τ→0 by the triangle inequality. For a geodesic distance, the distance d(μ,ν) is the length of the geodesic connecting μ and ν (solid blue line, while dashed connects μˉ and νˉ ).
Figure 12: Average error (evaluated over 1000 random GMM pattern pairs) vs variables required for two quantisation choices. GMMs are generated as in Section 5.1 , their density is initially sampled on a uniform grid of 1024 points. Signatures require twice the amount of variables for each grid size as we are specifying positions as well as weights (plot is adjusted accordingly). The signature quantisation is achieved through hierarchical agglomerative centroid-linkage clustering.
Preset
σΔμ
σp
95% of Δμkij are in
95% of σkij are in
Low
0.015
0.15
[−0.03,+0.03]
[0.81σki,1.23σki]
Medium
0.025
0.3
[−0.05,+0.05]
[0.66σki,1.52σki]
High
0.035
0.5
[−0.07,+0.07]
[0.50σki,2.00σki]
Table 2: Basis variation strength presets used, where two variations of the same peak can be found as far as 14% of the domain width apart.
Figure 13: Visual comparison of decompositions of one randomly generated GMM dataset (high variation preset), according to each tested method. Signatures are visualised by smoothing with a Gaussian kernel with σKDE=0.004 . Note that the mixture in the first column of (b), has a component of the first basis (blue) in which some peaks are found further to the left and others are found further to the right, when compared to the left column of (a). Also, note that the mixture in the third column of (b), has a component with a different set of non-uniform shifts. Therefore, for a decomposition method to be able to explain both of these mixtures with respect to the same basis, it must have the ability to perturb each peak independently.
Metric
Variability
Approach
AADPA (A,At)
RMSE (A,At)
RMSE (B,Bt)
IBCW (B,Bt)
×100
×1000
×100
NMF
5.783±2.469
7.909±3.421
1.742±0.569
1.294±0.602
Low
oNMF
8.300±2.921
10.882±3.722
4.611±1.454
3.559±1.507
Our approach
4.274±1.973
5.544±2.757
1.668±0.485
1.756±0.894
NMF
10.247±3.800
14.142±5.120
3.130±0.926
2.262±1.056
Table 3: Mean decomposition metrics over 400 random GMM datasets per variability preset. These were generated with K=3 and M=800 , where the density of each cj is initially sampled uniformly with N=256 then discretised with Nδ=64 . The best of eight decompositions with different initialisation (as measured by final cost), is taken for each dataset/method. The ground truth basis ( Bt ) is also similarly sampled (producing Bt ), as is the kde density of the inferred basis signature for basis RMSEs . The oNMF is performed with a maximum shift of 25 gridpoints and the inferred basis patterns are centered by shifting for 13 gridpoints before measuring RMSE (B,Bt) .
Figure 14: Visual comparison of basis separation (non-blind unmixing) according to the tested methods (one per row), of a Synthetic instance generator dataset with K=6 , m=M=1275 , Nδ=64 . To visualise signatures, σKDE=0.188 has been used, x domain ( 2θ ) has been truncated to [16,45] in these plots.
Metric
Approach
AADPA (A,At)
RMSE (A,At)
NMF
10.467
0.203
oNMF
6.825
0.118
Our approach
3.115
0.055
Table 4: Abundance derivation metrics for the compared approaches (in the separation-only/fixed-basis regime), over the synthetic instance generator dataset.
Figure 15: Visual comparison of decompositions of scene 1. To visualise signatures, σKDE=7.53 has been used.
Metric
Scene
Approach
AADPA (A,At)
RMSE (A,At)
RMSE (B,Bt)
IBCW (B,Bt)
×10
×1000
1
NMF
4.561
0.549
2.072
45.269
Our approach
4.437
0.595
1.947
41.683
2
NMF
11.697
1.486
1.830
35.212
Our approach
9.787
1.384
1.648
40.198
Table 5: Performance summary of NMF vs our method (in the fully blind regime), on a HI dataset. The best of 16 decompositions with different initial starts (as measured by final cost), is taken for each scene/method. oNMF has been omitted in this instance due to instability.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 16: Left: Simplex S2 , where weights on a static grid such as x=(0,0.5,1) can be represented. The repeated application of the shift-down operation (initially on e1 ) is illustrated. Right: The Dirac measures δx∈P(R) representing distributions with weights concentrated at a single point x , which we can continuously interpolate between.
This paper proposes a semisupervised geometric unmixing approach called minimum simplex semisupervised unmixing (MiSiSUn). The geometry of the data was incorporated for the first time into library-based unmixing using a simplex-volume-flavored penalty based on an archetypal analysis-type linear model. The experimental results were performed on two simulated datasets considering different levels of mixing ratios and spatial instruction at varying input noise. MiSiSUn considerably outperforms state-of-the-art semisupervised unmixing methods. The improvements vary from 1 dB to over 3 dB in different scenarios. The proposed method was also applied to a real dataset where visual interpretation is close to the geological map. MiSiSUn was implemented using PyTorch, which is open-source and available at https://github.com/BehnoodRasti/MiSiSUn. Moreover, we provide a dedicated Python package for Semisupervised Unmixing, which is open-source and includes all the methods used in the experiments for the sake of reproducibility.
Behnood Rasti, Bikram Koirala, Paul Scheunders
Faculty of Electrical Engineering and Computer Science, Technische Universität Berlin, 10623 Berlin, Germany · Berlin Institute for the Foundations of Learning and Data (BIFOLD), 10623 Berlin, Germany · Imec-VisionLab, Department of Physics, University of Antwerp, Belgium
Several foundation models dedicated to hyperspectral images have recently been made available. These models are trained on large unlabeled datasets and exhibit strong performance on many hyperspectral imaging tasks, such as classification or denoising. Nonetheless, their performance for hyperspectral unmixing -- the task of separating mixed spectra of overlapping materials in a hyperspectral image -- remain understudied. This might partly be due to the fact that most of them rely on vision transformer backbones, including patchification, leading to a feature resolution problem. While hyperspectral unmixing already arises from the low resolution of hyperspectral images, this patchification step potentially makes the problem even more ill-posed. Therefore, in this work, we aim to answer two questions: 1) how do foundation models perform in hyperspectral unmixing?; 2) how to tackle the feature-level loss of resolution? To answer the first question, we benchmark foundation models for unmixing, showing that they can reach state-of-the-art performance on four hyperspectral unmixing datasets. To answer the second question, we compare several feature upsampling approaches and empirically show that using a simple one can lead to high performance results. The code is available at https://gitlab.telecom-paris.fr/ring/hfm-hsu.git.
Edgard Dabier, Christophe Kervazo, Pietro Gori +1
LTCI, T´el´ecom Paris, Institut Polytechnique de Paris, Palaiseau, France
The Linear Mixing Model (LMM) dominates spectral unmixing for its simplicity, but fails under multiple scattering; existing nonlinear models compensate by applying a fixed regime uniformly across entire scenes. We propose Physics-Guided Regime Unmixing (PGRU), which estimates a pixel-wise scalar ξi∈[0,1] from observable physical features to activate nonlinear mixing only where justified. Residuals from the Generalized Bilinear Model (GBM), the Post-Nonlinear Mixing Model (PPNM), and Hapke are combined via learned attention, yielding interpretable regime maps. Experiments on Samson, Jasper Ridge, and Urban show consistent improvements over baselines, with physical coherence ρ>0.90.
Paula Pacheco, Pablo Granitto, Juan B. Cabral
GVT-CONAE, Centro Espacial Teófilo Tabanera, Argentina · CONICET, Argentina · FAMAF-UNC, Córdoba, Argentina +1