Multi-task Disentangling Fusion Mamba-CNN for MODIS Spectral-Temporal Unmixing with Endmember Variability Modeling

Motasem Alkayid; Yimin Zhu; Naser El-Sheimy; Lincoln Linlin Xu

cs.CVOct 27, 2026

Multi-task Disentangling Fusion Mamba-CNN for MODIS Spectral-Temporal Unmixing with Endmember Variability Modeling

Authors: Motasem Alkayid, Yimin Zhu, Naser El-Sheimy, Lincoln Linlin Xu

Organizations: Department of Geomatics Engineering, University of Calgary, Canada · Department of Geography, Faculty of Arts, The University of Jordan, Amman, Jordan

Abstract

Spectral-temporal unmixing of MODIS time series is essential for applications such as abundance mapping and spectral library construction, yet it remains challenging due to (i) spectral-temporal information coupling in MODIS data, (ii) intra-class endmember spectral variability and inter-class signature ambiguity, and (iii) the instability of naively fusing multiple expert representations. This paper presents the Multi- task Disentangling Fusion Mamba (MDF-Mamba) to address these limitations. To resolve spectral-temporal coupling, we design a disentangling architecture that pairs per-channel spectral experts with a dedicated bidirectional state-space (Mamba) temporal branch. Convolutional modules each process a single spectral channel, while the Mamba branch encodes the cross-channel seasonal trajectory, forming a multi-branch expert ensemble with soft inter-expert reweighting based on squeeze-and-excitation. For the integration of these spectral and temporal expert outputs, we introduce mHCFusion, adapting the manifold-constrained hyper-connection framework to merge the outputs in a stable and interpretable manner. Endmember variability is addressed by extending the linear mixing model with pixel-wise learnable spectral variability weights that capture intra-class endmember variation. Finally, we design a multi-task training objective that combines supervised abundance estimation with self-supervised spectral reconstruction and endmember regularization terms on the same training data, yielding a highly efficient framework for spectral-temporal unmixing. Experiments on MODIS time series covering the Alberta and Saskatchewan provinces show that MDF-Mamba achieves the best performance on 12 of 14 evaluation metrics across both datasets (7 of 7 on Alberta; 5 of 7 on Saskatchewan), ranking second on the remaining two.

Figures & tables

Type	Model	Abundance		Reconstruction		Classification
		MAE $\downarrow$	RMSE $\downarrow$	MSE $\downarrow$	SAD (rad) $\downarrow$	OA $\uparrow$	AA $\uparrow$	$\boldsymbol{\kappa}$ $\uparrow$
Traditional	FCLSU [ 3 ]	0.0633	0.1530	0.0106	0.2445	0.5256	0.3290	0.4256
	SGSNMF [ 6 ]	0.0703	0.1668	0.0118	0.2512	0.4734	0.3258	0.3750
	SUnSAL [ 5 ]	0.0666	0.1547	0.0104	0.2469	0.5124	0.2984	0.4035
CNN/AE	PGMSU [ 9 ]	0.0329	0.0842	0.0116	0.2344	0.6847	0.4152	0.6159
	SUnCNN [ 10 ]	0.0309	0.0823	0.0141	0.2678	0.6985	0.4308	0.6328

TABLE I: Quantitative performance comparison on the Alberta MODIS dataset. Bold indicates the best result; underline indicates the second best.

\downarrow

indicates lower is better;

\uparrow

indicates higher is better.

Type	Model	Abundance		Reconstruction		Classification
		MAE $\downarrow$	RMSE $\downarrow$	MSE $\downarrow$	SAD (rad) $\downarrow$	OA $\uparrow$	AA $\uparrow$	$\boldsymbol{\kappa}$ $\uparrow$
Traditional	FCLSU [ 3 ]	0.0729	0.2217	0.0126	0.2560	0.5572	0.2729	0.4495
	SGSNMF [ 6 ]	0.0691	0.2481	0.0312	0.3498	0.5229	0.1857	0.3822
	SUnSAL [ 5 ]	0.0807	0.2359	0.0132	0.2533	0.4607	0.2417	0.3476
CNN/AE	PGMSU [ 9 ]	0.0546	0.1748	0.0143	0.2617	0.6986	0.3832	0.6286
	SUnCNN [ 10 ]	0.0509	0.1685	0.0162	0.2836	0.7176	0.3874	0.6493

TABLE II: Quantitative performance comparison on the Saskatchewan MODIS dataset. Bold indicates the best result; underline indicates the second best.

\downarrow

indicates lower is better;

\uparrow

indicates higher is better.

Type	Model	Abundance		Reconstruction		Classification
		MAE $\downarrow$	RMSE $\downarrow$	MSE $\downarrow$	SAD (rad) $\downarrow$	OA $\uparrow$	AA $\uparrow$	$\boldsymbol{\kappa}$ $\uparrow$
Traditional	FCLSU [ 3 ]	0.7291	0.7526	0.0231	0.3375	0.5874	0.2166	0.4315
	SGSNMF [ 6 ]	0.6591	0.7217	0.0317	0.3540	0.6466	0.2887	0.5368
	SUnSAL [ 5 ]	0.7180	0.7464	0.0242	0.3364	0.5901	0.2304	0.4367
CNN/AE	PGMSU [ 9 ]	0.3726	0.4360	0.0145	0.2562	0.8984	0.5979	0.8724
	SUnCNN [ 10 ]	0.3001	0.3793	0.0183	0.2963	0.9134	0.6457	0.8913

TABLE III: Quantitative performance comparison on the Alberta filtered dataset (80% purity). Bold indicates the best result; underline indicates the second best.

\downarrow

indicates lower is better;

\uparrow

indicates higher is better.

Type	Model	Abundance		Reconstruction		Classification
		MAE $\downarrow$	RMSE $\downarrow$	MSE $\downarrow$	SAD (rad) $\downarrow$	OA $\uparrow$	AA $\uparrow$	$\boldsymbol{\kappa}$ $\uparrow$
Traditional	FCLSU [ 3 ]	0.0552	0.2006	0.0270	0.3438	0.6503	0.2004	0.4774
	SGSNMF [ 6 ]	0.0567	0.2113	0.0342	0.3576	0.6303	0.2353	0.4819
	SUnSAL [ 5 ]	0.0548	0.2052	0.0273	0.3429	0.6493	0.2008	0.4754
CNN/AE	PGMSU [ 9 ]	0.0312	0.1176	0.0191	0.2789	0.8539	0.5477	0.8080
	SUnCNN [ 10 ]	0.0296	0.1135	0.0227	0.3066	0.8615	0.5874	0.8175

TABLE IV: Quantitative performance comparison on the Saskatchewan filtered dataset (80% purity). Bold indicates the best result; underline indicates the second best.

\downarrow

indicates lower is better;

\uparrow

indicates higher is better.

Fig. 5: Spectral reconstruction evaluation on the Alberta test set. (a) Patch-mean original versus reconstructed spectra for dominant classes across the 138-band spectral-temporal sequence. (b) Per-feature reconstruction RMSE over the 23-step temporal index. (c) Pixel-level original versus reconstructed reflectance scatter, with overall MSE, RMSE, SAD, and $R^{2}$ annotated. (d) Per-class reconstruction SAD.

Fig. 6: Unified classification comparison on the Alberta dataset. Top two rows: Complete classification maps for all 13 models with a standardized 14-class colormap consistently maintained across the manuscript; white squares on the Ground Truth map mark twelve $60\times 60$ sub-regions (a-l) centered on distinct land cover classes. Bottom grid: Zoomed-in $13\times 12$ comparison of all 13 models within each highlighted region. Regions were selected to represent a diverse set of median-performing and challenging cases.

Fig. 10: Learned temporal endmember profiles and concordance analysis for the

P{=}12

active land cover classes, ordered from top to bottom by lowest RMSE (best prediction) to highest RMSE (worst prediction); the two sub-polar lichen-moss classes are omitted (Section IV-A). Columns 1 - 6: Per-branch temporal profiles (NDVI, EVI, Red, NIR, Blue, MIR) showing

R{=}10

variant spread (shaded), learned mean endmember (solid colored line), GT reference (solid black), and geographic initialization (dashed). Column 7 (Corr.): Per-class scatter of GT vs. learned mean endmember across all

L{=}138

spectral-temporal bands, colored by spectral branch, with the 1:1 identity line (dashed) and annotated Lin’s concordance correlation coefficient (CCC) [ 52 ] , RMSE, and SAD. CCC measures agreement with the 1:1 line and penalizes both offset and scale differences. Classes whose GT reference is nearly flat (Shrubland, Snow/Ice, Urban, Broadleaf Forest, Sub-polar Taiga) produce the highest RMSE and lowest CCC, consistent with poor endmember convergence being data-imposed rather than architectural. Y-axis auto-scaled per row; X-axis spans the full 2010 calendar year (23 composites).

Configuration	MAE $\downarrow$	OA $\uparrow$	$\boldsymbol{\kappa}$ $\uparrow$	Recon-MSE $\downarrow$	Recon-SAD $\downarrow$
Full MDF-Mamba	0.0255	0.7122	0.6575	0.0067	0.1843
$-$ Temporal/phenology branch (C1)	0.0305	0.7036	0.6415	0.0067	0.186
$-$ Learned fusion weighting (C2)	0.0372	0.6759	0.6009	0.0072	0.190
$-$ Endmember variability (C3)	0.0255	0.7122	0.6575	0.0474	0.481

TABLE V: Part I: component lesion study. One module of the single trained MDF-Mamba is disabled at inference (shared weights, zero retraining variance). Bold : full model.

Configuration	MAE $\downarrow$	OA $\uparrow$	$\boldsymbol{\kappa}$ $\uparrow$	Recon-MSE $\downarrow$	Endm-mSAD $\downarrow$	Endm-mRMSE $\downarrow$
Full MDF-Mamba (proposed)	0.0255	0.7122	0.6575	0.0067	0.00017	0.0458
(C3) Endmember variant count $R$ and initialization
$R{=}1$ (no variant bank, standard LMM)	0.0290	0.7012	0.6358	0.0568	0.0260	0.1111
$R{=}10$ , random init (no GT purity)	0.0294	0.7016	0.6372	0.0798	0.0150	0.6537
(C4) Multi-task reconstruction objective
No reconstruction loss ( $-$ Maha, $-$ SAD)	0.0264	0.7101	0.6515	0.0281	0.00231	0.1429

TABLE VI: Part II: design ablations on the Alberta test set. Each row retrains the full MDF-Mamba with one design decision changed; all other settings remain fixed. Bold : proposed configuration (not necessarily the best value in each column).

Seed	MAE $\downarrow$	RMSE $\downarrow$	OA $\uparrow$	$\boldsymbol{\kappa}$ $\uparrow$	Recon-MSE $\downarrow$	Recon-SAD $\downarrow$
42	0.0255	0.0654	0.7122	0.6575	0.0067	0.1843
123	0.0268	0.0681	0.7113	0.6492	0.0064	0.1845
7	0.0267	0.0679	0.7119	0.6499	0.0068	0.1874
Mean $\pm$ Std	$\mathbf{0.0263\pm 0.0006}$	$\mathbf{0.0671\pm 0.0012}$	$\mathbf{0.7118\pm 0.0004}$	$\mathbf{0.6522\pm 0.0038}$	$\mathbf{0.0066\pm 0.0002}$	$\mathbf{0.1854\pm 0.0014}$

TABLE VII: Multi-seed reproducibility on the Alberta test set. All three seeds are evaluated on the full test set under identical conditions. Mean and standard deviation computed over three independent training runs.

Model	Params (M)	GFLOPs
FCLSU	0.00	1.62
SGSNMF	0.00	3.16
SUnSAL	0.00	1.85
DeepTrans-HSU	26.82	1.04
PGMSU	2.15	8.82
SUnCNN	1.58	7.37

TABLE VIII: Computational complexity profile of all 11 baselines compared to MDF-Mamba. Measured on input shape

(1,138,64,64)

. Traditional non-parametric models (FCLSU, SGSNMF, SUnSAL) contain no trainable weights (Params = 0.00) and their GFLOPs are analytically estimated based on standard convergence iteration limits (e.g., ADMM, multiplicative updates). UNMamba GFLOPs are similarly estimated analytically.

Component	Params (M)	GFLOPs	Mem (MB)	patches/s
MDF-Mamba (full)	3.33	14.93	180.9	48.8
CNN Branches ( $\times$ 6)	2.761	11.32	-	-
Temporal Mamba	0.126	2.46	-	-
mHC Fusion	0.072	0.29	-	-
ELMM + Recon Head	0.122	0.00	-	-
Miscellaneous (stem, SE, FiLM)	0.249	0.86	-	-

TABLE IX: Internal computational profile of MDF-Mamba components. Throughput measured as patches per second (batch size 1).

Class	F1	Transfer quality
Cropland	0.934	✓ Excellent - stable agricultural phenology
Water	0.816	✓ Excellent - spectrally invariant
Needleleaf	0.776	✓ Good - consistent boreal canopy
Barren	0.758	✓ Good - low-reflectance signature stable
Grassland	0.688	Moderate
Wetland	0.533	Moderate

TABLE X: Class-level transfer metrics for the Alberta

\rightarrow

Saskatchewan cross-provincial evaluation (OA: 0.7485,

\kappa

: 0.6881, MAE: 0.0492). Classes marked ✓ (F1

>0.70

) transfer consistently; classes marked ✗ (F1

<0.30

) are limited by domain-specific spectral signatures rare or absent in the Alberta training distribution.

Method	OA	AA	$\kappa$	MAE	RMSE	SAD (rad)
DFMamba [ 58 ]	0.6658	0.4122	0.5923	0.0383	0.0939	1.5708
MambaLG [ 59 ]	0.6837	0.4324	0.6142	0.0366	0.0914	1.5708
MambaHSI [ 60 ]	0.6951	0.4442	0.6297	0.0342	0.0869	1.5708
MHSSMamba [ 61 ]	0.5253	0.2340	0.4057	0.0620	0.1260	1.5708
Mamba-in-Mamba [ 62 ]	0.5403	0.2471	0.4283	0.0593	0.1231	1.5708
MDF-Mamba (Ours)	0.7122	0.4721	0.6575	0.0255	0.0654	0.1843

TABLE XI: Quantitative comparison against recent generic Mamba models on the Alberta test set. Without dedicated physics constraints (ELMM), generic state-space architectures fail to achieve stable spectral reconstruction or accurate fractional abundance estimation.

Spectral Unmixing

See all 25 papers

UnMixMamba: Unmixing Guided State Space Subpixel Mapping of Crops from MODIS Time Series
Oct 28, 2026Motasem Alkayid, Yimin Zhu, Naser El-Sheimy +1Agricultural Remote Sensing Spectral Unmixing
Department of Geomatics Engineering, University of Calgary, Canada · Department of Geography, Faculty of Arts, The University of Jordan, Amman, Jordan
Unmixing-Guided Spatial-Spectral Mamba with Clustering Tokens for Hyperspectral Image Classification
Apr 10, 2026Yimin Zhu, Lincoln Linlin XuHyperspectral Unmixing Hyperspectral Image Classification
Department of Geomatics Engineering, University of Calgary, Canada
A Novel Geographic Graph-Regulated Disentangling Mamba Model with Sparse Tokens for Enhanced Tree Species Classification from MODIS Time Series
May 7, 2026Motasem Alkayid, Zhengsen Xu, Naser El-Sheimy +1Remote Sensing Image Understanding Mamba
Department of Geomatics Engineering, University of Calgary, Canada · Department of Geography, Faculty of Arts, The University of Jordan, Amman, Jordan

Oct 28, 2026J/K move · Enter open · S save

UnMixMamba: Unmixing Guided State Space Subpixel Mapping of Crops from MODIS Time Series

Motasem Alkayid, Yimin Zhu, Naser El-Sheimy, Lincoln Linlin Xu

Department of Geomatics Engineering, University of Calgary, Canada · Department of Geography, Faculty of Arts, The University of Jordan, Amman, Jordan

Agricultural Remote Sensing Spectral Unmixing Remote Sensing Image Super-Resolution State Space Models

Open paper AskPDF arXiv

Subpixel mapping (SPM) of crop types from \SI{250}{m} Moderate Resolution Imaging Spectroradiometer (MODIS) time series to field-scale Agriculture and Agri-Food Canada (AAFC) labels supports acreage estimation and agricultural monitoring. Three difficulties remain: (i)spectral-temporal coupling and mixed pixels that break direct coarse-to-fine classifiers; (ii)intra-class endmember variability that a single spectrum per crop cannot capture; and (iii)uniform refinement on the high-resolution (HR) grid, which treats every subpixel the same, so most of the effort goes to easy pixels while errors remain in the minority class. UnMixMamba addresses them as follows. For(i), a temporal-spectral-spatial decoupled (TSSD) factorization estimates simplex fractions and drives AAFC classification from the same mixture field, instead of mapping low-resolution (LR) observations directly to HR logits. For(ii), $\times 8$ abundance super-resolution and a learnable per-crop variant endmember bank reconstruct the MODIS cube at LR and HR under a linear mixing model. For(iii), the predicted crop fractions decide where a fixed Mamba token budget is spent, so HR refinement focuses on the minority class instead of the full grid. On Alberta 2015 under a fixed spatial-block protocol, UnMixMamba reaches OA 0.8397, mIoU 0.6640, AA 0.7837, and Kappa 0.7715, ahead of ST-DRes, ESPCN, FineCrop, AgriFM, and classical SPM baselines. The task is closed-set four-crop mapping inside a known agricultural mask.

Abstract

Spectral Unmixing

Explore similar work

UnMixMamba: Unmixing Guided State Space Subpixel Mapping of Crops from MODIS Time Series

Unmixing-Guided Spatial-Spectral Mamba with Clustering Tokens for Hyperspectral Image Classification

A Novel Geographic Graph-Regulated Disentangling Mamba Model with Sparse Tokens for Enhanced Tree Species Classification from MODIS Time Series