cs.CVSep 30, 2026
SaveHyperspectral Image Models: Technical Report
Organizations: Department of Information Technology Siddhartha Academy of Higher Education Vijayawada, Andhra Pradesh 521108, India · Department of Computer Science and Engineering Vellore Institute of Technology Bhopal, Madhya Pradesh 466114, India · Department of Computer and Information Sciences Indira Gandhi National Open University New Delhi 110068, India · Department of Computer Science and Engineering Tezpur University Tezpur, Assam 784028, India
Abstract
Hyperspectral remote sensing has advanced across diverse deep learning paradigms, including spectral spatial CNNs, Vision Transformers, Mamba, graph neural networks, Kolmogorov Arnold networks, and self supervised masked autoencoding. Yet progress remains hindered by fragmented repositories, incompatible tensor conventions, and non standardized evaluation. Hyperspectral Image Models addresses these challenges through a modular framework unifying 55 representative models across six paradigms with a common registry, automatic 4D/5D tensor adaptation, and standardized constructors. It integrates 24 benchmark scenes from Airborne, Spaceborne, UAV, and Mars CRISM sensors, with caching, label remapping, PCA, explicit band selection or raw spectra, optional spatial max pooling, and arbitrary PxP patch extraction. To prevent inflated accuracy from overlapping windows, it supports class balanced random partitioning and spatially disjoint regional blocking with Chebyshev guard bands that eliminate train test pixel overlap. Experiments use a single config.yaml with deterministic seeds and complete provenance, generating LaTeX benchmark tables and classification maps. Across 1,320 model scene evaluations and 6,600 seeded runs, scene difficulty dominates architecture, with mean accuracy ranging from 96.40% on Botswana to 56.70% on Houston 2018, versus a 15 point spread across paradigm means. No paradigm universally dominates, while sub 1 M parameter models can match architectures two orders of magnitude larger. Code is publicly available at https://github.com/Tanishq251/Hyperspectral-Image-Models.
Figures & tables
Figure 1: Conceptual landscape of hyperspectral image (HSI) classification across deep learning paradigms and Earth/planetary observation platforms. Left: High-dimensional hyperspectral cubes spanning continuous spectral wavelengths, from which spatial patches ( ) and spectral profiles are extracted. Center: Six major architectural paradigms evaluated within our framework, namely spectral–spatial 3D-CNNs, Vision Transformers (ViT), selective state-space models (Mamba/SSM), graph convolutional networks (GCN), Kolmogorov–Arnold networks (KAN), and self-supervised masked autoencoders (SSL). Right: Multiplatform operational modalities (Airborne, Spaceborne, UAV, and Mars planetary exploration) mapped to pixel-wise semantic land-cover classifications.
| Framework | Year | Models | Parad. | Scenes (Platform) | Key Architectural Coverage | Spatial Split | Auto-Download |
|---|---|---|---|---|---|---|---|
| DeepHyperX [ 76 ] | 2019 | 13 | 2 | 5 (Airborne) | Classical (SVM), 1D/2D/3D CNN | Fold-based | Script-based |
| HyperSTAR [ 80 ] | 2020 | 5 | 2 | 3 (Airborne) | Classical (SVM), 1D/2D/3D CNN | Predefined Disjoint | Manual |
| TorchGeo [ 77 ] | 2022 | Generic | Vision | Many (General EO) | Generic backbones (ResNet, ViT) | Grid-based | API-based |
| HyTAS [ 78 ] | 2024 | 12 | 1 | 5 (Airborne) | Vision & Spectral Transformers | Random only | Manual |
| Hyperspectral-Image-Models (ours) | 2026 | 55 | 6 | 24 (4 platforms) | CNN, Transformer, Mamba/SSM, GCN, KAN, Self-Supervised | Disjoint & Random | Hugging Face Hub |
Table 1: Comparison of the Hyperspectral-Image-Models framework against representative publicly available toolboxes and benchmarking frameworks in hyperspectral imaging and geospatial learning. Counts and capabilities reflect published literature and public releases. The comparison highlights scope, architectural coverage, and reproducibility infrastructure.
| Scene | Size ( ) | Bands | Cls | Labelled px | Min | Max | Imb. | Train % | Sensor |
| Airborne (14) | |||||||||
| Augsburg | 332 485 | 180 | 7 | 78,294 | 575 | 30,329 | 53 | 0.27 | DAS Specim |
| Berlin | 1723 476 | 244 | 8 | 464,671 | 6,672 | 268,642 | 40 | 0.05 | HyMap |
| Chikusei | 2517 2335 | 128 | 19 | 77,592 | – | – | – | 0.73 | Headwall Photonics |
| Dioni | 250 1376 | 176 | 12 | 20,024 | 150 | 6,374 | 42 | 1.80 | AVIRIS-NG |
| Houston 2013 | 349 1905 | 144 | 15 | 15,029 | 325 | 1,268 | 4 | 2.99 | ITRES CASI-1500 |
Table 2: The 24 scenes of the collection, grouped by acquisition platform. Dimensions, band and class counts, labelled-pixel counts and per-class sample counts are read from config/dataset.yaml ; class names are listed in Table 9 . Min and Max are the smallest and largest class of the scene and Imb. their ratio. Train % is the share of the annotation consumed by the protocol’s fixed budget of 30 training pixels per class, which varies by more than two orders of magnitude across the collection and is what makes a nominally identical protocol a different experiment from one scene to the next. Chikusei’s per-class breakdown is not recorded in the registry. † Indian Pines has a class of only 20 labelled pixels, so the standard 30/10 budget cannot be drawn from it; it is evaluated at 10/5 and its Train % is computed accordingly ( Section 3.5 ).
Figure 2: Ground-truth label maps for all 24 scenes in the collection, rendered directly from the dataset registry with each scene’s class palette and legend colours. Panels are shown at their native aspect ratio, which is why the classical airborne strips (Botswana, Houston 2013, Houston 2018, Dioni) appear tall and narrow next to the wider UAV and Mars CRISM mosaics. The spread of label density, from the sparse point-like annotations of Botswana and Houston 2018 to the fully labelled agricultural blocks of WHU Hi LongKou and Indian Pines, is itself a visual summary of the imbalance ratios in Table 2 .
Figure 3: The Hyperspectral-Image-Models execution path and modular framework architecture. A single declarative configuration file ( config.yaml ) controls dataset ingestion across 24 multiplatform scenes (14 Airborne, 1 Spaceborne, 6 UAV, 3 Planetary), spectral preprocessing (raw, PCA, or pooling), spatial patch extraction ( ), splitting strategies (balanced count, ratio, or disjoint blocks), and unified model execution across 55 catalog architectures spanning 6 paradigms. While the showcase evaluation in Section 5 uses a standardized reference setting ( , 30 PCA bands, 30/10 samples, 5 seeds) for fair comparison, the framework provides full user customizability across all pipeline stages, with automated emission of L a T e X tables and thematic classification maps.
| Subsystem | Core Component | Modular Capabilities |
|---|---|---|
| Dataset Layer | DatasetLoader | Unified retrieval and caching across 24 multi-platform scenes (Airborne, Spaceborne, UAV, Mars CRISM); automatic scaling, label remapping, background masking. |
| Preprocessing | HyperspectralDataset | Arbitrary spatial patch extraction ( to ); PCA dimensionality reduction to components, band pooling, explicit selection, or full raw spectra. |
| Split Engine | data_split.py | Class-balanced random sampling (fixed counts or fractional percentage ratios) and spatially disjoint component-level partitioning with a guard band that removes evaluation patches overlapping any training patch. |
| Model Registry | @register_model | Unified model ecosystem hosting 55 architectures across 6 paradigms (CNN, ViT, SSM/Mamba, GCN, KAN, MAE); auto-discovery; dynamic 4D/5D InputShapeWrapper . |
| Training | trainer.py | Shared loop with 6 optimizers (Adam, AdamW, SGD, RMSprop, Adagrad, Adadelta), plateau LR decay, validation early stopping, atomic checkpointing. |
| Evaluation | metrics.py | Strict held-out evaluation: OA, AA, Cohen’s , full confusion matrices, per-class accuracies, and cross-seed aggregation ( ). |
Table 3: The Hyperspectral-Image-Models software architecture and modular capability matrix. The library decouples data ingestion, preprocessing, partitioning, model registration, training, and evaluation into interchangeable, configuration-driven components.
| Setting | Value | Source |
|---|---|---|
| Input | ||
| Patch size | dataset.patch_size | |
| Patch stride | 1 | dataset.stride |
| Dimensionality reduction | PCA | preprocessing.dim_reduction_method |
| Retained components | 30 | preprocessing.num_pca_bands |
| Tensor layout | preprocessing.use_channel_dim | |
Table 4: The standardized unified protocol used to showcase benchmark results across all 55 models and 24 scenes in the Hyperspectral-Image-Models framework. Every value is read from config/config.yaml unless the source column says otherwise, and the configuration file is copied into each run directory for complete provenance. Holding these parameters constant enables fair, commensurable cross-model comparison; researchers can freely adapt any setting (including patch size, spectral band reduction, sampling budgets, spatial disjoint partitioning, and optimizer schedules) to suit their own model compositions and experimental protocols.
| Paradigm | Impl. | Years | Par. (M) | MACs (M) | Core mechanism |
|---|---|---|---|---|---|
| CNN | 10 | 2017–2026 | 0.012–1.34 | 1.7–102 | local weight sharing, 2D/3D spectral-spatial convolution |
| Transformer | 16 | 2021–2026 | 0.003–4.44 | 1.8–615 | multi-head self-attention over spectral or patch tokens |
| Mamba / SSM | 20 | 2024–2026 | 0.052–4.25 | 0.2–345 | input-dependent selective state-space scan, linear in length |
| Graph / GCN | 4 | 2024–2026 | 0.022–0.33 | 1.9–28 | message passing over a superpixel adjacency graph |
| KAN | 2 | 2024–2024 | 0.344–0.43 | 1.2–11 | learnable univariate B-spline functions on network edges |
| Self-supervised | 3 | 2024–2024 | 0.513–34.23 | 16.3–4195 | masked spectral-spatial reconstruction, then fine-tuning |
Table 5: The implemented collection by paradigm. Parameter and operation counts are measured under one fixed probe input and span three and four orders of magnitude respectively; every entry takes a hyperspectral patch as its only input and is trained under the single protocol of Table 4 . Table 8 in Appendix A gives the per-model breakdown with venues, identifiers and costs. The family rows and cost spans are aggregated automatically by the Hyperspectral-Image-Models framework from the same run records and structural profile used to emit Table 6 .
Figure 4: Mean overall accuracy across 24 scenes against publication year, coloured by paradigm, with a linear trend. The trend indicates the magnitude of the year effect relative to within-year variation; it is not intended for extrapolation.
| Indian Pines | Pavia University | WHU-Hi-LongKou | Nili Fossae | |||||
| Model | OA | AA | OA | AA | OA | AA | OA | AA |
| CNN | ||||||||
| SSRN | 84.71 3.9 | 91.99 1.4 | 97.81 0.5 | 98.09 0.4 | 98.36 0.3 | 97.44 0.4 | 96.69 1.0 | 95.29 1.8 |
| DBDA | 80.73 3.9 | 89.34 1.7 | 97.46 0.3 | 97.29 0.5 | 98.05 0.2 | 96.78 0.2 | 96.45 0.5 | 95.76 0.7 |
| DKDMN | 84.82 1.8 | 91.64 1.1 | 95.44 1.2 | 95.82 1.7 | 97.64 0.5 | 95.90 0.8 | 93.73 1.4 | 94.46 0.9 |
| ENL_FCN | 80.73 2.2 | 90.37 1.3 | 94.43 1.7 | 95.32 0.6 | 96.93 0.6 | 95.58 1.4 | 92.22 3.6 | 93.26 2.4 |
Table 6: Per-model overall and average accuracy on four representative scenes under the protocol of Table 4 : two classical airborne benchmarks, one UAV agricultural mosaic and one Mars CRISM scene. Each entry is the mean over 5 seeded runs with its sample standard deviation, and the best value in each column is in bold. Cohen’s is omitted for width and because it correlates with overall accuracy at across the benchmark; all three metrics are recorded for every run. Models are ordered within each family by their summed accuracy over the four scenes. The table is emitted automatically by the Hyperspectral-Image-Models framework from the completed runs of the same repository released with this paper.
Figure 5: Mean overall accuracy against (a) trainable parameters and (b) multiply-accumulate operations (GMACs), both on logarithmic axes and both measured under one fixed probe input. Annotations mark the architectures on the accuracy and cost frontier.
Figure 6: Arranged classification maps generated directly by the framework. (a) WHU Hi HanChuan, a UAV agricultural strip with narrow parcel boundaries. (b) Nili Fossae, a Mars CRISM scene with diffuse mineralogical transitions. Each row displays the ground-truth map alongside representative convolutional ( SACNet ), transformer ( SSFTTNet ), state-space ( SSMamba ), and graph-based ( MCTGCL ) model predictions with comprehensive class palettes and legends.
| Off target (pp) | |||||||
| Scene | Cls | Labelled px | Kept (%) | Split (%) | Mean | Worst | Overlap (%) |
| Airborne (14) | |||||||
| Augsburg | 7 | 78,294 | 94.4 | 49.5/31.9/18.6 | 2.6 | 10.8 | 58.9 |
| Berlin | 8 | 464,671 | 99.7 | 50.7/30.1/19.2 | 1.1 | 4.6 | 31.9 |
| Chikusei | 19 | 77,592 | 100.0 | 47.2/32.8/20.0 | 5.1 | 16.2 | 6.7 |
| Dioni | 12 | 20,024 | 98.9 | 49.8/29.6/20.6 | 1.8 | 6.8 | 27.6 |
Table 7: The disjoint protocol drawn on every scene at the default 50/30/20 targets ( , stride 1, seed 0), with the per-class breakdown in docs/DISJOINT_SPLIT_CLASSES.md . Kept is the share of labelled pixels that can centre an window; the rest lie within five pixels of the border and are never used, under any protocol. Split gives the realised train/validation/test shares of the kept patches, and the next two columns how far the individual classes land from their targets, averaged over classes and at the worst class, in percentage points. Overlap is the share of validation and test patches whose window shares at least one pixel with a training window when patches are assigned by their centre pixel alone; these are exactly the patches the guard band of Eq. 1 removes.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | Yr | Venue | Par. (M) | MACs (M) | Paper | |
| CNN (10 implemented) | ||||||
| SSRN | 2017 | TGRS | 0.136 | 42.00 | 11 | 10.1109/TGRS.2017.2755542 |
| HybridSN | 2019 | GRSL | 0.535 | 16.01 | 11 | 10.1109/LGRS.2019.2918719 |
| pResNet | 2019 | TGRS | 0.510 | 15.87 | 11 | 10.1109/TGRS.2018.2860125 |
| DBDA | 2020 | Remote Sens. | 0.113 | 47.29 | 11 | 10.3390/rs12030582 |
| ENL_FCN | 2020 | TGRS | 0.089 | 11.18 | 11 | 10.1109/TGRS.2020.3014286 |
Table 8: The complete model catalogue, grouped by family and ordered by publication year, transcribed from MODEL_CATALOG in models/registry.py and docs/MODELS.md . Parameters and multiply-accumulate operations are measured by model_info.py under one fixed probe input of shape at 16 classes, where is the patch column; they are structural properties, not performance measurements. Architectural credit for every entry belongs to its original authors.
| Scene | Classes |
|---|---|
| Augsburg | Forest; Residential Area; Industrial Area; Low Plants; Allotment; Commercial Area; Water |
| Berlin | Forest; Residential; Industrial; Low Plants; Soil; Allotment; Commercial; Water |
| Botswana | Water; Hippo grass; Floodplain grasses 1; Floodplain grasses 2; Reeds; Riparian; Firescar; Island interior; Acacia woodlands; Acacia shrublands; Acacia grasslands; Short mopane; Mixed mopane; Exposed soils |
| Chikusei | Water; Bare soil (farmland); Bare soil (park); Bare soil (roadside); Pavement (asphalt); Pavement (brick); Farmland (paddy); Farmland (other); Greenhouse; Grass (park); Grass (roadside); Tree (park); Tree (roadside); Forest; Building (low-rise); Building (high-rise); Building (factory); Power line; Swimming pool |
| Dioni | Dense Urban Fabric; Mineral Extraction Sites; Non Irrigated Arable Land; Fruit Trees; Olive Groves; Coniferous Forest; Dense Sclerophyllous Vegetation; Sparce Sclerophyllous Vegetation; Sparsely Vegetated Areas; Rocks and Sand; Water; Coastal Water |
| Holden | Analcime; Plagioclase; Prehnite; High-Ca Pyroxene; Serpentine; Margarite |
Table 9: Class names for every scene, as recorded in config/dataset.yaml and used by the framework’s dataset-aware visualisation. The unlabelled background class is not listed: the loader excludes it, and the remaining labels are remapped to a contiguous index range.
| Scene | Platform | Mean OA (%) | OA range (min–max) | Highest (model) | |
|---|---|---|---|---|---|
| Botswana | Spaceborne | 96.40 | 11.67 | 37.6–99.8 | Graph ( MS2GCAN ) |
| Pavia Centre | Airborne | 96.31 | 9.35 | 40.0–99.5 | Graph ( MS2GCAN ) |
| WHU-Hi-LongKou | UAV | 95.13 | 7.37 | 51.7–98.8 | Transformer ( MMFormer ) |
| Chikusei | Airborne | 95.03 | 13.48 | 11.8–99.4 | Mamba ( R2Mamba ) |
| Holden | Orbital | 94.82 | 9.29 | 39.5–99.1 | CNN ( SSRN ) |
| Utopia | Orbital | 94.30 | 9.89 | 38.2–98.8 | Mamba ( HyperMamba ) |
Table 10: Difficulty spectrum across all 24 scenes, ordered by the mean overall accuracy attained across the 55 architectures under the protocol of Table 4 . The dispersion column and the observed range give the spread across architectures on each scene, and the final column names the architecture attaining the highest value there.