HyperLoop: Hyperspectral-Assisted Loop Closure Detection in 3D-SLAM for Planetary Exploration

Annie Bhalla; Dennis Rotondi; Riccardo Giubilato; Kai O. Arras; Wolfgang Stuerzl

cs.CVOct 14, 2026

HyperLoop: Hyperspectral-Assisted Loop Closure Detection in 3D-SLAM for Planetary Exploration

Authors: Annie Bhalla, Dennis Rotondi, Riccardo Giubilato, Kai O. Arras, Wolfgang Stuerzl

Organizations: Institute for Artificial Intelligence (AI), University of Stuttgart, Universitätsstraße 32, 70569 Stuttgart, Germany. · German Aerospace Center (DLR), Institute of Robotics and Mechatronics, 82234 Oberpfaffenhofen, Germany.

Abstract

Autonomous navigation of planetary rovers is challenged by visually repetitive, featureless terrain. SLAM systems based on 3D Gaussian Splatting (3DGS) offer dense mapping but rely on RGB appearance descriptors that degrade under perceptual aliasing - a common failure mode in extraterrestrial environments where physically distinct locations appear visually identical. This paper presents HyperLoop, the first framework integrating hyperspectral imaging (HSI) into the loop closure module of a 3DGS-based SLAM system. Hyperspectral descriptors encode material-level spectral signatures complementing RGB features, providing robustness to illumination changes and visual repetitiveness. Two HSI descriptor families are introduced: Bag-of-Spectral-Words (BoSW), encoding spectral material distributions via TF-IDF weighted histograms, and PCA-K VLAD (PCAK-VLAD), capturing first-order residual statistics of PCA-compressed spectral signatures. Both are evaluated with SIFT-VLAD and NetVLAD RGB descriptors under Strict-AND and Weighted fusion strategies on a custom dataset from the Moon-Mars Outdoor Test Site (MMOTS) at DLR, comprising synchronized RGB-D and hyperspectral sequences across four planetary analog traverses (65.5 m total). A key finding is that the sensor's Red-NIR range (600-860 nm) provides limited discriminability for rock terrain: the best HSI fusion (SIFT-VLAD + BoSW, Strict-AND) achieves 5.04 cm ATE RMSE, within 1.8% of SIFT-VLAD alone (4.95 cm). The gain over NetVLAD (8.81 cm) is attributable to SIFT-VLAD, not HSI fusion. This motivates future adoption of SWIR sensors capturing diagnostic mineral absorption features absent from the 600-860 nm range for planetary SLAM missions.

Figures & tables

Figure 1: Overview of the HyperLoop framework. RGB and HSI streams are processed independently to yield complementary descriptors, which are combined via decision-level fusion before geometric verification and Pose Graph Optimization.

Figure 2: The SIFT-VLAD pipeline. SIFT keypoints provide 128-dimensional local descriptors; K-Means clustering builds a visual codebook; VLAD aggregates residuals per cluster into a global descriptor invariant to scale, rotation, and illumination.

Figure 3: HSI descriptor extraction pipelines. (a) BoSW constructs a TF-IDF weighted spectral word histogram; (b) PCAK-VLAD aggregates VLAD residuals in PCA-reduced spectral space. Both share the same preprocessing, PCA, and K-Means vocabulary.

Figure 4: Cumulative explained variance of the fitted PCA model on the spectral training set. Six principal components capture $\approx$ 95% of variance, justifying the $\mathbb{R}^{15}\!\to\!\mathbb{R}^{6}$ dimensionality reduction. The steep drop after PC 1 (80% variance) reflects the dominant residual spectral shape mode; after per-pixel intensity normalisation (Eq. 3 ), PC 1 captures the largest source of remaining spectral variation across the training set, not raw reflectance intensity, which has already been divided out.

Figure 5: Field-of-view mismatch between the RGB camera (84 ∘ $\times$ 84 ∘ , full frame) and the Ximea IMEC hyperspectral sensor (40 ∘ $\times$ 20 ∘ , white rectangle). The HSI sensor covers only a central sub-region of the RGB frame, motivating decision-level fusion rather than pixel-level registration.

Figure 6: Synchronised multi-modal data sample from MMOTS-WS. (a) RGB frame; (b) Depth frame; (c) HSI false-colour frame. Note the HSI frame covers a smaller field of view than the RGB pair.

Property	Value
Spectral bands	15 channels
Spectral range	600–860 nm (Red-NIR)
Sensor resolution	2048 $\times$ 1088 px (native)
Cube resolution	1020 $\times$ 540 $\times$ 15 (after mosaic de-interleaving)
Frame rate	up to 170 FPS (raw); operated at 8 Hz for HSI cubes
Pixel size	5.5 $\mu$ m

Table 1: Ximea IMEC Hyperspectral Camera Specifications

Figure 7: Ground-truth trajectory of the MMOTS-WS dataset. The loop closure is enforced by starting and ending at the same spatial location.

Dataset	RGB	HSI	Dur.(s)	Dist.(m)
MMOTS-WS	1072	1072	133.9	8
MMOTS-LL	1385	1385	172.9	15
MMOTS-RL	945	945	119.1	18
MMOTS-US	1540	1540	186.0	24.5
SS-SWEDEN	2100	2100	262.2	123.3
Total	7042	7042	874.1	188.8

Table 2: Overview of Collected Datasets

ID	RGB	HSI	Fusion
E1	NetVLAD (default)	–	–
E2	NetVLAD (tuned)	–	–
E3	SIFT-VLAD K=8	–	–
E4-A	NetVLAD	BoSW	Strict-AND
E4-B	SIFT-VLAD	BoSW	Strict-AND
E4-C	NetVLAD	PCAK-VLAD	Strict-AND

Table 3: Experiment Configuration Matrix

NV = NetVLAD, SV = SIFT-VLAD
Descriptor	Pos. $\,\mu$	Pos. $\,\sigma$	Neg. $\,\mu$	Neg. $\,\sigma$
NetVLAD	0.662	0.137	0.247	0.137
SIFT-VLAD K=8	0.729	0.105	0.404	0.221
SIFT-VLAD K=16	0.581	0.125	0.224	0.139
SIFT-VLAD K=32	0.446	0.123	0.144	0.105
SIFT-VLAD K=64	0.347	0.110	0.087	0.090

Table 4: Cosine Similarity Statistics for RGB Descriptors (MMOTS-WS)

Figure 8: Cosine similarity distributions (positive and negative pairs) for SIFT-VLAD K=8 and NetVLAD on the full MMOTS-WS dataset.

Panel of Figure 9 — Figure 9: Top: IDF weights per spectral word (K=8); spectral word 6 receives the highest IDF weight ( $\approx$ 1.6), identifying a rare material. Bottom: Raw mean spectra of all MMOTS-WS HSI keyframe cubes. Curves are largely similar in shape, differing mainly in intensity, indicating limited material discriminability in the 600–860 nm range.

Figure 10: Aligned ATE RMSE for tuned experiments E2–E5-D on MMOTS-WS. E3 (SIFT-VLAD, RGB-only) achieves the best overall result (4.95 cm); E4-B (SIFT-VLAD + BoSW, Strict-AND) achieves the best fusion result (5.04 cm), within 1.8% of E3. The gap to E2 (8.81 cm) is attributable to the RGB descriptor change (NetVLAD $\to$ SIFT-VLAD), not to HSI fusion. NV: NetVLAD; SV: SIFT-VLAD; PV: PCAK-VLAD; AND: Strict-AND; Wt.: Weighted.

Figure 11: Precision, Recall, and F1-score across all experiments, computed before geometric verification (descriptor-level retrieval only). E4-B achieves the highest precision ( $\approx$ 0.24); E5-C achieves the best F1-score balance ( $\approx$ 0.11). NV: NetVLAD; SV: SIFT-VLAD; PV: PCAK-VLAD; AND: Strict-AND; Wt.: Weighted.

Loop Closure Detection

Loop closure detection recognizes previously visited places in robot maps to correct accumulated localization drift and maintain global consistency.

See all 22 papers

Visual SLAM with DEM Anchoring for Lunar Surface Navigation
Mar 18, 2026Adam Dai, Guillem Casadesus Vila, Grace GaoSimultaneous Localization and Mapping Visual SLAM
Stanford University William F. Durand Building 496 Lomita Mall Stanford, CA 94305
Evaluation of Front-End Feature Matching for Lunar Visual SLAM
Oct 22, 2026Guillem Casadesus Vila, Grace GaoPlanetary Rover Robotics Local Feature Matching
Department of Aeronautics and Astronautics, Stanford University, Stanford, CA, United States
RawSLAM: Online HDR Gaussian SLAM from Linear Radiance
Sep 17, 2026Marina Orozco González, Luis Merino3D Gaussian Splatting 3DGS SLAM

Mar 18, 2026J/K move · Enter open · S save

Visual SLAM with DEM Anchoring for Lunar Surface Navigation

Adam Dai, Guillem Casadesus Vila, Grace Gao

Stanford University William F. Durand Building 496 Lomita Mall Stanford, CA 94305

Simultaneous Localization and Mapping Visual SLAM Robot Navigation Map-Based Localization

Open paper AskPDF arXiv

Future lunar missions will require autonomous rovers capable of traversing tens of kilometers across challenging terrain while maintaining accurate localization and producing globally consistent maps. However, the absence of global positioning systems, extreme illumination, and low-texture regolith make long-range navigation on the Moon particularly difficult, as visual-inertial odometry pipelines accumulate drift over extended traverses. To address this challenge, we present a stereo visual simultaneous localization and mapping (SLAM) system that integrates learned feature detection and matching with global constraints from digital elevation models (DEMs). Our front-end employs learning-based feature extraction and matching to achieve robustness to illumination extremes and repetitive terrain, while the back-end incorporates DEM-derived height and surface-normal factors into a pose graph, providing absolute surface constraints that mitigate long-term drift. We validate our approach using both simulated lunar traverse data generated in Unreal Engine and real Moon/Mars analog data collected from Mt. Etna. Results demonstrate that DEM anchoring consistently reduces absolute trajectory error compared to baseline SLAM methods, lowering drift in long-range navigation even in repetitive or visually aliased terrain.

Abstract

Loop Closure Detection

Explore similar work

Visual SLAM with DEM Anchoring for Lunar Surface Navigation

Evaluation of Front-End Feature Matching for Lunar Visual SLAM

RawSLAM: Online HDR Gaussian SLAM from Linear Radiance