DLRMamba: Distilling Low-Rank Mamba for Edge Multispectral Fusion Object Detection
Authors: Qianqian Zhang, Leon Tabaro, Ahmed M. Abdelmoniem, Junshe An
Organizations: National Space Science Center, Chinese Academy of Sciences, Beijing, 101499, China · School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing, 100049, China · School of Electronic Engineering and Computer Science, Queen Mary University of London, London, E1 4NS, UK · School of Astronomy and Space Science, University of Chinese Academy of Sciences, Beijing, 100049, China
Multispectral fusion object detection is a critical task for edge-based maritime surveillance and remote sensing, demanding both high inference efficiency and robust feature representation for high-resolution inputs. However, current State Space Models (SSMs) like Mamba suffer from significant parameter redundancy in their standard 2D Selective Scan (SS2D) blocks, which hinders deployment on resource-constrained hardware and leads to the loss of fine-grained structural information during conventional compression. To address these challenges, we propose the Low-Rank Two-Dimensional Selective Structured State Space Model (Low-Rank SS2D), which reformulates state transitions via matrix factorization to exploit intrinsic feature sparsity. Furthermore, we introduce a Structure-Aware Distillation strategy that aligns the internal latent state dynamics of the student with a full-rank teacher model to compensate for potential representation degradation. This approach substantially reduces computational complexity and memory footprint while preserving the high-fidelity spatial modeling required for object recognition. Extensive experiments on five benchmark datasets and real-world edge platforms, such as Raspberry Pi 5, demonstrate that our method achieves a superior efficiency-accuracy trade-off, significantly outperforming existing lightweight architectures in practical deployment scenarios.
Figures & tables
Fig. 1: Structural comparison of full-rank vs. low-rank SS2D. By significantly reducing computational overhead while maintaining representative power, the low-rank design opens up new avenues for efficient vision computing on resource-constrained edge devices.
Fig. 2: Overview of the VMamba backbone and SS2D module.
Fig. 3: Overview of the proposed DLRMamba framework. The proposed framework consists of four core components: (1) A pixel-level multispectral modality fusion module, which is designed to effectively fuse and process visible and infrared spectral information; (2) Low-Rank Structured State Space Modeling (Low-Rank SS2D), which is integrated to realize model lightweighting; (3) A structure-aware distillation (SAD) mechanism, including Singular Value Decomposition (SVD) Alignment (Matrix-level Distillation), Hidden State Sequence Alignment (Dynamic Distillation), and Feature Reconstruction (Output-level Distillation), which is proposed to compensate for performance degradation induced by model compression; (4) A detection head, which is used to output the final detection results.
Fig. 4: Sample RGB–IR pairs from five datasets (top: RGB; bottom: IR).
Method (Journal, Year)
Car
Pickup
Camping Car
Truck
Other
Tractor
Boat
Van
mAP50 (%) ↑
DMM ( TGRS 2025 ) [ 28 ]
84.2
78.8
79.0
65.7
56.2
72.3
72.3
72.5
75.0
C2DFF-Net ( TGRS 2025 ) [ 21 ]
91.8
85.9
87.2
74.4
58.7
78.6
62.2
91.3
79.8
MRN_lite ( TGRS 2025 ) [ 12 ]
92.2
87.7
83.9
73.9
68.1
80.4
54.9
72.1
76.7
LGA-YOLO ( JSTARS 2025 ) [ 13 ]
89.4
85.9
74.1
83.4
59.9
86.0
62.4
83.2
80.3
SuperYOLO ( TGRS 2023 ) [ 4 ]
91.1
85.7
79.3
70.2
57.3
80.4
60.2
76.5
75.1
Ours
91.5
81.0
82.0
91.9
78.2
80.0
91.9
81.3
84.7
TABLE I: Comparisons of different methods on the VEDAI dataset. The best result is shown in bold and the second best is shown with underline .
Method
mAP50 (%) ↑
ΔmAP50 (Method - Ours) (%)
Parameters (M) ↓
ΔParam (Method - Ours) (%)
DMM ( TGRS 2025 ) [ 28 ]
75.00
-9.70
87.97
+83.53
C2DFF-Net ( TGRS 2025 ) [ 21 ]
79.80
-4.90
6.58
+2.14
PG-DRFNet ( TCSVT 2025 ) [ 14 ]
84.06
-0.64
26.60
+22.16
MRN_lite ( TGRS 2025 ) [ 12 ]
76.70
-8.00
6.87
+2.43
DETR ( JSTARS 2024 ) [ 20 ]
77.30
-7.40
72.36
+67.92
LGA-YOLO ( JSTARS 2025 ) [ 13 ]
80.30
-4.40
10.83
+6.39
TABLE II: Comparison of detection accuracy and model size on the VEDAI dataset. The best result is shown in bold and the second best is shown with underline .
Fig. 5: Visual comparison of detection results produced by our and competing approaches on the VEDAI dataset under various challenging scenarios. Subfigures (a) and (d) illustrate detection performance in the presence of tree occlusions. Subfigure (b) presents results in an extremely dense scene containing numerous objects of different scales and categories. Subfigures (c) and (e) demonstrate cases where background objects exhibit high visual similarity to the objects. Red circles denote misclassified objects (incorrect category prediction), blue circles indicate false positives (detections of non-existent objects), and yellow circles represent missed detections of ground-truth objects.
Dataset
Method (Journal, Year)
mAP50 (%) ↑
Dataset
Method (Journal, Year)
mAP50 (%) ↑
FLIR
EI2Det ( TCSVT 2025 ) [ 19 ]
80.2
LLVIP
EI2Det ( TCSVT 2025 ) [ 19 ]
98.0
MMI-Det ( TCSVT 2024 ) [ 7 ]
79.8
DHANet ( TGRS 2025 ) [ 49 ]
97.7
ICAFusion ( PR 2024 ) [ 17 ]
77.5
PSFusion ( IF 2023 ) [ 16 ]
96.6
PSFusion ( IF 2023 ) [ 16 ]
75.8
YOLO-Adaptor ( TIV 2024 ) [ 50 ]
96.5
CDDFuse ( CVPR 2023 ) [ 51 ]
75.5
PIAFusion ( IF 2022 ) [ 52 ]
96.1
DATFuse ( TCSVT 2023 ) [ 53 ]
75.4
CDDFuse ( CVPR 2023 ) [ 51 ]
95.7
TABLE III: Comparisons of different methods on four datasets. The best result for each dataset is shown in bold , and the second-best result is shown in underline . The performance gap between our method and the best result is shown in blue.
Fig. 6: Precision–Recall curves of our method on the VEDAI, FLIR, DroneVehicle, and M3FD datasets, showing the detection performance for each class. The LLVIP dataset contains only the person class, so its PR curve is not shown.
No.
SS2D replaced
Params (MB) ↓
mAP50 (%) ↑
FPS ↑
I
0
17.1
81.5
0.4
II
0
9.0 ( -47.4% )
70.2 ( -13.9% )
1.0 ( +150.0% )
III
6
8.5 ( -50.3% )
75.5 ( -7.4% )
1.2 ( +200.0% )
TABLE IV: Comparison of our LowRank-SS2D method with mainstream existing lightweighting methods on the VEDAI dataset. (I: SS2D baseline; II: SS2D + pruning; III: SS2D + Our LowRank; FPS measured on Raspberry Pi 5, FP32; blue parentheses indicate percentage change relative to I).
No.
Method
Device
FPS ↑
Inference time (ms) ↓
I
Baseline
GPU A100
41.88
23.88
Ours
43.59
22.94
II
Baseline
GPU 4090
19.60
51.03
Ours
29.00
34.49
III
Baseline
Raspberry Pi 5
0.42
2367.67
Ours
2.30
434.78
TABLE V: Inference speed on different devices.
No.
Rank ratio
mAP50 (%) ↑
FPS (Raspberry Pi 5) ↑
I
0.65
80.20
0.64
II
0.60
79.60
0.98 ( +53.1% )
III
0.55
76.71
1.13 ( +76.6% )
IV
0.50
75.53
1.22 ( +90.6% )
TABLE VI: Effect of low-rank decomposition rank ratio on detection accuracy and Raspberry Pi 5 inference speed.
No.
SS2D replaced
Params (MB) ↓
mAP50 (%) ↑
FPS ↑
I
0
17.1
81.5
0.4
II
6
8.5
75.5
1.2
III
6
4.3
80.1
2.3
IV
6
4.3
84.7
2.3
TABLE VII: Ablation study of model compression and acceleration on VEDAI dataset (FPS on Raspberry Pi 5, FP32). I: Original SS2D (Baseline); II: LowRank-SS2D without distillation; III: LowRank-SS2D with distillation; IV: LowRank-SS2D with distillation and fine-tuning. The best result is shown in bold and the second best is shown with underline .
Fig. 7: Comparison of Grad-CAM heatmaps generated from the SS2D layer for the baseline and our method.
State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, 210008, Jiangsu, China · School of Elite Biomedical Engineers and the Institute for Interdisciplinary Intelligent Pharmacy, China Pharmaceutical University, Nanjing 211198, China