Organizations: MOE Key Lab of ICSP, Anhui Provincial Key Lab of Multimodal Cognitive Computation, IMIS Lab of Anhui Province, School of Computer Science and Technology, Anhui University, Hefei, China · School of Public Safety and Emergency Management, Anhui University of Science and Technology, Hefei 231131, China · College of Computing and Data Science, Nanyang Technological University, Singapore 639798 · Shaanxi Joint Laboratory of Artificial Intelligence, Shaanxi University of Science and Technology, Xi’an 710021, China · School of Software Engineering, Xi’an Jiaotong University, Xi’an 710049, China · Graduate School of Frontier Sciences, The University of Tokyo, Chiba, 277-8561, Japan
Change detection in optical remote sensing imagery is susceptible to illumination fluctuations, seasonal changes, and variations in surface land-cover materials. Relying solely on RGB imagery often produces pseudo-changes and leads to semantic ambiguity in features. Incorporating near-infrared (NIR) information provides heterogeneous physical cues that are complementary to visible light, thereby enhancing the discriminability of building materials and tiny structures while improving detection accuracy. However, existing multi-modal datasets generally lack high-resolution and accurately registered bi-temporal imagery, and current methods often fail to fully exploit the inherent heterogeneity between these modalities. To address these issues, we introduce the Large-scale Small-change Multi-modal Dataset (LSMD), a bi-temporal RGB-NIR building change detection benchmark dataset targeting small changes in realistic scenarios, providing a rigorous testing platform for evaluating multi-modal change detection methods in complex environments. Based on LSMD, we further propose the Multi-modal Spectral Complementarity Network (MSCNet) to achieve effective cross-modal feature fusion. MSCNet comprises three key components: the Neighborhood Context Enhancement Module (NCEM) to strengthen local spatial details, the Cross-modal Alignment and Interaction Module (CAIM) to enable deep interaction between RGB and NIR features, and the Saliency-aware Multisource Refinement Module (SMRM) to progressively refine fused features. Extensive experiments demonstrate that MSCNet effectively leverages multi-modal information and consistently outperforms existing methods under multiple input configurations, validating its efficacy for fine-grained building change detection. The source code will be made publicly available at: https://github.com/AeroVILab-AHU/LSMD
Figures & tables
Figure 1: Illustration of data distribution and challenging scenarios in realistic change detection. Note: unchanged images are excluded in (a) and (b) to focus on change-relevant statistics. (a) Comparison of the average change ratio per image between the proposed LSMD and other mainstream benchmarks. (b) Comparison of image proportions across different change ratios (1%–10%) between the proposed LSMD and existing benchmarks. (c) Visual examples of small-scale changes in large-scale scenes. (d) Visual examples of small buildings under vegetation backgrounds.
Figure 2: General architectural paradigm for MCD. Synchronous multi-modal imagery is provided at each temporal phase ( t1 and t2 ). The architecture employs dual-branch feature encoding and independent change perception to capture initial temporal difference cues for each modality. Finally, a Multi-modal Complementary Fusion mechanism is utilized for the synergistic integration of multi-source information, generating highly robust change detection results.
Figure 3: Samples and annotations from the LSMD dataset. (Left) Bi-temporal RGB and NIR image pairs with ground truth. (Right) Detailed illustrations of changed regions.
Dataset
Resolution
Image pairs
Image size (pixels)
t1 type
t2 type
Objects of interest
SMCD Dataset
LEVIR [ 2 ]
0.5m
637
1,024×1,024
RGB
RGB
Building
WHU [ 11 ]
0.2m
1
32,207×15,354
RGB
RGB
Building
CDD [ 13 ]
0.03m–1m
16,000
256×256
RGB
RGB
Multi-class
SYSU [ 34 ]
0.5m
20,000
256×256
RGB
RGB
Building
HCD Dataset
Table 1: Summary of widely used change detection datasets
Figure 4: Overall architecture of the proposed MSCNet. First, a Siamese backbone is employed to extract bi-temporal RGB and NIR features separately. Next, the Neighborhood Context Enhancement Module (NCEM) enhances the features to capture local change information. The Cross-modal Alignment and Interaction Module (CAIM) then integrates the RGB and NIR difference features to generate a more discriminative fused representation. Subsequently, the Saliency-aware Multisource Refinement Module (SMRM) leverages high-level semantic priors generated offline by RemoteSAM and, under the guidance of semantic masks, multi-modal difference features, and multi-scale context, progressively refines the fused features to restore spatial resolution and maintain cross-scale feature consistency. Finally, the features processed by these modules are used to generate the change detection results.
Figure 5: Structure of NCEM. Multi-level neighboring features are selectively aggregated and adaptively weighted to enhance local spatial details and contextual consistency.
Figure 6: Structure of CAIM. (a) Overall multi-modal fusion framework. (b) Bi-Directional Cross-Attention (BDCA). Cross-modal Channel Alignment and Fusion (CCAF), Adaptive Weighted Pooling (AWP), and BDCA are jointly employed to enable deep interaction between RGB texture features and NIR spectral features.
Datasets
DCILNet
S2CD
RHighNet
ConvFormer-CD
HRMNet
RFANet
CSI-Net
DGMA2-Net
A2Net
MSCNet (ours)
FLOPs(G)
41.47
9.55
66.51
5.14
12.17
3.16
367.25
18.10
3.05
4.49
Params(M)
26.80
3.26
96.81
37.72
13.46
2.86
62.18
37.10
3.78
6.40
RGB
Kappa
71.62
73.52
74.04
73.16
76.18
77.88
75.50
78.16
79.83
80.26
IoU
56.96
59.23
59.91
58.85
62.62
64.82
61.72
65.20
67.40
67.98
F1
72.58
74.40
74.93
74.10
77.01
78.66
76.33
78.93
80.52
80.94
Recall
68.29
68.86
71.48
71.24
74.85
76.49
72.78
77.95
77.04
77.45
Table 2: Quantitative comparison on the LSMD dataset using different input types. The best results are highlighted in red , the second-best results are blue , and the third-best results are orange .
Datasets
DCILNet
S2CD
RHighNet
ConvFormer-CD
HRMNet
RFANet
CSI-Net
DGMA2-Net
A2Net
MSCNet (ours)
SMARS(0.3)
Kappa
94.32
93.49
91.95
94.72
97.13
96.92
95.16
97.03
97.08
97.36
IoU
91.26
90.02
87.81
91.85
95.48
95.17
92.50
95.33
95.41
95.84
F1
95.43
94.75
93.51
95.75
97.69
97.53
96.11
97.61
97.65
97.88
Recall
95.45
93.48
92.67
95.50
97.45
97.63
96.02
97.58
97.51
97.85
Precision
95.41
96.05
94.36
96.01
97.92
97.43
96.20
97.64
97.80
97.91
SMARS(0.5)
Kappa
91.63
89.28
88.71
92.00
95.26
94.91
94.78
94.86
94.92
95.64
Table 3: Quantitative comparison on SMARS datasets with different resolutions. The best results are highlighted in red , the second-best results are blue , and the third-best results are orange .
Figure 7: Qualitative visualizations of different change detection approaches on the LSMD dataset under RGB-only configuration, with MSCNet using multi-modal (RGB+NIR) information for direct comparison. The rendered colors denote TP (white), FP (red), TN (black), and FN (blue).
Figure 8: Qualitative visualizations of different change detection approaches on the LSMD dataset under NIR-only configuration, with MSCNet using multi-modal (RGB+NIR) information for direct comparison. The rendered colors denote TP (white), FP (red), TN (black), and FN (blue).
Figure 9: Qualitative visualizations of different change detection approaches on the LSMD dataset under multi-modal (RGB+NIR) configuration. The rendered colors denote TP (white), FP (red), TN (black), and FN (blue).
Figure 10: Qualitative visualizations of different change detection approaches on the SMARS dataset at 0.3 m spatial resolution. The rendered colors denote TP (white), FP (red), TN (black), and FN (blue).
Figure 11: Qualitative visualizations of different change detection approaches on the SMARS dataset at 0.5 m spatial resolution. The rendered colors denote TP (white), FP (red), TN (black), and FN (blue).
Table 4: Ablation study of different modules on the LSMD dataset.
Figure 12: Heatmap visualizations of the proposed modules. (a) and (b) show the RGB t2 and NIR t2 feature maps extracted from the four stages of the backbone network, respectively. (c) and (d) present the RGB t2 and NIR t2 feature maps after applying the NCEM. (e) shows the feature maps after the CAIM. (f) illustrates the feature maps refined by the SMRM.
Institute of Geospatial Information, Information Engineering University, Zhengzhou, China · Academy of Digital China (Fujian), Fuzhou University, Fuzhou, China · The School of Electronics and Communication Engineering, Sun Yat-sen University, Shenzhen, China +1
MOE Key Laboratory of ICSP, IMIS Laboratory of Anhui Province, Anhui Provincial Key Laboratory of Multimodal Cognitive Computation, Zenmorn-AHU AI Joint Laboratory, School of Computer Science and Technology, Anhui University, Hefei 230601, China