MaskSCD: Bitemporal Mask Classification for Semantic Change Detection in Remote Sensing Images
Organizations: Helmholtz Institute Freiberg for Resource Technology, Helmholtz-Zentrum Dresden-Rossendorf, 09599 Freiberg, Germany · Chair of Data Science in Earth Observation, Technical University of Munich, Germany · Lancaster University, LA1 4YR Lancaster, U.K.
Abstract
Semantic change detection (SCD) identifies where the land surface has changed and which land-cover transition has taken place. Most deep SCD networks predict the change map and the land-cover map of each date with separate per-pixel decoders. These outputs need not agree, so a pixel can be labeled as changed although it has the same class at both dates, and per-pixel decisions fragment objects. We present MaskSCD, which formulates SCD as the mask classification of a bitemporal label-pair image. Each bitemporal segment, a region of constant land-cover class pair, is predicted as one mask shared by both dates together with one class per date, and the change map is derived from the predicted class pair. This formulation keeps the change and semantic maps consistent and the label space linear in the number of classes. A bitemporal masked-attention transformer decoder implements it by reading each segment in both images and relating the two readings, and a bitemporal fusion pyramid provides per-date and change-aware features. MaskSCD is evaluated on SECOND, Landsat-SCD and Hi-UCD, which span 0.1 to 30 m resolution and four to nine land-cover classes, against fourteen recent SCD models re-trained under one protocol. MaskSCD achieves the best separated kappa, semantic change and transition on all three datasets and exceeds the strongest competitor by about five points in mean separated kappa. It produces no contradictory change pixels, which account for nearly 30% of the predicted change of some multi-task networks. The code is available at https://github.com/EricYu97/MaskSCD.
Figures & tables
| Dataset | Sensor, resolution | Train / Val / Test | Size | Labels | |
|---|---|---|---|---|---|
| SECOND | aerial, 0.5 to 3 m | 2 672 / 296 / 1 694 | 512 | 7 | changed |
| Landsat-SCD | Landsat, 30 m | 1 431 / 477 / 477 | 416 | 5 | changed |
| Hi-UCD | aerial, 0.1 m | 4 800 / 944 / 6 176 | 256 | 10 | full |
| SCanNet | ClearSCD | LSAFNet | TripleS | HGINet | CPGNet | Change- Mamba | GSTM- SCD | FoBa | Mamba- FCS | PerASCD VM | PerASCD PerA | SemDINO | DMCTNet | MaskSCD | ||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SECOND | OA | 88.42 | 86.83 | 88.39 | 85.74 | 88.18 | 88.42 | 88.92 | 87.44 | 89.08 | 89.27 | 87.48 | 88.86 | 88.69 | 87.62 | 89.32 |
| mIoU | 73.86 | 70.37 | 73.79 | 69.39 | 73.02 | 73.60 | 74.17 | 73.05 | 74.33 | 74.77 | 72.07 | 74.37 | 73.86 | 73.88 | 74.98 | |
| SeK | 24.92 | 18.46 | 24.79 | 15.98 | 23.64 | 24.54 | 25.18 | 24.32 | 25.92 | 26.72 | 21.66 | 26.15 | 24.95 | 25.54 | 27.04 | |
| 65.24 | 59.26 | 65.16 | 56.42 | 64.33 | 64.94 | 65.25 | 64.53 | 66.27 | 66.92 | 62.38 | 66.50 | 65.38 | 64.88 | 67.11 | ||
| 57.50 | 50.51 | 57.37 | 46.66 | 56.80 | 57.28 | 57.30 | 56.61 | 59.02 | 59.75 | 54.16 | 59.22 | 57.73 | 56.56 | 59.94 | ||
| Landsat- SCD | OA | 97.45 | 93.00 | 97.56 | 91.04 | 95.18 | 96.18 | 96.15 | 94.93 | 96.32 | 96.48 | 95.19 | 93.61 | 96.84 | 96.88 | 98.04 |
| Method | Complexity | Mean accuracy over the three datasets (%) | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Params (M) | GFLOPs | Time (ms) | Memory (GB) | SeK | SeK (F) | SeK (P) | |||
| SCanNet | 27.9 | 265.2 | 18.4 | 0.49 | 34.13 | 65.34 | 58.82 | 34.91 | 32.04 |
| ClearSCD | 5.8 | 20.0 | 28.0 | 0.14 | 23.41 | 61.95 | 54.34 | 23.96 | 11.25 |
| LSAFNet | 28.4 | 520.3 | 26.3 | 0.45 | 37.28 | 69.63 | 63.85 | 37.48 | 29.44 |
| TripleS | 8.6 | 30.3 | 33.4 | 0.16 | 18.92 | 57.62 | 49.37 | 19.56 | 8.02 |
| HGINet | 27.7 | 58.2 | 43.4 | 0.26 | 32.33 | 68.34 | 61.47 | 32.49 | 13.39 |
| Method | SECOND | Landsat-SCD | Hi-UCD |
|---|---|---|---|
| SCanNet | 0.41 | 1.30 | 28.97 |
| ClearSCD | 5.49 | 6.13 | 5.90 |
| LSAFNet | 0.27 | 0.14 | 14.97 |
| TripleS | 10.09 | 8.17 | 11.80 |
| HGINet | 1.47 | 1.95 | 5.40 |
| CPGNet | 2.20 | 2.18 | 4.46 |
| No. | Variants | Metrics | |||
|---|---|---|---|---|---|
| mIoU | SeK | ||||
| #01 | MaskSCD | 81.39 | 42.09 | 73.98 | 68.75 |
| Decoder components | |||||
| #02 | w/o masked attention | 80.51 | 40.35 | 71.91 | 66.51 |
| #03 | w/o joint self-attention | 80.52 | 40.75 | 73.00 | 68.02 |
| #04 | w/o masked attention and joint self-attention | 79.99 | 39.21 | 70.92 | 65.24 |
Remote Sensing Change Detection
Remote sensing change detection finds and interprets differences across Earth-observation images acquired at different times.
See all 100 papers