nnMNet: Baseline for Martian Terrain Semantic Segmentation
Organizations: Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan 701, Taiwan
Abstract
Semantic segmentation is a crucial task for understanding Mars, the most Earth-like planet in our solar system. However, it is challenging because the Martian surface is highly unstructured and complex, making accurate pixel-level prediction and fine-grained annotation difficult. Recent advancements in deep learning have introduced numerous methods and datasets to address these challenges. Nevertheless, the field lacks a robust, publicly available, and reproducible baseline, as well as a unified benchmark to facilitate fair evaluations. In this work, we present nnMNet, a new baseline model designed for Martian terrain semantic segmentation. Building upon nnWNet, we integrate linear attention to better capture global context and employ lightweight convolutions to reduce computational overhead. To bridge the gap between local and global representations, we introduce the Spatially-Aware Fusion Block (SAFB), which augments and combines features with diverse characteristics. Furthermore, we establish a new benchmark by curating and standardizing three high-quality datasets for thorough evaluation. nnMNet achieves new state-of-the-art 86.61%, 83.25%, and 88.24% mIoU on SynMars-TW, SynMars-Air, and MarsScapes, respectively. Our code, models, and datasets are publicly available at https://github.com/dereklee0310/nnMNet.
Figures & tables
| Dataset | Source | Type | Annotation Method | Scale | Classes |
|---|---|---|---|---|---|
| AI4Mars [ 59 ] | MER, MSL | Real | Crowdsourcing | 16.39k | 4 |
| MER-Seg [ 60 ] | MER | Real | Average Labeling | 1.06k | 9 |
| MSL-Seg [ 60 ] | MSL | Real | Average Labeling | 4.13k | 9 |
| Mars [ 61 ] | MSL | Real | Sparse Labeling | 6k | 9 |
| MarsScapes [ 62 , 35 ] | MSL | Real | Semi-automatic | 20.8k | 9 |
| SynMars [ 5 ] | TW-1 | Synthetic | VisionBlender | 60k | 1 |
| Class | IoU (%) | ||||
|---|---|---|---|---|---|
| SynMars-TW | SynMars-Air | MarsScapes | MSL-Seg | Mars | |
| Sky | 98.43 | 99.54 | 83.06 | — | 83.83 |
| Ridge | 93.49 | 81.86 | 73.59 | — | 82.38 |
| Soil / Martian Soil | 98.79 | 99.11 | 86.89 | 17.24 | 30.39 |
| Sand / Sands | 77.73 | 65.33 | 73.91 | 41.00 | 37.69 |
| Bedrock | 68.53 | 70.18 | 75.13 | 59.11 | 57.87 |
| Method | Big rock | Small rock | Gravel | Bedrock | Ridge | Sand | Soil | Sky | mIoU (%) | Params (M) | FLOPs (G) | FPS |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| BiseNetV2 [ 90 ] | 83.58 | 22.09 | 0.01 | 84.43 | 95.95 | 68.02 | 96.09 | 99.41 | 68.70 | 3.35 | 12.31 | 501.13 |
| DeepLabv3+ [ 21 ] | 88.80 | 43.98 | 1.30 | 89.46 | 96.95 | 80.25 | 97.33 | 99.52 | 74.70 | 12.32 | 54.27 | 192.78 |
| SegFormer-B0 [ 12 ] | 89.74 | 42.86 | 1.41 | 91.06 | 95.56 | 84.43 | 97.65 | 99.30 | 75.25 | 3.75 | 8.50 | 302.28 |
| SegNeXt-T [ 23 ] | 89.24 | 31.43 | 0.23 | 89.41 | 95.88 | 83.83 | 97.16 | 99.36 | 73.32 | 4.26 | 6.45 | 221.09 |
| Light4Mars-B [ 6 ] | 89.81 | 44.37 | 0.09 | 91.70 | 96.65 | 84.78 | 97.84 | 99.47 | 76.70 | 2.56 | 9.42 | 160.51 |
| nnUNet [ 15 ] | 94.31 | 78.10 | 56.61 | 76.44 | 94.77 | 81.73 | 99.08 | 99.36 | 85.05 | 65.22 | 60.20 | 274.69 |
| Method | Big rock | Small rock | Gravel | Bedrock | Ridge | Sand | Soil | Sky | mIoU (%) | Params (M) | FLOPs (G) | FPS |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| BiseNetV2 [ 90 ] | 84.74 | 29.14 | 0.00 | 90.00 | 98.27 | 96.63 | 98.66 | 99.77 | 74.65 | 3.35 | 12.31 | 501.13 |
| DeepLabv3+ [ 21 ] | 87.86 | 43.31 | 0.34 | 92.55 | 98.71 | 95.54 | 99.03 | 99.81 | 77.15 | 12.32 | 54.27 | 192.78 |
| SegFormer-B0 [ 12 ] | 90.56 | 52.67 | 1.09 | 94.33 | 98.83 | 98.24 | 99.26 | 99.75 | 79.34 | 3.75 | 8.50 | 302.28 |
| SegNeXt-T [ 23 ] | 91.36 | 43.62 | 0.07 | 94.02 | 98.97 | 98.21 | 99.10 | 99.83 | 78.15 | 4.26 | 6.45 | 221.09 |
| Light4Mars-B [ 6 ] | 89.98 | 49.81 | 5.02 | 94.38 | 98.94 | 97.95 | 99.26 | 99.83 | 79.40 | 2.56 | 9.42 | 160.51 |
| nnUNet [ 15 ] | 89.40 | 77.94 | 43.04 | 77.62 | 87.58 | 74.39 | 99.31 | 98.01 | 80.91 | 65.22 | 60.20 | 274.69 |
| Method | Soil | Bedrock | Gravel | Sand | Big Rock | Ridge | Sky | Rover | Unlabeled | mIoU (%) | Params (M) | FLOPs (G) | FPS |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| BiseNetV2 [ 90 ] | 89.30 | 84.20 | 74.16 | 81.16 | 63.43 | 88.68 | 95.47 | 77.78 | 79.22 | 81.49 | 3.35 | 6.16 | 623.24 |
| DeepLabv3+ [ 21 ] | 85.94 | 80.52 | 66.04 | 78.15 | 57.36 | 83.70 | 91.68 | 63.75 | 72.03 | 75.47 | 12.32 | 27.14 | 300.22 |
| SegFormer-B0 [ 12 ] | 91.12 | 86.59 | 78.25 | 84.16 | 67.50 | 92.46 | 96.61 | 88.64 | 84.25 | 85.51 | 3.72 | 3.70 | 395.43 |
| SegNeXt-T [ 23 ] | 91.71 | 87.21 | 79.81 | 84.18 | 67.98 | 93.00 | 96.92 | 88.11 | 83.87 | 85.87 | 4.26 | 3.23 | 252.14 |
| Light4Mars-B [ 6 ] | 90.78 | 86.59 | 77.11 | 83.64 | 66.24 | 91.35 | 95.97 | 83.68 | 81.96 | 84.15 | 2.56 | 4.82 | 282.06 |
| nnUNet [ 15 ] | 93.28 | 87.45 | 85.24 | 84.67 | 83.19 | 80.73 | 90.13 | 97.85 | 80.00 | 86.95 | 47.64 | 29.84 | 387.84 |
| Method | mIoU (%) | Params (M) | FLOPs (G) |
|---|---|---|---|
| nn W Net [ 13 ] | 81.17 ( 0.00) | 7.05 | 42.54 |
| Pooling RALA [ 81 ] | 81.62 ( 0.45) | 7.21 | 43.91 |
| + RoPE [ 87 ] | 82.05 ( 0.43) | 7.21 | 43.91 |
| + APE [ 43 ] | 81.64 ( 0.41) | 9.18 | 43.91 |
| + CPE [ 68 , 84 ] | 81.79 ( 0.26) | 7.22 | 44.00 |
| + LePE [ 85 ] | 81.78 ( 0.27) | 7.22 | 44.00 |
| token mixer | mIoU (%) | Params (M) | FLOPs (G) |
|---|---|---|---|
| RALA [ 81 ] | 82.55 | 5.71 | 39.81 |
| Efficient Self-Attention [ 12 ] | 81.99 | 6.47 | 44.11 |
| Average Pooling [ 75 ] | 82.13 | 5.55 | 38.46 |
| kernel size | mIoU (%) | Params (M) | FLOPs (G) |
|---|---|---|---|
| 3 3 | 82.55 | 5.71 | 39.81 |
| 5 5 | 82.36 | 5.77 | 40.56 |
| 7 7 | 82.33 | 5.85 | 41.69 |
| reduction ratio | mIoU (%) | Params (M) | FLOPs (G) |
|---|---|---|---|
| 1 | 82.57 | 6.16 | 42.57 |
| 0.5 | 82.54 | 5.86 | 40.73 |
| 0.25 | 82.55 | 5.71 | 39.81 |
| 0.125 | 82.38 | 5.64 | 39.95 |
| w/o SAFB | 82.43 | 5.56 | 37.71 |
| Model | Iterations | mIoU (%) | Params (M) | FLOPs (G) | FPS |
|---|---|---|---|---|---|
| nn W Net | 50K | 80.02 | 7.05 | 42.54 | 92.58 |
| nn M Net | 50K | 82.55 | 5.71 | 39.81 | 63.33 |
| nn M Net † | 50K | 82.66 | 4.23 | 39.32 | 100.27 |
| nn W Net | 250K | 84.01 | 7.05 | 42.54 | 92.58 |
| nn M Net | 250K | 86.61 | 5.71 | 39.81 | 63.33 |
| nn M Net † | 250K | 86.12 | 4.23 | 39.32 | 100.27 |
Semantic Segmentation
Semantic segmentation assigns a class label to every image pixel, enabling detailed scene understanding across domains and sensors.
See all 399 papers