DualSwinFusionSeg: Multimodal Martian Landslide Segmentation via Dual Swin Transformer with Multi-Scale Fusion and UNet++

Shahriar Kabir; Abdullah Muhammed Amimul Ehsan; Istiak Ahmmed Rifti; Md Kaykobad Reza

cs.CVMar 14, 2026

DualSwinFusionSeg: Multimodal Martian Landslide Segmentation via Dual Swin Transformer with Multi-Scale Fusion and UNet++

Authors: Shahriar Kabir, Abdullah Muhammed Amimul Ehsan, Istiak Ahmmed Rifti, Md Kaykobad Reza

Organizations: Bangladesh University of Engineering and Technology, Bangladesh · University of California, Riverside, USA

Abstract

Automated segmentation of Martian landslides, particularly in tectonically active regions such as Valles Marineris,is important for planetary geology, hazard assessment, and future robotic exploration. However, detecting landslides from planetary imagery is challenging due to the heterogeneous nature of available sensing modalities and the limited number of labeled samples. Each observation combines RGB imagery with geophysical measurements such as digital elevation models, slope maps, thermal inertia, and contextual grayscale imagery, which differ significantly in resolution and statistical properties. To address these challenges, we propose DualSwinFusionSeg, a multimodal segmentation architecture that separates modality-specific feature extraction and performs multi-scale cross-modal fusion. The model employs two parallel Swin Transformer V2 encoders to independently process RGB and auxiliary geophysical inputs, producing hierarchical feature representations. Corresponding features from the two streams are fused at multiple scales and decoded using a UNet++ decoder with dense nested skip connections to preserve fine boundary details. Extensive ablation studies evaluate modality contributions, loss functions, decoder architectures, and fusion strategies. Experiments on the MMLSv2 dataset from the PBVS 2026 Mars-LS Challenge show that modality-specific encoders and simple concatenation-based fusion improve segmentation accuracy under limited training data. The final model achieves 0.867 mIoU and 0.905 F1 on the development benchmark and 0.783 mIoU on the held-out test set, demonstrating strong performance for multimodal planetary surface segmentation.

Figures & tables

Figure 1: DualSwinFusionSeg: two SwinV2-Small encoders process RGB and AUX inputs into four-level feature hierarchies; per-scale features are concatenated,reduced via

1\times 1

convolution, and decoded by UNet+,followed by a Conv

1\times 1

head and bilinear upsampling to the final segmentation mask.

Hyperparameter	Value	Hyperparameter	Value
Optimiser	AdamW	Epochs / fold	50
Learning rate	$2{\times}10^{-4}$	Batch size	16
Weight decay	$10^{-4}$	Precision	FP16
LR warm-up	3 epochs	LR schedule	Cosine
EMA decay ( $\gamma$ )	0.995	CV folds	5

Table 1: Training hyperparameters used across all experiments.

Figure 2: Qualitative segmentation results.

Phase	mIoU	IoU ${}_{\text{fg}}$	IoU ${}_{\text{bg}}$	F1	Prec.	Recall
Development	0.867	0.827	0.907	0.905	0.900	0.911
Test	0.783	0.677	0.890	0.800	0.760	0.860

Table 2: Official CodaBench evaluation for the final submission.

Configuration	mIoU	F1
SwinTiny + FPN, RGB + DEM (4 channels)†	0.672	-
SwinTiny + FPN, all 7 channels†	0.743	0.821
SwinTiny + FPN, class-weighted BCE + Dice†	0.762	0.840
MMSFormer [ 17 ] (B2+B1 encoders), all 7 channels†	0.716	-
DualSwinTiny + FPN, gated cross-modal fusion†	0.793	0.848
SwinTiny + FPN (CodaBench development leaderboard)	0.837	0.884

Table 3: Progressive improvement across experimental configurations. Metrics marked (†) are single-fold results on the competition validation split; all other entries are official CodaBench leaderboard scores. Configurations marked (

\star

) are novel architecture explorations that provide design insights but do not surpass the final UNet++ configuration on mIoU.

Type	Transform	Setting
Geometric	Horizontal / Vertical flip	$p=0.5$ each
	Random rotation( $90^{\circ}$ )	$p=0.5$
	Affine transform	$\pm 5\%$ shift, $[0.9,1.1]\!\times$ scale, $\pm 20^{\circ}$ , $p{=}0.5$
Photometric (RGB only)	Gaussian blur	kernel $3$ – $7$ , $\sigma$ auto, $p{=}0.15$
	Brightness & Contrast	limit $\pm 0.2$ each, $p{=}0.3$

Table 4: Online data augmentation pipeline applied during training.

Modalities	Channels	Val mIoU	Val Dice
RGB + DEM	4	0.672	0.758
RGB + Thermal	4	0.661	0.750
RGB + DEM + Slope	5	0.721	0.798
RGB + DEM + Thermal	5	0.691	0.778
All 7 channels	7	0.743	0.821

Table 5: Ablation study on modality combinations.

ID	Loss function	Val mIoU	Val Dice
$\mathcal{L}_{1}$	BCE	0.728	0.806
$\mathcal{L}_{2}$	Dice	0.752	0.833
$\mathcal{L}_{3}$	BCE + Dice	0.758	0.833
$\mathcal{L}_{4}$	BCE( $w^{+}$ ) + Dice	0.762	0.840
$\mathcal{L}_{5}$	Focal [ 8 ]	0.744	0.818
$\mathcal{L}_{6}$	Focal + Dice	0.748	0.827

Table 6: Ablation study on the loss function.

Decoder	Val mIoU	Val F1
FPN [ 7 ]	0.800	0.849
UPerNet [ 23 ]	0.818	0.865
SegFormer MLP [ 24 ]	0.806	0.857
DeepLabV3+ [ 3 ]	0.801	0.846
UNet++ [ 26 ]	0.824	0.868

Table 7: Ablation study on the decoder choice.

Decoder	Cross-modal fusion	mIoU	F1
FPN	Concatenation-based	0.836	0.878
UPerNet	Concatenation-based	0.822	0.867
UNet++	Concatenation-based	0.814	0.857
UNet++	Weighted-sum	0.803	0.850
UPerNet	Weighted-sum	0.824	0.868

Table 8: Ablation study on cross-modal fusion strategy.

Decoder	mIoU	IoU ${}_{\text{fg}}$	IoU ${}_{\text{bg}}$	Prec.	Recall	F1
UNet++	0.836	0.776	0.895	0.895	0.850	0.872
Hybrid	0.828	0.769	0.887	0.856	0.899	0.877

Table 9: Hybrid decoder vs. plain UNet++.

Config.	IA	IE	DO	Attention-gated fusion	CV mIoU	CV F1
E10-a	-	ECA	-	ECA-gated	$0.806\pm 0.009$	$0.873\pm 0.009$
E10-b	-	ECA	-	CBAM-gated	$0.805\pm 0.008$	$0.872\pm 0.009$
E11	ECA	ECA	CBAM	CBAM-gated	$0.802\pm 0.011$	$0.871\pm 0.010$
E12	ECA	ECA	ECA	ECA-gated	$0.805\pm 0.008$	$0.872\pm 0.008$
E13	ECA	ECA	SE	SE-gated	$0.803\pm 0.006$	$0.871\pm 0.007$
E9	ECA	SE	CBAM	ECA-gated	$0.805\pm 0.008$	$0.872\pm 0.008$

Table 10: Ablation study on channel attention. IA = input-level attention; IE = intra-encoder attention; DO = decoder-output attention.

TTA Strategy	Val mIoU	$\Delta$ mIoU	Inf. time
No TTA	0.8278	$+$ 0.0000	0.9 s
Horizontal flip	0.8272	$-$ 0.0006	1.0 s
Vertical flip	0.8276	$-$ 0.0002	1.0 s
H-flip + V-flip (2-view)	0.8283	$+$ 0.0005	1.2 s
4-view ( $90^{\circ}$ rotations)	0.8271	$-$ 0.0007	1.2 s
8-view (D4 symmetry group)	0.8231	$-$ 0.0047	1.5 s

Table 11: Ablation study on test-time augmentation.

Training configuration	Val mIoU	Val F1	Best epoch
EMA ( $\gamma=0.995$ )	0.8278	0.8692	28
Standard (no EMA)	0.8178	0.8611	12

Table 12: EMA vs. standard training.

Multimodal Semantic Segmentation

See all 27 papers

Global-Local Contextual Progressive Expansion Network for Martian Landslide Segmentation in Multimodal Remote Sensing Imagery
Sep 11, 2026Leo Thomas Ramos, Sidike Paheding, Abel A. Reyes-Angulo +4Multi-Angular Remote Sensing Remote Sensing Image Segmentation
Computer Vision Center, Universitat Aut`onoma de Barcelona, Barcelona, 08193, Spain · Department of Computer Science and Engineering, Fairfield University, Fairfield, CT 06824, USA · Michigan Technological University, Houghton, MI 49931, USA +3
MANTLE: A Framework for Adaptive In-Situ Planetary Perception Using a Modular Uplink Principle
Aug 28, 2026Pranav Durai, Gary DoranSpace Robotics Remote Sensing Image Segmentation
Stanford Center for Innovation in In Vivo Imaging Stanford University School of Medicine 318 Campus Drive, Room S040, Stanford, CA 94305, USA · Jet Propulsion Laboratory California Institute of Technology 4800 Oak Grove Drive, Pasadena, CA 91109, USA
nnMNet: Baseline for Martian Terrain Semantic Segmentation
Aug 30, 2026Ming-Han Lee, Chi-Yeh ChenSemantic Segmentation Remote Sensing Image Segmentation
Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan 701, Taiwan

Sep 11, 2026J/K move · Enter open · S save

Global-Local Contextual Progressive Expansion Network for Martian Landslide Segmentation in Multimodal Remote Sensing Imagery

Leo Thomas Ramos, Sidike Paheding, Abel A. Reyes-Angulo, Rajaneesh A.+3

Computer Vision Center, Universitat Aut`onoma de Barcelona, Barcelona, 08193, Spain · Department of Computer Science and Engineering, Fairfield University, Fairfield, CT 06824, USA · Michigan Technological University, Houghton, MI 49931, USA+3

Multi-Angular Remote Sensing Remote Sensing Image Segmentation Semantic Segmentation Remote Sensing

Open paper AskPDF arXiv

Automated landslide segmentation on Mars is one of the important tasks for understanding its surface processes, and all will aid in future space exploration. However, it remains a relatively underexplored open challenge because landslide morphology is highly variable, foreground regions are often sparse or irregular, and orbital observations combine heterogeneous spectral and topographic cues. In this context, this work investigates the capability of deep learning to address Martian landslide segmentation through an extensive assessment of modern neural segmentation models. To the best of our knowledge, this is the first study to provide such a comprehensive exploration in this domain. We further propose TransCPLES, a U-shaped network that couples Contextual Progressive Layer Expansion feature extraction with Transformer-based contextual reasoning, enabling the model to capture local geomorphic patterns and broader spatial dependencies for more reliable landslide delineation. Experiments on MMLSv2, a seven-band multimodal Martian landslide dataset, show that TransCPLES achieves the best overall performance when evaluated on geographically distinct samples, with consistent delineation across different landslide extents, stable foreground discrimination, and a favorable balance between accuracy and computational cost compared with several state-of-the-art convolutional, attention-based, and Transformer-based segmentation models. With this work, we hope to provide a useful reference and encourage further research and development in deep learning for planetary remote sensing. Code will be available after publication.

Abstract

Multimodal Semantic Segmentation

Explore similar work

Global-Local Contextual Progressive Expansion Network for Martian Landslide Segmentation in Multimodal Remote Sensing Imagery

MANTLE: A Framework for Adaptive In-Situ Planetary Perception Using a Modular Uplink Principle

nnMNet: Baseline for Martian Terrain Semantic Segmentation