BuildMamba: A Visual State-Space Based Model for Multi-Task Building Segmentation and Height Estimation from Satellite Images

Sinan U. Ulu; A. Enes Doruk; I. Can Yagmur; Bahadir K. Gunturk; Oguz Hanoglu; Hasan F. Ates

cs.CVMar 9, 2026

BuildMamba: A Visual State-Space Based Model for Multi-Task Building Segmentation and Height Estimation from Satellite Images

Authors: Sinan U. Ulu, A. Enes Doruk, I. Can Yagmur, Bahadir K. Gunturk, Oguz Hanoglu, Hasan F. Ates

Organizations: Graduate School of Engineering, Ozyegin university, 34794 Istanbul, Türkiye. · Department of Computer Science, University of Rochester, Rochester, NY 14627 USA. · School of Engineering and Natural Sciences, Istanbul Medipol University, 34810 Istanbul, Türkiye. · Huawei Turkiye R&D Center, 34768 Istanbul, Türkiye.

Abstract

Accurate building segmentation and height estimation from single-view RGB satellite imagery are fundamental for urban analytics, yet remain ill-posed due to structural variability and the high computational cost of global context modeling. While current approaches typically adapt monocular depth architectures, they often suffer from boundary bleeding and systematic underestimation of high-rise structures. To address these limitations, we propose BuildMamba, a unified multi-task framework designed to exploit the linear-time global modeling of visual state-space models. Motivated by the need for stronger structural coupling and computational efficiency, we introduce three modules: a Mamba Attention Module for dynamic spatial recalibration, a Spatial-Aware Mamba-FPN for multi-scale feature aggregation via gated state-space scans, and a Mask-Aware Height Refinement module using semantic priors to suppress height artifacts. Extensive experiments demonstrate that BuildMamba establishes a new performance upper bound across three benchmarks. Specifically, it achieves an IoU of 0.93 and RMSE of 1.77~~m on DFC23 benchmark, surpassing state-of-the-art by 0.82~~m in height estimation. Simulation results confirm the model's superior robustness and scalability for large-scale 3D urban reconstruction.

Figures & tables

Fig. 1 : Overview of the proposed BuildMamba architecture for joint building segmentation and height estimation.

Fig. 2 : Overview of BuildMamba pipeline.

Fig. 3 : Illustration of 2D-Selective-Scan (SS2D) [ 27 ] .

Panel of Figure 4 — Fig. 4 : Overview of the Mamba Attention Module (MAM).

Fig. 5 : Overview of the S-MambaFPN module.

Fig. 6 : Overview of the Mask-Aware Height Refinement (MHR) module.

Metric	Train/Val	Test
Total Buildings	177,408	58,399
Avg. Buildings	31.61	22.18
Building Pixel Ratio	0.24	0.29
Maximum Height (m)	309	328
Mean Height (m)	14.85	18.69
Std. Deviation (m)	17.55	20.56

TABLE I : Huawei BHE train/val and test set statistics.

Fig. 7 : Sample images from different cities of Huawei dataset.

		DFC19		DFC23		Huawei BHE
Model	Input	IoU $\uparrow$	F1 $\uparrow$	IoU $\uparrow$	F1 $\uparrow$	IoU $\uparrow$	F1 $\uparrow$
BuildFormer	RGB	0.79	0.88	0.78	0.88	0.42	0.60
DSMNet	RGB	0.45	0.62	0.85	0.92	0.42	0.60
ST-UNet	RGB	0.84	0.89	0.82	0.90	0.52	0.68
FarSeg++	RGB	0.82	0.90	0.83	0.91	0.52	0.68
UANet	RGB	0.89	0.94	0.92	0.96	0.53	0.69

TABLE II : Semantic segmentation performance comparison across DFC19, DFC23, and Huawei BHE datasets. The best and second-best results are highlighted in bold and underlined, respectively.

		DFC19				DFC23				Huawei BHE
Model	Input	RMSE $\downarrow$	$\delta_{1}\uparrow$	$\delta_{2}\uparrow$	$\delta_{3}\uparrow$	RMSE $\downarrow$	$\delta_{1}\uparrow$	$\delta_{2}\uparrow$	$\delta_{3}\uparrow$	RMSE $\downarrow$	$\delta_{1}\uparrow$	$\delta_{2}\uparrow$	$\delta_{3}\uparrow$
LUMNet	RGB+Mask	1.040	0.942	0.963	0.971	3.620	0.746	0.806	0.833	8.521	0.818	0.854	0.887
DSMNet	RGB	3.880	0.443	0.476	0.500	3.389	0.746	0.805	0.849	10.413	0.609	0.641	0.671
IM2HEIGHT	RGB	4.899	0.201	0.203	0.205	7.210	0.678	0.679	0.681	12.301	0.385	0.385	0.385
IM2ELEVATION	RGB	4.952	0.774	0.774	0.774	6.501	0.759	0.764	0.772	11.841	0.792	0.798	0.802
HTC-DC	RGB	1.227	0.934	0.946	0.950	2.592	0.827	0.860	0.873	10.621	0.479	0.512	0.544

TABLE III : Height estimation performance comparison: RMSE (m) and Deltas (

\delta_{n}

). Among models with RGB-only input, the best and second-best results are highlighted in bold and underlined, respectively.

Panel of Figure 11 — Fig. 8 : Visual comparison of segmentation results on DFC19 dataset.

Panel of Figure 12 — Fig. 9 : Visual comparison of segmentation results on Huawei BHE dataset.

Panel of Figure 13 — Fig. 10 : Visual comparison of height estimation results DFC23 dataset

Panel of Figure 14 — Fig. 11 : Visual comparison of height estimation results on Huawei BHE dataset

VMamba	MAM	S-MambaFPN	MHR	IoU	RMSE (m)	Params (M)
				0.785	-	49.2
✓				0.852	1.20	58.2
✓	✓			0.871	1.17	65.2
✓	✓	✓		0.896	1.10	71.2
✓	✓	✓	✓	0.897	1.06	74.3

TABLE IV : Ablation Study on DFC19 Dataset. The best result is highlighted in bold.

Fig. 12 : Qualitative analysis of the MHR module on the DFC19 dataset.

	DFC19		DFC23		Params (M)
	IoU	RMSE	IoU	RMSE
ResNet-101	0.862	1.11	0.891	1.91	60.2
Swin-S	0.881	1.09	0.908	1.85	74.4
Mamba (ours)	0.897	1.05	0.927	1.80	74.3

TABLE V : Analysis of backbone on DFC19 and DFC23 datasets using IoU and RMSE (m) metrics.

	DFC19		DFC23		Params (M)
	IoU	RMSE	IoU	RMSE
Without S-MambaFPN	0.872	1.16	0.909	1.98	68.3
With S-MambaFPN	0.897	1.05	0.927	1.80	74.3

TABLE VI : Analysis of S-MambaFPN on DFC19 and DFC23 datasets using IoU and RMSE (m) metrics.

Method	DFC19 RMSE (m)	DFC23 RMSE (m)
w/o Refinement	1.10	1.84
w Refinement	1.06	1.77

TABLE VII : Effect of the mask-aware height refinement on DFC19 and DFC23 datasets.

Remote Sensing Image Segmentation

Remote sensing image segmentation labels land cover and objects in satellite, aerial, radar, and multispectral imagery.

See all 220 papers

The First Assessment of PhiSat-2 Imagery for Monocular Building Height Estimation
Mar 31, 2026Yanjiao Song, Bowen Cai, Timo Balz +4Remote Sensing Image Understanding Building Footprint Extraction
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University, Wuhan, 430079, Hubei, China · Department of Geomatics Engineering, Institut Teknologi Sepuluh Nopember, Surabaya, 60111, East Java, Indonesia · Department of Informatics and Information Technology, College of Natural and Applied Sciences (CoNAS), Sokoine University of Agriculture, Morogoro, 3038, Morogoro, Tanzania +2
Morphology-Guided Cross-Task Coupling for Joint Building Height and Footprint Estimation
May 6, 2026Jinzhen Han, JinByeong Lee, Jisung Kim +1Building Footprint Extraction Multi-Task Learning
Department of Civil, Architectural and Environmental System Engineering, Sungkyunkwan University, 2066 Seobu-ro, Jangan-gu, Suwon, 16419, Republic of Korea · School of Geography, University of Leeds, Leeds, LS2 9JT, United Kingdom
UniBuild: Unified Building Mapping From Multi-Source Optical Remote Sensing Imagery With Detail Decoding and Geometry Regularization
Sep 29, 2026Wei Huang, Chenying Liu, Yilei Shi +1Remote Sensing Image Segmentation Building Footprint Extraction
Chair of Data Science in Earth Observation, Technical University of Munich, 80333 Munich, Germany

Mar 31, 2026J/K move · Enter open · S save

The First Assessment of PhiSat-2 Imagery for Monocular Building Height Estimation

Yanjiao Song, Bowen Cai, Timo Balz, Zhenfeng Shao+3

State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University, Wuhan, 430079, Hubei, China · Department of Geomatics Engineering, Institut Teknologi Sepuluh Nopember, Surabaya, 60111, East Java, Indonesia · Department of Informatics and Information Technology, College of Natural and Applied Sciences (CoNAS), Sokoine University of Agriculture, Morogoro, 3038, Morogoro, Tanzania+2

Remote Sensing Image Understanding Building Footprint Extraction

Open paper AskPDF arXiv

PhiSat-2 provides multispectral observations at a 4.75m spatial resolution across geographically diverse scenes, offering a new source of remote sensing imagery for monocular building-height estimation. To comprehensively assess its utility, this study introduces the PhiSat-2--Height Dataset (PHDataset) and the Two-Stream Ordinal Network (TSONet). PHDataset integrates global PhiSat-2 imagery with open building-height references and contains 9475 co-registered patch pairs from 26 cities worldwide. In TSONet, multiscale decoupled height representation (MDHR) separates adaptive bin generation from spatial assignment, while footprint-conditioned height reconstruction (FCHR) incorporates footprint guidance at feature and reconstruction levels. Under city-disjoint evaluation, TSONet reduces building RMSE and MAE by 3.64% and 4.49%, respectively, relative to the most accurate comparison method originally designed for building-height estimation, while maintaining similar height-derived footprint IoU. Ablations support the combined design for height accuracy and footprint consistency. PhiSat-2 reduces building RMSE by 3.73% and 6.05% relative to the evaluated Sentinel-2 and Landsat 8 configurations, respectively, with all seven bands providing the lowest building-height errors among its tested band subsets. LiDAR-based testing supports PHDataset-style supervision in a held-out city, while selected patch comparisons show finer urban height patterns than coarser global products. These findings highlight the value of PhiSat-2 spatial-spectral information for urban building height mapping, with PHDataset and TSONet providing a foundation for further development.

Abstract

Remote Sensing Image Segmentation

Explore similar work

The First Assessment of PhiSat-2 Imagery for Monocular Building Height Estimation

Morphology-Guided Cross-Task Coupling for Joint Building Height and Footprint Estimation

UniBuild: Unified Building Mapping From Multi-Source Optical Remote Sensing Imagery With Detail Decoding and Geometry Regularization