Agentic Building-Aware Satellite Gaussian Splatting for Auditable Urban DSM Reconstruction
Authors: Wentao Sun, Zhengsen Xu, Yiping Chen, John S. Zelek, Jonathan Li
Organizations: University of Waterloo, Department of Systems Design Engineering, Waterloo, Canada · University of Calgary, Department of Geomatics Engineering, Calgary, Canada · Sun Yat-sen University, School of Geospatial Engineering and Science, Zhuhai, China
Urban-scale 3D reconstruction from satellite imagery supports disaster response, city monitoring, and geospatial digital twins, yet neural rendering methods typically optimize average visual fidelity rather than the structures that analysts inspect first: buildings. We present an agentic building-aware satellite Gaussian Splatting workflow that uses Segment Anything-derived building masks as semantic priors and an Agentic Reconstruction Controller to select, verify, and record DSM reconstruction policies. On the DFC2019 JAX_004 scene, building-aware weighting reduces building-region DSM MAE from 0.844 m to 0.806 m, showing that semantic priors can shift reconstruction capacity toward analyst-critical regions. A staged schedule provides a balanced operating point, improving full-scene MAE from 1.362 m to 1.349 m while retaining a building gain. Across four JAX scenes, the Agent selects validated policies for both general DSM and building-focused DSM objectives, and produces building-inventory metadata and per-scene decision records. The system combines semantic priors, policy selection, region-specific DSM metrics, and DSM-derived GIS surface products for auditable urban 3D analysis.
Figures & tables
Family
Main emphasis
Difference in this work
Sat-NeRF / EO-NeRF
RPC NeRF, shadows, DSM/NVS
Adds building-task prior and Agent policy
SAT-NGP / EOGS
Efficient satellite neural/GS reconstruction
Uses EOGS as engine, adds semantic product control
Recent satellite GS
Season, shadow, sparse-view, or generalization modules
Selects AOI-specific building-aware DSM products
Surface-oriented GS
Explicit surface and mesh extraction
Reports DSM-derived GIS surface previews and audit records
Table 1: External method positioning. The comparison is by system role rather than a direct SOTA leaderboard because the proposed contribution is a semantic-prior and Agent-control layer around EOGS.
Figure 1: System overview. Multi-view satellite images and SAM-derived building masks feed candidate EOGS reconstructions. The Agentic Reconstruction Controller observes mask QA and inventory metadata, acts by selecting a policy, verifies region-specific DSM metrics, and reports an auditable product bundle.
Deployment item
System contract
Target user
Geospatial analyst inspecting urban DSMs and 3D surfaces
Input
Multi-view satellite images, camera models, optional SAM-derived masks
Select baseline, staged, or building-specialist semantic schedule
Agent verifier
Report selected policy, full/building/non-building metrics, and decision trace
Table 2: Deployment contract for the system workflow.
Scene
Variant
Full MAE
Bldg. MAE
Veg. MAE
Non-bldg. MAE
JAX_004
Baseline
1.3625
0.8440
3.2068
1.4238
JAX_004
SAM 1.3
1.3615
0.8326
3.2307
1.4241
JAX_004
SAM 2.0
1.3757
0.8061
3.1939
1.4432
JAX_004
SAM 2.5
1.3515
0.8077
3.1493
1.4159
JAX_004
Staged
1.3486
0.8347
3.1710
1.4094
JAX_068
Baseline
1.0926
1.0478
1.6069
1.1266
Table 3: DSM MAE in meters. JAX_004 shows the building-region gain, while the additional scenes support readiness-policy selection for semantic weighting.
Figure 2: Building/global operating points and cross-scene regional DSM deltas. Deltas below zero indicate improvement over each baseline.
Figure 3: Mask-QA diagnostics. Coverage provides an interpretable prior-quality signal, and validation ablations complete the operating-policy decision.
Scene
Bldg. area
Components
SAM cov.
General policy
Full MAE
Building policy
Bldg. MAE
JAX_004
6,910 m 2 / 10.5%
63
0.131
Staged
1.3486
Static 2.0
0.8061
JAX_068
28,222 m 2 / 43.1%
9
0.405
Baseline
1.0926
Baseline
1.0478
JAX_214
32,959 m 2 / 50.3%
3
0.442
Baseline
1.7493
Baseline
1.3635
JAX_260
5,626 m 2 / 8.6%
22
0.127
Baseline
1.5502
Baseline
0.8303
Table 4: Agent-generated AOI reconstruction report. Building area and component count are benchmark-grid inventory metadata (CLS=6, 0.5 m grid, connected components ≥ 25 m 2 ). The Agent selects the validated policy for both general DSM and building-focused DSM objectives on all four scenes.
School of Computer Science, China University of Geosciences (Wuhan), Wuhan, 430074, China · School of Architecture and Urban Planning, Shenzhen University, Shenzhen, 518060, China · Guangdong Key Laboratory of Urban Informatics, Shenzhen University, Shenzhen, 518060, China +1