Indoor Asset Detection in Large Scale 360° Drone-Captured Imagery via 3D Gaussian Splatting

Monica Tang; Avideh Zakhor

cs.CVApr 7, 2026

Indoor Asset Detection in Large Scale 360° Drone-Captured Imagery via 3D Gaussian Splatting

Authors: Monica Tang, Avideh Zakhor

Organizations: UC Berkeley

Abstract

We present an approach for object-level detection and segmentation of target indoor assets in 3D Gaussian Splatting (3DGS) scenes, reconstructed from 360° drone-captured imagery. We introduce a 3D object codebook that jointly leverages mask semantics and spatial information of their corresponding Gaussian primitives to guide multi-view mask association and indoor asset detection. By integrating 2D object detection and segmentation models with semantically and spatially constrained merging procedures, our method aggregates masks from multiple views into coherent 3D object instances. Experiments on two large indoor scenes demonstrate reliable multi-view mask consistency, improving F1 score by 65% over state-of-the-art baselines, and accurate object-level 3D indoor asset detection, achieving an 11% mAP gain over baseline methods.

Figures & tables

Figure 2: Overview of our proposed pipeline.

Panel of Figure 3 — Figure 3: Depth-based processing to determine 3D Gaussians corresponding to a 2D mask. (a) Estimated depth image computed via [ 25 ] . The lamp object shown in (b) and (c) is circled in red. (b) Results from GAGA’s [ 27 ] depth processing method. Gaussians belonging to the ceiling are erroneously included in the lamp object’s set of Gaussians. (c) Results from our depth processing method.

Panel of Figure 4 — Figure 4: Sparse point clouds reconstructed by COLMAP [ 37 ] for both datasets. Camera positions are displayed in red, tracking the drone trajectory during data capture.

Cory 3rd Floor Class Labels
door	drinking fountain	exit sign	fire alarm
fire extinguisher	lamp	television	window
Cory 307 Office Class Labels
bench	bottle	chair	computer keyboard
computer monitor	computer mouse	cupboard	desk
door	exit sign	file cabinet	fire alarm

Table 1: Indoor asset class labels. A total of 8 classes are used for the Cory 3rd Floor dataset and 29 classes for Cory 307 Office.

Pipeline Step	Symbol	Threshold
2B Semantic-Constrained Overlap	$\tau_{overlap}$	0.2
2C $1^{\text{st}}$ Low-Weight Gaussian Filtering	$\tau_{filter1}$	0.4
2D Spatial Merging Overlap	$\tau_{spatial}$	0.3
2E $2^{\text{nd}}$ Low-Weight Gaussian Filtering	$\tau_{filter2}$	0.3
3A Object Filtering	$\tau_{object}$	0.8

Table 2: Thresholds used for each pipeline step from Fig. 2

Method	mIoU	Precision	Recall	F1	Runtime $\downarrow$
	Cory 3rd Floor
Ours	75.84	82.61	84.07	83.33	10min
GAGA [ 27 ]	$9.28$	$73.33$	$9.73$	$17.19$	39min
	Cory 307 Office
Method	mIoU	Precision	Recall	F1	Runtime $\downarrow$
Ours	66.01	67.05	72.54	69.69	1hr 33min

Table 3: Evaluation results for 2D mask association.

	Cory 3rd Floor	Cory 307 Office
Ground Truth	$113$	$244$
Ours	$115$	$264$
GAGA [ 27 ]	$15$	$12$

Table 4: Number of unique masks that appear in the 2D mask-association test sets

Panel of Figure 5 — Figure 5: Mask association results produced by GAGA [ 27 ] using inputs from SAM’s segment-everything mode.

Panel of Figure 6 — Figure 6: Plots of temporal F1 performance comparing our mask association method against GAGA’s [ 27 ] , computed for image batch sizes of 10, 20, and 50. F1 Scores computed from the full test set, reported in Tab. 3 , are plotted as horizontal lines.

	Cory 3rd Floor		Cory 307 Office
Method	mAP $\uparrow$	mLAMR $\downarrow$	mAP $\uparrow$	mLAMR $\downarrow$
Ours	41.78	69.77	41.62	63.92
OWLv2 [ 29 ]	$28.15$	$78.30$	$33.27$	$73.19$

Table 5: Object detection evaluation, comparing our results and baseline OWLv2 [ 29 ] detections.

Panel of Figure 7 — Figure 7: Qualitative results of object detection for the Cory 3rd Floor dataset.

Pipeline	mIoU	Prec.	Rec.	F1	mAP	mLAMR $\downarrow$	mIoU	Prec.	Rec.	F1	mAP	mLAMR $\downarrow$
	Cory 3rd Floor						Cory 307 Office
	2D Mask Association				Object Detection		2D Mask Association				Object Detection
Full	75.84	82.61	84.07	83.33	41.78	69.77	66.01	67.05	72.54	69.69	41.62	63.92
w/o depth processing	48.85	79.22	53.98	64.21	15.07	89.78	52.36	67.49	56.15	61.30	9.69	88.87
w/o semantic constraint	72.36	81.82	79.65	80.72	37.71	72.00	47.47	70.79	51.64	59.72	28.08	75.46
w/o $1^{\text{st}}$ filtering	75.84	82.61	84.07	83.33	39.41	71.94	65.61	66.92	71.31	69.05	41.95	63.46

Table 6: Ablation study for pipeline components. Cells are colored from best (green) to worst (red) per column.

3D Object Detection

3D object detection identifies objects and estimates their locations, sizes, and orientations from images, point clouds, radar, or fused sensors.

See all 181 papers

PointGauss: Point Cloud-Guided Multi-Object Segmentation for Gaussian Splatting
Aug 1, 2025Wentao Sun, Yiping Chen, John S. Zelek +13D Gaussian Splatting 3DGS Segmentation
University of Waterloo, Waterloo, ON N2L 3G1, Canada · Sun Yat-Sen University, Zhuhai 519082, China
Seed2GS: Camera-Free, Training-Free Object Extraction from 3D Gaussian Scenes via a Single Reference-View Grounding
Aug 12, 2026Zongjian Ding, Yudong Gao, Jiale Liu +73D Gaussian Splatting Open-Vocabulary 3D Segmentation
University of Chinese Academy of Sciences · Institute of Information Engineering, Chinese Academy of Sciences · 4The Hong Kong University of Science and Technology +2
OP2GS: Object-Aware 3D Gaussian Splatting with Dual-Opacity Primitives
May 19, 2026Guiyu Liu, Niklas Vaara, Janne Mustaniemi +23D Gaussian Splatting Open-Vocabulary 3D Segmentation
Center for Machine Vision and Signal Analysis, University of Oulu, Finland · Aalto University, Finland

Aug 1, 2025J/K move · Enter open · S save

PointGauss: Point Cloud-Guided Multi-Object Segmentation for Gaussian Splatting

Wentao Sun, Yiping Chen, John S. Zelek, Jonathan Li

University of Waterloo, Waterloo, ON N2L 3G1, Canada · Sun Yat-Sen University, Zhuhai 519082, China

3D Gaussian Splatting 3DGS Segmentation 3D Semantic Segmentation Remote Sensing 3D Instance Segmentation

Open paper AskPDF arXiv

While 3D Gaussian Splatting (3DGS) has established new standards for high-fidelity 3D scene modeling, interpreting massive, unstructured Gaussian primitives into meaningful geospatial entities remains a critical challenge for remote sensing and urban applications. Existing 2D-to-3D distillation methods suffer from projection ambiguities, geometric inconsistencies, and computational bottlenecks when applied to complex, large-scale topographies. Furthermore, current benchmarks lack natively aligned 2D-3D ground truth tailored for splatting-based representations. To address these limitations, we propose PointGauss, a unified 3D-native framework for robust semantic parsing and instance segmentation. By treating Gaussian primitives as unstructured point sets, PointGauss leverages Point Transformer V3 (PTv3) to extract scale-invariant geometric features directly from Gaussian attributes, effectively resolving projection ambiguities. To ensure scalability in building-scale scenes, we introduce an adaptive region-of-interest cropping strategy and an instance-aware distance-constrained rasterization pipeline for pixel-level, view-consistent projection. Additionally, we present SplatSeg-360, the first rigorous cross-scale benchmark tailored for 3DGS, comprising 32 complex scenes with full 360

^\circ

coverage and over 6,300 natively aligned 2D-3D masks. Extensive experiments demonstrate that PointGauss operates in real-time and achieves state-of-the-art performance. Notably, it attains approximately 90% 3D-mIoU in large-scale building scenarios and roughly 80% 2D-mIoU in view-consistent 2D instance segmentation, outperforming baseline methods by 16%. (Code)

Abstract

3D Object Detection

Explore similar work

PointGauss: Point Cloud-Guided Multi-Object Segmentation for Gaussian Splatting

Seed2GS: Camera-Free, Training-Free Object Extraction from 3D Gaussian Scenes via a Single Reference-View Grounding

OP2GS: Object-Aware 3D Gaussian Splatting with Dual-Opacity Primitives