Cut a Wall, See a Wall: Identity Substitution in Occluded Point-Cloud Perception for Construction Robots

Tsung-Lung Yang

cs.ROOct 25, 2026

Cut a Wall, See a Wall: Identity Substitution in Occluded Point-Cloud Perception for Construction Robots

Authors: Tsung-Lung Yang

Organizations: National University

Abstract

Construction robots must label building elements they can only partly see. We show that when occlusion at test time removes whole regions of a scene, rather than parts of an object as in training, point-cloud segmentation networks do not simply miss the hidden elements: they relabel them, and the errors collapse onto a single sink class. With fixed synthetic layouts the column absorbs up to 70% of all test points; with randomized layouts the sink moves to the stair, predicted at 4.5-5.4x its true share across three training seeds. Baseline mIoU falls to 0.15-0.27 (fixed) and 0.36-0.72 (randomized), and up to 269 whole objects vanish in a single test set. Appending 14 hand-computed geometric features per point (GPE) restores mIoU to 0.54-0.92 and removes 64-100% of whole-object losses in every run, for PointNet++ and a PTv3-style transformer in both layout regimes; on real S3DIS scans it removes every whole-wall loss (10 to 0). In an as-built column check computed from the predictions, GPE raises the share of correctly checked views from 17-58% to 71-92%. Feature permutation on fixed, randomized and real scenes explains the pattern: each class relies on whichever cue is stable in its training data, position for height-banded slabs and beams, intrinsic shape for walls and stairs once their position varies. Occlusion augmentation is a strong alternative on mIoU, but in every setting it loses more whole objects than GPE (e.g., 25-35 vs. 0).

Figures & tables

Fig. 1: The two scene families: (a) fixed layout, one column randomized; (c) randomized layout, everything drawn per scene. (b,d) An unseen scene-level cut removes the back and top of the whole scene (grey); in training, only the column itself was ever cut.

		mIoU			objects lost
layout	model	seen	test2	test3	test2	test3
fixed	PN++	.85 $\to$ .95	.20 $\to$ .79	.15 $\to$ .65	269 $\to$ 3	202 $\to$ 0
fixed	PTv3	.96 $\to$ .98	.27 $\to$ .67	.17 $\to$ .54	107 $\to$ 17	123 $\to$ 16
random	PN++	.90 $\to$ .96	.54 $\to$ .89	.36 $\to$ .82	184 $\to$ 17	237 $\to$ 9
random	PTv3	.95 $\to$ .98	.71 $\to$ .92	.55 $\to$ .90	21 $\to$ 4	26 $\to$ 9

TABLE I: mIoU and whole-object losses, baseline

\to

+GPE. Seen = test1 (object cuts, as in training); test2/test3 = unseen scene-level cuts. Randomized layouts: mean of three training seeds (ranges in Sec. IV ).

setting	test	slab	column ( $p$ )	beam	wall ( $p$ )	stair
fixed PN++	2	3 $\to$ 0	18 $\to$ 1 (7.6e-5)	109 $\to$ 0	49 $\to$ 0 (3.6e-15)	90 $\to$ 2
fixed PN++	3	22 $\to$ 0	15 $\to$ 0 (6.1e-5)	66 $\to$ 0	43 $\to$ 0 (2.3e-13)	56 $\to$ 0
fixed PTv3	2	0 $\to$ 0	48 $\to$ 17 (7.9e-9)	25 $\to$ 0	25 $\to$ 0 (6.0e-8)	9 $\to$ 0
fixed PTv3	3	7 $\to$ 0	31 $\to$ 13 (4.0e-5)	25 $\to$ 1	33 $\to$ 1 (4.7e-10)	27 $\to$ 1
rand. PN++	2	0 $\to$ 0	84 $\to$ 22 (7.0e-18)	9 $\to$ 1	87 $\to$ 0 (1.3e-26)	1 $\to$ 1
rand. PN++	3	0 $\to$ 0	115 $\to$ 9 (6.7e-31)	3 $\to$ 0	116 $\to$ 0 (2.4e-35)	2 $\to$ 1

TABLE II: Whole-object losses per class, baseline

\to

+GPE (exact McNemar

p

for column and wall). Randomized rows: first training seed.

layout	model	test	missed	alarm	correct
fixed	PN++	2	14 $\to$ 2	44 $\to$ 12	45 $\to$ 88
fixed	PN++	3	11 $\to$ 3	34 $\to$ 17	58 $\to$ 82
fixed	PTv3	3	38 $\to$ 13	33 $\to$ 17	42 $\to$ 71
random	PN++	2	51 $\to$ 9	20 $\to$ 0	35 $\to$ 88 (81–94)
random	PN++	3	82 $\to$ 15	5 $\to$ 0	17 $\to$ 77 (63–90)
random	PTv3	2	21 $\to$ 7	19 $\to$ 0	56 $\to$ 88 (87–89)

TABLE III: As-built column check from the predictions, baseline

\to

+GPE (%). Missed: visible columns with

<

50% of points labelled column. Alarm: views with a phantom vertical column member. Correct: views with neither. Randomized rows: mean over three seeds (range of correct for GPE).

Fig. 2: Where the points go (PointNet++). Share of all test points per class: ground truth, baseline prediction and GPE prediction. The baseline over-predicts one sink class, the column in fixed layouts and the stair in randomized layouts; GPE restores the true shares. Annotations give the sink’s over-prediction factor. Randomized panels: first training seed (the other two give 3.5–4.0 $\times$ on test2 and 4.7–5.4 $\times$ on test3).

Fig. 3: Permutation importance of the PointNet++ GPE model: mean drop in per-view recall when one channel group is shuffled within a view (3 repeats, paired chunk order, all test2 views, first training seed; S3DIS: all 139 test views). Bold = the largest carrier of each class.

setting	mIoU	objects lost	columns lost	walls lost
fixed, test2	.20 / .76–.81 / .78	271 / 14–19 / 3	18 / 10–14 / 1	50 / 0–4 / 0
fixed, test3	.15 / .62–.73 / .65	202 / 25–35 / 0	15 / 14–16 / 0	43 / 2–6 / 0
rand., test2	.55 / .69 / .88	185 / 63 / 24	80 / 22 / 21	93 / 24 / 0
rand., test3	.36 / .65 / .79	235 / 68 / 10	115 / 24 / 9	115 / 31 / 0

TABLE IV: Augmentation control (PointNet++). Baseline / +augmentation / +GPE. Fixed-layout augmentation: two training seeds. One joint evaluation run, so counts differ from Table II by a few views.

Fig. 4: Augmentation versus GPE (PointNet++). Whiskers: the two training seeds of the fixed-layout augmentation model. On mIoU (left) augmentation can match or beat GPE depending on the seed and regime; on whole-object losses (right, symlog scale) GPE is lower in every setting.

class	IoU	views	objects lost	$p$
ceiling	.80 $\to$ .87	139	0 $\to$ 0	–
floor	.74 $\to$ .90	104	0 $\to$ 0	–
wall	.33 $\to$ .55	139	10 $\to$ 0	.002
beam	.40 $\to$ .40	136	27 $\to$ 23	.50 (n.s.)
column	.09 $\to$ .17	77	18 $\to$ 8	.064
mIoU	.47 $\to$ .58

TABLE V: S3DIS Area 1 (PointNet++). IoU, baseline

\to

+GPE, and whole-object losses with exact McNemar

p

3D Point Cloud Segmentation

3D point cloud segmentation assigns semantic, instance, or part labels to points for scene understanding, mapping, robotics, and scientific measurement.

See all 70 papers

CRAB: Cuboid-based Representation for As-built BIM from Point Clouds
Oct 21, 2026Haoran Xiong, Jing Wang, Shanshan Pan +3Building Information Modeling
Guangdong Provincial Key Laboratory of Visual Media and Multidimensional Intelligence, College of Computer Science and Software Engineering, Shenzhen University, China, Shenzhen · University of Guelph, Canada, Guelph
TopoField: Technical Report for ECCV 2026 SceneFun3D Challenge
Oct 27, 2026Zhiwei Chen, Zhiheng Fu, Zixu Li +33D Point Cloud Segmentation 3D Instance Segmentation
Shandong University
Transformer-based Monte Carlo Localization in Construction Meshes
Sep 25, 2026Linus Kramer, William Talbot, Olga Vysotska +1Robot Localization LiDAR Localization
Robotic Systems Laboratory, ETH Zurich, Zurich 8092, Switzerland

Oct 21, 2026J/K move · Enter open · S save

CRAB: Cuboid-based Representation for As-built BIM from Point Clouds

Haoran Xiong, Jing Wang, Shanshan Pan, Zihao Yan+2

Guangdong Provincial Key Laboratory of Visual Media and Multidimensional Intelligence, College of Computer Science and Software Engineering, Shenzhen University, China, Shenzhen · University of Guelph, Canada, Guelph

Building Information Modeling

Open paper AskPDF arXiv

As-built BIM reconstruction from point clouds demands semantic, volumetric, parametric, and editable building components, beyond 2D layouts or planar abstractions. Yet existing scan-to-BIM pipelines often rely on handcrafted rules, heuristic optimization, or manual correction, while learning-based methods fall short of recovering wall thickness, opening geometry, and component-level BIM structure. We introduce CRAB, a BIM-ready cuboid reconstruction framework that encodes walls, doors, windows, beams, and columns as compact architectural cuboids with IFC-compatible semantics and parameters. CRAB bridges point clouds and editable BIM by combining ProposalNet, which predicts coarse component cuboids, with AssemblyNet, a relation-aware Flow Matching model that refines geometry, predicts structural relationships, and enables automatic structural correction and human-guided editing. To address limited cuboid-level supervision, we extend Structured3D with component-wise cuboid annotations and construct cuboid-based procedural scenes. Experiments show CRAB surpasses floor plan and plane-based alternatives in component completeness, geometric accuracy, and generalization. For interior wall thickness estimation, CRAB achieves a mean error of 2.80 cm, with 79.50% of matched walls within 3 cm. The project page is available at https://vcc.tech/research/2026/CRAB.

Abstract

3D Point Cloud Segmentation

Explore similar work

CRAB: Cuboid-based Representation for As-built BIM from Point Clouds

TopoField: Technical Report for ECCV 2026 SceneFun3D Challenge

Transformer-based Monte Carlo Localization in Construction Meshes