cs.ROOct 25, 2026

Cut a Wall, See a Wall: Identity Substitution in Occluded Point-Cloud Perception for Construction Robots

Authors: Tsung-Lung Yang

Organizations: National University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Construction robots must label building elements they can only partly see. We show that when occlusion at test time removes whole regions of a scene, rather than parts of an object as in training, point-cloud segmentation networks do not simply miss the hidden elements: they relabel them, and the errors collapse onto a single sink class. With fixed synthetic layouts the column absorbs up to 70% of all test points; with randomized layouts the sink moves to the stair, predicted at 4.5-5.4x its true share across three training seeds. Baseline mIoU falls to 0.15-0.27 (fixed) and 0.36-0.72 (randomized), and up to 269 whole objects vanish in a single test set. Appending 14 hand-computed geometric features per point (GPE) restores mIoU to 0.54-0.92 and removes 64-100% of whole-object losses in every run, for PointNet++ and a PTv3-style transformer in both layout regimes; on real S3DIS scans it removes every whole-wall loss (10 to 0). In an as-built column check computed from the predictions, GPE raises the share of correctly checked views from 17-58% to 71-92%. Feature permutation on fixed, randomized and real scenes explains the pattern: each class relies on whichever cue is stable in its training data, position for height-banded slabs and beams, intrinsic shape for walls and stairs once their position varies. Occlusion augmentation is a strong alternative on mIoU, but in every setting it loses more whole objects than GPE (e.g., 25-35 vs. 0).

Figures & tables

Fig. 1: The two scene families: (a) fixed layout, one column randomized; (c) randomized layout, everything drawn per scene. (b,d) An unseen scene-level cut removes the back and top of the whole scene (grey); in training, only the column itself was ever cut.
  mIoUobjects lost
layoutmodelseentest2test3test2test3
fixedPN++.85 →\to .95.20 →\to .79.15 →\to .65269 →\to 3202 →\to 0
fixedPTv3.96 →\to .98.27 →\to .67.17 →\to .54107 →\to 17123 →\to 16
randomPN++.90 →\to .96.54 →\to .89.36 →\to .82184 →\to 17237 →\to 9
randomPTv3.95 →\to .98.71 →\to .92.55 →\to .9021 →\to 426 →\to 9
TABLE I: mIoU and whole-object losses, baseline →\to +GPE. Seen = test1 (object cuts, as in training); test2/test3 = unseen scene-level cuts. Randomized layouts: mean of three training seeds (ranges in Sec. IV ).
settingtestslabcolumn ( pp )beamwall ( pp )stair
fixed PN++23 →\to 018 →\to 1 (7.6e-5)109 →\to 049 →\to 0 (3.6e-15)90 →\to 2
fixed PN++322 →\to 015 →\to 0 (6.1e-5)66 →\to 043 →\to 0 (2.3e-13)56 →\to 0
fixed PTv320 →\to 048 →\to 17 (7.9e-9)25 →\to 025 →\to 0 (6.0e-8)9 →\to 0
fixed PTv337 →\to 031 →\to 13 (4.0e-5)25 →\to 133 →\to 1 (4.7e-10)27 →\to 1
rand. PN++20 →\to 084 →\to 22 (7.0e-18)9 →\to 187 →\to 0 (1.3e-26)1 →\to 1
rand. PN++30 →\to 0115 →\to 9 (6.7e-31)3 →\to 0116 →\to 0 (2.4e-35)2 →\to 1
TABLE II: Whole-object losses per class, baseline →\to +GPE (exact McNemar pp for column and wall). Randomized rows: first training seed.
layoutmodeltestmissedalarmcorrect
fixedPN++214 →\to 244 →\to 1245 →\to 88
fixedPN++311 →\to 334 →\to 1758 →\to 82
fixedPTv3338 →\to 1333 →\to 1742 →\to 71
randomPN++251 →\to 920 →\to 035 →\to 88 (81–94)
randomPN++382 →\to 155 →\to 017 →\to 77 (63–90)
randomPTv3221 →\to 719 →\to 056 →\to 88 (87–89)
TABLE III: As-built column check from the predictions, baseline →\to +GPE (%). Missed: visible columns with << 50% of points labelled column. Alarm: views with a phantom vertical column member. Correct: views with neither. Randomized rows: mean over three seeds (range of correct for GPE).
Fig. 2: Where the points go (PointNet++). Share of all test points per class: ground truth, baseline prediction and GPE prediction. The baseline over-predicts one sink class, the column in fixed layouts and the stair in randomized layouts; GPE restores the true shares. Annotations give the sink’s over-prediction factor. Randomized panels: first training seed (the other two give 3.5–4.0 ×\times on test2 and 4.7–5.4 ×\times on test3).
Fig. 3: Permutation importance of the PointNet++ GPE model: mean drop in per-view recall when one channel group is shuffled within a view (3 repeats, paired chunk order, all test2 views, first training seed; S3DIS: all 139 test views). Bold = the largest carrier of each class.
settingmIoUobjects lostcolumns lostwalls lost
fixed, test2.20 / .76–.81 / .78271 / 14–19 / 318 / 10–14 / 150 / 0–4 / 0
fixed, test3.15 / .62–.73 / .65202 / 25–35 / 015 / 14–16 / 043 / 2–6 / 0
rand., test2.55 / .69 / .88185 / 63 / 2480 / 22 / 2193 / 24 / 0
rand., test3.36 / .65 / .79235 / 68 / 10115 / 24 / 9115 / 31 / 0
TABLE IV: Augmentation control (PointNet++). Baseline / +augmentation / +GPE. Fixed-layout augmentation: two training seeds. One joint evaluation run, so counts differ from Table II by a few views.
Fig. 4: Augmentation versus GPE (PointNet++). Whiskers: the two training seeds of the fixed-layout augmentation model. On mIoU (left) augmentation can match or beat GPE depending on the seed and regime; on whole-object losses (right, symlog scale) GPE is lower in every setting.
classIoUviewsobjects lostpp
ceiling.80 →\to .871390 →\to 0–
floor.74 →\to .901040 →\to 0–
wall.33 →\to .5513910 →\to 0.002
beam.40 →\to .4013627 →\to 23.50 (n.s.)
column.09 →\to .177718 →\to 8.064
mIoU.47 →\to .58   
TABLE V: S3DIS Area 1 (PointNet++). IoU, baseline →\to +GPE, and whole-object losses with exact McNemar pp .
More in this topic

3D Point Cloud Segmentation

3D point cloud segmentation assigns semantic, instance, or part labels to points for scene understanding, mapping, robotics, and scientific measurement.

See all 70 papers

Explore similar work

CardsList
Oct 21, 2026

CRAB: Cuboid-based Representation for As-built BIM from Point Clouds

As-built BIM reconstruction from point clouds demands semantic, volumetric, parametric, and editable building components, beyond 2D layouts or planar abstractions. Yet existing scan-to-BIM pipelines often rely on handcrafted rules, heuristic optimization, or manual correction, while learning-based methods fall short of recovering wall thickness, opening geometry, and component-level BIM structure. We introduce CRAB, a BIM-ready cuboid reconstruction framework that encodes walls, doors, windows, beams, and columns as compact architectural cuboids with IFC-compatible semantics and parameters. CRAB bridges point clouds and editable BIM by combining ProposalNet, which predicts coarse component cuboids, with AssemblyNet, a relation-aware Flow Matching model that refines geometry, predicts structural relationships, and enables automatic structural correction and human-guided editing. To address limited cuboid-level supervision, we extend Structured3D with component-wise cuboid annotations and construct cuboid-based procedural scenes. Experiments show CRAB surpasses floor plan and plane-based alternatives in component completeness, geometric accuracy, and generalization. For interior wall thickness estimation, CRAB achieves a mean error of 2.80 cm, with 79.50% of matched walls within 3 cm. The project page is available at https://vcc.tech/research/2026/CRAB.

Haoran Xiong, Jing Wang, Shanshan Pan +3

Guangdong Provincial Key Laboratory of Visual Media and Multidimensional Intelligence, College of Computer Science and Software Engineering, Shenzhen University, China, Shenzhen · University of Guelph, Canada, Guelph

Building Information Modeling
AskPDF
Oct 27, 2026

TopoField: Technical Report for ECCV 2026 SceneFun3D Challenge

Functional segmentation requires locating small interactive parts, such as knobs, buttons, and sockets, in indoor point clouds and returning confidence-ranked instance masks. A point-wise semantic predictor produces a continuous probability field, whereas the benchmark evaluates a discrete set of scored instances. This creates a substantial field-to-instance gap. We present TopoField, a deterministic decoder built on top of VOLT. TopoField first converts 13 test-time forward passes into a semantic consensus field, then decodes instance candidates from the consensus geometry, and finally ranks the candidates in a class-aware manner. The augmentation stage retains the processed-point identity of every fragment prediction, accumulates probabilities in a common point array, normalizes by coverage count, and fuses the resulting caches. The decoding stage separates high-confidence cores from broader support regions, constructs a symmetric radius-pruned point graph, and uses a multi-source graph-geodesic partition to split support regions that contain multiple seeds. On 561 ground-truth instances across 30 validation scenes, an IoU-0.5-valid candidate exists for 67.2% of instances, and top-1 coverage is 20.0%. These complementary measurements motivate joint attention to candidate construction and ranking. TopoField achieves 28.286% AP, 41.383% AP50, and 47.558% AP25. TopoField achieved second place on the ECCV 2026 SceneFun3D Challenge.

Zhiwei Chen, Zhiheng Fu, Zixu Li +3

Shandong University

3D Point Cloud Segmentation3D Instance Segmentation3D Scene Understanding
AskPDF
Sep 25, 2026

Transformer-based Monte Carlo Localization in Construction Meshes

To be able to perform inspection or digitization tasks, mobile robots on construction sites must be able to localize themselves reliably with respect to a global reference frame that is shared with a building map. Similar room layouts and low-texture surfaces pose a challenge for existing LiDAR- and vision-based localization methods. We approach this problem with a LiDAR-based global relocalization system that estimates the robot's pose relative to a building mesh and combines a PointNet++ encoder with a place recognition decoder, whose outputs serve as a learned observation model within a Monte Carlo Localization (MCL) framework. The pipeline is trained exclusively on synthetic LiDAR scans obtained by simulating the robot's sensors inside the building mesh. Our approach is robust in ambiguous environments due to an uncertainty-aware decoder that scales positional likelihoods and a resampling strategy that injects model hypotheses into the particle set, enabling recovery from potential particle depletion. Evaluations on real-world datasets show that our method outperforms both diffusion-based and ScanContext++ baselines while maintaining fast inference (18 ms per call), demonstrating the practicality of synthetic-data training for mesh-referenced global localization in construction robotics.

Linus Kramer, William Talbot, Olga Vysotska +1

Robotic Systems Laboratory, ETH Zurich, Zurich 8092, Switzerland

Robot LocalizationLiDAR LocalizationMap-Based LocalizationRobotic ConstructionLiDAR Place Recognition
AskPDF