VoxelFix: Post-Hoc Semantic Correction of Completed 3D Voxel Maps

Sunesh Praveen Raja Sundarasami; Taehyoung Kim; Johannes Scherer; Tomaž Cotič; Sivasubiramaniam Subbiah; Andreas Greiner; Paul Spannaus; Sebastian Houben

cs.CVSep 4, 2026

VoxelFix: Post-Hoc Semantic Correction of Completed 3D Voxel Maps

Authors: Sunesh Praveen Raja Sundarasami, Taehyoung Kim, Johannes Scherer, Tomaž Cotič, Sivasubiramaniam Subbiah, Andreas Greiner, Paul Spannaus, Sebastian Houben

Organizations: Fraunhofer IVI · Hochschule Bonn-Rhein-Sieg · University of Bologna · FAU ·

Abstract

Semantic 3D maps are increasingly constructed automatically for aerial robotics by integrating learned semantic predictions into 3D representations. While this avoids costly manual 3D annotation, errors in the perception and mapping pipeline can persist in the resulting map, reducing its reliability for downstream autonomous tasks. Existing 3D semantic map refinement methods either rely on the original observations, treat occupancy as part of the prediction problem, or apply non-learned local regularization to completed maps. Instead, we study post-hoc semantic correction, asking whether semantic accuracy can be recovered directly from the completed map while keeping its geometry and occupancy fixed. We introduce \method, a graph-based model that corrects voxel labels based on local geometry and neighboring semantic information. To obtain training pairs, we corrupt contiguous regions of annotated OccuFly maps according to class confusions observed in upstream maps. We evaluate \method on completed OccuFly maps generated from predictions of four independently trained 2D segmentation models. \method consistently improves mIoU by 4.23--5.00 percentage points, with gains broadly distributed across the evaluated semantic classes and particularly strong improvements for tree, roof, and wall. Results on an independently reconstructed out-of-distribution aerial scene further suggest that the learned correction can transfer beyond the environments seen during training.

Figures & tables

Figure 1: Map-only post-hoc semantic correction. One common semantic mapping pipeline predicts 2D semantics and back-projects them using depth and camera poses before aggregating the observations into a semantic voxel map. Upstream perception and mapping errors can persist in the completed map. Our setting begins after map construction: source images, depth maps, camera poses, and intermediate mapping state are unavailable, while voxel geometry and occupancy remain fixed. VoxelFix uses only the completed voxel geometry and semantic labels to correct inconsistent semantics.

Method	No Obs.	No Pose	Map	Fixed	Learned
SemanticFusion [ 32 ]	✗	✗	✗	✓	✗
P2Net [ 33 ]	✗	✗	✗	✓	✓
OccFiner [ 37 ]	✗	✗	✗	✗	✓
ESSC-RM [ 54 ]	✓	✓	✗ †	✗	✓
CRF (baseline)	✓	✓	✓	✓	✗
KNN (baseline)	✓	✓	✓	✓	✗

Table 1: Comparison of approaches for improving semantic map quality. No Obs. and No Pose indicate that original observations and pose information are not required; Map denotes operation on a completed semantic map, and Fixed indicates that geometry and occupancy are not modified.

Figure 2: VoxelFix. Occupied voxels form a fixed

k

-NN graph whose node descriptors combine local geometry

\mathbf{g}_{i}

, frozen volumetric embeddings

\mathbf{z}_{i}

, current labels

\hat{y}_{i}

, and class-conditioned geometric agreement features

a_{i},\boldsymbol{\delta}_{i}

. The geometric branch attends to geometric edge attributes

\mathbf{e}_{ij}

, while the semantic branch additionally uses

\mathbf{s}_{ij}

containing class compatibility from the co-occurrence prior

\mathbf{A}

and label-consistency cues. Their outputs are fused by a voxel-wise gate for semantic relabeling. Dashed paths denote training-only corruption, prototype updates, and auxiliary error-detection supervision.

Panel of Figure 3 — Figure 3: Qualitative map-level correction on the fixed OccuFly test scene. From left to right: uncorrected Radix input, KNN, CRF, geometry heuristic, MinkUNet, VoxelFix, and ground truth. VoxelFix more consistently corrects spatially coherent semantic errors while preserving surrounding labels compared with local smoothing and geometry-only refinement.

Method	mIoU $\uparrow$	$\Delta$ mIoU $\uparrow$	Acc. $\uparrow$	ECR $\uparrow$	DR $\downarrow$
Radix (input)	25.80	–	60.46	–	–
KNN	26.14	+0.34	60.89	5.12	2.64
CRF	26.19	+0.39	61.19	10.65	5.75
Geometry Heuristic	28.60	+2.80	66.27	24.51	6.41
MinkUNet	28.52 $\pm$ 1.17	+2.72 $\pm$ 1.17	65.83 $\pm$ 2.34	16.16 $\pm$ 7.09	1.69 $\pm$ 0.82
VoxelFix (Ours)	30.66 $\pm$ 0.55	+4.86 $\pm$ 0.55	68.34 $\pm$ 0.62	25.36 $\pm$ 1.67	3.55 $\pm$ 0.58

Table 2: Map-level semantic correction on the fixed OccuFly test scene using SegFormer-MiT-B3 upstream predictions. Learned models report mean

\pm

std over eight independently trained cross-validation splits.

\Delta

mIoU is measured relative to the uncorrected Radix input map.

Method	Road	Tree	Vehicle	Grass	Wall	Roof	Obstacle	Gravel	Person	Dirt	Bicycle	mIoU	Acc.

Radix (input)	68.53	42.85	42.41	40.25	18.18	52.97	9.37	6.83	0.00	2.43	0.00	25.80	60.46
KNN	69.50	42.49	43.75	40.75	18.16	53.38	10.28	6.91	0.00	2.36	0.00	26.14	60.89
CRF	71.24	41.39	44.71	40.68	17.14	53.53	11.31	6.71	0.00	1.42	0.00	26.19	61.19
Geometry Heuristic	70.73	66.17	31.89	49.07	17.21	52.56	12.02	12.27	0.00	2.71	0.00	28.60	66.27
MinkUNet	70.48 $\pm$ 0.18	61.02 $\pm$ 8.73	45.14 $\pm$ 0.51	46.16 $\pm$ 2.97	19.05 $\pm$ 1.07	53.65 $\pm$ 0.60	8.93 $\pm$ 0.23	6.84 $\pm$ 0.07	0.00	2.48 $\pm$ 0.10	0.00	28.52 $\pm$ 1.17	65.83 $\pm$ 2.34

Table 3: Per-class IoU (%) and overall performance on the fixed OccuFly test scene using SegFormer-MiT-B3 upstream predictions. Only classes present in the test scene are shown. Best and second-best results are shown in bold and underlined , respectively.

Upstream model	Input mIoU	Corrected mIoU $\uparrow$	$\Delta$ mIoU $\uparrow$
SegFormer-MiT-B3	25.80	30.66 $\pm$ 0.55	+4.86 $\pm$ 0.55
UPerNet-SwinV2-T	22.64	27.25 $\pm$ 0.74	+4.61 $\pm$ 0.74
UPerNet-ConvNeXtV2-T	24.88	29.11 $\pm$ 0.47	+4.23 $\pm$ 0.47
Mask2Former-Swin-T	23.70	28.69 $\pm$ 0.41	+5.00 $\pm$ 0.41

Table 4: Effect of the upstream segmentation model on map-level correction on the fixed OccuFly test scene. Corrected results show mean

\pm

std over eight cross-validation splits.

Method	mIoU $\uparrow$	$\Delta$ mIoU $\uparrow$	ECR $\uparrow$	DR $\downarrow$
Radix (input)	30.95	–	–	–
KNN	31.98	+1.04	11.95	1.65
CRF	31.95	+1.00	25.88	3.72
Geometry Heuristic	31.10	+0.16	7.01	1.87
MinkUNet	31.58	+0.63	6.41	0.72
VoxelFix (Ours)	32.66	+1.71	15.65	1.03

Table 5: OOD map-level correction on an independently reconstructed scene using Mask2Former upstream predictions. The scene is not used during training or model selection.

Panel of Figure 4 — Figure 4: Qualitative map-level correction on the OOD aerial scene using Mask2Former upstream predictions. From left to right: uncorrected Radix input, KNN, CRF, geometry heuristic, MinkUNet, VoxelFix, and ground truth. VoxelFix corrects several semantic inconsistencies on previously unseen scene geometry while largely preserving correctly labeled regions.

Variant	mIoU $\uparrow$	$\Delta$ mIoU $\uparrow$	ECR $\uparrow$
w/o UNet embeddings (6D geometry only)	27.43 $\pm$ 1.43	+3.18 $\pm$ 0.36	12.31 $\pm$ 2.01
single-layer GAT	27.65 $\pm$ 1.43	+3.39 $\pm$ 0.21	13.79 $\pm$ 1.60
w/o geometric branch	27.43 $\pm$ 1.47	+3.17 $\pm$ 0.49	14.77 $\pm$ 3.00
w/o gated fusion	27.88 $\pm$ 1.47	+3.63 $\pm$ 0.39	15.81 $\pm$ 2.68
reversed noise curriculum (0.05 $\to$ 0.50)	25.80 $\pm$ 1.35	+1.55 $\pm$ 0.19	6.73 $\pm$ 1.24
low-noise curriculum (0.10 $\to$ 0.02)	25.41 $\pm$ 1.20	+1.15 $\pm$ 0.28	4.95 $\pm$ 1.16

Table 6: Ablation of the main VoxelFix components on the fixed OccuFly test scene. Results report mean

\pm

std across the four upstream segmentation models, evaluated on a single held-out split of the 7/1/1 protocol.

Parameter	Value
Graph neighbors $k$	30
Structure-tensor neighbors $k_{\mathrm{geo}}$	50
Density radius $r$	3 m
Shared hidden dimension $d$	32
GATv2 layers per branch	2
Attention heads per layer	2

Table 7: VoxelFix architectural configuration.

Parameter	Value
Optimizer	AdamW
Initial learning rate	$1\times 10^{-3}$
Weight decay	$1\times 10^{-4}$
Epochs	500 max., early stopping (patience 20)
LR scheduler	ReduceLROnPlateau ( $\times 0.5$ , patience 5)
Batch / subgraph size	1 scene graph / tile

Table 8: VoxelFix training configuration.

Stage	Configuration / time
Hardware	1 $\times$ NVIDIA A100 80 GB
STPLS3D pretraining	$\sim$ 8 h
OccuFly fine-tuning	$\sim$ 1 h
VoxelFix training	$\sim$ 3 h per split

Table 9: Training compute and approximate wall-clock cost.

Parameter	Value
Sparse U-Net backbone	MinkowskiEngine sparse U-Net, additive skip connections
Input voxel features	6D geometric descriptor (height, planarity, linearity, sphericity, verticality, density)
Encoder stages	4 (1 stride-1 stem + 3 stride-2 blocks)
Channel configuration	$6\!\rightarrow\!32\!\rightarrow\!64\!\rightarrow\!128\!\rightarrow\!256$ (encoder), $256\!\rightarrow\!128\!\rightarrow\!64\!\rightarrow\!32$ (decoder)
Output embedding dimension	16
Voxel size / quantization	0.5 m

Table 10: Geometry-encoder architecture.

Parameter	Value
Training split	60 / 6 / 1 scenes (train / val / test, random)
Number of classes	8 (grouped)
Optimizer	AdamW
Learning rate	$1\times 10^{-3}$
Weight decay	$1\times 10^{-4}$
Number of epochs	1000 max., early stop (patience 250)

Table 11: STPLS3D pretraining configuration for the geometry encoder.

Parameter	Value
Optimizer	AdamW
Learning rate	$5\times 10^{-4}$ , cosine decay after 5-epoch warmup
Weight decay	$1\times 10^{-5}$
Number of epochs	200 max., early stop (patience 40)
Loss	Cross-entropy
Class weighting	Effective-number ( $\beta=0.999$ ), clipped to $[0.25,4.0]$

Table 12: OccuFly fine-tuning configuration for the geometry encoder.

Figure 5: Distribution of semantic substitutions for synthetic corruption. The heatmap visualizes the transition probabilities derived from our heuristically constructed class-transition mapping. Plausible substitutions are restricted to the most prominent voxel-level confusions observed in the training scenes, resulting in a targeted prior for realistic synthetic errors.

Parameter	Value
Connectivity / neighborhood definition	Euclidean ball around a random seed voxel (KD-tree radius query), not graph-connected growth
Minimum patch size	radius $2.5$ voxels ( $1.25$ m)
Maximum patch size	radius $7.5$ voxels ( $3.75$ m)
Patch-size distribution	radius $=5\times u$ , $u\sim\mathcal{U}(0.5,1.5)$
Maximum patches per map	$\lfloor N_{\mathrm{target}}/50\rfloor$ , $N_{\mathrm{target}}=\text{rate}\times\|V\|$
Overlapping patches allowed	Yes; patch centers sampled independently, no overlap exclusion

Table 13: Synthetic corruption parameters.

Panel of Figure 6 — Figure 6: Examples of confusion-guided synthetic corruptions used for training. Spatially contiguous voxel regions are selected from clean semantic maps and relabeled according to class-transition probabilities derived from voxel-level upstream confusion statistics. Unlike independent label flipping, the resulting corruptions form coherent erroneous regions resembling those observed in completed upstream maps.

Dataset	unlabeled	road	dirt	gravel	rock	grass	vegetation	tree	ground-obstacle	animals	person	bicycle	vehicle	water	boat	building	roof	sky	drone	train-track	cable	cable-tower	wind-turbine-blade	wind-turbine-tower	walkway	bridge	airplanes
CART [ 24 ]	$\checkmark$	$\checkmark$	$\checkmark$		$\checkmark$		$\checkmark$	$\checkmark$			$\checkmark$		$\checkmark$	$\checkmark$		$\checkmark$		$\checkmark$
ICG [ 20 ]	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$		$\checkmark$	$\checkmark$
OkutSwiss [ 38 ]	$\checkmark$	$\checkmark$					$\checkmark$		$\checkmark$		$\checkmark$		$\checkmark$	$\checkmark$		$\checkmark$				$\checkmark$
UAVid [ 31 ]	$\checkmark$	$\checkmark$					$\checkmark$	$\checkmark$			$\checkmark$		$\checkmark$	$\checkmark$		$\checkmark$	$\checkmark$
UDD [ 6 ]	$\checkmark$	$\checkmark$					$\checkmark$						$\checkmark$	$\checkmark$		$\checkmark$	$\checkmark$
VDD [ 3 ]	$\checkmark$	$\checkmark$					$\checkmark$						$\checkmark$	$\checkmark$		$\checkmark$	$\checkmark$

Table 14: Semantic class coverage across the datasets used to train the upstream segmentation models. The unlabeled class is mapped to the ignore index and is not part of the 26-class taxonomy.

Method	Backbone	mIoU
		Val	Unseen
SegFormer [ 51 ]	MiT-B3	66.56	38.91
UPerNet [ 50 ]	Swin-v2-T	66.53	38.11
UPerNet [ 50 ]	ConvNeXt-v2-T	69.23	38.05
Mask2Former [ 7 ]	Swin-T	69.55	36.60

Table 15: Performance of the upstream segmentation models on the validation and unseen datasets.

Table 16: Semantic taxonomy alignment from upstream segmentation to VoxelFix. The 26-class upstream predictions are first aligned with the OccuFly taxonomy and subsequently reduced to the 12 classes used for VoxelFix training and evaluation. Classes without a corresponding target class are ignored.

Parameter	Value
Number of neighbors	5
Distance metric	Euclidean, in metres
Include query voxel	Yes (counted within $k=5$ )
Distance weighting	None (uniform vote)
Number of iterations	1
Tie-breaking rule	Keep the voxel’s current label

Table 17: KNN baseline parameters.

Parameter	Value
Unary potential	Upstream scalar confidence on the input label, remaining mass uniform over the other 11 classes
Pairwise neighborhood $k$	30 (self excluded)
Spatial standard deviation $\sigma$	$1.5$ m
Edge weighting	Gaussian, $\exp(-d^{2}/2\sigma^{2})$ , row-normalized
Pairwise weight (Potts)	$15.0$
Compatibility term	Potts: class-independent, uniform penalty for unlike labels

Table 18: CRF baseline parameters.

Parameter	Value
Ground separation (CSF)	Cloth resolution $1.0$ , rigidness $5$ , classification threshold $0.3$ m, 2500 iterations
Ground correction	KNN majority vote over valid ground classes; $k=50$ , widened to $k=100$ when too few valid neighbors are found
Above-ground clustering	DBSCAN, $\epsilon=0.5$ m, minimum 5 samples
Cluster relabeling	Majority vote within each cluster; dense components merged into a single cluster

Table 19: Geometry-based heuristic parameters.

Panel of Figure 7 — Figure 7: Qualitative correction on held-out OccuFly validation scenes using SegFormer-MiT-B3 upstream predictions. Each scene is corrected by the model of the split in which it was held out. From left to right: uncorrected Radix input, VoxelFix, and ground truth.

Parameter	Value
Architecture	Sparse U-Net (MinkowskiEngine), 4 stride-2 stages, 8 channels throughout, additive skips; 20,395 parameters
Input features	12-D one-hot of the input label map only (no geometry, no embeddings)
Voxel size	$0.5$ m
Optimizer	AdamW
Learning rate	$1\times 10^{-3}$ , ReduceLROnPlateau ( $\times 0.5$ , patience 5)
Weight decay	$1\times 10^{-4}$

Table 20: MinkUNet baseline configuration.

Method	Road	Tree	Vehicle	Grass	Wall	Roof	Obstacle	Gravel	Person	Dirt	mIoU

Radix (input)	76.57	5.65	23.07	87.41	34.92	54.92	21.68	0.00	5.00	0.28	30.95
KNN	78.75	6.52	26.33	88.50	35.75	56.32	23.25	0.00	3.95	0.47	31.98
CRF	80.35	8.91	30.56	90.31	32.38	57.84	19.13	0.00	0.00	0.00	31.95
Geometry Heuristic	78.94	5.14	25.90	85.67	33.68	56.50	19.94	0.00	5.00	0.27	31.10
MinkUNet	79.18	6.03	26.53	87.35	34.92	55.01	20.72	0.00	5.75	0.28	31.58

Table 21: Per-class IoU (%) on the independently reconstructed OOD scene using single-split Mask2Former upstream predictions. Only classes present in the scene are shown; mIoU is averaged over these 10 classes and is therefore not directly comparable to the in-distribution values in Tab. 3 . Best and second-best results are shown in bold and underlined , respectively.

Panel of Figure 8 — Figure 8: Qualitative correction on the independently reconstructed OOD scene using Mask2Former upstream predictions. From left to right: uncorrected Radix input, VoxelFix, and ground truth. Wall and roof regions are recovered along the building structures, and spurious water labels on the grass surface are removed. Remaining errors concentrate in structures for which the completed map carries no distinguishing evidence.

Split	Training scenes	Validation scene	Fixed test scene
1	1, 2, 7, 3, 9, 4, 5	6	8
2	6, 2, 7, 3, 9, 4, 5	1	8
3	6, 1, 7, 3, 9, 4, 5	2	8
4	6, 1, 2, 3, 9, 4, 5	7	8
5	6, 1, 2, 7, 9, 4, 5	3	8
6	6, 1, 2, 7, 3, 4, 5	9	8

Table 22: OccuFly train/validation/test composition for the eight 7/1/1 cross-scene splits.

3D Semantic Segmentation

3D semantic segmentation assigns meaning to points, voxels, meshes, or reconstructed scene elements for spatial understanding.

See all 137 papers

Point-Level Refinable Semantic Mapping for Spatial Understanding
Oct 22, 2026Taimeng Fu, Yi Du, Zitong Zhan +43D Semantic Mapping Semantic Mapping
Spatial AI & Robotics Lab, Department of Computer Science and Engineering, University at Buffalo, USA.
VEOcc++: Learning Persistent Voxel Feature Maps for Embodied Online Semantic Occupancy Mapping and Offline Refinement
Oct 20, 2026Ruoyu Wang, Jiahan Li, Yong Liu +1Semantic Occupancy Mapping 3D Mapping
APRIL Lab, Institute of Cyber-Systems and Control, Zhejiang University, China
FUS3DMaps: Scalable and Accurate Open-Vocabulary Semantic Mapping by 3D Fusion of Voxel- and Instance-Level Layers
May 5, 2026Timon Homberger, Finn Lukas Busch, Jesús Gerardo Ortega Peimbert +2Open-Vocabulary 3D Segmentation 3D Semantic Segmentation
Division of Robotics, Perception, and Learning, KTH Royal Institute of Technology, Sweden

Oct 22, 2026J/K move · Enter open · S save

Point-Level Refinable Semantic Mapping for Spatial Understanding

Taimeng Fu, Yi Du, Zitong Zhan, Qiwei Du+3

Spatial AI & Robotics Lab, Department of Computer Science and Engineering, University at Buffalo, USA.

3D Semantic Mapping Semantic Mapping Robotic Spatial Memory 3D Scene Understanding

Open paper AskPDF arXiv

Open-vocabulary 3D semantic maps have emerged as a key spatial memory representation for long-horizon robot autonomy, enabling robots to reason about objects, their spatial relationships, and past observations across time. However, existing object-centric semantic maps are primarily designed for static environments and rely on an append-only point integration strategy, where newly observed points are accumulated but previously integrated points are rarely reevaluated. Consequently, perception errors accumulate over time, and outdated points persist after objects move or disappear. Although recent dynamic mapping methods update object-level representations in changing scenes, they do not explicitly revise noisy or obsolete points within each object. We propose an online refinable semantic mapping framework that continuously refines object representations through probabilistic point-level confidence updating. Each object point maintains a confidence score that is updated online using multi-view temporal evidence: consistent observations reinforce reliable points, while inconsistent segmentation masks, depth measurements, and visibility cues suppress spurious and outdated points. Combined with object tracking and re-identification to preserve instance identities under motion and relocation, our framework enables object-centric semantic maps to be continuously refined rather than monotonically accumulated. Experiments demonstrate improved mapping accuracy over the previous method by 3.46% on Replica and 2.55% on ScanNet, show robustness in dynamic environments and under perception noise, and provide an effective spatial memory for object navigation and embodied question-answering with success rates of 82.4% and 76.7%.

Abstract

3D Semantic Segmentation

Explore similar work

Point-Level Refinable Semantic Mapping for Spatial Understanding

VEOcc++: Learning Persistent Voxel Feature Maps for Embodied Online Semantic Occupancy Mapping and Offline Refinement

FUS3DMaps: Scalable and Accurate Open-Vocabulary Semantic Mapping by 3D Fusion of Voxel- and Instance-Level Layers