GaussianSSC: Triplane-Guided Directional Gaussian Fields for 3D Semantic Completion

Ruiqi Xian; Jing Liang; He Yin; Xuewei Qi; Dinesh Manocha

cs.ROMar 23, 2026

GaussianSSC: Triplane-Guided Directional Gaussian Fields for 3D Semantic Completion

Authors: Ruiqi Xian, Jing Liang, He Yin, Xuewei Qi, Dinesh Manocha

Abstract

We present GaussianSSC, a two-stage, grid-native and triplane-guided approach to semantic scene completion (SSC) that injects the benefits of Gaussians without replacing the voxel grid or maintaining a separate Gaussian set. We introduce Gaussian Anchoring, a sub-pixel, Gaussian-weighted image aggregation over fused FPN features that tightens voxel--image alignment and improves monocular occupancy estimation. We further convert point-like voxel features into a learned per-voxel Gaussian field and refine triplane features via a triplane-aligned Gaussian--Triplane Refinement module that combines local gathering (target-centric) and global aggregation (source-centric). This directional, anisotropic support captures surface tangency, scale, and occlusion-aware asymmetry while preserving the efficiency of triplane representations. On SemanticKITTI~\cite{behley2019semantickitti}, GaussianSSC improves Stage~~1 occupancy by +1.0% Recall, +2.0% Precision, and +1.8% IoU over state-of-the-art baselines, and improves Stage~~2 semantic prediction by +1.8% IoU and +0.8% mIoU.

Figures & tables

Fig. 1: Our GaussianSSC takes a monocular input, learns semantics efficiently via a triplane representation, then refines these features with Gaussian primitives to capture geometry and fine details, before fusing into a voxel space for 3D semantic scene completion.

Fig. 2: Overview of Our GaussianSSC: We present GaussianSSC, a two‑stage pipeline for semantic scene completion. In Stage 1, it predicts an occupancy map from a monocular image, which serves as a structural prior for Stage 2. In Stage 2, we instantiate Gaussian embeddings at voxel locations, gate them by the occupancy priors, and condition three orthogonal triplanes to multi‑scale image features. We then decode a 3D Gaussian per voxel and splat it into the triplanes to perform Gaussian–Triplane refinement to produce stronger semantic features (see Figure 4 ). The refined triplane features are lifted and merged back to voxel space, where a semantic head predicts the final dense semantic map.

Fig. 3: Illustration of Stage1: Stage 1: From a monocular image, we build a query‑conditioned triplane to obtain voxel descriptors, then apply Gaussian Anchoring—per‑voxel Gaussian windowing on a fused FPN map—to gather sub‑pixel image evidence, fuse it via a gated residual, and predict occupancy with a lightweight 3D head.

Fig. 4: Illustration of Gaussian-Triplane Refinement: We associate each voxel with a Gaussian centered at the voxel and project this Gaussian onto the three orthogonal triplanes. On each plane, we refine the feature at the projected mean with two complementary steps: (i) local gathering, which anchors the feature by aggregating neighboring evidence within the Gaussian field; and (ii) global aggregation, which shares semantic information from all other locations whose Gaussians cover that point.

Fig. 5: Visualization results: The figure compares semantic maps generated by different approaches. The reference LiDAR sample is constructed from multiple consecutive LiDAR frames. Blue boxes highlight cases where our method completes the scene more effectively, while red boxes indicate more accurate semantic estimations than other approaches.

Method	Recall	Precision	IoU
VoxFormer [ 8 ]	61.5	71.1	52.0
ETFormer [ 12 ]	71.5	77.8	59.4
GaussianSSC (Ours)	72.5 (+1.0)	79.8 (+2.0)	61.2 (+1.8)

TABLE I: Results of Stage 1 On SemanticKITTI test: Our approach, GaussianSSC, outperforms existing methods by at least 1.8% in IoU and 2% in precision for occupancy map estimation.

Methods	IoU	mIoU	road	sidewalk	parking	other-ground	building	car	truck	bicycle	motorcycle	other-vehicle	vegetation	trunk	terrain	person	bicyclist	motorcyclist	fence	pole	traffic-sign
MonoScene [ 13 ]	34.1	11.0	54.7	27.1	24.8	5.7	14.4	18.8	3.3	0.5	0.7	4.4	14.9	2.4	19.5	1.0	1.4	0.4	11.1	3.3	2.1
TPVFormer [ 19 ]	34.2	11.2	55.1	27.2	27.4	6.5	14.8	19.2	3.7	1.0	0.5	2.3	13.9	2.6	20.4	1.1	2.4	0.3	11.0	2.9	1.5
VoxFormer [ 8 ]	42.9	12.2	53.9	25.3	21.1	5.6	19.8	20.8	3.5	1.0	0.7	3.7	22.4	7.5	21.3	1.4	2.6	0.2	11.1	5.1	4.9
OccFormer [ 24 ]	34.5	12.3	55.9	30.3	31.5	6.5	15.7	21.6	1.2	1.5	1.7	3.2	16.8	3.9	21.3	2.2	1.1	0.2	11.9	3.8	3.7
MonoOcc [ 37 ]	-	13.8	55.2	27.8	25.1	9.7	21.4	23.2	5.2	2.2	1.5	5.4	24.0	8.7	23.0	1.7	2.0	0.2	13.4	5.8	6.4
Symphonies [ 38 ]	42.1	15.0	58.4	29.3	26.9	11.7	24.7	23.6	3.2	3.6	2.6	5.6	24.2	10.0	23.1	3.2	1.9	2.0	16.1	7.7	8.0

TABLE II: Quantitative comparison on SemanticKITTI [ 1 ] . GaussianSSC achieves 53.2/17.1 IoU/mIoU ( +1.8/+0.8 over prior camera-only best) and yields top accuracy on multiple structural categories (e.g., building, car, truck, vegetation, terrain, and pole).

Method	IoU $\uparrow$	mIoU $\uparrow$	Latency (ms) $\downarrow$	Peak Mem. (GB) $\downarrow$
VoxFormer [ 8 ]	50.0	14.9	309.5 $\pm$ 63.0	3.8
Symphonics [ 38 ]	50.5	15.3	612.0 $\pm$ 57.8	4.3
ETFormer [ 12 ]	51.4	16.3	356.6 $\pm$ 61.7	2.3
GaussianSSC (Ours)	53.2	17.1	409.8 $\pm$ 56.3	2.8

TABLE III: Accuracy and efficiency on SemanticKITTI. All results are reported in FP32 with batch size 1 on an RTX A6000.

Method	Recall	Precision	IoU
Triplane Baseline	71.5	77.8	59.4
+ our Gaussian anchoring	72.2 (+0.7)	79.4 (+1.6)	60.8 (+1.4)
+ negative sampling	72.5 (+0.3)	79.8 (+0.4)	61.2 (+0.4)

TABLE IV: Stage-1 ablation on SemanticKITTI. Compared with the baseline, Gaussian anchoring improves occupancy prediction, and negative sampling provides additional gains.

Method	IoU $\uparrow$	mIoU $\uparrow$
Triplane Baseline	51.4	16.3
+ Stage-2 (global-only, $\beta{=}0$ )	52.2 (+0.8)	16.6 (+0.3)
+ Stage-2 (local+global, $\beta{=}0.5$ )	52.6 (+1.2)	16.8 (+0.5)
+ Stage-2 (local-only, $\beta{=}1$ )	52.2 (+0.8)	16.7 (+0.4)
+ Stage-1 & Stage-2 (local+global, $\beta{=}0.5$ )	53.2 (+1.8)	17.1 (+0.8)

TABLE V: Stage-2 ablation on SemanticKITTI. We vary

\beta

to mix global aggregation (

\beta{=}0

) and local gathering (

\beta{=}1

);

\beta{=}0.5

performs best. Stage-1 anchoring further boosts completion.

3D Scene Completion

3D scene completion infers missing geometry, occupancy, semantics, and appearance from partial images, point clouds, or depth scans.

See all 41 papers

Learning Adaptive Semantic Gaussian Allocation for 3D Occupancy
Jul 24, 2026Kanglin Ning, Yiran Zhao, Wenrui Li +43D Gaussian Splatting Semantic Occupancy Mapping
Harbin Institute of Technology · The Suzhou Research Institute of HIT · The PengChengLab
Sparsity-Aware Voxel Attention and Foreground Modulation for 3D Semantic Scene Completion
Apr 7, 2026Yu Xue, Longjun Gao, Yuanqi Su +23D Scene Completion 3D Scene Understanding
Xi’an Jiaotong University, China
Bridging 3D Gaussians and Semantic Occupancy for Comprehensive Open-Vocabulary Scene Understanding from Unposed Images
Jul 2, 2026Hu Zhu, Bohan Li, Xianda Guo +53D Gaussian Splatting Open-Vocabulary 3D Segmentation
Department of Computing, The Hong Kong Polytechnic University, Hong Kong, SAR, China · Eastern Institute of Technology, Ningbo, China · Shanghai Jiao Tong University, Shanghai, China +4

Jul 24, 2026J/K move · Enter open · S save

Learning Adaptive Semantic Gaussian Allocation for 3D Occupancy

Kanglin Ning, Yiran Zhao, Wenrui Li, Houde Quan+3

Harbin Institute of Technology · The Suzhou Research Institute of HIT · The PengChengLab

3D Gaussian Splatting Semantic Occupancy Mapping 3D Occupancy Prediction

Open paper AskPDF arXiv

Semantic 3D Gaussians provide a compact representation for 3D semantic occupancy prediction by rendering semantic primitives into a voxel volume under voxel-wise supervision. Recent methods have improved the modeling ability and efficiency of this representation through more flexible primitive shapes, geometry-guided initialization, and progressive densification. However, these advances mainly determine how primitives are represented, initialized, or added, and do not explicitly address how to select the most useful Gaussians when their total number must be limited to control memory and computation. This imbalance creates an allocation bottleneck: redundant Gaussians remain in simple regions, while difficult regions receive insufficient semantic support. We propose the Semantic Gaussian Allocation Transformer (SAGFormer), which uses Gaussian attributes and local geometric-semantic features to score candidates and select a fixed final Gaussian set. Experiments on nuScenes-SurroundOcc and SSCBench-KITTI-360 show that SAGFormer improves occupancy prediction under the evaluated protocols and yields more semantically consistent and better-utilized Gaussian representations. Under similar final counts and raw coverage, it reduces semantic mixing, strengthens class-consistent voxel support, and produces fewer unused Gaussians. The results indicate that explicit capacity allocation is a useful complement to Gaussian refinement for semantic occupancy prediction.

Abstract

3D Scene Completion

Explore similar work

Learning Adaptive Semantic Gaussian Allocation for 3D Occupancy

Sparsity-Aware Voxel Attention and Foreground Modulation for 3D Semantic Scene Completion

Bridging 3D Gaussians and Semantic Occupancy for Comprehensive Open-Vocabulary Scene Understanding from Unposed Images