Generalizing Visual Geometry Priors to Sparse Gaussian Occupancy Prediction

Changqing Zhou; Yueru Luo; Changhao Chen

cs.CVFeb 25, 2026

Generalizing Visual Geometry Priors to Sparse Gaussian Occupancy Prediction

Authors: Changqing Zhou, Yueru Luo, Changhao Chen

Organizations: The Hong Kong University of Science and Technology (Guangzhou) · The Chinese University of Hong Kong, Shenzhen

Abstract

Accurate 3D scene understanding is essential for embodied intelligence, with occupancy prediction emerging as a key task for reasoning about both objects and free space. Existing approaches largely rely on depth priors (e.g., DepthAnything) but make only limited use of 3D cues, restricting performance and generalization. Recently, visual geometry models such as VGGT have shown strong capability in providing rich 3D priors, but similar to monocular depth foundation models, they still operate at the level of visible surfaces rather than volumetric interiors, motivating us to explore how to more effectively leverage these increasingly powerful geometry priors for 3D occupancy prediction. We present GPOcc, a framework that leverages generalizable visual geometry priors (GPs) for monocular occupancy prediction. Our method extends surface points inward along camera rays to generate volumetric samples, which are represented as Gaussian primitives for probabilistic occupancy inference. To handle streaming input, we further design a training-free incremental update strategy that fuses per-frame Gaussians into a unified global representation. Experiments on Occ-ScanNet and EmbodiedOcc-ScanNet demonstrate significant gains: GPOcc improves mIoU by +9.99 in the monocular setting and +11.79 in the streaming setting over prior state of the art. Under the same depth prior, it achieves +6.73 mIoU while running 2.65 $\times$ faster. These results highlight that GPOcc leverages geometry priors more effectively and efficiently. Code will be released at https://github.com/JuIvyy/GPOcc.

Figures & tables

Panel of Figure 1 — Figure 1: Comparison of monocular occupancy prediction pipelines. ISO [ 39 ] formulates depth estimation as a multi-class classification problem, using the predicted depth distributions to lift 2D image features into dense 3D volumes, which are then processed by a 3D U-Net for occupancy prediction. EmbodiedOcc [ 34 ] , by contrast, initializes random 3D anchors and applies cross-attention to aggregate image features, predicting Gaussian primitives that are splatted into voxels. Many of these Gaussians fall in empty regions, shown as gray primitives. In contrast, GPOcc employs ray-based volumetric sampling to generate sparse Gaussians concentrated on or within objects, producing a compact and efficient representation for occupancy inference.

Figure 2: Overview of GPOcc. Given an input RGB image, visual geometry priors (GPs) predicts surface points and extracts 3D-aware features. These surface points guide ray-based volumetric sampling to estimate interior points, which serve as Gaussian centers. The extracted features are combined with learnable embeddings to predict Gaussian attributes, and the resulting primitives are splatted to infer occupancy probabilistically. Monocular predictions are incrementally integrated into a global memory bank, enabling coherent large-scale occupancy construction.

Figure 3: Comparison of Gaussian representations. (a) EmbodiedOcc, where gray indicates Gaussians predicted as empty. A substantial portion of primitives are placed in empty space, resulting in an inefficient representation. (b) Our method is much more compact where Gaussians are concentrated in occupied regions.

Method	IoU	$\blacksquare$ ceiling	$\blacksquare$ floor	$\blacksquare$ wall	$\blacksquare$ window	$\blacksquare$ chair	$\blacksquare$ bed	$\blacksquare$ sofa	$\blacksquare$ table	$\blacksquare$ tvs	$\blacksquare$ furniture	$\blacksquare$ objects	mIoU
TPVFormer [ 5 ]	33.39	6.96	32.97	14.41	9.10	24.01	41.49	45.44	28.61	10.66	35.37	25.31	24.94
GaussianFormer [ 6 ]	40.91	20.70	42.00	23.40	17.40	27.0	44.30	44.80	32.70	15.30	36.70	25.00	29.93
MonoScene [ 2 ]	41.60	15.17	44.71	22.41	12.55	26.11	27.03	35.91	28.32	6.57	32.16	19.84	24.62
ISO [ 39 ]	42.16	19.88	41.88	22.37	16.98	29.09	42.43	42.00	29.60	10.62	36.36	24.61	28.71
Surroundocc [ 31 ]	42.52	18.90	49.30	24.80	18.00	26.80	42.00	44.10	32.90	18.60	36.80	26.90	30.83
EmbodiedOcc [ 34 ]	53.55	39.60	50.40	41.40	31.70	40.90	55.00	61.40	44.00	36.10	53.90	42.20	45.15

Table 1: Monocular prediction performance on the Occ-ScanNet dataset.

Method	IoU	$\blacksquare$ ceiling	$\blacksquare$ floor	$\blacksquare$ wall	$\blacksquare$ window	$\blacksquare$ chair	$\blacksquare$ bed	$\blacksquare$ sofa	$\blacksquare$ table	$\blacksquare$ tvs	$\blacksquare$ furniture	$\blacksquare$ objects	mIoU
TPVFormer [ 5 ]	35.88	1.62	30.54	12.03	13.22	35.47	51.39	49.79	25.63	3.60	43.15	16.23	25.70
SurroundOcc [ 31 ]	37.04	12.70	31.80	22.50	22.00	29.90	44.70	36.50	24.60	11.50	34.40	18.20	26.27
GaussianFormer [ 6 ]	38.02	17.00	33.60	21.50	21.70	29.40	47.80	37.10	24.30	15.50	36.20	16.80	27.36
SplicingOcc [ 34 ]	49.01	31.60	38.80	35.50	36.30	47.10	54.50	57.20	34.40	32.50	51.20	29.10	40.74
EmbodiedOcc [ 34 ]	51.52	22.70	44.60	37.40	38.00	50.10	56.70	59.70	35.40	38.40	52.00	32.90	42.53
EmbodiedOcc++ [ 23 ]	52.20	27.90	43.90	38.70	40.60	49.00	57.90	59.20	36.80	37.80	53.50	34.10	43.60

Table 2: Embodied prediction performance on the EmbodiedOcc-ScanNet dataset.

Figure 4: Qualitative comparison on monocular occupancy prediction. (a) shows the input RGB images, (b) the ground-truth occupancy, (c) the predictions of EmbodiedOcc [ 34 ] , (d) the predictions of our method, and (e) the visualization of the Gaussian primitives predicted by our method. Compared to EmbodiedOcc, our framework produces more accurate and complete reconstructions, while the Gaussian representation provides interpretable intermediate geometry.

Figure 5: Qualitative results on streaming inputs. Our incremental update strategy progressively integrates information from sequential frames. The predictions become increasingly complete as more frames are observed, demonstrating the effectiveness of our streaming design.

Model	IoU	mIoU	FPS	#Params
ISO [ 39 ]	42.16	28.71	3.63	303.05M
EmbodiedOcc [ 34 ]	53.55	45.15	10.66	231.45M
Ours-DPT	56.96	51.88	28.22	97.95M
Ours-VGGT	63.14	56.19	5.26	942.31M

Table 3: Comparison on the Occ-ScanNet dataset in terms of accuracy, efficiency, and model complexity. DPT and VGGT refer to [ 37 ] and [ 26 ] , respectively. All FPS values are measured on the same computer with NVIDIA A800 GPU, averaged over 1000 runs after an initial warm-up of 100 iterations.

$K$	mIoU	IoU	#Gaussians
1	47.88	53.10	3079
2	52.65	56.35	2310
4	55.28	60.35	2731
8	55.76	61.67	3940
16	56.19	63.14	5876
32	56.72	63.84	20206

Table 4: Ablation on the number of sampled points

K

per ray. Increasing

K

improves accuracy, but gains saturate beyond

K{=}16

despite a sharp increase in the number of Gaussians.

Threshold	mIoU	IoU	#Gaussians
$>0.01$	56.19	63.14	5876
$>0.02$	55.21	62.08	3509
$>0.03$	54.59	61.45	2468
$>0.04$	54.29	61.09	1931
$>0.05$	54.16	60.84	1612
$>0.06$	54.02	60.54	1398

Table 5: Ablation on the opacity threshold

\tau

. Lower thresholds (e.g.,

>0.01

) perform best; larger thresholds prune too many Gaussians and degrade accuracy.

3D Occupancy Prediction

3D occupancy prediction estimates the geometry and semantics of occupied and free space around an agent from camera, LiDAR, or radar data.

See all 113 papers

GPOcc++: Unified Sparse Gaussian Occupancy Prediction with Visual Geometry Priors
Jul 15, 2026Changqing Zhou, Yueru Luo, Yulan Guo +33D Occupancy Prediction 3D Scene Representation
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511453, China · The Chinese University of Hong Kong, Shenzhen, China · The School of Electronics and Communication Engineering, Sun Yat-sen University, Shenzhen 518107, China +3
VGOcc: Learning Visual-Geometric Gaussians for Vision-Centric 3D Driving Occupancy Prediction
Jul 20, 2026Junhong Lin, Xianda Guo, Kangli Wang +4Autonomous Driving 3D Occupancy Prediction
1SECE, Peking University · School of Computer Science, Wuhan University · 3Great Wall Motor
VG3S: Visual Geometry Grounded Gaussian Splatting for Semantic Occupancy Prediction
Mar 6, 2026Xiaoyang Yan, Muleilan Pei, Shaojie Shen3D Occupancy Prediction 3D Gaussian Splatting
Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology, Hong Kong SAR, China

Jul 15, 2026J/K move · Enter open · S save

GPOcc++: Unified Sparse Gaussian Occupancy Prediction with Visual Geometry Priors

Changqing Zhou, Yueru Luo, Yulan Guo, Bing Wang+2

The Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511453, China · The Chinese University of Hong Kong, Shenzhen, China · The School of Electronics and Communication Engineering, Sun Yat-sen University, Shenzhen 518107, China+3

3D Occupancy Prediction 3D Scene Representation

Open paper AskPDF arXiv

Accurate 3D scene understanding is fundamental to embodied intelligence and autonomous driving, where 3D occupancy provides a unified representation of objects, structures, and free space. However, recovering such a complete volumetric representation from visual observations remains challenging, particularly in occluded and unobserved regions. Visual geometry priors offer strong and generalizable geometric cues for addressing this challenge, but their outputs are inherently surface-centric, whereas occupancy prediction requires reasoning about volumetric interiors and free space. To bridge this gap, we introduce GPOcc, which transforms visual geometry priors into occupancy-aware sparse Gaussian representations for efficient and expressive volumetric scene modeling. Building on GPOcc, GPOcc++ models multi-view observations and temporal sequences within a unified framework, allowing spatial and temporal evidence to be handled through the same representation. We further extend GPOcc++ from indoor scenes to outdoor occupancy prediction. Extensive experiments on both indoor and outdoor benchmarks demonstrate consistently strong performance across both multi-view and temporal settings, together with favorable efficiency and generalization. Code will be released at https://github.com/JuIvyy/GPOcc.

Abstract

3D Occupancy Prediction

Explore similar work

GPOcc++: Unified Sparse Gaussian Occupancy Prediction with Visual Geometry Priors

VGOcc: Learning Visual-Geometric Gaussians for Vision-Centric 3D Driving Occupancy Prediction

VG3S: Visual Geometry Grounded Gaussian Splatting for Semantic Occupancy Prediction