OccAny: Generalized Unconstrained Urban 3D Occupancy

Anh-Quan Cao; Tuan-Hung Vu

cs.CVMar 24, 2026

OccAny: Generalized Unconstrained Urban 3D Occupancy

Authors: Anh-Quan Cao, Tuan-Hung Vu

Organizations: Valeo.ai, Paris, France

Abstract

Relying on in-domain annotations and precise sensor-rig priors, existing 3D occupancy prediction methods are limited in both scalability and out-of-domain generalization. While recent visual geometry foundation models exhibit strong generalization capabilities, they were mainly designed for general purposes and lack one or more key ingredients required for urban occupancy prediction, namely metric prediction, geometry completion in cluttered scenes and adaptation to urban scenarios. We address this gap and present OccAny, the first unconstrained urban 3D occupancy model capable of operating on out-of-domain uncalibrated scenes to predict and complete metric occupancy coupled with segmentation features. OccAny is versatile and can predict occupancy from sequential, monocular, or surround-view images. Our contributions are three-fold: (i) we propose the first generalized 3D occupancy framework with (ii) Segmentation Forcing that improves occupancy quality while enabling mask-level prediction, and (iii) a Novel View Rendering pipeline that infers novel-view geometry to enable test-time view augmentation for geometry completion. Extensive experiments demonstrate that OccAny outperforms all visual geometry baselines on 3D occupancy prediction task, while remaining competitive with in-domain self-supervised methods across three input settings on two established urban occupancy prediction datasets. Our code is available at https://github.com/valeoai/OccAny .

Figures & tables

Figure 1: OccAny is a generalized 3D occupancy model that is trained once and can operate on out-of-domain sequential, monocular, or surround-view urban images. It produces SAM2-like features, enabling promptable segmentation.

Figure 2: OccAny Training is done in two stages: (i) 3D Reconstruction infers 3D scene using ${N_{rec}}$ reconstruction frames and (ii) Novel-View Rendering renders geometry of ${N_{rnd}}$ new views having camera poses $\{\mathbf{T}_{j}\}_{j=1}^{N_{rnd}}$ . Segmentation Forcing with SAM2 features helps regularize and improve geometry prediction. The scene memory $\mathbf{M}$ is dynamically updated during reconstruction, while during rendering, the final scene memory output from the reconstruction stage is used without updating

Figure 3: OccAny inference undergoes two stages: (i) 3D reconstruction to retrieve ${N_{rec}}$ pointmaps with predicted camera poses $\{\mathbf{v}_{i}\}_{i=1}^{{N_{rec}}}$ , and (ii) novel-view rendering with TTVA sampled along the trajectory of $\{\mathbf{v}_{i}\}_{i=1}^{{N_{rec}}}$ . 3D occupancy is obtained by aggregating all pointmaps and voxelizing them with trilinear interpolation.

	Input	MUSt3R	VGGT †	CUT3R*	AnySplat* †	OccAny (ours)	GT
Sequence

Surround-view

Figure 4: Occupancy predictions of OccAny and baselines on a sequence and a surround view. We visualize here predicted voxels. For qualitative analysis, we overlay the semantic ground-truth colors on predicted voxels to better highlight class-wise gains. False positive voxels are painted in gray without any overlayed color. Compared to baselines, our occupancy predictions are denser and more accurate.

Method	Venue	Semantic KITTI				Occ3D-NuScenes
Method	Venue	Res.	Prec.	Rec.	IoU	Res.	Prec.	Rec.	IoU
MUSt3R [ 3 ]	CVPR’25	512x160	18.38	25.58	11.97	512x288	19.27	28.60	13.01
CUT3R [ 65 ]	CVPR’25	512x160	25.72	21.11	13.11	512x288	24.69	16.57	11.01
CUT3R* [ 65 ]	CVPR’25	512x160	27.05	27.92	15.93	512x288	29.44	30.50	17.62
VGGT † [ 63 ]	CVPR’25	518x168	36.35	22.62	15.20	518x294	38.34	26.23	18.45
AnySplat † [ 29 ]	TOG’25	518x168	18.22	35.62	11.67	518x294	26.67	36.93	18.33

Table 1: Sequence setting. Occupancy prediction on SemanticKITTI and Occ3D-NuScenes.

Test	Method	Venue	Res.	Prec.	Rec.	IoU
in-domain	MonoScene [ 6 ]	CVPR’22	1220x370	13.15	40.22	11.18
	SceneRF [ 7 ]	ICCV’23	1220x370	17.28	40.96	13.84
	SelfOcc [ 23 ]	CVPR’24	1220x370	34.83	37.31	21.97
	Splatter Image [ 56 ]	CVPR’24	1220x370	11.30	53.93	10.30
	Hi-Gaussian [ 75 ]	ICCV’25	1220x370	17.39	59.72	15.56
	OccNeRF [ 83 ]	TIP’25	1220x370	35.25	39.27	22.81

Table 2: Monocular setting. Occupancy results with Monocular input on SemanticKITTI following [ 23 , 7 ] . Results for MonoScene and Splatter Image are taken from [ 7 , 75 ] .

Test	Method	Venue	Res.	Prec.	Rec.	IoU
in-domain	SelfOcc [ 23 ]	CVPR’24	800x450	–	–	45.01
	OccNeRF [ 83 ]	TIP’25	672x336	57.20	55.47	39.20
	DistillNeRF [ 64 ]	NeuRIPS’24	400×228	–	–	29.11
	SimpleOcc [ 15 ]	TIV’24	672x336	41.91	64.02	33.92
	GaussTR [ 28 ]	CVPR’25	896x504	–	–	45.19
out-of-domain	MUSt3R [ 3 ]	CVPR’25	512x288	20.79	28.29	13.61

Table 3: Surround-view setting . More results are in Tab. 8 .

Figure 5: Qualitative ablation shows the gains from Segmentation Forcing and Novel-View Rendering . Voxel colorization follows Fig. 4 . The two proposed strategies significantly improve the density and the accuracy of occupancy predictions.

Method	Venue	Semantic KITTI sequence			Occ3D-NuScenes surround-view
Method	Venue	Res.	mIoU	mIoU sc	Res.	mIoU	mIoU sc
MUSt3R [ 3 ] + SAM2 [ 47 ]	CVPR’25	512x160	3.22	5.96	512x288	2.43	3.84
CUT3R [ 65 ] + SAM2 [ 47 ]	CVPR’25	512x160	4.15	6.72	512x288	2.40	2.75
CUT3R* [ 65 ] + SAM2 [ 47 ]	CVPR’25	512x160	4.53	8.18	512x288	3.06	3.99
VGGT † [ 63 ] + SAM2 [ 47 ]	CVPR’25	518x294	3.47	6.76	518x294	4.39	6.49
AnySplat † [ 29 ] + SAM2 [ 47 ]	TOG’25	518x168	3.37	6.83	518x294	3.96	5.97

Table 4: Semantic Occupancy Prediction with GSAM2 [ 48 ] .

Method	Semantic KITTI sequence						Occ3D-NuScenes surround-view
	Res.	Pre.	Rec.	IoU	mIoU	mIoU sc	Res.	Pre.	Rec.	IoU	mIoU	mIoU sc
OccAny	512x160	36.79	46.70	25.91	7.28	13.53	512x288	45.04	58.54	34.15	6.66	10.32
OccAny+	512x160	38.12	49.14	27.33	6.48	13.30	512x288	46.38	54.66	33.49	7.20	11.50

Table 5: Changing the base foundation models used in OccAny to DA3 [ 36 ] and SAM3 [ 8 ] results in the OccAny+ variant.

Panel of Table 6 — Table 6: Ablation results on SemanticKitti. The “geo-aware” stands for applying geometry confidence maps $\mathbf{C}$ in the segmentation forcing loss ( cf . Eq. 3 ).

Figure 6: Ablating NVR inference on SemanticKITTI

Panel of Figure 7 — Figure 7: PCA visualization of our segmentation features of multi-view sequences. Low-resolution features capture high-level semantics ( e.g . , separating cars, buildings, and roads), while high-resolution features capture low-level details such as boundaries and textures. Features remain consistent across different views.

Input	Instance Seg.	Input	Instance Seg.

Figure 8: Instance segmentation of cars with OccAny’s features.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Sem. feat.	Params	Semantic KITTI sequence			Occ3D-NuScenes surround-view
Method	Sem. feat.	Params	Res.	mIoU	mIoU sc	Res.	mIoU	mIoU sc
OccAny	Distilled	623M	512x160	7.28	13.53	512x288	6.66	10.32
OccAny+	Distilled	651M	512x160	6.48	13.30	512x288	7.20	11.50
OccAny	Pretrained	864M	512x160	7.67	13.75	512x288	7.42	10.78
OccAny+	Pretrained	1.08B	512x160	8.03	13.17	512x288	9.45	12.22

Appendix

Table 7: Using pretrained segmentation features to boost semantic performance. OccAny+ is the variant using DA3 and SAM3 base models. Parameter counts reflect the forward path from the input to the predicted pointmaps and segmentation features. Note that using "pretrained" semantic features incurs a higher parameter cost due to the use of pretrained encoder.

		Method	Extr.	Intr.	Fixed Ratio	Fixed Rig	GT LiDAR	Sem. Adapt.	GT Occ.	IoU	mIoU
Occ3D-NuScenes (ext. Tab. 3 )	in-domain	SimpleOcc	Req.	Req.	Req.	Req.	Req.	Req.	–	33.92	7.05
		DistillNeRF	Req.	Req.	Req.	Req.	Req.	–	–	29.11	8.93
		SelfOcc	Req.	Req.	Req.	Req.	–	Req.	–	45.01	9.30
		POP-3D	Req.	Req.	Req.	Req.	Req.	Req.	–	28.17	9.31
		OccNeRF	Req.	Req.	Req.	Req.	–	Req.	–	39.20	9.53
		GaussianOcc	–	Req.	Req.	Req.	–	Req.	–	51.22	9.94

Appendix

Table 8: Detailed surround-view results. OccAny+ is the variant using DA3 and SAM3 base models.

Method	Semantic KITTI							Occ3D-NuScenes
	Res.	sequence			monocular			Res.	sequence			surround-view
	Res.	Prec.	Rec.	IoU	Prec.	Rec.	IoU	Res.	Prec.	Rec.	IoU	Prec.	Rec.	IoU
OccAny ${}_{\text{depth completion}}$	$512\times 160$	24.59	44.55	18.82	21.59	37.55	15.89	$512\times 288$	29.80,	36.09	19.51	30.57	39.32	20.77
OccAny	$512\times 160$	36.79	46.79	25.91	45.64	33.66	24.03	$512\times 288$	36.09	40.39	23.55	45.04	58.54	34.15

Appendix

Table 9: Novel-View Rendering vs . Depth Completion. Occupancy prediction results on SemanticKITTI and Occ3D-NuScenes show the effectiveness of Novel-View Rendering .

Label	Method	Venue	Occ3D-NuScenes surround-view				SemanticKITTI monocular
Label	Method	Venue	Res.	Prec.	Rec.	IoU	Res.	Prec.	Rec.	IoU
Occ	CVT-Occ [ 81 ] (Trained on Occ3D-Waymo )	ECCV’24	$1600\times 900$	35.38	25.86	17.56	$1220\times 370$	8.97	34.92	7.69
	CVT-Occ [ 81 ] (Trained on Occ3D-Waymo )	ECCV’24	$960\times 540$	29.15	28.33	16.78	$960\times 292$	8.92	36.84	7.73
	CVT-Occ [ 81 ] (Trained on Occ3D-NuScenes )	ECCV’24	in-domain	–	–	–	$1220\times 370$	11.73	59.97	9.43
	ALOcc [ 11 ] (Trained on Occ3D-NuScenes )	ICCV’25	in-domain	–	–	–	$704\times 256$	16.34	53.06	14.28
LiDAR	OccAny	–	$512\times 288$	45.04	58.54	34.15	$512\times 160$	45.64	33.66	24.03

Appendix

Table 10: Generalization results of fully-supervised methods. Occ label is denser through temporal accumulation of LiDAR point-clouds and subsequent post-processing, whereas the LiDAR label remains sparser at each timestep. OccAny works out of the box in any evaluation settings with different inference areas, voxel resolutions and sensor configurations. In contrast, other methods require manual code modifications to align testing and training conditions. Beyond being more versatile, OccAny clearly demonstrates superior generalization.

Method	ADE (m)
GeoCalib [ 60 ] + DroidSLAM [ 57 ] + DA2 [ 78 ]	1.63
DA3 large + DA3 metric large	2.44
OccAny+	1.86

Appendix

Table 11: Ego Vehicle Trajectory Prediction.

#Aug. Frames	1	2	4	8	10	20	50	100	200
Time (s)	0.052	0.057	0.085	0.172	0.227	0.542	1.406	2.786	5.572
Mem. (GB)	1.175	1.920	3.422	6.471	8.005	9.936	14.314	17.013	22.418

Appendix

Table 12: NVR inference complexity , measured on one A100 GPU.

Method	Train. GPUs	Train. time	Recon. time (ms)	Render time (ms)	Params (M)
CUT3R	8 $\times$ A100	$\approx$ 30 days	240.0	259.8	793.3
VGGT	64 $\times$ A100	$>$ 9 days	222.2	–	1157.9
AnySplat	16 $\times$ A800	$\approx$ 2 days	251.7	17.2	1190.7
OccAny	16 $\times$ A100	$\approx$ 1.5 days	93.8	123.2	651.1

Appendix

Table 13: Model size and speed. Train times are from the original papers. Inference times are measured in the surround setting with 6 input views and 6 render views.

	Input	MUSt3R	VGGT †	CUT3R*	AnySplat* †	OccAny (ours)	GT
Sequence

Appendix

Figure 9: Occupancy predictions of OccAny and baselines on sequential data. We visualize here predicted voxels. For qualitative analysis, we overlay the semantic ground-truth colors on predicted voxels to better highlight class-wise gains. False positive voxels are painted in gray without any overlayed color. Compared to baselines, our occupancy predictions are denser and more accurate.

	Input	MUSt3R	VGGT †	CUT3R*	AnySplat* †	OccAny (ours)	GT
Surround-view

Appendix

Figure 10: Occupancy predictions of OccAny and baselines on surround-view data. Voxel colorization follows Fig. 9 . Compared to baselines, our occupancy predictions are denser and more accurate.

Figure 11: Qualitative ablation on Semantic KITTI shows the gains from Segmentation Forcing and Novel-View Rendering . Voxel colorization follows Fig. 9 . The two proposed strategies significantly improve the density and the accuracy of occupancy predictions.

Figure 12: Qualitative ablation on Occ3D-NuScenes shows the gains from Segmentation Forcing and Novel-View Rendering . Voxel colorization follows Fig. 9 . The two proposed strategies significantly improve the density and the accuracy of occupancy predictions.

Panel of Figure 13 — Figure 13: PCA visualization of predicted feature maps. Low-resolution features capture high-level semantics (e.g., separating cars, buildings, and roads), while high-resolution features capture low-level details such as boundaries and textures. Features remain consistent across different views.

3D Occupancy Prediction

3D occupancy prediction estimates the geometry and semantics of occupied and free space around an agent from camera, LiDAR, or radar data.

See all 113 papers

Aug 9, 2026

OccAnyScene: Towards Unified Indoor-Outdoor 3D Occupancy Prediction

3D occupancy prediction is fundamental to scene understanding, yet existing 3D semantic occupancy methods are typically specialized to fixed scene types and occupancy protocols. We introduce Cross-Scene 3D Semantic Occupancy Prediction, a new task setting which requires a single model to handle heterogeneous indoor and outdoor scenes with varying cameras, spatial ranges, voxel specifications, and semantic taxonomies. This setting poses a fundamental challenge: achieving metric-consistent yet scene-adaptive image-to-3D lifting across varying camera configurations and scene scales. To address this challenge, we propose OccAnyScene, a pixel-frustum-centered Gaussian framework built upon a pretrained depth foundation model. Specifically, the framework employs Pixel-Aligned Frustum Feature Aggregation to construct a camera-aware frustum query for each feature pixel, and Frustum-Parameterized Gaussian Construction to decode each query into multiple Gaussians whose positions and sizes are constrained by the predicted pixel depth and corresponding frustum geometry. OccAnyScene sets new state-of-the-art results, achieving 59.92% mIoU on the indoor Occ-ScanNet and 23.06% mIoU on the outdoor SurroundOcc-nuScenes.

Junjie Liu, Wanshui Gan, Zitong Dai +6

Pengcheng Laboratory · SUSTech · Shanghai Artificial Intelligence Laboratory +1

3D Gaussian Splatting 3D Occupancy Prediction

Ask PDF

Feb 25, 2026

Generalizing Visual Geometry Priors to Sparse Gaussian Occupancy Prediction

Accurate 3D scene understanding is essential for embodied intelligence, with occupancy prediction emerging as a key task for reasoning about both objects and free space. Existing approaches largely rely on depth priors (e.g., DepthAnything) but make only limited use of 3D cues, restricting performance and generalization. Recently, visual geometry models such as VGGT have shown strong capability in providing rich 3D priors, but similar to monocular depth foundation models, they still operate at the level of visible surfaces rather than volumetric interiors, motivating us to explore how to more effectively leverage these increasingly powerful geometry priors for 3D occupancy prediction. We present GPOcc, a framework that leverages generalizable visual geometry priors (GPs) for monocular occupancy prediction. Our method extends surface points inward along camera rays to generate volumetric samples, which are represented as Gaussian primitives for probabilistic occupancy inference. To handle streaming input, we further design a training-free incremental update strategy that fuses per-frame Gaussians into a unified global representation. Experiments on Occ-ScanNet and EmbodiedOcc-ScanNet demonstrate significant gains: GPOcc improves mIoU by +9.99 in the monocular setting and +11.79 in the streaming setting over prior state of the art. Under the same depth prior, it achieves +6.73 mIoU while running 2.65

\times

faster. These results highlight that GPOcc leverages geometry priors more effectively and efficiently. Code will be released at https://github.com/JuIvyy/GPOcc.

Changqing Zhou, Yueru Luo, Changhao Chen

The Hong Kong University of Science and Technology (Guangzhou) · The Chinese University of Hong Kong, Shenzhen

3D Occupancy Prediction

Ask PDF

Jul 15, 2026

GPOcc++: Unified Sparse Gaussian Occupancy Prediction with Visual Geometry Priors

Accurate 3D scene understanding is fundamental to embodied intelligence and autonomous driving, where 3D occupancy provides a unified representation of objects, structures, and free space. However, recovering such a complete volumetric representation from visual observations remains challenging, particularly in occluded and unobserved regions. Visual geometry priors offer strong and generalizable geometric cues for addressing this challenge, but their outputs are inherently surface-centric, whereas occupancy prediction requires reasoning about volumetric interiors and free space. To bridge this gap, we introduce GPOcc, which transforms visual geometry priors into occupancy-aware sparse Gaussian representations for efficient and expressive volumetric scene modeling. Building on GPOcc, GPOcc++ models multi-view observations and temporal sequences within a unified framework, allowing spatial and temporal evidence to be handled through the same representation. We further extend GPOcc++ from indoor scenes to outdoor occupancy prediction. Extensive experiments on both indoor and outdoor benchmarks demonstrate consistently strong performance across both multi-view and temporal settings, together with favorable efficiency and generalization. Code will be released at https://github.com/JuIvyy/GPOcc.

Changqing Zhou, Yueru Luo, Yulan Guo +3

The Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511453, China · The Chinese University of Hong Kong, Shenzhen, China · The School of Electronics and Communication Engineering, Sun Yat-sen University, Shenzhen 518107, China +3

3D Occupancy Prediction 3D Scene Representation

Ask PDF

Abstract

3D Occupancy Prediction

Explore similar work

OccAnyScene: Towards Unified Indoor-Outdoor 3D Occupancy Prediction

Generalizing Visual Geometry Priors to Sparse Gaussian Occupancy Prediction

GPOcc++: Unified Sparse Gaussian Occupancy Prediction with Visual Geometry Priors