OPUS-V2: Bridging the Gap between Sparse Points and Dense Voxels

Jiabao Wang; Qiang Meng; Liujiang Yan; Ke Wang; Qibin Hou; Ming-Ming Cheng

cs.CVAug 29, 2026

OPUS-V2: Bridging the Gap between Sparse Points and Dense Voxels

Authors: Jiabao Wang, Qiang Meng, Liujiang Yan, Ke Wang, Qibin Hou, Ming-Ming Cheng

Organizations: Nankai University, Tianjin, 300350, China. · Momenta, Beijing, 102200, China. · KargoBot, Beijing, 102200, China.

Abstract

The point-based occupancy prediction paradigm has achieved an attractive trade-off between accuracy and efficiency by modeling 3D space sparsely. However, its predictions inherently mismatch the dense voxel-based occupancy required by self-driving systems, necessitating hand-crafted heuristics during training and inference that limit final performance. To overcome these limitations, we propose OPUS-V2, a novel framework built upon the pioneering OPUS (occupancy prediction using a sparse set) point-based approach. OPUS-V2 incorporates a lightweight point-voxel transformation (PVT) module behind the decoder to adaptively map sparse predictions into the dense voxel space, eliminating the need for suboptimal operations and improving model accuracy. Furthermore, our architecture decouples feature and occupancy generation processes, allowing OPUS-V2 to adapt to arbitrary occupancy resolutions. OPUS-V2 achieves a state-of-the-art rayIoU of 44.0 on the Occ3D dataset. On the more challenging OpenOccupancy dataset, it attains a competitive 16.4 mIoU while running in real time at 20.6 FPS.

Figures & tables

Table 1

Figure 1: Comparison of OPUS and OPUS-V2 pipelines. (a) OPUS needs suboptimal operations, such as a nearest-neighbor strategy in training and handcrafted post-processing in testing. In contrast, (b) OPUS-V2 leverages PVT to adaptively transform sparse predictions into the voxel space, facilitating direct interaction between predictions and ground truth.

Figure 2: Comparison between OPUS-V2 and previous methods on the Occ3D and OpenOccupancy datasets, evaluated using both mIoU and rayIoU metrics. FPS measurements were conducted on an A100 GPU with PyTorch FP32 precision, except for ALOcc [ 15 ] , for which FPS was measured on a 4090 GPU and is included for reference.

Figure 3: Numbers of mislabeled voxels in post-processing on the Occ3D dataset. We count mislabeled voxels whose occupancy status is correctly represented by predicted points but misclassified in post-processing.

Figure 4: OPUS-V2 employs a three-stage pipeline: (1) an image encoder to extract multi-view image features, (2) a series of decoders to produce 3D sparse points with corresponding latent features, and (3) a lightweight PVT module that adaptively transforms these sparse features into dense voxel space to produce the final occupancy predictions.

Figure 5: Training objectives in OPUS-V2. We separately supervise the sparse and dense predictions of decoders and PVT by the center points of occupied voxels and semantic labels of voxelized annotations.

Model	$Q$	$S$	$F$	#points in each stage
Model	$Q$	$S$	$F$	$R_{1}$	$R_{2}$	$R_{3}$	$R_{4}$	$R_{5}$
OPUS-V2-T	600	4	32	8	16	32	64	128
OPUS-V2-S	1200	2	32	4	8	16	32	64
OPUS-V2-M	2400	2	32	2	4	8	16	32
OPUS-V2-L	4800	2	32	1	2	4	8	16

Table 1: Configurations for different models.

Q

S

F

represent the numbers of queries, sampling points in each query, and channel size of predicted point features.

R_{i}

indicates the number of points predicted by a single query in the

i

-th decoder.

Methods	Input	Backbone	Image Size	Epoch	Vis. Mask	mIoU	$\text{rayIoU}_{\text{1m}}$	$\text{rayIoU}_{\text{2m}}$	$\text{rayIoU}_{\text{4m}}$	$\text{rayIoU}_{\text{}}$	FPS
RenderOcc [ 50 ]	C	Swin-B	$1408\times 512$	12	$\checkmark$	24.5	13.4	19.6	25.5	19.5	-
BEVFormer [ 51 ]	C	R101	$1600\times 900$	24	$\checkmark$	39.3	26.1	32.9	38.0	32.4	3.0
BEVDet-Occ [ 52 ]	C	R50	$704\times 256$	90	$\checkmark$	36.1	23.6	30.0	35.1	29.6	2.6
BEVDet-Occ (8f) [ 52 ]	C	R50	$704\times 384$	90	$\checkmark$	39.3	26.6	33.1	38.2	32.6	0.8
FB-Occ (16f) [ 4 ]	C	R50	$704\times 256$	80 ∗	$\checkmark$	39.1	26.7	34.1	39.7	33.5	10.3
SparseOcc (8f) [ 7 ]	C	R50	$704\times 256$	12	-	30.9	28.0	34.7	39.4	34.0	17.3

Table 2: Occupancy prediction performance on the Occ3D dataset. The best result in each column is highlighted in bold. Models using temporal fusion over 8 and 16 frames are denoted by ‘8f’ and ‘16f,’ respectively. Some models implement CBGS [ 49 ] , which extends the training duration by nearly 4.5 times (marked with ∗ ). All FPS results are sourced from their corresponding papers. Most were measured on an A100 GPU unless otherwise noted; ALOcc (marked with † ) was measured on an RTX 4090.

Method	Input	IoU	mIoU	barrier	bicycle	bus	car	const. veh.	motorcycle	pedestrian	traffic cone	trailer	truck	drive. suf.	other flat	sidewalk	terrain	manmade	vegetation	FPS
MonoScene [ 3 ]	C	18.4	6.9	7.1	3.9	9.3	7.2	5.6	3.0	5.9	4.4	4.9	4.2	14.9	6.3	7.9	7.4	10.0	7.6	-
TPVFormer [ 6 ]	C	15.3	7.8	9.3	4.1	11.3	10.1	5.2	4.3	5.9	5.3	6.8	6.5	13.6	9.0	8.3	8.0	9.2	8.2	-
C-CONet [ 2 ]	C	20.1	12.8	13.2	8.1	15.4	17.2	6.3	11.2	10.0	8.3	4.7	12.1	31.4	18.8	18.7	16.3	4.8	8.2	-
SparseOcc [ 48 ]	C	21.8	14.1	16.1	9.3	15.1	18.6	7.3	9.4	11.2	9.4	7.2	13.0	31.8	21.7	20.7	18.8	6.1	10.6	-
3DSketch [ 56 ]	C&D	25.6	10.7	12.0	5.1	10.7	12.4	6.5	4.0	5.0	6.3	8.0	7.2	21.8	14.8	13.0	11.8	12.0	21.2	-
AICNet [ 57 ]	C&D	23.8	10.6	11.5	4.0	11.8	12.3	5.1	3.8	6.2	6.0	8.2	7.5	24.1	13.0	12.8	11.5	11.6	20.2	-

Table 3: Occupancy prediction performance on the OpenOccupancy dataset. The best result in each column is highlighted in bold. The results of previous methods are taken from the papers of OpenOccupancy and SparseOcc proposed by Tang et al. FPS results of OPUS-V2 are measured on an A100 GPU.

models	Image Size	IoU	mIoU	Mem. (GB)	FPS
SparseOcc [ 48 ]	$704\times 256$	21.8	14.1	13.0	-
SparseOcc [ 48 ]	$1600\times 900$	20.4	14.6	-	-
OPUS-V2-T	$704\times 256$	24.8	15.1	2.3	24.2
OPUS-V2-T	$1600\times 900$	26.2	15.7	11.6	6.9

Table 4: Comparison of the sparse voxel-based method for different input image resolutions. The results of SparseOcc by Tang et al. were obtained from their original paper. FPS results were measured on a RTX 4090 GPU.

index	NN	PVT	$\text{rayIoU}_{\text{1m}}$	$\text{rayIoU}_{\text{2m}}$	$\text{rayIoU}_{\text{4m}}$	$\text{rayIoU}_{\text{}}$
(I)	✓		30.3	38.0	43.1	37.2
(II)	✓	✓	34.0	41.3	46.2	40.6
(III)		✓	34.6	41.7	46.5	41.0

Table 5: Ablation studies for different training and testing strategies. ‘NN’ means using the proxy constructed by the nearest-neighbor in classification training. ‘PVT’ means employing the PVT module in model.

$\mathbf{f}$	$\mathbf{p}_{\text{c}}$	$\mathbf{p}_{\text{v}}$	$\text{rayIoU}_{\text{1m}}$	$\text{rayIoU}_{\text{2m}}$	$\text{rayIoU}_{\text{4m}}$	$\text{rayIoU}_{\text{}}$
✓			33.6	41.0	45.9	40.2
✓	✓		34.5	41.6	46.5	40.9
✓	✓	✓	34.6	41.7	46.5	41.0

Table 6: Ablation studies for components in point features.

\mathbf{f}

denotes the semantic features predicted by decoders;

\mathbf{p}_{\text{c}}

and

\mathbf{p}_{\text{v}}

denote relative positions from corresponding query centers and voxel centers, respectively.

Figure 6: Varying the number of decoders. All models were trained for 50 epochs. FPS results were measured on an A100 GPU.

Figure 7: Convergence of focal loss and chamfer distance loss with weights of chamfer distance loss set to 0.5 and 1.0.

Figure 8: Visualizations of the sparse point-based and dense voxel-based predictions of OPUS-V2.

Figure 9: Results of the original OPUS and OPUS-V2 trained on Occ3D and OpenOccupancy.

3D Occupancy Prediction

3D occupancy prediction estimates the geometry and semantics of occupied and free space around an agent from camera, LiDAR, or radar data.

See all 114 papers

Jul 6, 2026

SparseOcc++: Geometry-Aware Sparse Latent Representation for Semantic Occupancy Prediction

Vision-based 3D semantic occupancy prediction is essential for autonomous driving, yet dense voxel representations waste computation on largely empty space, while BEV and TPV projections compromise fine-grained 3D structure. Fully sparse representations offer an attractive alternative, but existing methods, including SparseOcc, entangle scene completion with semantic prediction by indiscriminately propagating high-dimensional features into empty regions and applying voxel-wise classification. This creates excessive activations, computational overhead, and geometric ambiguity. We present SparseOcc++, a geometry-aware sparse framework that explicitly decouples scene completion from semantic segmentation. SparseOcc++ reformulates completion as signed-distance regression on sparse anchor voxels through a scene completion field (SCF). To model complex outdoor geometry robustly, it combines orthogonal decomposition with discretized distance learning. A geometry-guided propagation module then converts the SCF into a complete volumetric scene and restricts semantic segmentation to geometrically verified regions. Experiments establish new state of the art: SparseOcc++ improves IoU by 2.3 points and is 3.9x faster than SparseOcc on nuScenes, while achieving a 5.9x speedup over OccFormer on SemanticKITTI.

Pin Tang, Zhongdao Wang, Guoqing Wang +2

MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, Shanghai 200240, China · Huawei Noah’s Ark Lab, China

3D Reconstruction Autonomous Driving Perception 3D Semantic Segmentation Semantic Occupancy Mapping 3D Occupancy Prediction 3D Scene Completion

Ask PDF

Mar 28, 2025

Streaming Dense Voxel Representations for 3D Occupancy Prediction

In this paper, we explore dense voxel streaming for accurate and efficient 3D occupancy prediction. While dense voxel representations offer fine-grained spatial details and streaming paradigm enables efficient temporal processing, naively combining the two introduces key challenges: (i) warping-induced distortions caused by interpolation used for temporal alignment, and (ii) degraded dynamic object representations due to motion misalignment and detail loss in image-to-voxel projection. To address these, we propose StreamOcc, a novel framework that utilizes two aggregation strategies. Specifically, it first refines propagated voxel features to reduce warping artifacts before temporal accumulation, and then selectively injects instance-level query features encoding dynamic-object semantics into the corresponding occupied voxel regions, preserving temporally consistent modeling while strengthening dynamic object representations. Unlocking effective dense voxel streaming, StreamOcc achieves state-of-the-art performance on SurroundOcc-benchmark and Occ3D-nuScenes under real-time constraints, outperforming the prior best methods by +1.3/2.5 and +1.5/2.0 in (overall/dynamic object) mIoU, respectively, while running at 83.3 ms per frame with only 2.8 GB of memory. The project page is available at https://moonseokha.github.io/StreamOcc/.

Seokha Moon, Janghyun Baek, Yujin Jeong +5

1Korea University · 5NAVER LABS · † Work done during an internship at NAVER LABS. +4

3D Occupancy Prediction 3D Representation Learning

Ask PDF

Jun 30, 2026

Semantic Occupancy Prediction with Dual Range-Voxel Representation

LiDAR-based 3D semantic occupancy prediction, which aims to provide accurate and comprehensive scene representation, is crucial for autonomous driving systems. As point clouds suffer from sparsity and incompleteness, leading to insufficient semantic learning and difficult occupancy perception, existing methods often stack multi-sweep point clouds to obtain dense spatial information. However, such a naive strategy also results in efficiency (e.g., additional computational burden) and robustness (e.g., pose transformation noise) concerns, which hinder their practical applications. In this work, we propose a Dual Range-Voxel Representation (DRVR) that leverages the range-view context and voxel-view geometry of single-sweep point clouds for 3D semantic occupancy prediction, eliminating the concerns associated with the multi-sweeps. Specifically, we use the range-view encoder to extract the compact context of the scene. To fully exploit the spatial information, we design a geometry-aware voxel-view encoder that extracts multi-scale voxel-view features separately and combines them for better geometric occupancy prediction. Moreover, we propose a range-voxel fusion module to cooperate range- and voxel-view features via voxel-to-range and range-to-voxel fusions. Extensive experiments on nuScenes-Occupancy, SemanticKITTI and SemanticPOSS show the superiority of our method. Especially on nuScenes-Occupancy, our single-sweep DRVR achieves 5.4% improvement in mIoU and 2.1x acceleration compared to the multi-sweep method.

Sitao Chen, Zhuangwei Zhuang, Hui Luo +3

School of Software Engineering, South China University of Technology, Guangzhou 510006, China · State Key Laboratory of Optical Field Manipulation Science and Technology, Institute of Optics and Electronics, CAS, Chengdu 610209, China

Autonomous Driving Perception Semantic Occupancy Mapping 3D Occupancy Prediction Feature Fusion

Ask PDF

Abstract

3D Occupancy Prediction

Explore similar work

SparseOcc++: Geometry-Aware Sparse Latent Representation for Semantic Occupancy Prediction

Streaming Dense Voxel Representations for 3D Occupancy Prediction

Semantic Occupancy Prediction with Dual Range-Voxel Representation