SelectOccFlow: Selective Spatiotemporal Aggregation for 3D Occupancy and Scene Flow Prediction

Yuhang Wang; Kai Luo; Yuanfan Zheng; Kailun Yang

cs.CVOct 3, 2026

SelectOccFlow: Selective Spatiotemporal Aggregation for 3D Occupancy and Scene Flow Prediction

Authors: Yuhang Wang, Kai Luo, Yuanfan Zheng, Kailun Yang

Organizations: School of Artificial Intelligence and Robotics and the National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, China · National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, China

Abstract

Comprehensive 3D scene understanding for autonomous driving requires modeling geometry, semantics, and motion. However, camera-based occupancy and scene flow prediction are sensitive to unreliable spatial and temporal aggregation, caused by semantically incompatible image features, misaligned historical observations, and incomplete voxel structures. To address this issue, we propose SelectOccFlow, a selective spatiotemporal aggregation framework that progressively refines contextual evidence across image, temporal, and voxel domains. To obtain semantically compatible image evidence, we design Semantic-Guided Sampling (SGS) to regulate feature sampling with semantic priors. Since reliable image evidence alone cannot resolve temporal inconsistency, we then present State-Conditioned Temporal Aggregation (SCTA) to selectively retrieve historical evidence according to voxel states. To further enhance the structural completeness of voxel representations, we introduce Extent-Aware Spatial Aggregation (ESA), which exploits directional structural support to refine foreground geometry. Experiments on OpenOcc demonstrate that SelectOccFlow achieves a state-of-the-art OccScore of 44.9, improving the previous best by +4.2%. It also maintains competitive occupancy performance on Occ3D-nus and improves the mean OccScore under nuScenes-C corruptions by +11.1%, demonstrating improved robustness to visual corruptions. The source code will be made publicly available at https://github.com/muchen1021/SelectOccFlow.

Figures & tables

Fig. 1: Overview of SelectOccFlow. (a) Overall pipeline for 3D occupancy and scene flow prediction. (b) SGS performs semantic-guided image-space sampling before lifting. (c) SCTA adapts historical aggregation to voxel states. (d) ESA exploits directional extents for spatial feature refinement.

Method	Sup.	Backbone	Input Size	RayIoU 1m (%) $\uparrow$	RayIoU 2m (%) $\uparrow$	RayIoU 4m (%) $\uparrow$	RayIoU (%) $\uparrow$	mAVE $\downarrow$	OccScore $\uparrow$
OccNeRF-L [ 38 ]	C+L	ResNet-101	256 $\times$ 704	16.6	29.3	49.2	31.7	1.59	28.5
RenderOcc [ 39 ]	L	Swin-Base	512 $\times$ 1408	20.3	32.7	49.9	36.7	1.63	33.0
LetOccFlow-L [ 6 ]	C+L	ConvNeXt-Base	512 $\times$ 1408	25.5	39.7	56.3	40.5	1.45	36.4
CascadeFlow [ 40 ]	3D	ResNet-50	256 $\times$ 704	33.5	40.6	45.3	39.6	0.47	40.9
F-Occ [ 41 ]	3D	Intern-T	256 $\times$ 704	33.9	40.7	45.2	39.9	0.493	41.0
FB-Occ † [ 42 ]	3D	ResNet-50	256 $\times$ 704	31.9	39.2	43.7	38.3	0.954	34.9

TABLE I: Comparison of RayIoU (%), mAVE, and OccScore on OpenOcc [ 1 ] . † denotes results reproduced with the official code and evaluated under the same protocol. Sup.: C/L/3D denote camera, LiDAR, and 3D occupancy supervision.

Method	Backbone	Input Size	RayIoU 1,2,4m $\uparrow$			RayIoU $\uparrow$
BEVFormer [ 24 ]	R101	900 $\times$ 1600	26.1	32.9	38.0	32.4
RenderOcc [ 39 ]	Swin-B	512 $\times$ 1408	13.4	19.6	25.5	19.5
BEVDet-Occ [ 43 ]	R50	256 $\times$ 704	23.6	30.0	35.1	29.6
FB-Occ [ 42 ]	R50	256 $\times$ 704	33.0	39.9	44.0	39.0
SparseOcc [ 27 ]	R50	256 $\times$ 704	29.1	35.8	40.3	35.1
COTR [ 9 ]	R50	256 $\times$ 704	36.3	41.7	45.1	41.0

TABLE II: Comparison of RayIoU (%) performance on Occ3D-nus [ 3 ] . All methods are trained without the camera-visible mask.

Method	Backbone	Input Size	mIoU (%)	$\blacksquare$ others	$\blacksquare$ barrier	$\blacksquare$ bicycle	$\blacksquare$ bus	$\blacksquare$ car	$\blacksquare$ cons. veh.	$\blacksquare$ motor.	$\blacksquare$ pedes.	$\blacksquare$ tfc. cone	$\blacksquare$ trailer	$\blacksquare$ truck	$\blacksquare$ drv. surf.	$\blacksquare$ other flat	$\blacksquare$ sidewalk	$\blacksquare$ terrain	$\blacksquare$ manmade	$\blacksquare$ vegetation
BEVFormer ∗ [ 24 ]	R101	900 $\times$ 1600	26.9	5.9	37.8	17.9	40.4	42.4	7.4	23.9	21.8	21.0	22.4	30.7	55.4	28.4	36.0	28.1	20.0	17.7
CTF-Occ ∗ [ 3 ]	R101	900 $\times$ 1600	28.5	8.1	39.3	20.6	38.3	42.2	16.9	24.5	22.7	21.1	23.0	31.1	53.3	33.8	38.0	33.2	20.8	18.0
TPVFormer ∗ [ 10 ]	R101	900 $\times$ 1600	27.8	7.2	38.9	13.7	40.8	45.9	17.2	20.0	18.9	14.3	26.7	34.2	55.7	35.5	37.6	30.7	19.4	16.8
OSP [ 45 ]	R101	900 $\times$ 1600	41.2	11.0	49.0	27.7	50.2	56.0	23.0	31.0	30.9	30.3	35.6	41.2	82.1	42.6	51.9	55.1	44.8	38.2
FB-Occ [ 42 ]	R50	256 $\times$ 704	39.8	13.8	44.5	27.1	46.2	49.7	24.6	27.4	28.5	28.2	33.7	36.5	81.7	44.1	52.6	56.9	42.6	38.1
ViewFormer [ 13 ]	R50	256 $\times$ 704	41.9	12.9	50.1	27.9	44.6	52.9	22.4	29.6	28.0	29.2	35.2	39.4	84.7	49.4	57.4	59.7	47.4	40.6

TABLE III: Comparison of mIoU (%) performance on Occ3D-nus [ 3 ] .

\dagger

denotes results obtained by retraining with the official code. ∗ indicates models trained without the camera-visible mask.

Method	Bright	Dark	Fog	Snow	Motion	Quant	Avg.
OpenOcc [ 1 ] OccScore $\uparrow$
FB-Occ † [ 42 ]	28.0	17.0	27.3	5.3	24.4	22.8	20.8
VoxelSplat † [ 23 ]	29.5	17.4	29.9	3.7	22.8	22.6	21.0
STCOcc [ 4 ]	33.3	18.1	32.7	9.8	29.5	27.6	25.2
ALOcc [ 5 ]	33.4	23.1	33.1	4.9	30.2	27.3	25.3
Ours	36.5	21.1	36.2	12.7	31.9	30.2	28.1

TABLE IV: Robustness comparison on OpenOcc [ 1 ] and Occ3D-nus [ 3 ] under nuScenes-C [ 16 ] degradations.

\dagger

denotes results obtained by retraining with the official code.

Fig. 2: Qualitative comparison of 3D semantic occupancy under clean and corrupted visual inputs. Clean samples are from OpenOcc [ 1 ] , while Fog , Dark , and Motion samples are from nuScenes-C [ 16 ] . From left to right, we show multi-view images, Ground Truth, Ours, ALOcc [ 5 ] , and STCOcc [ 4 ] . Red circles and blue boxes highlight movable objects and enlarged regions, showing clearer boundaries and more complete structures.

	SGS	SCTA	ESA	RayIoU (%) $\uparrow$	mAVE $\downarrow$	OccScore $\uparrow$
A				38.9	0.525	39.8
B	✓			39.7	0.487	40.8
C		✓		41.5	0.408	43.3
D			✓	39.4	0.512	40.3
E		✓	✓	42.7	0.419	44.2
F	✓		✓	40.2	0.489	41.3

TABLE V: Ablation study of architectural components on OpenOcc [ 1 ] .

Fig. 3: Qualitative comparison of scene flow predictions on OpenOcc [ 1 ] . Results from ALOcc [ 5 ] and STCOcc [ 4 ] are included for comparison. Colors encode flow magnitude and direction, red arrows indicate motion direction, and colored dots denote movable objects moving slowly or remaining nearly stationary. Green ellipses highlight potential ego-vehicle interaction regions.

Temporal	Motion	RayIoU (%) $\uparrow$	mAVE $\downarrow$	OccScore $\uparrow$
		40.2	0.489	41.3
✓		43.2	0.432	44.6
✓	✓	43.1	0.394	44.9

TABLE VI: Ablation study of SCTA on OpenOcc [ 1 ] .

Scene Flow Estimation

See all 21 papers

SelfOccFlow: Towards end-to-end self-supervised 3D Occupancy Flow prediction
Feb 27, 2026Xavier Timoneda, Markus Herb, Fabian Duerr +13D Occupancy Prediction Autonomous Driving
Xavier Timoneda, Markus Herb, and Fabian Duerr are with the Onboard Fusion team at CARIAD SE, Volkswagen Group, Ingolstadt, Germany. · Daniel Goehring is with the Dahlem Center for Machine Learning and Robotics group at Freie Universität Berlin, Germany.
SparseOcc++: Geometry-Aware Sparse Latent Representation for Semantic Occupancy Prediction
Jul 6, 2026Pin Tang, Zhongdao Wang, Guoqing Wang +23D Reconstruction Autonomous Driving Perception
MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, Shanghai 200240, China · Huawei Noah’s Ark Lab, China
$M^2$ -Occ: Resilient 3D Semantic Occupancy Prediction for Autonomous Driving with Incomplete Camera Inputs
Mar 10, 2026Kaixin Lin, Kunyu Peng, Di Wen +3Multi-View Learning Autonomous Driving Perception
School of Artificial Intelligence and Robotics and the National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, China · Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology, Germany

Feb 27, 2026J/K move · Enter open · S save

SelfOccFlow: Towards end-to-end self-supervised 3D Occupancy Flow prediction

Xavier Timoneda, Markus Herb, Fabian Duerr, Daniel Goehring

Xavier Timoneda, Markus Herb, and Fabian Duerr are with the Onboard Fusion team at CARIAD SE, Volkswagen Group, Ingolstadt, Germany. · Daniel Goehring is with the Dahlem Center for Machine Learning and Robotics group at Freie Universität Berlin, Germany.

3D Occupancy Prediction Autonomous Driving Self-Supervised Learning

Open paper AskPDF arXiv

Estimating 3D occupancy and motion at the vehicle's surroundings is essential for autonomous driving, enabling situational awareness in dynamic environments. Existing approaches jointly learn geometry and motion but rely on expensive 3D occupancy and flow annotations, velocity labels from bounding boxes, or pretrained optical flow models. We propose a self-supervised method for 3D occupancy flow estimation that eliminates the need for human-produced annotations or external flow supervision. Our method disentangles the scene into separate static and dynamic signed distance fields and learns motion implicitly through temporal aggregation. Additionally, we introduce a strong self-supervised flow cue derived from features' cosine similarities. We demonstrate the efficacy of our 3D occupancy flow method on SemanticKITTI, KITTI-MOT, and nuScenes.

Abstract

Scene Flow Estimation

Explore similar work

SelfOccFlow: Towards end-to-end self-supervised 3D Occupancy Flow prediction

SparseOcc++: Geometry-Aware Sparse Latent Representation for Semantic Occupancy Prediction

M2M^2M2-Occ: Resilient 3D Semantic Occupancy Prediction for Autonomous Driving with Incomplete Camera Inputs

$M^2$ -Occ: Resilient 3D Semantic Occupancy Prediction for Autonomous Driving with Incomplete Camera Inputs