SelfOccFlow: Towards end-to-end self-supervised 3D Occupancy Flow prediction

Xavier Timoneda; Markus Herb; Fabian Duerr; Daniel Goehring

cs.CVFeb 27, 2026

SelfOccFlow: Towards end-to-end self-supervised 3D Occupancy Flow prediction

Authors: Xavier Timoneda, Markus Herb, Fabian Duerr, Daniel Goehring

Organizations: Xavier Timoneda, Markus Herb, and Fabian Duerr are with the Onboard Fusion team at CARIAD SE, Volkswagen Group, Ingolstadt, Germany. · Daniel Goehring is with the Dahlem Center for Machine Learning and Robotics group at Freie Universität Berlin, Germany.

Abstract

Estimating 3D occupancy and motion at the vehicle's surroundings is essential for autonomous driving, enabling situational awareness in dynamic environments. Existing approaches jointly learn geometry and motion but rely on expensive 3D occupancy and flow annotations, velocity labels from bounding boxes, or pretrained optical flow models. We propose a self-supervised method for 3D occupancy flow estimation that eliminates the need for human-produced annotations or external flow supervision. Our method disentangles the scene into separate static and dynamic signed distance fields and learns motion implicitly through temporal aggregation. Additionally, we introduce a strong self-supervised flow cue derived from features' cosine similarities. We demonstrate the efficacy of our 3D occupancy flow method on SemanticKITTI, KITTI-MOT, and nuScenes.

Figures & tables

Fig. 1: Similarity flow self-supervision on SemanticKITTI . We provide explicit flow supervision by comparing the dynamic BEV features at current time $t$ and its adjacent frames $t\!\pm\!1$ aligned by ego motion. The flow pseudo-labels $\mathbf{f}^{s}_{t^{-}},\mathbf{f}^{s}_{t^{+}}$ are obtained from the cosine similarities between each cell in the current timestep and its $N\!\times\!N$ neighbors at $t\!\pm\!1$ .

Fig. 2: Overview of our method. a) Occupancy-Flow model. Baseline pipeline used for both training and inference. The multi-view images from time $t$ are processed by a 2D backbone, and their features are fused in the 3D encoder. Half of the encoder’s attention heads produce static BEV features $\beta^{s}_{t}$ , and the other half produce dynamic ones $\beta^{d}_{t}$ . Both $\beta^{s}_{t}$ and $\beta^{d}_{t}$ are fused with their previous frame features $\beta^{s}_{t-1}$ , $\beta^{d}_{t-1}$ aligned by ego motion. The fused static features $\beta^{s}_{t_{\text{hist}}}$ are used to predict the static SDF $\phi^{s}_{t}$ , while the fused dynamic features $\beta^{d}_{t_{\text{hist}}}$ are used to predict the dynamic SDF $\phi^{d}_{t}$ , and flow $\mathbf{f}$ . b) Temporal aggregation. Static field predictions $\phi^{s}_{t}$ are aggregated with $\phi^{s}_{t-1}$ and $\phi^{s}_{t+1}$ , aligned by ego motion. For the dynamic field $\phi^{d}_{t}$ , the sampling positions at $\phi^{d}_{t-1}$ and $\phi^{d}_{t+1}$ are first warped by the flow $\mathbf{f}$ , enabling implicit flow learning. c) Ray-based supervision of aggregated fields $\bar{\phi}^{s}_{t},\bar{\phi}^{d}_{t}$ using photometric $\mathcal{L}_{photo}$ and LiDAR $\mathcal{L}_{lidar}$ losses. $\mathcal{L}_{photo}$ is applied to the blended field $\phi^{b}_{t}$ using camera rays from time $t$ only. $\mathcal{L}_{lidar}$ supervises dynamic predictions $\phi^{d}_{t}$ using dynamic LiDAR rays from $t$ , and static predictions $\phi^{s}_{t}$ using static LiDAR rays from $t\pm k$ neighbors, leveraging its stationary nature. d) Similarity flow supervision using auto-generated flow pseudo-labels $\mathbf{f}^{s}_{t^{-}},\mathbf{f}^{s}_{t^{+}}$ , obtained as the argmax of the cosine similarity between $\beta^{d}_{t}$ and $\beta^{d}_{t\pm 1}$ over $N\!\times\!N$ neighboring cells.

Fig. 3: 3D occupancy comparison on SemanticKITTI [ 1 ] . We draw white boxes on areas with notable changes. Our model predicts better occupancy for small dynamic objects, and infers better geometries in occluded regions such as behind cars.

Method	3D Occupancy
Method	RayIoU 1m,2m,4m $\uparrow$			RayIoU $\uparrow$
MonoScene [ 42 ]	26.00	36.50	49.09	37.19
SceneRF [ 35 ]	20.65	35.74	56.35	37.58
OccNeRF [ 10 ]	23.22	40.17	62.57	41.99
SelfOcc [ 7 ]	23.13	35.85	50.31	36.27
LetOccFlow [ 6 ]	28.62	45.69	66.95	47.06

TABLE I: 3D occupancy prediction on SemanticKITTI [ 1 ] . For OccNeRF [ 10 ] , we use the results from LetOccFlow’s [ 6 ] adaptation to SemanticKITTI [ 1 ] .

Fig. 4: 3D occupancy flow results on KITTI-MOT [ 2 ] . Flow legend displayed at the top right corner.

Method	Supervision	3D Occupancy & Occupancy Flow
Method	Supervision	DE $\downarrow$	EPE $\downarrow$	$\text{DE}_{\text{FG}}\!\downarrow$	$\text{EPE}_{\text{FG}}\!\downarrow$	$\text{D}_{5\%}\!\downarrow$	$\text{Fl}_{10\%}\!\downarrow$	$\text{SF}_{10\%}\!\downarrow$
OccNeRF-C⁢ [ 10 ]	C+O	3.547	6.986	7.976	10.445	0.204	0.272	0.306
LetOccFlow-C⁢ [ 6 ]	C+O	3.431	3.529	6.429	7.635	0.215	0.084	0.118
OccNeRF-L⁢ [ 10 ]	C+L+O	2.667	8.131	4.040	11.284	0.137	0.309	0.326
LetOccFlow-L⁢ [ 6 ]	C+L+O	2.610	3.488	3.106	7.510	0.154	0.083	0.105
Ours	C+L	1.907	6.788	2.435	11.638	0.112	0.171	0.182

TABLE II: 3D occupancy flow prediction on KITTI-MOT [ 2 ] . Ours - SemKITTI is trained on SemanticKITTI [ 1 ] and evaluated on KITTI-MOT [ 2 ] without fine-tuning (not directly comparable, shown to illustrate generalization).

Fig. 5: 3D occupancy flow results on nuScenes [ 3 ] . Flow colors are the same as in KITTI-MOT.

Method	3D Occupancy & Occupancy Flow
Method	RayIoU 1m,2m,4m $\uparrow$			RayIoU $\uparrow$	mAVE $\downarrow$
OccNet [ 41 ]	29.28	39.68	50.02	39.66	1.61
OccNeRF-C⁢ [ 10 ]	9.93	19.06	35.84	21.61	1.53
LetOccFlow-C⁢ [ 6 ]	17.49	28.52	44.33	30.12	1.42
RenderOcc⁢ [ 8 ]	20.27	32.68	49.92	36.67	1.63
OccNeRF-L⁢ [ 10 ]	16.62	29.25	49.17	31.68	1.59

TABLE III: 3D occupancy flow prediction on nuScenes [ 3 ] . OccNet is trained with 3D labels. RenderOcc, OccNeRF-L, LetOccFlow-L and ours use LiDAR supervision. All methods except ours (and OccNet) use optical flow supervision.

Method	Params (M) $\downarrow$	FPS $\uparrow$ (V100)	FLOPs (G) $\downarrow$
LetOccFlow [ 6 ]	253.3	1.04	3202
Ours	32.4	3.78	405

TABLE IV: Model complexity and inference efficiency on SemanticKITTI [ 1 ] . The values for LetOccFlow [ 6 ] are obtained experimentally using their official checkpoint.

Method	$\text{DE}\!\downarrow$	$\text{EPE}\!\downarrow$	$\text{D}_{\text{5\%}}\!\downarrow$	$\text{Fl}_{\text{10\%}}\!\downarrow$	$\text{SF}_{\text{10\%}}\!\downarrow$
Ours (w/o T.A.)	1.948	7.074	0.116	0.179	0.191
Ours (w/o dyn. T.A.)	1.909	6.861	0.114	0.177	0.188
Ours	1.907	6.788	0.112	0.171	0.182

TABLE V: Ablation study on KITTI-MOT [ 2 ] . Individual contributions of static and dynamic temporal aggregation.

3D Occupancy Prediction

3D occupancy prediction estimates the geometry and semantics of occupied and free space around an agent from camera, LiDAR, or radar data.

See all 113 papers

SelectOccFlow: Selective Spatiotemporal Aggregation for 3D Occupancy and Scene Flow Prediction
Oct 3, 2026Yuhang Wang, Kai Luo, Yuanfan Zheng +1Scene Flow Estimation 3D Occupancy Prediction
School of Artificial Intelligence and Robotics and the National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, China · National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, China
TeFlow: Enabling Multi-frame Supervision for Self-Supervised Feed-forward Scene Flow Estimation
Feb 22, 2026Qingwen Zhang, Chenhan Jiang, Xiaomeng Zhu +4Scene Flow Estimation Self-Supervised Learning
KTH Royal Institute of Technology · Hong Kong University of Science and Technology · Scania +2
TriO: Tri-Modal Unsupervised Occupancy World Model for Anything Perception
Sep 25, 2026Quinlan Sykora, Sourav Biswas, Christopher Diehl +3Unsupervised Learning Autonomous Driving Perception
Waabi · University of Toronto

Oct 3, 2026J/K move · Enter open · S save

SelectOccFlow: Selective Spatiotemporal Aggregation for 3D Occupancy and Scene Flow Prediction

Yuhang Wang, Kai Luo, Yuanfan Zheng, Kailun Yang

School of Artificial Intelligence and Robotics and the National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, China · National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, China

Scene Flow Estimation 3D Occupancy Prediction 3D Scene Understanding

Open paper AskPDF arXiv

Comprehensive 3D scene understanding for autonomous driving requires modeling geometry, semantics, and motion. However, camera-based occupancy and scene flow prediction are sensitive to unreliable spatial and temporal aggregation, caused by semantically incompatible image features, misaligned historical observations, and incomplete voxel structures. To address this issue, we propose SelectOccFlow, a selective spatiotemporal aggregation framework that progressively refines contextual evidence across image, temporal, and voxel domains. To obtain semantically compatible image evidence, we design Semantic-Guided Sampling (SGS) to regulate feature sampling with semantic priors. Since reliable image evidence alone cannot resolve temporal inconsistency, we then present State-Conditioned Temporal Aggregation (SCTA) to selectively retrieve historical evidence according to voxel states. To further enhance the structural completeness of voxel representations, we introduce Extent-Aware Spatial Aggregation (ESA), which exploits directional structural support to refine foreground geometry. Experiments on OpenOcc demonstrate that SelectOccFlow achieves a state-of-the-art OccScore of 44.9, improving the previous best by +4.2%. It also maintains competitive occupancy performance on Occ3D-nus and improves the mean OccScore under nuScenes-C corruptions by +11.1%, demonstrating improved robustness to visual corruptions. The source code will be made publicly available at https://github.com/muchen1021/SelectOccFlow.

Abstract

3D Occupancy Prediction

Explore similar work

SelectOccFlow: Selective Spatiotemporal Aggregation for 3D Occupancy and Scene Flow Prediction

TeFlow: Enabling Multi-frame Supervision for Self-Supervised Feed-forward Scene Flow Estimation

TriO: Tri-Modal Unsupervised Occupancy World Model for Anything Perception