SelfOccFlow: Towards end-to-end self-supervised 3D Occupancy Flow prediction

Xavier Timoneda; Markus Herb; Fabian Duerr; Daniel Goehring

cs.CVFeb 27, 2026

SelfOccFlow: Towards end-to-end self-supervised 3D Occupancy Flow prediction

Authors: Xavier Timoneda, Markus Herb, Fabian Duerr, Daniel Goehring

Organizations: Xavier Timoneda, Markus Herb, and Fabian Duerr are with the Onboard Fusion team at CARIAD SE, Volkswagen Group, Ingolstadt, Germany. · Daniel Goehring is with the Dahlem Center for Machine Learning and Robotics group at Freie Universität Berlin, Germany.

Abstract

Estimating 3D occupancy and motion at the vehicle's surroundings is essential for autonomous driving, enabling situational awareness in dynamic environments. Existing approaches jointly learn geometry and motion but rely on expensive 3D occupancy and flow annotations, velocity labels from bounding boxes, or pretrained optical flow models. We propose a self-supervised method for 3D occupancy flow estimation that eliminates the need for human-produced annotations or external flow supervision. Our method disentangles the scene into separate static and dynamic signed distance fields and learns motion implicitly through temporal aggregation. Additionally, we introduce a strong self-supervised flow cue derived from features' cosine similarities. We demonstrate the efficacy of our 3D occupancy flow method on SemanticKITTI, KITTI-MOT, and nuScenes.

Figures & tables

Fig. 1: Similarity flow self-supervision on SemanticKITTI . We provide explicit flow supervision by comparing the dynamic BEV features at current time $t$ and its adjacent frames $t\!\pm\!1$ aligned by ego motion. The flow pseudo-labels $\mathbf{f}^{s}_{t^{-}},\mathbf{f}^{s}_{t^{+}}$ are obtained from the cosine similarities between each cell in the current timestep and its $N\!\times\!N$ neighbors at $t\!\pm\!1$ .

Fig. 2: Overview of our method. a) Occupancy-Flow model. Baseline pipeline used for both training and inference. The multi-view images from time $t$ are processed by a 2D backbone, and their features are fused in the 3D encoder. Half of the encoder’s attention heads produce static BEV features $\beta^{s}_{t}$ , and the other half produce dynamic ones $\beta^{d}_{t}$ . Both $\beta^{s}_{t}$ and $\beta^{d}_{t}$ are fused with their previous frame features $\beta^{s}_{t-1}$ , $\beta^{d}_{t-1}$ aligned by ego motion. The fused static features $\beta^{s}_{t_{\text{hist}}}$ are used to predict the static SDF $\phi^{s}_{t}$ , while the fused dynamic features $\beta^{d}_{t_{\text{hist}}}$ are used to predict the dynamic SDF $\phi^{d}_{t}$ , and flow $\mathbf{f}$ . b) Temporal aggregation. Static field predictions $\phi^{s}_{t}$ are aggregated with $\phi^{s}_{t-1}$ and $\phi^{s}_{t+1}$ , aligned by ego motion. For the dynamic field $\phi^{d}_{t}$ , the sampling positions at $\phi^{d}_{t-1}$ and $\phi^{d}_{t+1}$ are first warped by the flow $\mathbf{f}$ , enabling implicit flow learning. c) Ray-based supervision of aggregated fields $\bar{\phi}^{s}_{t},\bar{\phi}^{d}_{t}$ using photometric $\mathcal{L}_{photo}$ and LiDAR $\mathcal{L}_{lidar}$ losses. $\mathcal{L}_{photo}$ is applied to the blended field $\phi^{b}_{t}$ using camera rays from time $t$ only. $\mathcal{L}_{lidar}$ supervises dynamic predictions $\phi^{d}_{t}$ using dynamic LiDAR rays from $t$ , and static predictions $\phi^{s}_{t}$ using static LiDAR rays from $t\pm k$ neighbors, leveraging its stationary nature. d) Similarity flow supervision using auto-generated flow pseudo-labels $\mathbf{f}^{s}_{t^{-}},\mathbf{f}^{s}_{t^{+}}$ , obtained as the argmax of the cosine similarity between $\beta^{d}_{t}$ and $\beta^{d}_{t\pm 1}$ over $N\!\times\!N$ neighboring cells.

Fig. 3: 3D occupancy comparison on SemanticKITTI [ 1 ] . We draw white boxes on areas with notable changes. Our model predicts better occupancy for small dynamic objects, and infers better geometries in occluded regions such as behind cars.

Method	3D Occupancy
Method	RayIoU 1m,2m,4m $\uparrow$			RayIoU $\uparrow$
MonoScene [ 42 ]	26.00	36.50	49.09	37.19
SceneRF [ 35 ]	20.65	35.74	56.35	37.58
OccNeRF [ 10 ]	23.22	40.17	62.57	41.99
SelfOcc [ 7 ]	23.13	35.85	50.31	36.27
LetOccFlow [ 6 ]	28.62	45.69	66.95	47.06

TABLE I: 3D occupancy prediction on SemanticKITTI [ 1 ] . For OccNeRF [ 10 ] , we use the results from LetOccFlow’s [ 6 ] adaptation to SemanticKITTI [ 1 ] .

Fig. 4: 3D occupancy flow results on KITTI-MOT [ 2 ] . Flow legend displayed at the top right corner.

Method	Supervision	3D Occupancy & Occupancy Flow
Method	Supervision	DE $\downarrow$	EPE $\downarrow$	$\text{DE}_{\text{FG}}\!\downarrow$	$\text{EPE}_{\text{FG}}\!\downarrow$	$\text{D}_{5\%}\!\downarrow$	$\text{Fl}_{10\%}\!\downarrow$	$\text{SF}_{10\%}\!\downarrow$
OccNeRF-C⁢ [ 10 ]	C+O	3.547	6.986	7.976	10.445	0.204	0.272	0.306
LetOccFlow-C⁢ [ 6 ]	C+O	3.431	3.529	6.429	7.635	0.215	0.084	0.118
OccNeRF-L⁢ [ 10 ]	C+L+O	2.667	8.131	4.040	11.284	0.137	0.309	0.326
LetOccFlow-L⁢ [ 6 ]	C+L+O	2.610	3.488	3.106	7.510	0.154	0.083	0.105
Ours	C+L	1.907	6.788	2.435	11.638	0.112	0.171	0.182

TABLE II: 3D occupancy flow prediction on KITTI-MOT [ 2 ] . Ours - SemKITTI is trained on SemanticKITTI [ 1 ] and evaluated on KITTI-MOT [ 2 ] without fine-tuning (not directly comparable, shown to illustrate generalization).

Fig. 5: 3D occupancy flow results on nuScenes [ 3 ] . Flow colors are the same as in KITTI-MOT.

Method	3D Occupancy & Occupancy Flow
Method	RayIoU 1m,2m,4m $\uparrow$			RayIoU $\uparrow$	mAVE $\downarrow$
OccNet [ 41 ]	29.28	39.68	50.02	39.66	1.61
OccNeRF-C⁢ [ 10 ]	9.93	19.06	35.84	21.61	1.53
LetOccFlow-C⁢ [ 6 ]	17.49	28.52	44.33	30.12	1.42
RenderOcc⁢ [ 8 ]	20.27	32.68	49.92	36.67	1.63
OccNeRF-L⁢ [ 10 ]	16.62	29.25	49.17	31.68	1.59

TABLE III: 3D occupancy flow prediction on nuScenes [ 3 ] . OccNet is trained with 3D labels. RenderOcc, OccNeRF-L, LetOccFlow-L and ours use LiDAR supervision. All methods except ours (and OccNet) use optical flow supervision.

Method	Params (M) $\downarrow$	FPS $\uparrow$ (V100)	FLOPs (G) $\downarrow$
LetOccFlow [ 6 ]	253.3	1.04	3202
Ours	32.4	3.78	405

TABLE IV: Model complexity and inference efficiency on SemanticKITTI [ 1 ] . The values for LetOccFlow [ 6 ] are obtained experimentally using their official checkpoint.

Method	$\text{DE}\!\downarrow$	$\text{EPE}\!\downarrow$	$\text{D}_{\text{5\%}}\!\downarrow$	$\text{Fl}_{\text{10\%}}\!\downarrow$	$\text{SF}_{\text{10\%}}\!\downarrow$
Ours (w/o T.A.)	1.948	7.074	0.116	0.179	0.191
Ours (w/o dyn. T.A.)	1.909	6.861	0.114	0.177	0.188
Ours	1.907	6.788	0.112	0.171	0.182

TABLE V: Ablation study on KITTI-MOT [ 2 ] . Individual contributions of static and dynamic temporal aggregation.

3D Occupancy Prediction

3D occupancy prediction estimates the geometry and semantics of occupied and free space around an agent from camera, LiDAR, or radar data.

See all 113 papers

Oct 3, 2026

SelectOccFlow: Selective Spatiotemporal Aggregation for 3D Occupancy and Scene Flow Prediction

Comprehensive 3D scene understanding for autonomous driving requires modeling geometry, semantics, and motion. However, camera-based occupancy and scene flow prediction are sensitive to unreliable spatial and temporal aggregation, caused by semantically incompatible image features, misaligned historical observations, and incomplete voxel structures. To address this issue, we propose SelectOccFlow, a selective spatiotemporal aggregation framework that progressively refines contextual evidence across image, temporal, and voxel domains. To obtain semantically compatible image evidence, we design Semantic-Guided Sampling (SGS) to regulate feature sampling with semantic priors. Since reliable image evidence alone cannot resolve temporal inconsistency, we then present State-Conditioned Temporal Aggregation (SCTA) to selectively retrieve historical evidence according to voxel states. To further enhance the structural completeness of voxel representations, we introduce Extent-Aware Spatial Aggregation (ESA), which exploits directional structural support to refine foreground geometry. Experiments on OpenOcc demonstrate that SelectOccFlow achieves a state-of-the-art OccScore of 44.9, improving the previous best by +4.2%. It also maintains competitive occupancy performance on Occ3D-nus and improves the mean OccScore under nuScenes-C corruptions by +11.1%, demonstrating improved robustness to visual corruptions. The source code will be made publicly available at https://github.com/muchen1021/SelectOccFlow.

Yuhang Wang, Kai Luo, Yuanfan Zheng +1

School of Artificial Intelligence and Robotics and the National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, China · National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, China

Scene Flow Estimation 3D Occupancy Prediction 3D Scene Understanding

Ask PDF

Feb 22, 2026

TeFlow: Enabling Multi-frame Supervision for Self-Supervised Feed-forward Scene Flow Estimation

Self-supervised feed-forward methods for scene flow estimation offer real-time efficiency, but their supervision from two-frame point correspondences is unreliable and often breaks down under occlusions. Multi-frame supervision has the potential to provide more stable guidance by incorporating motion cues from past frames, yet naive extensions of two-frame objectives are ineffective because point correspondences vary abruptly across frames, producing inconsistent signals. In the paper, we present TeFlow, enabling multi-frame supervision for feed-forward models by mining temporally consistent supervision. TeFlow introduces a temporal ensembling strategy that forms reliable supervisory signals by aggregating the most temporally consistent motion cues from a candidate pool built across multiple frames. Extensive evaluations demonstrate that TeFlow establishes a new state-of-the-art for self-supervised feed-forward methods, achieving performance gains of up to 33% on the challenging Argoverse 2 and nuScenes datasets. Our method performs on par with leading optimization-based methods, yet speeds up 150 times. The code is open-sourced at https://github.com/Kin-Zhang/TeFlow along with trained model weights.

Qingwen Zhang, Chenhan Jiang, Xiaomeng Zhu +4

KTH Royal Institute of Technology · Hong Kong University of Science and Technology · Scania +2

Scene Flow Estimation Self-Supervised Learning

Ask PDF

Sep 25, 2026

TriO: Tri-Modal Unsupervised Occupancy World Model for Anything Perception

We present TriO, a multi-modal unsupervised world model that predicts 4D occupancy, obstacle segmentation, flow and LiDAR. In contrast to prior work, TriO utilizes three distinct sensor modalities (camera, LiDAR, and RADAR) as both inputs and sources of self-supervision, eliminating the need for additional human annotations. Thanks to its novel supervision, the model is able to segment any occupancy from the drivable surface, overcoming the limitations of existing open-set methods in handling long-tail objects. TriO achieves state-of-the-art results in multiple 3D and 4D tasks, including occupancy, flow, and LiDAR prediction, as well as zero-shot road obstacle segmentation across multiple datasets such as Argoverse 2, and Spotting the Unexpected.

Quinlan Sykora, Sourav Biswas, Christopher Diehl +3

Waabi · University of Toronto

Unsupervised Learning Autonomous Driving Perception 3D Occupancy Prediction World Models for Autonomous Driving Multimodal Sensor Fusion

Ask PDF

Abstract

3D Occupancy Prediction

Explore similar work

SelectOccFlow: Selective Spatiotemporal Aggregation for 3D Occupancy and Scene Flow Prediction

TeFlow: Enabling Multi-frame Supervision for Self-Supervised Feed-forward Scene Flow Estimation

TriO: Tri-Modal Unsupervised Occupancy World Model for Anything Perception