OccStress: Stress-Testing the 4D Occupancy Forecasting Chain

Yu Zheng; Jie Hu; Jiaqi Xiong; Ruiping Liu; Junwei Zheng; Kailun Yang; Jiaming Zhang

cs.CVDec 17, 2025

OccStress: Stress-Testing the 4D Occupancy Forecasting Chain

Authors: Yu Zheng, Jie Hu, Jiaqi Xiong, Ruiping Liu, Junwei Zheng, Kailun Yang, Jiaming Zhang

Organizations: Hunan University · ETH Zurich

Abstract

Occupancy world models use historical occupancy states to forecast future 3D scenes, but their robustness under corrupted temporal inputs remains poorly understood. Existing evaluations primarily emphasize clean forecasting accuracy and provide limited evidence about how errors enter, persist, and propagate through the occupancy perception-forecasting chain. This paper introduces OccStress, a robustness stress-testing benchmark for the occupancy forecasting chain. OccStress contains 21 corruption families with 61 severity configurations and 10,827 strict temporal anchors across 3 datasets. OccStress covers both 3D occupancy perception and 4D occupancy forecasting through two complementary tracks. This design separates model-mediated pipeline errors under standardized sensor stressors from the intrinsic sensitivity of 4D forecasting models to corrupted occupancy states. OccStress further defines temporal injection protocols to test whether errors in the current state, recent history, or earlier history affect future forecasts differently. OccStress provides aggregate metrics for evaluating robustness along the occupancy forecasting chain. Experiments with five 4D forecasters reveal that current occupancy models are substantially affected by both upstream prediction errors and direct state corruptions, and that clean performance alone is an insufficient description of source- and position-specific robustness. Code, data, and evaluation tools are available at https://insailab.org/OccStress.

Figures & tables

Figure 1 : Overview of the 4D occupancy forecasting chain studied in OccStress . Occupancy world models rely on temporal occupancy states for future forecasting, while errors from upstream perception or direct state perturbations can persist and propagate across the forecasting horizon.

Figure 2 : OccStress data distribution .

Figure 3 : Position-wise temporal sensitivity.

Figure 5 : Qualitative cases of I 2 -World on OccStress-nuScenes under the hard Current-only protocol.

Appendix figures & tables25 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Severity	Bright	Dark	Fog	Snow	Motion	Quant	Crash	Frame
COTR	easy	36.44	24.15	31.85	15.32	36.77	38.44	28.02	30.46
	mid	30.69	20.35	29.98	7.01	25.80	29.36	15.12	16.22
	hard	27.26	14.48	27.61	5.74	21.18	16.53	13.26	9.41
ALOcc	easy	40.36	29.47	36.86	16.71	39.58	41.15	36.09	38.75
	mid	34.39	25.10	35.24	6.62	27.94	32.26	26.58	28.59
	hard	30.36	18.06	33.34	5.68	24.05	17.27	20.61	20.05

Appendix

Table 7 : Full 3D image-corruption results by severity. Each row reports semantic mIoU under one severity level. Method blocks should be compared only within the same input-modality setting in the main paper.

Method	Severity	Fog	Wet	Snow	Motion	Beam	Cross	Echo	Sensor
OccFusion	light	21.90	23.36	21.99	22.62	21.44	22.03	22.13	17.31
	moderate	17.68	23.01	21.11	21.25	18.91	19.86	21.23	14.14
	heavy	10.17	22.59	20.23	19.84	16.24	17.84	19.17	11.52
FusionOcc	light	43.47	45.68	45.38	40.73	45.31	43.37	45.31	44.74
	moderate	39.09	45.76	44.45	36.61	44.69	41.42	45.00	42.92
	heavy	28.90	45.76	43.19	33.13	43.36	39.81	43.93	41.89

Appendix

Table 8 : Full 3D point-cloud-corruption results by severity for the camera–LiDAR baselines with completed point-cloud corruption runs. Cross denotes crosstalk and Sensor denotes cross-sensor corruption.

Method	Source	IoU metric	$\mathbf{CS}$	$\mathbf{RS}$
ALOcc	Image	Occupied IoU	74.81	61.24
ViewFormer	Image	Geometric IoU	71.29	54.56
SparseOcc	Image	RayIoU	36.75	16.45
OccFusion	Image	Binary IoU	36.44	34.04
OccFusion	Point cloud	Binary IoU	36.44	30.79

Appendix

Table 9: Additional 3D IoU-style metrics emitted by baseline-specific evaluators. These auxiliary metrics use the definitions of the original evaluators and are therefore reported for audit completeness rather than cross-method ranking.

CS

is the clean score and

RS

is averaged over the completed corruptions in the indicated source track.

Figure 6 : 3D upstream BEV sequence for a shared scene. Rows show ground truth, STCOcc camera-only occupancy under hard fog, and SDGOcc point-cloud-fusion occupancy under heavy LiDAR fog. Columns cover the temporal window from $t{-}4$ to $t{+}6$ . Only history and current states serve as forecasting inputs. Future columns are shown for visual reference.

Figure 7 : STCOcc camera-only BEV outputs for the same current frame under clean input and representative hard camera corruptions. The fixed crop makes source-side structural changes directly comparable across corruptions.

Figure 8 : Current-frame BEV error maps against ground truth. Green denotes correctly predicted occupied cells, red missing occupied cells, blue hallucinated occupied cells, and orange semantic class swaps. The panels separate geometric failures from semantic confusion for STCOcc hard fog and SDGOcc heavy LiDAR fog.

Forecaster	Upstream	$S$	$CS^{S}$	$RS^{S}_{\forall}$	$RS^{S}_{\rm Cur}$	$RS^{S}_{\rm Burst}$	$RS^{S}_{\rm Hist}$
I 2 -World	ALOcc	I	9.29	7.21	6.27	6.28	9.07
GenieDrive	ALOcc	I	8.86	7.14	6.34	6.26	8.81
I 2 -World	FusionOcc	I	10.42	8.91	8.18	8.17	10.37
I 2 -World	FusionOcc	P	10.42	9.93	9.70	9.70	10.38
GenieDrive	FusionOcc	I	9.52	8.16	7.54	7.47	9.46
GenieDrive	FusionOcc	P	9.52	9.14	8.97	8.94	9.49

Appendix

Table 14: Additional 4D upstream-chain results on OccStress-nuScenes. Semantic mIoU (%) averages the physical 1s, 2s, and 3s horizons.

S

identifies the corrupted sensor source, not the estimator’s full input modality.

RS^{S}_{\forall}

averages all 72 corrupted conditions, and each protocol column averages eight families and three severities.

Dataset	$CS^{I}$	$RS^{I}_{\forall}$	$RS^{I}_{\rm Cur}$	$RS^{I}_{\rm Burst}$	$RS^{I}_{\rm Hist}$
OccStress-nuScenes	13.18	11.74	12.44	11.79	10.99
OccStress-Waymo	3.69	3.00	2.70	2.66	3.64
OccStress-CARLA	4.59	3.57	3.06	3.05	4.59

Appendix

Table 15: SparseWorld-TC camera-source robustness on the common H4/F6 inventories: 4,519 OccStress-nuScenes, 5,978 OccStress-Waymo, and 330 OccStress-CARLA anchors. Semantic mIoU (%) averages 1s, 2s, and 3s.

RS^{I}_{\forall}

averages 72 corrupted configurations, and each protocol column averages eight families and three severities.

Setting	Original	Fine-tuned	$\Delta$
Semantic	32.90	32.84	-0.05
Dropout	34.29	33.25	-1.05
Hole	39.34	38.08	-1.26
Traffic (held out)	38.07	37.77	-0.30

Appendix

Table 19: Corruption-aware fine-tuning of I 2 -World on 4,519 OccStress-nuScenes anchors. Semantic mIoU (%) averages 1s, 2s, and 3s. Content-corruption rows also average three severities and three protocols. Traffic uses its single mirrored-sequence protocol.

\Delta

is fine-tuned minus original, computed before rounding.

Method	others	barrier	bicycle	bus	car	constr. veh.	motor.	ped.	traffic cone	trailer	truck	drive. surf.	other flat	sidewalk	terrain	manmade	veget.	mIoU
OccWorld	12.3	21.1	8.4	20.8	19.9	12.9	7.8	8.9	8.5	13.2	17.7	35.1	24.0	24.1	20.7	17.2	20.1	17.21
DOME	27.0	32.9	17.6	25.0	25.1	32.0	16.1	9.4	17.3	23.7	27.1	42.7	33.7	32.5	29.3	29.1	34.1	26.75
COME	35.3	42.6	24.3	30.5	30.8	35.1	27.0	20.2	26.1	29.6	28.0	53.3	43.8	41.5	40.3	38.0	40.3	34.52
I 2 -World	40.9	50.2	26.1	35.0	36.0	39.1	26.8	18.6	31.2	36.4	40.7	62.3	50.7	49.7	50.3	42.4	39.9	39.78
GenieDrive	46.1	51.7	29.9	36.5	38.5	43.9	24.2	21.3	36.8	39.5	41.3	66.1	55.0	55.3	53.1	43.4	41.5	42.60

Appendix

Table 20 : Clean 4D forecasting by semantic class. Class entries are per-class semantic IoU (%) averaged over the 1s, 2s, and 3s forecasting horizons on the 4,519-anchor population. mIoU averages the 17 occupied classes, excluding free space. The colored squares follow the OccWorld Occ3D BEV palette.

Figure 9 : Position-wise relative mIoU drop (%) averaged over ten hard settings. Rows within each panel are input times and columns are future times, in seconds relative to the anchor. N/A means that the model does not observe that state.

Quantity	Estimate	95% CI
$mRS$	23.63	[22.92, 24.30]
$CRF_{2}$	25.71	[24.97, 26.43]
$CS^{O}-mRS$	16.15	[15.50, 16.81]
$CS^{I}-RS^{I}_{\forall}$	1.98	[1.85, 2.11]
$CS^{P}-RS^{P}_{\forall}$	2.11	[1.99, 2.23]
$CS^{O}-RS^{O}_{\forall}$	3.54	[3.34, 3.74]

Appendix

Table 22: Scene-clustered uncertainty estimates for I 2 -World on OccStress-nuScenes. Differences are computed within each paired bootstrap replicate.

$S$	$CS^{S}$	$RS^{S}_{\forall}$	$RS^{S}_{\rm Cur}$	$RS^{S}_{\rm Burst}$	$RS^{S}_{\rm Hist}$
I	17.28 [16.54, 18.04]	15.31 [14.63, 15.99]	14.44 [13.80, 15.10]	14.50 [13.85, 15.16]	16.98 [16.25, 17.73]
P	21.44 [20.43, 22.47]	19.33 [18.43, 20.26]	18.36 [17.50, 19.25]	18.34 [17.48, 19.23]	21.29 [20.29, 22.31]
O	39.78 [38.66, 40.87]	36.24 [35.23, 37.22]	34.57 [33.61, 35.51]	34.53 [33.57, 35.47]	39.62 [38.49, 40.71]

Appendix

Table 23 : Source-wise and source-conditioned protocol scores with 95% CIs. Sources

I/P/O

use STCOcc, SDGOcc, and ground-truth occupancy, respectively.

$S$	Family	mIoU [95% CI]
I	Brightness	16.03 [15.32, 16.74]
I	Crash	15.45 [14.76, 16.17]
I	Quant	15.81 [15.11, 16.53]
I	Fog	15.80 [15.13, 16.49]
I	Frame	15.26 [14.59, 15.95]
I	Dark	14.76 [14.10, 15.43]

Appendix

Table 24 : Corruption-family scores with 95% CIs. Each ordinary family averages nine severity–protocol configurations. Traffic has one configuration.

$S$	Family	Severity	Cur	Burst	Hist
I	Brightness	E	16.65 [15.91, 17.39]	16.66 [15.92, 17.40]	17.23 [16.49, 18.00]
I	Brightness	M	15.31 [14.59, 16.02]	15.39 [14.66, 16.10]	17.14 [16.39, 17.89]
I	Brightness	H	14.36 [13.65, 15.04]	14.43 [13.72, 15.11]	17.09 [16.35, 17.85]
I	Crash	E	16.07 [15.32, 16.81]	16.09 [15.35, 16.84]	17.08 [16.33, 17.84]
I	Crash	M	15.11 [14.37, 15.85]	15.16 [14.41, 15.91]	16.96 [16.22, 17.72]
I	Crash	H	12.98 [12.32, 13.66]	13.01 [12.35, 13.69]	16.63 [15.90, 17.38]

Appendix

Table 25 : All corrupted-configuration scores with 95% CIs. Together with the three clean scores in Table 23 , these cover all 184 configurations. E/M/H denotes increasing severity. Traffic’s single fixed-protocol result is shown across the three protocol columns.

Family	Changed voxels	CV	Pairwise Jaccard
Semantic	2,939.6	0.69%	0.310
Dropout	12,996.6	0.19%	0.447
Hole	246.7	2.24%	0.868

Appendix

Table 28: Five-realization sensitivity of stochastic manual corruptions. CV is computed across realization-level mean changed-voxel counts. Jaccard pools full-grid changed-voxel intersections and unions over all 5,119 frames for each pair, then averages the ten realization pairs.

Figure 10 : Clean source quality and average robustness across the manual, camera-only, and point-cloud-fusion tracks. The figure separates downstream forecasting robustness from the clean-source reference performance.

Figure 11 : Upstream 3D occupancy corruption propagated into 4D forecasting. Camera-only panels use STCOcc hard corruptions; point-cloud-fusion panels use SDGOcc heavy corruptions. Each cell reports horizon-averaged semantic mIoU under the corresponding temporal protocol.

Figure 12 : Upstream corruption severity sweeps averaged over temporal protocols. Camera-only columns correspond to easy, mid, and hard severities; point-cloud-fusion columns correspond to light, moderate, and heavy severities.

Figure 13 : BEV visualization of manual occupancy-state corruptions on a shared scene. Each row compares the clean input with easy, moderate, and hard variants using the same crop, projection, and Occ3D semantic color map.

Panel of Figure 14 — Figure 14 : Manual occupancy-state corruption diagnostics. Each heatmap cell reports horizon-averaged semantic mIoU; higher is better.

Panel of Figure 15 — Figure 15 : Manual-track horizon and traffic diagnostics. Traffic is a structured scene transformation and is therefore shown separately from noisy-input state corruptions.

Figure 16 : Matched I 2 -World and GenieDrive predictions under the Current-only hard protocol. Each row uses one shared scene and anchor.

Figure 17 : Matched I 2 -World and GenieDrive predictions under the Recent-burst hard protocol, using the same anchors as Figure 16 .

Figure 18 : Matched I 2 -World and GenieDrive predictions under the History-only hard protocol, using the same anchors as Figure 16 .

Explore similar work

Jul 27, 2026cs.CV

InterOCF: Spatio-Temporal 2D-3D Interaction for Camera-Only 4D Occupancy Forecasting

Camera-only 4D occupancy forecasting enables autonomous vehicles to predict future 3D semantic scenes solely from historical multi-view images, which is critical for driving safety. Even though current methods have achieved good performance, the strong spatial-temporal modeling between the input multi-view frames is still underexplored, which limits the performance of those methods in future 4D forecasting. To address this gap, we introduce a novel framework, InterOCF, for 4D occupancy forecasting that jointly models temporal dynamics in both 3D voxel-based representations and multi-view segmentation sequences, while explicitly incorporating feature interaction between the 2D and 3D branches. Our framework incorporates three core components: 1) A 3D Spatio-Temporal (3DST) module that learns volumetric dynamics from historical voxel states to predict future voxel states; 2) A 2D Spatio-Temporal (2DST) module employing an auxiliary multi-view temporal segmentation forecasting task to enhance temporal semantic dynamics; 3) A Spatio-Temporal Interaction Modeling (STIM) module that enables feature interaction between 2D and 3D representations. Experiments on the nuScenes, Lyft-Level5, and nuScenes-Occupancy datasets show that InterOCF consistently outperforms existing baseline approaches.

Qi Zhang, Xinquan Yu, Kaiyi Zhang +1

College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China.

3D Semantic Occupancy Prediction Occupancy Voxel 3D Scene Understanding

Ask PDF

Oct 7, 2026cs.CV

LighTROcc: Lightweight 4D Occupancy Forecasting via Instance-Centric 3D Gaussians

Forecasting future 3D occupancy from surround-view cameras is essential for autonomous driving, yet existing approaches rely on dense voxel or bird's-eye-view representations whose cost grows rapidly with spatial resolution and prediction horizon. Because these representations do not explicitly maintain object identities, they also struggle to preserve instance consistency over time. We present LighTROcc, a lightweight instance-centric framework that represents movable objects with a compact set of learned queries and predicts present and future occupancy in a single forward pass. LighTROcc localizes each query through attention-guided forward lifting, combining image-space cross-attention, query-specific depth, and camera geometry to estimate its 3D center. Each instance is modeled as a mixture of anisotropic 3D Gaussians and propagated across future steps using predicted displacements, producing continuous, temporally consistent occupancy forecasts. Experiments on nuScenes and supplemented nuScenes-Occupancy show that LighTROcc outperforms the evaluated dense and instance-wise baselines in instance-level forecasting accuracy while maintaining strong voxel-level occupancy quality. Across different model configurations, LighTROcc achieves a favorable balance between forecasting accuracy and computational efficiency, demonstrating the potential of compact instance-centric modeling for camera-based 4D occupancy forecasting.

Hwanhee Jung, SeungHyeon Kim, Inkyu Koo +3

3D Semantic Occupancy Prediction 3D Object Detection Dense Prediction

Ask PDF

Mar 9, 2026cs.CV

OccTrack360: 4D Panoptic Occupancy Tracking from Surround-View Fisheye Cameras

Understanding dynamic 3D environments in a spatially continuous and temporally consistent manner is fundamental for robotics and autonomous driving. While recent advances in occupancy prediction provide a unified representation of scene geometry and semantics, progress in 4D panoptic occupancy tracking remains limited by the lack of benchmarks that support surround-view fisheye sensing, long temporal sequences, and instance-level voxel tracking. To address this gap, we present OccTrack360, a new benchmark for 4D panoptic occupancy tracking from surround-view fisheye cameras. OccTrack360 provides substantially longer and more diverse sequences (174~2234 frames) than prior benchmarks, together with principled voxel visibility annotations, including an all-direction occlusion mask and an MEI-based fisheye field-of-view mask. To establish a strong fisheye-oriented baseline, we further propose Focus on Sphere Occ (FoSOcc), a framework that addresses two core challenges in fisheye occupancy tracking: distorted spherical projection and inaccurate voxel-space localization. FoSOcc includes a Center Focusing Module (CFM) to enhance instance-aware spatial localization through supervised focus guidance, and a Fisheye-based Enhanced Lifting (FEL) that extends perspective lifting to fisheye imaging under the Unified Projection Model. Extensive experiments on Occ3D-Waymo and OccTrack360 show that our method improves occupancy tracking quality with notable gains on geometrically regular categories, and establishes a strong baseline for future research on surround-view fisheye 4D occupancy tracking. The benchmark and source code will be made publicly available at https://github.com/YouthZest-Lin/OccTrack360.

Yongzhi Lin, Kai Luo, Yuanfan Zheng +4

School of Artificial Intelligence and Robotics and the National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, Changsha, China · Ant Group, Hangzhou, China · State Key Laboratory of Extreme Photonics and Instrumentation, Zhejiang University, Hangzhou, China

3D Semantic Occupancy Prediction Camera Voxel 4D Autonomous Driving

Ask PDF