4DRC-OCC: Robust Semantic Occupancy Prediction Through Fusion of 4D Radar and Camera

David Ninfa; Andras Palffy; Holger Caesar

cs.CVMar 8, 2026

4DRC-OCC: Robust Semantic Occupancy Prediction Through Fusion of 4D Radar and Camera

Authors: David Ninfa, Andras Palffy, Holger Caesar

Organizations: Cognitive Robotics, Delft University of Technology · Perciv AI

Abstract

Autonomous driving requires robust perception across diverse environmental conditions, yet 3D semantic occupancy prediction remains challenging under adverse weather and lighting. In this work, we present the first study combining 4D radar and camera data for 3D semantic occupancy prediction. Our fusion leverages the complementary strengths of both modalities: 4D radar provides reliable range, velocity, and angle measurements in challenging conditions, while cameras contribute rich semantic and texture information. We further show that integrating depth cues from camera pixels enables lifting 2D images to 3D, improving scene reconstruction accuracy. Additionally, we introduce a fully automatically labeled dataset for training semantic occupancy models, substantially reducing reliance on costly manual annotation. Experiments demonstrate the robustness of 4D radar across diverse scenarios, highlighting its potential to advance autonomous vehicle perception.

Figures & tables

Fig. 1: Adverse lighting conditions significantly impact the camera-only baseline; however, our network still detects the cyclist through radar-camera fusion.

Fig. 2: The architecture of 4DRC-OCC: The main network, Version A , processes radar and camera separately in a multi-scale manner, reaching a higher level of abstraction before merging them in voxel space to produce the final prediction. Versions B and Version C build upon version A by incorporating additional radar depth cues at different stages, enhancing the camera image and further assisting the lifting mechanism. This allows for a more refined fusion of radar and camera information, improving performance.

Fig. 3: The process of auto-labeling dense occupancy pseudo-ground truth involves capturing dense lidar data (1), performing semantic segmentation (2), and separately extracting and accumulating dynamic (3) and static objects (4). Both point clouds are transformed into the world coordinate system (5) before being voxelized and refined to create the final occupancy labels (6).

Version	mIoU	weighted mIoU	others (0.00%)	barrier (0.03%)	bicycle (0.15%)	bus (0.04%)	car (2.82%)	constr. veh. (0.02%)	motorcycle (0.01%)	pedestrian (0.10%)	traffic cone (0.00%)	trailer (0.05%)	truck (0.51%)	driv. surf. (28.45%)	other flat (0.89%)	sidewalk (5.70%)	terrain (3.11%)	manmade (40.81%)	vegetation (17.32%)
Radar Only	11.3	26.6	-	0.0	10.4	5.6	17.9	3.3	0.0	0.3	-	1.3	5.1	49.0	10.6	19.1	8.1	17.2	21.1
Baseline	11.7	24.2	-	1.5	12.2	3.6	23.7	0.2	0.1	7.1	-	3.3	7.0	48.6	10.8	16.0	12.2	13.5	16.0
Baseline-ft	12.1	24.6	-	0.0	12.6	5.4	27.0	1.2	0.1	7.5	-	2.4	7.2	48.7	8.3	21.3	10.1	14.2	14.9
Version AN-ft	14.2	31.0	-	0.0	15.7	4.0	31.1	2.2	0.0	11.0	-	1.0	5.7	52.4	10.7	21.0	13.3	23.7	22.0
Version AN	14.5	30.0	-	0.7	15.2	3.2	27.9	0.7	1.5	14.9	-	3.8	9.4	51.0	9.6	20.7	13.3	22.1	22.7
Version A	15.5	31.2	-	0.0	23.0	5.9	31.0	0.2	0.5	12.0	-	2.7	9.6	52.7	12.6	21.6	14.1	22.8	23.7

TABLE I: Semantic occupancy prediction performance on the Perciv-scenes dataset.

Fig. 4: 4DRC-OCC improves prediction accuracy in poor lighting conditions by integrating data from 4D radar and cameras.

Fig. 5: Depth association strategies enhance predictions by improving spatial accuracy and semantic understanding in complex environments.

Version	occupied
Baseline	35.1
Baseline-ft	35.5
Radar Only	39.1
Version AN	41.0
Version A	42.3
Version AN-ft	42.7

TABLE II: Occupancy performance on the perciv-scenes dataset.

3D Occupancy Prediction

3D occupancy prediction estimates the geometry and semantics of occupied and free space around an agent from camera, LiDAR, or radar data.

See all 113 papers

RadarGSOcc: Doppler-Aware 3D Semantic Occupancy Prediction Using 4D Radar Tensor with Geometric-Semantic Feature Extraction
Oct 13, 2026Christofel Rio Goenawan, Dong-Hee Paek, Seung-Hyun Kong3D Occupancy Prediction Autonomous Driving Perception
Robotics Program, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Korea · CCS Graduate School of Mobility, KAIST, Daejeon, Korea · Department of Future Mobility, Korea University, Sejong, Korea
4DR360: State Reasoning for Joint 3D Detection and Occupancy Prediction in 4D Radar-Camera Full-Scene Perception
Jul 10, 2026Xiaokai Bai, Lianqing Zheng, Runwei Guan +3Camera-Radar Fusion Millimeter-Wave Radar
College of Information Science and Electronic Engineering, Zhejiang University. · School of Automotive Studies, Tongji University. · Thrust of Artificial Intelligence, Hong Kong University of Science and Technology.
VLMFusionOcc3D: VLM Assisted Multi-Modal 3D Semantic Occupancy Prediction
Mar 3, 2026A. Enes Doruk, Hasan F. Ates3D Occupancy Prediction Autonomous Driving Perception

Oct 13, 2026J/K move · Enter open · S save

RadarGSOcc: Doppler-Aware 3D Semantic Occupancy Prediction Using 4D Radar Tensor with Geometric-Semantic Feature Extraction

Christofel Rio Goenawan, Dong-Hee Paek, Seung-Hyun Kong

Robotics Program, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Korea · CCS Graduate School of Mobility, KAIST, Daejeon, Korea · Department of Future Mobility, Korea University, Sejong, Korea

3D Occupancy Prediction Autonomous Driving Perception mmWave Radar Sensing

Open paper AskPDF arXiv

3D semantic occupancy prediction is crucial for autonomous driving, but camera-based methods degrade in low-light conditions and LiDAR-based methods are sensitive to adverse weather. 4D radar is more robust, yet its sparsity and the high cost of radar tensor processing limit its use. We propose Radar Geometric-Semantic Occupancy Prediction (RadarGSOcc), an efficient framework that directly processes 4D radar tensors by separately extracting geometric and semantic features, including Doppler information, with a deformable attention-based geometric refinement module as its key component. We also introduce the K-Radar 3DSemOcc benchmark, the first large-scale benchmark for 3D semantic occupancy prediction using 4D radar tensors, built on the K-Radar dataset with refined LiDAR-based annotations. RadarGSOcc predicts 15 classes within a 51.2m range and achieves 38.2% IoU, 7.6% mIoU, and 25.5% weighted mIoU, outperforming RadarOcc by 17% relative (25.5 vs. 21.9) in weighted mIoU while using fewer parameters and lower computation. It substantially improves truck prediction while trading off car accuracy, outperforms camera-based approaches, and narrows the gap with LiDAR-based methods. To assess robustness without per-frame ground truth in adverse weather, we further propose a novel proxy metric, Environment-Based KL Divergence (EBKL), under which 4D radar proves a reliable and efficient modality for 3D semantic occupancy prediction. The dataset and code will be publicly available.

Abstract

3D Occupancy Prediction

Explore similar work

RadarGSOcc: Doppler-Aware 3D Semantic Occupancy Prediction Using 4D Radar Tensor with Geometric-Semantic Feature Extraction

4DR360: State Reasoning for Joint 3D Detection and Occupancy Prediction in 4D Radar-Camera Full-Scene Perception

VLMFusionOcc3D: VLM Assisted Multi-Modal 3D Semantic Occupancy Prediction