TFusionOcc: T-Primitive Based Object-Centric Multi-Sensor Fusion Framework for 3D Occupancy Prediction
Organizations: Australian Centre for Robotics (ACFR) at the University of Sydney (NSW, Australia).
Abstract
The prediction of 3D semantic occupancy enables autonomous vehicles (AVs) to perceive the fine-grained geometric and semantic scene structure for safe navigation and decision-making. Existing methods mainly rely on either voxel-based representations, which incur redundant computation over empty regions, or object-centric Gaussian primitives, whose ellipsoidal per-primitive geometry can limit representation efficiency for complex, non-convex, and asymmetric structures under a fixed primitive and computational budget. In this paper, we present TFusionOcc, a T-primitive-based object-centric multi-sensor fusion framework for 3D semantic occupancy prediction. Specifically, we introduce a family of Student's t-distribution-based T-primitives, including the plain T-primitive, T-Superquadric, and deformable T-Superquadric with inverse warping, where the deformable T-Superquadric serves as the key geometry-enhancing primitive. We further develop a unified probabilistic formulation based on the Student's t-distribution and the T-mixture model (TMM) to jointly model occupancy and semantics, and design a tightly coupled multi-stage fusion architecture to integrate camera and LiDAR cues effectively. Extensive experiments on nuScenes demonstrate competitive performance against recent state-of-the-art methods and strong performance among object-centric multi-sensor approaches, while additional evaluations on nuScenes-C show robust performance across most corruption scenarios. Further evaluation on the separate SemanticKITTI benchmark demonstrates cross-dataset applicability. The code will be available at: https://github.com/DanielMing123/TFusionOcc
Figures & tables
| Methods | Modality | mIoU | person | bicyclist | motorcyclist. | bicycle | motorcycle | traf.-sign | car | truck | other-veh. | vegetation | trunk | fence | road | sidewalk | parking | other-grnd | building | terrain | pole |
| JS3C-Net [ 52 ] | C | 10.31 | 0.67 | 0.27 | 0.00 | 0.00 | 0.00 | 1.45 | 24.65 | 4.41 | 6.15 | 18.11 | 4.33 | 3.94 | 50.49 | 23.74 | 11.94 | 0.07 | 15.03 | 26.86 | 3.77 |
| MonoScene [ 36 ] | C | 11.08 | 1.86 | 1.20 | 0.00 | 0.61 | 0.45 | 2.25 | 23.26 | 6.98 | 1.48 | 17.89 | 2.81 | 5.84 | 56.52 | 26.72 | 14.27 | 0.46 | 14.09 | 29.64 | 4.14 |
| VoxFormer [ 53 ] | C | 12.35 | 1.78 | 3.32 | 0.00 | 0.59 | 0.51 | 4.18 | 25.79 | 5.63 | 3.77 | 24.39 | 5.08 | 7.64 | 54.76 | 26.35 | 15.50 | 0.70 | 17.65 | 29.96 | 7.11 |
| LMSCNet [ 46 ] | L | 6.70 | 0.00 | 0.00 | 0.00 | 0.0 | 0.0 | 0.00 | 18.33 | 0.00 | 0.0 | 13.66 | 0.02 | 1.21 | 40.68 | 18.22 | 4.38 | 0.00 | 10.31 | 20.54 | 0.00 |
| M-CONet [ 40 ] | C+L | 17.25 | 2.07 | 2.38 | 0.00 | 0.66 | 2.2 | 14.09 | 33.25 | 19.46 | 5.99 | 32.56 | 16.01 | 12.25 | 58.00 | 29.44 | 19.33 | 0.02 | 27.85 | 37.31 | 14.88 |
| OccFusion [ 6 ] | C+L | 21.64 | 2.33 | 3.07 | 0.00 | 2.00 | 3.39 | 14.36 | 45.72 | 19.41 | 7.92 | 41.21 | 19.18 | 15.74 | 65.73 | 36.46 | 21.35 | 0.00 | 39.32 | 46.28 | 27.60 |
3D Occupancy Prediction
3D occupancy prediction estimates the geometry and semantics of occupied and free space around an agent from camera, LiDAR, or radar data.
See all 124 papers