Fusion4CA: Boosting 3D Object Detection via Comprehensive Image Exploitation
Organizations: IRMV Lab, the Department of Automation, Shanghai Jiao Tong University.
Abstract
Nowadays, an increasing number of works fuse LiDAR and RGB data in the bird's-eye view (BEV) space for 3D object detection in autonomous driving systems. However, existing methods suffer from over-reliance on the LiDAR branch, with insufficient exploration of RGB information. To tackle this issue, we propose Fusion4CA, which is built upon the classic BEVFusion framework and dedicated to fully exploiting visual input with plug-and-play components. Specifically, a contrastive alignment module is designed to calibrate image features with 3D geometry, and a camera auxiliary branch is introduced to mine RGB information sufficiently during training. For further performance enhancement, we leverage an off-the-shelf cognitive adapter to make the most of pretrained image weights, and integrate a standard coordinate attention module into the fusion stage as a supplementary boost. Experiments on the nuScenes dataset demonstrate that our method achieves 69.7% mAP with only 6 training epochs and a mere 3.48% increase in inference parameters, yielding a 1.2% improvement over the baseline which is fully trained for 20 epochs. Extensive experiments in a simulated lunar environment further validate the effectiveness and generalization of our method. Our code will be released through Fusion4CA.
Figures & tables
| Method | Reference | Mod. | mAP | NDS | Car | Truck | C.V. | Bus | T.L. | B.R. | M.T. | B.C. | Ped. | T.C. |
| Results on the validation data set | ||||||||||||||
| BEVFusion [ 20 ] | ICRA 2023 | L | 64.7 | 69.3 | 86.9 | 61.0 | 27.3 | 72.5 | 41.8 | 69.6 | 71.7 | 56.3 | 86.6 | 73.2 |
| BEVFusion [ 20 ] | ICRA 2023 | L+C | 68.5 | 71.4 | 89.2 | 64.6 | 30.4 | 75.4 | 42.5 | 72.0 | 78.5 | 65.3 | 88.2 | 79.5 |
| Fusion4CA (Ours) | - | L+C | 69.7 | 72.1 | 89.7 | 66.2 | 31.9 | 77.3 | 43.6 | 72.3 | 79.5 | 66.3 | 89.5 | 80.3 |
| Results on the test data set | ||||||||||||||
| Order | ConAlign | CamAux | CoordAtt | CogAdp | mAP | mAP | NDS | NDS | mATE | mASE | mAOE |
| 01 | 64.7 | - | 69.4 | - | 0.291 | 0.254 | 0.302 | ||||
| 02 | 67.0 | +2.3 | 70.4 | +1.0 | 0.291 | 0.256 | 0.330 | ||||
| 03 | 68.7 | +4.0 | 71.5 | +2.1 | 0.285 | 0.256 | 0.308 | ||||
| 04 | 64.6 | -0.1 | 69.4 | +0.0 | 0.297 | 0.255 | 0.294 | ||||
| 05 | 68.9 | +4.2 | 71.5 | +2.1 | 0.281 | 0.255 | 0.319 | ||||
| Method | Reference | mAP | NDS | Meteor | Platform | mATE | mASE | mAOE |
| IS-Fusion [ 37 ] | CVPR 2024 | 71.0 | 66.9 | 74.6 | 67.5 | 0.105 | 0.073 | 0.683 |
| BEVFusion [ 20 ] | ICRA 2023 | 88.8 | 81.6 | 84.9 | 92.8 | 0.096 | 0.043 | 0.146 |
| Fusion4CA (Ours) | - | 90.9 | 82.7 | 86.8 | 95.0 | 0.091 | 0.035 | 0.153 |
3D Object Detection
3D object detection identifies objects and estimates their locations, sizes, and orientations from images, point clouds, radar, or fused sensors.
See all 180 papers