PALIN: Pixel-Aligned LiDAR-NIR Multimodal 3D Object Detection with Benchmark
Organizations: Thomas
Abstract
3D object detection is critical for intelligent transportation systems. LiDAR has been widely adopted for accurate geometric measurements; however, purely geometric representations remain ambiguous under sparse observations and occlusions, while cross-sensor multimodal fusion suffers from calibration and synchronization errors. To overcome these challenges, we leverage an Ouster LiDAR to construct PALIN (Pixel-Aligned LiDAR-NIR), a large-scale intrinsically pixel-aligned multimodal dataset jointly capturing geometry and NIR intensity, comprising approximately 8,000 annotated frames from static urban intersections and diverse mobile suburban scenes. Building upon this dataset, we propose CMambaFusion, a novel fusion framework consisting of a four-directional Cross-Mamba module for bidirectional cross-modal interaction and a Gated Concat-Mamba module for joint feature refinement. Extensive experiments demonstrate that PALIN enables effective multimodal learning, and CMambaFusion achieves superior accuracy-efficiency trade-offs over both LiDAR-only and representative fusion baselines.
Figures & tables
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
| Dataset | Year | Platform | LiDAR Sensor | Image Modality | Native Align. | Ann. Frames | Seqs. | Det. Classes | Object Attributes | |
| KITTI [ 42 , 6 ] | 2012 | Vehicle | HDL-64E | RGB 2 | 7,481 | — | 8 | Occ., Trunc., Diff. | ||
| nuScenes [ 9 ] | 2020 | Vehicle | HDL-32E | RGB 6 | 40,000 | 1,000 | 23 | Occ., Motion | ||
| Waymo [ 7 ] | 2020 | Vehicle | LiDAR (fused) | RGB 5 | 200,000 | 1,000 | 4 | Diff.(L1/L2), Heading | ||
| PandaSet [ 8 ] | 2021 | Vehicle | Hesai Pandar64 | RGB 6 | 8,240 | 103 | 28 | Occ., Heading | ||
| PALIN (Ours) | 2026 | Bus + Road | OS1-128 | NIR | 8,587 | 11 | Occ., Diff. (E/M/H), Yaw + Modality Dominance |