Geometric Refinement and Distributional Alignment for Sim2Real Roadside LiDAR Detection
Organizations: Columbia University
Abstract
We describe our approach to the LUMPI track of the UrbanTwin Sim2Real LiDAR Challenge, in which a 3D object detector must be trained exclusively on synthetic data and evaluated on unseen real roadside LiDAR, while the synthetic point clouds are simultaneously scored for distributional realism. Our final entry scores 0.4793 combined, from a detection mAP of 0.1333 and a realism score of 0.9126. Our contribution is not a new architecture. Working from a PointPillars baseline, we obtained most of our gains from three sources. First, a diagnosis that small object failure in roadside LiDAR is a localisation problem rather than a detection problem, which led to a parameter free geometric refinement worth 86 percent relative on pedestrian AP. Second, an audit of our own post processing which revealed that a shipped rule was destroying valid predictions, the repair of which produced our single largest detection gain at 335 percent relative on the Van class. Third, treating the realism metrics analytically rather than as a black box, which showed that one of the four components is a closed form function of nine summary statistics. We also report an experimental methodology that we believe is the more transferable result. Because mean AP decomposes into independent per class terms, a single submission can carry several mutually controlled experiments, and a submission assembled entirely from previously measured components has a score that is known before it is sent. We predicted our final score to four decimal places prior to submission. We report negative results at equal length, and give a full account of data provenance and compliance in Section 8, including a full account of every mechanism contributing to the submitted score.
Figures & tables
| corrected component | Person AP change |
|---|---|
| size only | +11 pp |
| z only | -0 pp |
| yaw only | +2 pp |
| xy centre only | +44 pp |
| class | n matched | dz | length ratio | width ratio |
|---|---|---|---|---|
| Person | 802 | +0.001 | 1.111 | 1.095 |
| Car | 2494 | -0.093 | 0.945 | 0.905 |
| Bicycle | 554 | -0.048 | 0.897 | 0.930 |
| Truck | 33 | +0.113 | 1.488 | 1.223 |
| quantity | value |
|---|---|
| Combined score | 0.4793 |
| Detection (normalised) | 0.1904 |
| 3D mAP @ IoU 0.5, R40 | 0.1333 |
| BEV mAP @ IoU 0.5, R40 | 0.1461 |
| Realism (normalised) | 0.9126 |
| Predictions submitted | 18,085 on 2,175 ground truth boxes |
| class | AP | ground truth instances |
|---|---|---|
| Car | 0.6430 | 1258 |
| Person | 0.1491 | 512 |
| Bicycle | 0.0782 | 82 |
| Van | 0.0317 | 79 |
| Motorcycle | 0.0202 | 27 |
| Truck | 0.0107 | 192 |
| metric | value | normalised |
|---|---|---|
| Chamfer Distance | 2.1473 | 0.9947 |
| Maximum Mean Discrepancy | 4.1803e-04 | 0.9992 |
| Earth Mover’s Distance | 1.6458 | 0.6708 |
| Frechet Point cloud Distance | 0.7104 | 0.9858 |
| lever | evidence | verdict |
|---|---|---|
| Bus class | five donor classes tested, plus resizing and relabelling. Every variant returned exactly 0.0000 | Unreachable. Of 11 Bus objects inspected, 2 are buses, 8 are two wheeler sized and 1 is car sized. Costs one seventh of mAP by construction |
| Score thresholding | tested in both directions | Recall is model limited, not threshold limited |
| Learned refiners | two built, one on real ground truth and one synthetic only | Synthetic cross validation showed 75 to 88 percent error reduction, but real transfer was flat at +0.0015. The sim2real gap, not compliance, was the binding constraint |
| CenterPoint [4] and anchor free heads | root caused | Initial failure was misconfiguration, specifically a pillar encoder, a four times coarser feature stride and a vestigial velocity head. Corrected, it remained no better than the anchor based baseline |
| Voxel resolution 0.10 | trained and evaluated | Net negative for the whole model. Retained only as a per class hybrid |
| Density rebalancing | three training runs, approximately 30 GPU hours | The first gained Bicycle 54 percent, the second failed, the third was a clean negative with mAP falling from 0.2211 to 0.1935. Lever exhausted |
Lidar-Based 3D Object Detection
Lidar-based 3D object detection locates and classifies objects in point clouds, supporting perception for autonomous vehicles, robots, and other mobile systems.
See all 60 papers