RAPA: Ray-Aware Physics Augmentation for Robust Camera-LiDAR Calibration
Organizations: Department of Mechanical Engineering, Yonsei University, Seoul, Republic of Korea · Yonsei Institute for Embodied Intelligence, Yonsei University, Seoul, Republic of Korea
Abstract
Camera-LiDAR calibration is essential for reliable perception in robotic and autonomous systems. Most target-based calibration methods rely on checkerboard observations, but accurate checkerboard feature extraction from LiDAR point clouds remains challenging due to measurement sparsity. Existing approaches either require manual annotation of 3D corners or depend on geometric feature extraction methods, both of which can become unreliable when checkerboard features are insufficiently observed. In this paper, we formulate checkerboard localization as a 2D detection problem using cylindrical LiDAR reflectivity images. To overcome the limited availability of labeled training data, we propose ray-aware physics augmentation (RAPA), which generates diverse synthetic samples from a single checkerboard observation. The generated data are used to improve detector robustness and generalization under varying sensing conditions. Across four independently captured indoor and outdoor locations, the proposed synthetic augmentation exceeds the strongest evaluated image-space augmentation baseline by 16.1-43.1 percentage points in [email protected]. Compared with training-free target-based methods on 242 observations, RAPA yields the largest number of valid target localizations, with 96.4% of its returned detections yielding valid target poses. These reliable localizations support downstream camera-LiDAR calibration across all four evaluated locations. Overall, RAPA addresses a key bottleneck in target-based calibration by enabling automated LiDAR-side target localization with only a single annotated target observation.
Figures & tables
| Training condition | Indoor1 | Indoor2 | Outdoor1 | Outdoor2 | |
|---|---|---|---|---|---|
| Predefined Good1 (single run) | |||||
| Good1 repeated (no aug.) | .405/.119 | .097/.048 | .198/.056 | .022/.010 | |
| HSV jitter Aug50 | .495/.141 | .102/.047 | .178/.066 | .065/.029 | |
| Horizontal flip Aug50 | .538/.174 | .132/.076 | .292/.087 | .059/.027 | |
| Deep ChArUco Aug50 | .623/.285 | .413/.182 | .563/.206 | .167/.056 | |
| Good1+Syn50 (R0) | .784/.524 | .844/.489 | .779/.565 | .483/.290 | |
| Dataset | Method | Returned | Valid | ( ∘ ) | (cm) |
|---|---|---|---|---|---|
| Indoor1 | RAPA | 50/59 | 46/59 | 1.37/5.14 | 2.55/3.25 |
| Zhou et al. | 27/59 | 21/59 | 1.60/5.10 | 2.40/2.98 | |
| ACSC | 51/59 | 38/59 | 1.39/5.12 | 3.10/3.54 | |
| Indoor2 | RAPA | 49/64 | 47/64 | 1.67/3.20 | 3.34/3.79 |
| Zhou et al. | 8/64 | 1/64 | – | – | |
| ACSC | 32/64 | 12/64 | 1.66/3.21 | 3.41/3.84 |
| Experiment | [email protected] | [email protected]:.95 | Valid | ( ∘ ) | (cm) |
|---|---|---|---|---|---|
| Target-configuration transfer | |||||
| Large ChArUco, Outdoor2 | .682 | .439 | 30/55 | 1.11/1.42 | 3.64/4.36 |
| Large ChArUco, Indoor2 | .892 | .621 | 35/55 | 0.97/1.13 | 3.76/4.43 |
| LiDAR-sensor adaptation | |||||
| VLP-16, Indoor1 | .888 | .391 | 52/59 | 1.85/2.58 | 5.50/6.23 |
| Extension beyond checkerboards | |||||