SparseGraph-I2P: Consistency-Guided Hypothesis Selection for Efficient Image-to-Point-Cloud Registration
Organizations: School of Computing, Institute of Science Tokyo · School of Electronic Science & Engineering, Nanjing University
Abstract
Coarse-to-fine image-to-point-cloud (I2P) registration risks discarding correct but ambiguous patch pairs, while retaining more candidates repeats fine matching over overlapping patches. SparseGraph-I2P aims to improve registration recall by increasing the inlier ratio at the pose solver, using sensor or monocular depth. A Coarse Hypothesis Graph combines appearance with depth-derived first- and second-order compatibility to select a fixed budget of patch pairs. Sparse Fine Matching decodes each queried pixel once and skips pixel--point comparisons covered by an ancestor patch paired with the same point-cloud region. With sensor depth, depth-aware filtering ranks correspondences by SC2 spectral support for a single EPnP solve. With monocular depth, depth or inverse depth is calibrated from correspondence geometry before filtering. SparseGraph-I2P achieves the highest registration recall and lowest latency among the compared methods on RGB-D Scenes V2 and 7-Scenes with sensor depth. With predicted depth, it retains the highest recall on RGB-D Scenes V2 at an additional calibration cost. Code will be released.
Figures & tables
| Coarse Hypothesis Graph | Sparse Fine Matching | Depth-aware Filtering | RGB-D V2 | 7-Scenes | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Variant | On-demand | Nested | IR | FMR | RR | Lat. | IR | FMR | RR | Lat. | ||
| (a) | – | – | – | – | 32.4 | 91.1 | 55.4 | 138.0 | 50.1 | 92.1 | 74.1 | 92.8 |
| (b) | – | – | – | 81.2 | 91.1 | 73.7 | 30.0 | 83.3 | 92.1 | 86.5 | 37.0 | |
| (c) | – | 78.4 | 90.5 | 69.4 | 24.8 | 81.8 | 91.1 | 84.4 | 31.9 | |||
| (d) | – | – | 90.3 | 85.1 | 85.0 | 46.4 | 86.1 | 91.5 | 88.4 | 42.4 | ||
| (e) | – | 88.8 | 83.6 | 84.8 | 35.0 | 85.3 | 92.0 | 88.2 | 38.5 | |||
| RGB-D V2 | 7-Scenes | |||||||
|---|---|---|---|---|---|---|---|---|
| Setting | IR | FMR | RR | Lat. | IR | FMR | RR | Lat. |
| 256 | 87.9 | 92.4 | 84.3 | 28.4 | 84.4 | 91.6 | 86.9 | 35.9 |
| 512 | 88.7 | 90.0 | 85.1 | 29.4 | 84.6 | 91.9 | 87.8 | 36.7 |
| 768 | 88.8 | 87.1 | 85.3 | 31.8 | 85.1 | 91.9 | 88.1 | 36.6 |
| 1536 | 88.9 | 75.1 | 86.0 | 38.7 | 85.0 | 91.7 | 88.1 | 37.1 |
| 2048 | 89.0 | 68.1 | 86.1 | 46.1 | 85.3 | 90.8 | 87.6 | 38.7 |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| All | Registered | ||||||
|---|---|---|---|---|---|---|---|
| Method | FMR | IR | RR | RRE | RTE | RRE | RTE |
| FreeReg | 100.0 | 60.4 | 65.8 | 8.21 | 2.91 | 4.64 | 1.62 |
| Diff 2 I2P | 99.7 | 66.7 | 77.5 | 2.79 | 1.84 | 1.88 | 1.10 |
| 2D3D-MATR | 98.8 | 89.7 | 71.6 | 9.65 | 3.13 | 3.06 | 1.27 |
| Ours | 99.4 | 81.0 | 85.1 | 3.71 | 1.84 | 2.53 | 1.36 |