cs.CVOct 19, 2026
SavePlanarCityLoc: Breaking Silhouette Ambiguity in LoD Aerial Localization via 3D Planes
Organizations: Technical University of Munich · University of Cambridge · ETH Zürich · Munich Center for Machine Learning ·
Abstract
Aerial visual localization is crucial for autonomous drone navigation in urban environments where GNSS signals are often degraded. Lightweight textureless Level-of-Detail (LoD) city models provide a geometric map alternative to textured assets. Existing silhouette-based methods rely on image-space alignment, which can leave different camera poses indistinguishable and favor incorrect poses when segmentation errors distort the building outlines. We present PlanarCityLoc, which moves beyond silhouette overlap by leveraging the planar geometry of building facades and roofs. Following geometry-aware initialization, we match image-side planar patches to LoD model planes and use the resulting 3D-to-2D correspondences to refine pose with Perspective-n-Point (PnP). The framework uses off-the-shelf geometry and segmentation models without dataset-specific training or fine-tuning. Experiments on UAVD4L-LoDv2 and Swiss-EPFLv2 achieve state-of-the-art localization recall among the evaluated off-the-shelf LoD-based methods. By grounding image-space alignment in 3D planes, PlanarCityLoc enables more accurate aerial localization with lightweight, textureless city maps.
Figures & tables
Figure 1: PlanarCityLoc overview. (a) Silhouette-based localization compares predicted building masks with renderings of the LoD model. (b) PlanarCityLoc grounds image-side planar patches in 3D through explicit correspondences with LoD planes. (c) Schematic score landscapes illustrate ambiguity in image-space overlap and a score maximum displaced from the correct pose. Plane-based refinement adds geometric constraints to move beyond image-space score maximization. Curves are illustrative, not measured method comparisons.
Figure 2: Overview of PlanarCityLoc. (a) We extract building masks, surface normals, depth, and structural edges from query image with foundation models, then (b) form 2D planar patches. (c) A coarse pose is obtained through altitude initialization, cost-volume search, and multi-scale particle filtering. (d) We iteratively match visible LoD planes to image patches and refine the pose using PnP-RANSAC on the resulting 3D-to-2D correspondences. (e) Equal-weight rank aggregation of complementary metrics selects the final pose.
| UAVD4L-LoDv2 query splits | Swiss-EPFLv2 query splits | |||||||||||||||||
| in-Traj. | out-of-Traj. | in-Place | out-of-Place | |||||||||||||||
| Method | Vision Model | Setup | 2m-2 ∘ | 3m-3 ∘ | 5m-5 ∘ | T.e./R.e. | 2m-2 ∘ | 3m-3 ∘ | 5m-5 ∘ | T.e./R.e. | 2m-2 ∘ | 3m-3 ∘ | 5m-5 ∘ | T.e./R.e. | 2m-2 ∘ | 3m-3 ∘ | 5m-5 ∘ | T.e./R.e. |
| Released priors | – | – | 0.0 | 0.0 | 4.3 | 6.48/1.63 | 0.0 | 0.0 | 0.4 | 11.12/1.33 | 0.0 | 0.0 | 0.6 | 17.59/3.86 | 0.0 | 0.0 | 1.1 | 17.88/3.94 |
| Textured Mesh-based Models | ||||||||||||||||||
| UAVLoc [ 39 ] | LoFTR | O | 84.98 | 88.09 | 88.90 | 0.81 / 0.29 | 91.56 | 92.02 | 92.11 | 0.79 / 0.20 | 26.40 | 46.21 | 62.22 | 3.06 / 1.86 | 68.87 | 81.00 | 84.96 | 1.12 / 1.08 |
| UAVLoc [ 39 ] | RoMa | O | 93.27 | 93.70 | 93.77 | 0.78 / 0.25 | 95.03 | 95.53 | 95.53 | 0.73 / 0.22 | 45.95 | 66.77 | 80.73 | 2.08 / 1.29 | 89.18 | 89.68 | 98.84 | 0.77 / 1.04 |
Table 1: Quantitative comparison on UAVD4L-LoDv2 and Swiss-EPFLv2. Setup: O ff-the-shelf, I n-domain training, C ross-dataset training, or GT oracle masks. C ∗ uses the InsLoD-Loc training set [ 26 ] . Recall is in %. T.e./R.e. are median translation (m) and rotation ( ∘ ) errors. For evaluated runs, recall requires both pose thresholds. Best/second-best distinct values per column within each setup group are bold/underlined. : adapted re-runs (see Sec. 4.1 ).
Figure 3: Qualitative comparison on UAVD4L-LoDv2 and Swiss-EPFLv2. Red overlays show LoD edges at each estimated pose. Labels give translation (m) / rotation ( ∘ ) errors. Similar visual alignments can correspond to different pose errors.
Figure 4: Pose estimates through PlanarCityLoc on UAVD4L-LoDv2. Each row shows the query image and map overlays at the ground-truth, sensor-prior, particle-filter, and final selected poses. Labels give translation (m) / rotation ( ∘ ) errors.
| Map | 2m–2 ∘ | 3m–3 ∘ | 5m–5 ∘ | T.e. (m) | R.e. ( ∘ ) |
|---|---|---|---|---|---|
| LoD1 | 23.8 / 44.8 | 54.1 / 74.3 | 82.2 / 93.0 | 2.85 / 2.13 | 1.26 / 0.95 |
| LoD3 | 75.4 / 80.6 | 90.8 / 95.1 | 97.2 / 99.0 | 1.28 / 1.26 | 0.68 / 0.69 |
Table 2: LoD1 vs. LoD3. Full UAVD4L-LoDv2 splits (in-Traj. / out-of-Traj.). Joint recalls (%) and median errors.
| Selection rule | 2m–2 ∘ | 3m–3 ∘ | 5m–5 ∘ | T.e. (m) | R.e. ( ∘ ) |
|---|---|---|---|---|---|
| Particle filter only | 36.0 / 60.8 | 67.7 / 90.6 | 82.8 / 98.5 | 2.36 / 1.81 | 1.30 / 0.98 |
| Max PnP inliers | 65.8 / 74.8 | 84.1 / 91.1 | 95.4 / 98.2 | 1.49 / 1.37 | 0.82 / 0.76 |
| Max mask IoU | 67.4 / 80.7 | 88.7 / 95.3 | 96.8 / 99.1 | 1.54 / 1.32 | 0.79 / 0.75 |
| Max edge score | 76.6 / 79.6 | 90.6 / 94.6 | 97.1 / 99.0 | 1.25 / 1.33 | 0.69 / 0.77 |
| Max composite score | 72.3 / 81.2 | 89.7 / 96.0 | 97.2 / 99.0 | 1.39 / 1.30 | 0.72 / 0.74 |
| Equal-weight ranks | 75.4 / 80.6 | 90.8 / 95.1 | 97.2 / 99.0 | 1.28 / 1.26 | 0.68 / 0.69 |
Table 3: Pose selection. Full UAVD4L-LoDv2 splits (in-Traj. / out-of-Traj.), using identical candidate pools. Best/second-best values at shown precision are bold/underlined.
Figure 5: Prior pose sensitivity on UAVD4L-LoDv2. Curves show means of split-wise median errors for the same 300 queries per split. Synthetic priors use expanded search. Native-prior points are separate references.
| Model | Backbone | 2m–2 ∘ | 3m–3 ∘ | 5m–5 ∘ | T.e. (m) | R.e. ( ∘ ) |
|---|---|---|---|---|---|---|
| Metric3D v2 | ViT-S | 33.0 / 55.0 | 60.0 / 86.0 | 82.0 / 94.0 | 2.55 / 1.87 | 1.25 / 0.89 |
| Metric3D v2 | ViT-L | 31.0 / 58.0 | 62.0 / 78.0 | 87.0 / 90.0 | 2.41 / 1.80 | 1.33 / 0.96 |
| Metric3D v2 | ViT-G2 | 58.0 / 59.0 | 81.0 / 81.0 | 95.0 / 89.0 | 1.70 / 1.73 | 0.91 / 0.90 |
| MoGe-2 | ViT-S | 75.0 / 84.0 | 94.0 / 94.0 | 98.0 / 99.0 | 1.14 / 1.37 | 0.68 / 0.78 |
| MoGe-2 | ViT-B | 82.0 / 83.0 | 97.0 / 93.0 | 98.0 / 98.0 | 1.05 / 1.23 | 0.65 / 0.65 |
| MoGe-2 | ViT-L | 87.0 / 84.0 | 96.0 / 97.0 | 97.0 / 100.0 | 1.09 / 1.25 | 0.61 / 0.62 |
Table 4: Geometry predictors. Each model supplies both normals and depth. SAM3 masks are unchanged. All rows use the same 100 queries per UAVD4L-LoDv2 split (in-Traj. / out-of-Traj.). Recall uses joint thresholds. T.e./R.e. are medians. Bold/underline mark best/second-best at shown precision.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset / map | Split | Stage | Queries | 1m–1 ∘ | 2m–2 ∘ | 3m–3 ∘ | 5m–5 ∘ | T.e. | R.e. |
|---|---|---|---|---|---|---|---|---|---|
| UAVD4L-LoDv2 / LoD3 | in-Traj. | Coarse search | 1604 | 1.4 | 27.6 | 59.9 | 79.4 | 2.62 | 1.49 |
| UAVD4L-LoDv2 / LoD3 | in-Traj. | Particle filter | 1604 | 3.1 | 36.0 | 67.7 | 82.8 | 2.36 | 1.30 |
| UAVD4L-LoDv2 / LoD3 | in-Traj. | PlanarCityLoc | 1604 | 36.3 | 75.4 | 90.8 | 97.2 | 1.28 | 0.68 |
| UAVD4L-LoDv2 / LoD3 | out-of-Traj. | Coarse search | 2192 | 2.7 | 34.9 | 74.1 | 95.3 | 2.35 | 1.22 |
| UAVD4L-LoDv2 / LoD3 | out-of-Traj. | Particle filter | 2192 | 8.8 | 60.8 | 90.6 | 98.5 | 1.81 | 0.98 |
| UAVD4L-LoDv2 / LoD3 | out-of-Traj. | PlanarCityLoc | 2192 | 35.5 | 80.6 | 95.1 | 99.0 | 1.26 | 0.69 |
Table A1: Full-split staged results. Recalls are joint percentages. Translation/rotation errors are medians in meters/degrees. Every stage is evaluated on the same queries within a split.
Figure A1 : Scene context and planar support on Swiss-EPFLv2. Left: views near each split’s median map-patch count and predicted-to-rendered ratio. Right: full-split distributions at the particle-filter pose. Boxes show interquartile ranges, lines show medians, and whiskers show the full range. The ratio axis is logarithmic.
Figure A2 : Plane extraction qualitative comparison. LoD primitives against predicted plane extractions. Patch granularity varies with normal consistency and minimum region size. RQ and PQ denote Recognition Quality and Panoptic Quality, respectively.
Figure A3 : Mask segmentation examples. Query images and predicted masks illustrate incomplete building coverage and non-building regions included by off-the-shelf segmentation. Such errors can weaken coarse alignment and the planar evidence available for refinement.
| Configuration | 2m–2 ∘ | 3m–3 ∘ | 5m–5 ∘ | T.e. (m) | R.e. ( ∘ ) |
|---|---|---|---|---|---|
| Mask only | 62.0 / 66.0 | 87.0 / 82.0 | 95.0 / 94.0 | 1.39 / 1.50 | 0.81 / 0.81 |
| Mask + edges | 74.0 / 76.0 | 92.0 / 91.0 | 98.0 / 97.0 | 1.20 / 1.42 | 0.75 / 0.71 |
| Mask + depth | 60.0 / 71.0 | 80.0 / 89.0 | 92.0 / 97.0 | 1.66 / 1.49 | 0.95 / 0.74 |
| All three cues | 75.0 / 84.0 | 94.0 / 94.0 | 98.0 / 99.0 | 1.14 / 1.37 | 0.68 / 0.78 |
Table A2: Coarse scoring components. Mask, edge, and depth cues in coarse search and particle filtering. Other stages are unchanged. Matched 100-query UAVD4L-LoDv2 subsets (in-Traj. / out-of-Traj.). Joint recalls (%) and median errors. Best/second-best values are bold/underlined.
| Configuration | 2m–2 ∘ | 3m–3 ∘ | 5m–5 ∘ | T.e. (m) | R.e. ( ∘ ) |
|---|---|---|---|---|---|
| Normal consistency | 74.0 / 79.0 | 94.0 / 96.0 | 98.0 / 100.0 | 1.30 / 1.41 | 0.78 / 0.79 |
| Depth continuity | 31.0 / 64.0 | 72.0 / 92.0 | 93.0 / 97.0 | 2.36 / 1.67 | 1.25 / 0.77 |
| Normals + depth | 75.0 / 84.0 | 94.0 / 94.0 | 98.0 / 99.0 | 1.14 / 1.37 | 0.68 / 0.78 |
Table A3: Plane extraction criteria. Normal and depth consistency for predicted and rendered planes. Matched 100-query UAVD4L-LoDv2 subsets (in-Traj. / out-of-Traj.). Joint recalls (%) and median errors. Best/second-best values are bold/underlined.
| Configuration | 2m–2 ∘ | 3m–3 ∘ | 5m–5 ∘ | T.e. (m) | R.e. ( ∘ ) |
|---|---|---|---|---|---|
| Overlap only | 54.0 / 76.0 | 89.0 / 96.0 | 96.0 / 98.0 | 1.80 / 1.41 | 1.02 / 0.86 |
| Centroid only | 64.0 / 72.0 | 85.0 / 91.0 | 97.0 / 99.0 | 1.50 / 1.59 | 0.91 / 0.79 |
| Overlap + centroid | 75.0 / 84.0 | 94.0 / 94.0 | 98.0 / 99.0 | 1.14 / 1.37 | 0.68 / 0.78 |
Table A4: PnP matching strategies. Overlap, centroid, and sequential correspondences. Matched 100-query UAVD4L-LoDv2 subsets (in-Traj. / out-of-Traj.). Joint recalls (%) and median errors. Best/second-best values are bold/underlined.
Figure A4 : Comprehensive prior noise evaluation. Joint recall and median errors under synthetic pose perturbations on UAVD4L-LoDv2, using 300 queries per split and expanded search ranges. Native-prior points are separate references on the same queries.
Figure A5 : Visual examples of map completeness. Overlays of rendered LoD geometry for the intact map and representative removal conditions. Groups are selected by projected area, not physical building volume.
| Configuration | 2m–2 ∘ | 3m–3 ∘ | 5m–5 ∘ | T.e. (m) | R.e. ( ∘ ) |
|---|---|---|---|---|---|
| SAM3 | 75.0 / 84.0 | 94.0 / 94.0 | 98.0 / 99.0 | 1.14 / 1.37 | 0.68 / 0.78 |
| EoMT | 40.0 / 48.0 | 54.0 / 61.0 | 69.0 / 70.0 | 2.51 / 2.26 | 1.46 / 1.06 |
Table A5: Building segmenters. SAM 3 and EoMT with fixed MoGe-2 ViT-S geometry. Matched 100-query UAVD4L-LoDv2 subsets (in-Traj. / out-of-Traj.). Joint recalls (%) and median errors. Best/second-best values are bold/underlined.
| Configuration | 2m–2 ∘ | 3m–3 ∘ | 5m–5 ∘ | T.e. (m) | R.e. ( ∘ ) |
|---|---|---|---|---|---|
| Intact map | 75.0 / 84.0 | 94.0 / 94.0 | 98.0 / 99.0 | 1.14 / 1.37 | 0.68 / 0.78 |
| Remove smallest | 79.0 / 82.0 | 94.0 / 98.0 | 98.0 / 99.0 | 1.18 / 1.41 | 0.68 / 0.73 |
| Remove middle | 71.0 / 77.0 | 89.0 / 95.0 | 94.0 / 99.0 | 1.41 / 1.46 | 0.72 / 0.81 |
| Remove largest | 52.0 / 69.0 | 70.0 / 84.0 | 84.0 / 95.0 | 1.87 / 1.47 | 1.02 / 0.76 |
Table A6: Map completeness. Matched 100-query UAVD4L-LoDv2 subsets (in-Traj. / out-of-Traj.). Groups of nearby mesh surfaces are removed by projected area. Queries without sufficient groups retain the intact map. Joint recalls (%) and median errors. Best/second-best values are bold/underlined.
More in this topic
Robot Localization
Robot localization estimates a robot’s position and orientation from onboard sensors, maps, and observations of its surroundings.
See all 177 papers