PlanarCityLoc: Breaking Silhouette Ambiguity in LoD Aerial Localization via 3D Planes
Authors: Oussema Dhaouadi, Shao Jie Hu-Chen, Jakob Gollreiter, Samir Jusufi, Nikolas Rieger, Caroline Wendlandt, Christian Wender, Jingfeng Mao, Guangming Wang, Zuria Bauer, Marc Pollefeys, Daniel Cremers, Olaf Wysocki
Organizations: Technical University of Munich · University of Cambridge · ETH Zürich · Munich Center for Machine Learning · LMU Munich · Konrad Zuse School of Excellence in Reliable AI (relAI) · BMW Group · Microsoft
Aerial visual localization is crucial for autonomous drone navigation in urban environments where GNSS signals are often degraded. Lightweight textureless Level-of-Detail (LoD) city models provide a geometric map alternative to textured assets. Existing silhouette-based methods rely on image-space alignment, which can leave different camera poses indistinguishable and favor incorrect poses when segmentation errors distort the building outlines. We present PlanarCityLoc, which moves beyond silhouette overlap by leveraging the planar geometry of building facades and roofs. Following geometry-aware initialization, we match image-side planar patches to LoD model planes and use the resulting 3D-to-2D correspondences to refine pose with Perspective-n-Point (PnP). The framework uses off-the-shelf geometry and segmentation models without dataset-specific training or fine-tuning. Experiments on UAVD4L-LoDv2 and Swiss-EPFLv2 achieve state-of-the-art localization recall among the evaluated off-the-shelf LoD-based methods. By grounding image-space alignment in 3D planes, PlanarCityLoc enables more accurate aerial localization with lightweight, textureless city maps.
Figures & tables
Figure 1: PlanarCityLoc overview. (a) Silhouette-based localization compares predicted building masks with renderings of the LoD model. (b) PlanarCityLoc grounds image-side planar patches in 3D through explicit correspondences with LoD planes. (c) Schematic score landscapes illustrate ambiguity in image-space overlap and a score maximum displaced from the correct pose. Plane-based refinement adds geometric constraints to move beyond image-space score maximization. Curves are illustrative, not measured method comparisons.
Figure 2: Overview of PlanarCityLoc. (a) We extract building masks, surface normals, depth, and structural edges from query image Iq with foundation models, then (b) form 2D planar patches. (c) A coarse pose ξf is obtained through altitude initialization, cost-volume search, and multi-scale particle filtering. (d) We iteratively match visible LoD planes to image patches and refine the pose using PnP-RANSAC on the resulting 3D-to-2D correspondences. (e) Equal-weight rank aggregation of complementary metrics selects the final pose.
UAVD4L-LoDv2 query splits
Swiss-EPFLv2 query splits
in-Traj.
out-of-Traj.
in-Place
out-of-Place
Method
Vision Model
Setup
2m-2 ∘
3m-3 ∘
5m-5 ∘
T.e./R.e.
2m-2 ∘
3m-3 ∘
5m-5 ∘
T.e./R.e.
2m-2 ∘
3m-3 ∘
5m-5 ∘
T.e./R.e.
2m-2 ∘
3m-3 ∘
5m-5 ∘
T.e./R.e.
Released priors
–
–
0.0
0.0
4.3
6.48/1.63
0.0
0.0
0.4
11.12/1.33
0.0
0.0
0.6
17.59/3.86
0.0
0.0
1.1
17.88/3.94
Textured Mesh-based Models
UAVLoc [ 39 ]
LoFTR
O
84.98
88.09
88.90
0.81 / 0.29
91.56
92.02
92.11
0.79 / 0.20
26.40
46.21
62.22
3.06 / 1.86
68.87
81.00
84.96
1.12 / 1.08
UAVLoc [ 39 ]
RoMa
O
93.27
93.70
93.77
0.78 / 0.25
95.03
95.53
95.53
0.73 / 0.22
45.95
66.77
80.73
2.08 / 1.29
89.18
89.68
98.84
0.77 / 1.04
Table 1: Quantitative comparison on UAVD4L-LoDv2 and Swiss-EPFLv2. Setup: O ff-the-shelf, I n-domain training, C ross-dataset training, or GT oracle masks. C ∗ uses the InsLoD-Loc training set [ 26 ] . Recall is in %. T.e./R.e. are median translation (m) and rotation ( ∘ ) errors. For evaluated runs, recall requires both pose thresholds. Best/second-best distinct values per column within each setup group are bold/underlined. † : adapted re-runs (see Sec. 4.1 ).
Figure 3: Qualitative comparison on UAVD4L-LoDv2 and Swiss-EPFLv2. Red overlays show LoD edges at each estimated pose. Labels give translation (m) / rotation ( ∘ ) errors. Similar visual alignments can correspond to different pose errors.
Figure 4: Pose estimates through PlanarCityLoc on UAVD4L-LoDv2. Each row shows the query image and map overlays at the ground-truth, sensor-prior, particle-filter, and final selected poses. Labels give translation (m) / rotation ( ∘ ) errors.
Map
2m–2 ∘
3m–3 ∘
5m–5 ∘
T.e. (m)
R.e. ( ∘ )
LoD1
23.8 / 44.8
54.1 / 74.3
82.2 / 93.0
2.85 / 2.13
1.26 / 0.95
LoD3
75.4 / 80.6
90.8 / 95.1
97.2 / 99.0
1.28 / 1.26
0.68 / 0.69
Table 2: LoD1 vs. LoD3. Full UAVD4L-LoDv2 splits (in-Traj. / out-of-Traj.). Joint recalls (%) and median errors.
Selection rule
2m–2 ∘
3m–3 ∘
5m–5 ∘
T.e. (m)
R.e. ( ∘ )
Particle filter only
36.0 / 60.8
67.7 / 90.6
82.8 / 98.5
2.36 / 1.81
1.30 / 0.98
Max PnP inliers
65.8 / 74.8
84.1 / 91.1
95.4 / 98.2
1.49 / 1.37
0.82 / 0.76
Max mask IoU
67.4 / 80.7
88.7 / 95.3
96.8 / 99.1
1.54 / 1.32
0.79 / 0.75
Max edge score
76.6 / 79.6
90.6 / 94.6
97.1 / 99.0
1.25 / 1.33
0.69 / 0.77
Max composite score
72.3 / 81.2
89.7 / 96.0
97.2 / 99.0
1.39 / 1.30
0.72 / 0.74
Equal-weight ranks
75.4 / 80.6
90.8 / 95.1
97.2 / 99.0
1.28 / 1.26
0.68 / 0.69
Table 3: Pose selection. Full UAVD4L-LoDv2 splits (in-Traj. / out-of-Traj.), using identical candidate pools. Best/second-best values at shown precision are bold/underlined.
Figure 5: Prior pose sensitivity on UAVD4L-LoDv2. Curves show means of split-wise median errors for the same 300 queries per split. Synthetic priors use expanded search. Native-prior points are separate references.
Model
Backbone
2m–2 ∘
3m–3 ∘
5m–5 ∘
T.e. (m)
R.e. ( ∘ )
Metric3D v2
ViT-S
33.0 / 55.0
60.0 / 86.0
82.0 / 94.0
2.55 / 1.87
1.25 / 0.89
Metric3D v2
ViT-L
31.0 / 58.0
62.0 / 78.0
87.0 / 90.0
2.41 / 1.80
1.33 / 0.96
Metric3D v2
ViT-G2
58.0 / 59.0
81.0 / 81.0
95.0 / 89.0
1.70 / 1.73
0.91 / 0.90
MoGe-2
ViT-S
75.0 / 84.0
94.0 / 94.0
98.0 / 99.0
1.14 / 1.37
0.68 / 0.78
MoGe-2
ViT-B
82.0 / 83.0
97.0 / 93.0
98.0 / 98.0
1.05 / 1.23
0.65 / 0.65
MoGe-2
ViT-L
87.0 / 84.0
96.0 / 97.0
97.0 / 100.0
1.09 / 1.25
0.61 / 0.62
Table 4: Geometry predictors. Each model supplies both normals and depth. SAM3 masks are unchanged. All rows use the same 100 queries per UAVD4L-LoDv2 split (in-Traj. / out-of-Traj.). Recall uses joint thresholds. T.e./R.e. are medians. Bold/underline mark best/second-best at shown precision.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset / map
Split
Stage
Queries
1m–1 ∘
2m–2 ∘
3m–3 ∘
5m–5 ∘
T.e.
R.e.
UAVD4L-LoDv2 / LoD3
in-Traj.
Coarse search
1604
1.4
27.6
59.9
79.4
2.62
1.49
UAVD4L-LoDv2 / LoD3
in-Traj.
Particle filter
1604
3.1
36.0
67.7
82.8
2.36
1.30
UAVD4L-LoDv2 / LoD3
in-Traj.
PlanarCityLoc
1604
36.3
75.4
90.8
97.2
1.28
0.68
UAVD4L-LoDv2 / LoD3
out-of-Traj.
Coarse search
2192
2.7
34.9
74.1
95.3
2.35
1.22
UAVD4L-LoDv2 / LoD3
out-of-Traj.
Particle filter
2192
8.8
60.8
90.6
98.5
1.81
0.98
UAVD4L-LoDv2 / LoD3
out-of-Traj.
PlanarCityLoc
2192
35.5
80.6
95.1
99.0
1.26
0.69
Appendix
Table A1: Full-split staged results. Recalls are joint percentages. Translation/rotation errors are medians in meters/degrees. Every stage is evaluated on the same queries within a split.
Figure A1 : Scene context and planar support on Swiss-EPFLv2. Left: views near each split’s median map-patch count and predicted-to-rendered ratio. Right: full-split distributions at the particle-filter pose. Boxes show interquartile ranges, lines show medians, and whiskers show the full range. The ratio axis is logarithmic.
Figure A2 : Plane extraction qualitative comparison. LoD primitives against predicted plane extractions. Patch granularity varies with normal consistency and minimum region size. RQ and PQ denote Recognition Quality and Panoptic Quality, respectively.
Figure A3 : Mask segmentation examples. Query images and predicted masks illustrate incomplete building coverage and non-building regions included by off-the-shelf segmentation. Such errors can weaken coarse alignment and the planar evidence available for refinement.
Configuration
2m–2 ∘
3m–3 ∘
5m–5 ∘
T.e. (m)
R.e. ( ∘ )
Mask only
62.0 / 66.0
87.0 / 82.0
95.0 / 94.0
1.39 / 1.50
0.81 / 0.81
Mask + edges
74.0 / 76.0
92.0 / 91.0
98.0 / 97.0
1.20 / 1.42
0.75 / 0.71
Mask + depth
60.0 / 71.0
80.0 / 89.0
92.0 / 97.0
1.66 / 1.49
0.95 / 0.74
All three cues
75.0 / 84.0
94.0 / 94.0
98.0 / 99.0
1.14 / 1.37
0.68 / 0.78
Appendix
Table A2: Coarse scoring components. Mask, edge, and depth cues in coarse search and particle filtering. Other stages are unchanged. Matched 100-query UAVD4L-LoDv2 subsets (in-Traj. / out-of-Traj.). Joint recalls (%) and median errors. Best/second-best values are bold/underlined.
Configuration
2m–2 ∘
3m–3 ∘
5m–5 ∘
T.e. (m)
R.e. ( ∘ )
Normal consistency
74.0 / 79.0
94.0 / 96.0
98.0 / 100.0
1.30 / 1.41
0.78 / 0.79
Depth continuity
31.0 / 64.0
72.0 / 92.0
93.0 / 97.0
2.36 / 1.67
1.25 / 0.77
Normals + depth
75.0 / 84.0
94.0 / 94.0
98.0 / 99.0
1.14 / 1.37
0.68 / 0.78
Appendix
Table A3: Plane extraction criteria. Normal and depth consistency for predicted and rendered planes. Matched 100-query UAVD4L-LoDv2 subsets (in-Traj. / out-of-Traj.). Joint recalls (%) and median errors. Best/second-best values are bold/underlined.
Configuration
2m–2 ∘
3m–3 ∘
5m–5 ∘
T.e. (m)
R.e. ( ∘ )
Overlap only
54.0 / 76.0
89.0 / 96.0
96.0 / 98.0
1.80 / 1.41
1.02 / 0.86
Centroid only
64.0 / 72.0
85.0 / 91.0
97.0 / 99.0
1.50 / 1.59
0.91 / 0.79
Overlap + centroid
75.0 / 84.0
94.0 / 94.0
98.0 / 99.0
1.14 / 1.37
0.68 / 0.78
Appendix
Table A4: PnP matching strategies. Overlap, centroid, and sequential correspondences. Matched 100-query UAVD4L-LoDv2 subsets (in-Traj. / out-of-Traj.). Joint recalls (%) and median errors. Best/second-best values are bold/underlined.
Figure A4 : Comprehensive prior noise evaluation. Joint recall and median errors under synthetic pose perturbations on UAVD4L-LoDv2, using 300 queries per split and expanded search ranges. Native-prior points are separate references on the same queries.
Figure A5 : Visual examples of map completeness. Overlays of rendered LoD geometry for the intact map and representative removal conditions. Groups are selected by projected area, not physical building volume.
Configuration
2m–2 ∘
3m–3 ∘
5m–5 ∘
T.e. (m)
R.e. ( ∘ )
SAM3
75.0 / 84.0
94.0 / 94.0
98.0 / 99.0
1.14 / 1.37
0.68 / 0.78
EoMT
40.0 / 48.0
54.0 / 61.0
69.0 / 70.0
2.51 / 2.26
1.46 / 1.06
Appendix
Table A5: Building segmenters. SAM 3 and EoMT with fixed MoGe-2 ViT-S geometry. Matched 100-query UAVD4L-LoDv2 subsets (in-Traj. / out-of-Traj.). Joint recalls (%) and median errors. Best/second-best values are bold/underlined.
Configuration
2m–2 ∘
3m–3 ∘
5m–5 ∘
T.e. (m)
R.e. ( ∘ )
Intact map
75.0 / 84.0
94.0 / 94.0
98.0 / 99.0
1.14 / 1.37
0.68 / 0.78
Remove smallest
79.0 / 82.0
94.0 / 98.0
98.0 / 99.0
1.18 / 1.41
0.68 / 0.73
Remove middle
71.0 / 77.0
89.0 / 95.0
94.0 / 99.0
1.41 / 1.46
0.72 / 0.81
Remove largest
52.0 / 69.0
70.0 / 84.0
84.0 / 95.0
1.87 / 1.47
1.02 / 0.76
Appendix
Table A6: Map completeness. Matched 100-query UAVD4L-LoDv2 subsets (in-Traj. / out-of-Traj.). Groups of nearby mesh surfaces are removed by projected area. Queries without sufficient groups retain the intact map. Joint recalls (%) and median errors. Best/second-best values are bold/underlined.
Autonomous Unmanned Aerial Vehicle (UAV) navigation in urban environments requires precise pose estimation. Because Global Navigation Satellite System (GNSS) signals are often unreliable in cities, visual localization serves as a crucial complement. Within this domain, textureless Level-of-Detail (LoD) city models provide a lightweight and privacy-preserving substitute for heavy 3D textured meshes. However, localizing against abstract geometry is challenging. Existing pipelines typically score 2D silhouette overlaps, which can suffer from severe geometric ambiguities. We present MatchCityLoc, a framework that shifts this paradigm to direct feature matching from a noisy 6-DoF pose prior. By extracting surface normals from the query image using a zero-shot foundation model and matching them against rendered LoD normals, we establish explicit 3D-to-2D correspondences. This enables a Perspective-n-Point (PnP) pose solve, which we iteratively refine through a render-and-match loop. Experiments demonstrate strong pose recall and accurate translation and rotation among the evaluated off-the-shelf LoD-based methods with very few renderings. By bridging the modality gap between aerial imagery and texture-free geometry, MatchCityLoc enables accurate localization using lightweight, privacy-preserving city models.
Oussema Dhaouadi, Zuria Bauer, Johannes Michael Meier +3
University of Cambridge · Technical University of Munich · ETH Zürich +2
Aerial 6DoF localization typically relies on precise GNSS signals or radiometrically rich 3D reconstructions, limiting scalability and on-board deployment. We propose SemCityLoc, a semantic-geometric alignment system that reframes aerial pose estimation as structured surface registration between foundation-model-derived visual priors and standardized LoD-compliant 3D city models. Instead of matching sparse contours or dense texture, our method aligns semantic surfaces and monocular depth with lightweight semantic 3D building models, increasing pose discriminability in repetitive and occluded urban environments. To enable accurate evaluation, we introduce SemCityLockeD, the first real-world benchmark combining centimeter-accurate UAV poses with standardized LoD1--LoD3 semantic city models and challenging low-altitude imagery. Experiments demonstrate substantial improvements over existing map-based approaches, improving recall by up to 36% and reducing mean positional error from 9.89m to 2.62m in challenging urban canyons. Our results indicate that semantically structured geometry provides sufficient and scalable constraints for high-precision aerial localization without radiometric scene reconstructions. The code and data are available at https://albertchen98.github.io/SemCityLoc.
Jingfeng Mao, Xuyang Chen, Qilin Zhang +6
Technical University of Munich, Munich, Germany · CV4DT, University of Cambridge, Cambridge, United Kingdom · Munich Center for Machine Learning, Munich, Germany +3
We present LoD-Loc v3, a novel method for generalized aerial visual localization in dense urban environments. While prior work LoD-Loc v2 achieves localization through semantic building silhouette alignment with low-detail city models, it suffers from two key limitations: poor cross-scene generalization and frequent failure in dense building scenes. Our method addresses these challenges through two key innovations. First, we develop a new synthetic data generation pipeline that produces InsLoD-Loc - the largest instance segmentation dataset for aerial imagery to date, comprising 100k images with precise instance building annotations. This enables trained models to exhibit remarkable zero-shot generalization capability. Second, we reformulate the localization paradigm by shifting from semantic to instance silhouette alignment, which significantly reduces pose estimation ambiguity in dense scenes. Extensive experiments demonstrate that LoD-Loc v3 outperforms existing state-of-the-art (SOTA) baselines, achieving superior performance in both cross-scene and dense urban scenarios with a large margin. The project is available at https://nudt-sawlab.github.io/LoD-Locv3/.
Shuaibang Peng, Juelin Zhu, Xia Li +4
National University of Defense Technology · Northwestern Polytechnical University