MatchCityLoc: Aerial Localization in LoD City Models via Surface Normal Matching

Oussema Dhaouadi; Zuria Bauer; Johannes Michael Meier; Marc Pollefeys; Daniel Cremers; Olaf Wysocki

cs.CVOct 19, 2026

MatchCityLoc: Aerial Localization in LoD City Models via Surface Normal Matching

Authors: Oussema Dhaouadi, Zuria Bauer, Johannes Michael Meier, Marc Pollefeys, Daniel Cremers, Olaf Wysocki

Organizations: University of Cambridge · Technical University of Munich · ETH Zürich · Munich Center for Machine Learning ·

Abstract

Autonomous Unmanned Aerial Vehicle (UAV) navigation in urban environments requires precise pose estimation. Because Global Navigation Satellite System (GNSS) signals are often unreliable in cities, visual localization serves as a crucial complement. Within this domain, textureless Level-of-Detail (LoD) city models provide a lightweight and privacy-preserving substitute for heavy 3D textured meshes. However, localizing against abstract geometry is challenging. Existing pipelines typically score 2D silhouette overlaps, which can suffer from severe geometric ambiguities. We present MatchCityLoc, a framework that shifts this paradigm to direct feature matching from a noisy 6-DoF pose prior. By extracting surface normals from the query image using a zero-shot foundation model and matching them against rendered LoD normals, we establish explicit 3D-to-2D correspondences. This enables a Perspective-n-Point (PnP) pose solve, which we iteratively refine through a render-and-match loop. Experiments demonstrate strong pose recall and accurate translation and rotation among the evaluated off-the-shelf LoD-based methods with very few renderings. By bridging the modality gap between aerial imagery and texture-free geometry, MatchCityLoc enables accurate localization using lightweight, privacy-preserving city models.

Figures & tables

Figure 1: (a) Prior silhouette alignment extracts a 2D mask from the UAV image and evaluates thousands of rendered LoD masks to score poses. (b) MatchCityLoc shifts this paradigm to iterative render-and-match by extracting surface normals and matching them directly against a rendered LoD normal map. These matches are passed to an explicit 3D pose solver (PnP), enabling pose refinement with a few renderings.

Figure 2: MatchCityLoc pipeline overview. (1) Given a UAV query image and a noisy 6-DoF pose prior $\boldsymbol{\xi}_{prior}$ , we predict a dense surface normal map $\hat{\mathbf{N}}$ zero-shot. (2) We then iteratively refine the pose $\boldsymbol{\xi}^{(t)}$ up to $N$ times by rendering an expected normal map $\mathbf{N}_{r}^{(t)}$ from the LoD model, matching $\hat{\mathbf{N}}$ against $\mathbf{N}_{r}^{(t)}$ to establish dense 2D correspondences $\mathcal{C}^{(t)}$ , lifting them to 3D via rendered depth, and solving for $\boldsymbol{\xi}^{(t+1)}$ with PnP-RANSAC.

Figure 3: Iterative render-and-match. The query image and its predicted normals $\hat{\mathbf{N}}$ are matched against the rendered normals $\mathbf{N}_{r}^{(t)}$ at each step $t$ . LoD edge overlays at the resulting pose $\boldsymbol{\xi}^{(t)}$ show progressive alignment. Translation error (T.e.) and rotation error (R.e.) are reported in meters and degrees, respectively. This example shows three updates. The general loop allows at most $N$ updates, with early stopping.

			UAVD4L-LoD								Swiss-EPFL
			in-Traj.				out-of-Traj.				in-Place				out-of-Place
Method	Vision Model	Setup	2m-2 ${}^{\circ}\!\uparrow$	3m-3 ${}^{\circ}\!\uparrow$	5m-5 ${}^{\circ}\!\uparrow$	T.e./R.e. $\!\downarrow$	2m-2 ${}^{\circ}\!\uparrow$	3m-3 ${}^{\circ}\!\uparrow$	5m-5 ${}^{\circ}\!\uparrow$	T.e./R.e. $\!\downarrow$	2m-2 ${}^{\circ}\!\uparrow$	3m-3 ${}^{\circ}\!\uparrow$	5m-5 ${}^{\circ}\!\uparrow$	T.e./R.e. $\!\downarrow$	2m-2 ${}^{\circ}\!\uparrow$	3m-3 ${}^{\circ}\!\uparrow$	5m-5 ${}^{\circ}\!\uparrow$	T.e./R.e. $\!\downarrow$
Sensor priors	–	–	0.00	0.00	4.30	6.48/1.63	0.00	0.00	0.36	11.12/1.33	0.00	0.00	0.56	17.59/3.86	0.00	0.00	1.06	17.88/3.94
Textured Mesh-based Models
UAVLoc [ 30 ]	LoFTR	O	84.98	88.09	88.90	0.81 / 0.29	91.56	92.02	92.11	0.79 / 0.20	26.40	46.21	62.22	3.06 / 1.86	68.87	81.00	84.96	1.12 / 1.08
UAVLoc [ 30 ]	RoMa	O	93.27	93.70	93.77	0.78 / 0.25	95.03	95.53	95.53	0.73 / 0.22	45.95	66.77	80.73	2.08 / 1.29	89.18	89.68	98.84	0.77 / 1.04

Table 1: Visual localization on UAVD4L-LoD and Swiss-EPFL queries. Our results and adapted re-runs (

\dagger

) use LoD3 and LoD2 maps, respectively. Published rows retain source protocols, including the LoD1 extensions UAVD4L-LoDv2 and Swiss-EPFLv2 for LoD-Loc v2, LoD-Loc v3, and MC-Loc. T.e./R.e.: median translation (m) and rotation ( ∘ ). ‡ SemCityLoc reports mean translation/yaw. Bold/underline: best/second within each group, not a matched-map ranking across source protocols. Oracle evaluation settings are in the supplement. Setup: O ff-the-shelf, I n-domain training, C* : training on InsLoD-Loc [ 22 ] , GT : ground-truth masks.

Figure 5: Qualitative comparison across UAVD4L-LoD, Swiss-EPFL, and SemCityLockeD. Each panel blends LoD surface normals rendered at the predicted pose over the UAV image (red: LoD edges). T.e./R.e.: translation error (m) and rotation error ( ∘ ) vs. ground truth. † LoD-Loc v2 [ 35 ] and SemCityLoc [ 18 ] with off-the-shelf masks. ‡ LoD-Loc v3 [ 22 ] with oracle masks.

Figure 6: Challenging failure modes of silhouette alignment. Each panel blends LoD normals at the predicted pose over the UAV image (red: LoD edges). Top (Swiss-EPFL): construction-site temporal mismatch. Bottom (SemCityLockeD): facade-saturated view where a single wall fills the frame. T.e./R.e.: translation error (m) and rotation error ( ∘ ). † off-the-shelf masks. ‡ oracle masks.

		in-Traj. / out-of-Traj.
UAV	LoD	2m-2 ${}^{\circ}\!\uparrow$	3m-3 ${}^{\circ}\!\uparrow$	5m-5 ${}^{\circ}\!\uparrow$	T.e./R.e. $\!\downarrow$
RGB	Depth	57.61 / 61.22	64.03 / 67.93	70.95 / 71.94	1.40/0.78 / 1.35/0.65
RGB	Normals	77.81 / 83.90	83.23 / 89.10	88.53 / 92.02	0.74 / 0.40 / 0.74 / 0.30
Depth	Normals	73.38 / 79.24	83.29 / 86.82	91.21 / 92.84	1.07/0.59 / 1.02/0.48
Depth	Depth	83.42 / 88.32	89.65 / 94.75	94.08 / 97.63	0.81/0.42 / 0.85/0.35
Normals ∗	Normals ∗	93.64 / 95.48	96.20 / 97.99	98.69 / 99.27	0.58 / 0.31 / 0.64 / 0.27

Table 3: Matching modalities (UAVD4L-LoD, LoD3). Normal-to-normal matching performs best. T.e./R.e.: median translation (m) and rotation ( ∘ ) error.

	in-Traj. / out-of-Traj.
Matching algorithm	2m-2 ${}^{\circ}\!\uparrow$	3m-3 ${}^{\circ}\!\uparrow$	5m-5 ${}^{\circ}\!\uparrow$	T.e./R.e. $\!\downarrow$
SPP + SuperGlue	73.38 / 66.83	81.42 / 79.11	87.09 / 84.95	1.20/0.70 / 1.36/0.68
XFeat	26.93 / 18.11	44.20 / 30.98	62.09 / 51.41	3.50/1.65 / 4.78/2.01
LightGlue	53.99 / 52.01	63.59 / 64.78	73.94 / 73.36	1.78/1.00 / 1.89/0.94
LoFTR	62.22 / 59.95	70.51 / 66.97	78.49 / 70.57	1.28/0.75 / 1.41/0.68
DKM	16.90 / 15.15	22.13 / 19.21	30.42 / 22.67	6.03/1.49 / 9.58/1.31

Table 4: Matching algorithms (UAVD4L-LoD, LoD3). MINIMA outperforms the evaluated RGB matchers. T.e./R.e.: median translation (m) and rotation ( ∘ ) error.

		in-Traj. / out-of-Traj.
Model	Backbone	2m-2 ${}^{\circ}\!\uparrow$	3m-3 ${}^{\circ}\!\uparrow$	5m-5 ${}^{\circ}\!\uparrow$	T.e./R.e. $\!\downarrow$
Metric3D	ViT-S	81.36 / 89.64	87.47 / 95.57	91.77 / 97.90	0.84/0.49 / 0.81/0.36
Metric3D	ViT-L	79.18 / 88.50	86.78 / 94.80	92.58 / 97.58	0.84/0.55 / 0.84/0.43
Metric3D	ViT-G	95.45 / 94.34	97.63 / 97.26	99.50 / 98.45	0.54 / 0.30 / 0.67/ 0.27
MoGe-2 ∗	ViT-S ∗	93.33 / 95.21	96.20 / 97.95	98.13 / 98.95	0.58/0.31 / 0.66 /0.28
MoGe-2	ViT-B	94.39 / 95.80	97.07 / 98.49	98.32 / 99.18	0.55/ 0.30 / 0.64 / 0.26

Table 5: Normal predictor models (UAVD4L-LoD, LoD3). All rows cover the complete splits at four updates. MoGe-2 ViT-S is competitive with the heavier backbones. T.e./R.e.: median translation (m) and rotation ( ∘ ) error.

City Model	2m-2 ${}^{\circ}\!\uparrow$	3m-3 ${}^{\circ}\!\uparrow$	5m-5 ${}^{\circ}\!\uparrow$	T.e./R.e. $\!\downarrow$
LoD1 (Prismatic)	14.09	26.04	40.45	7.00/3.26
LoD2 (Roof shapes)	59.12	68.41	77.16	1.45 / 1.14
LoD3 (Detailed)	70.33	79.94	86.34	0.87 / 0.81

Table 6: Sensitivity to Level-of-Detail (SemCityLockeD). LoD1 flat-block models lack the surface normal variation needed for matching, causing a large recall drop. Performance recovers strongly with LoD2, confirming the pipeline requires and exploits explicit 3D geometry. T.e./R.e.: median translation (m) and rotation ( ∘ ) error.

Figure 7: Impact of LoD detail on a close-facade view (SemCityLockeD). Top: query UAV image and MatchCityLoc overlays with LoD2 vs. LoD3 (LoD normals + red edges at predicted pose). Bottom: MoGe-predicted query normals and LoD normals rendered at the pose prior $\boldsymbol{\xi}_{prior}$ . T.e./R.e.: translation (m) and rotation ( ∘ ) errors vs. ground truth.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A1 : Matching modality qualitative ablation. Alternative modality pairs for matching. The normals-to-normals configuration aligns the projected geometry with the building structure. T.e./R.e.: per-frame translation (m) and rotation ( ∘ ) error.

Figure A2 : Matching algorithm qualitative comparison. Comparison of dense and sparse matchers on the UAVD4L-LoD dataset. RoMa [ 7 ] with MINIMA [ 23 ] weights produces geometrically consistent correspondences. T.e./R.e.: per-frame translation (m) and rotation ( ∘ ) error.

Stage	Time (s) $\downarrow$
Once per query
Normal prediction (MoGe-2 ViT-S)	0.28
Per refinement update
Rendering / preparation (nvdiffrast)	0.70
Dense matching (MINIMA / RoMa)	0.73
PnP refinement	0.01

Appendix

Table A1: MatchCityLoc runtime (UAVD4L-LoD, LoD3). Per-update averages include normal and depth rendering at each pose. Normal prediction is a separate one-time cost.

Panel of Figure 16 — Figure A3 : Prior pose noise robustness. Component-wise translation and Euler-angle perturbations of the ground-truth pose, with roll/pitch clipping as described in the text. Close-range urban geometry in SemCityLockeD limits recovery at the largest perturbations compared to UAVD4L-LoD [ 30 , 36 ] . The blue region represents prior pose improvement using our MatchCityLoc method. Metrics are reported with noise in meters and degrees, and recall in percent.

Figure A4 : Map completeness qualitative analysis. Overlays of rendered footprints when specific building instances are removed. Dense normal matching remains robust on UAVD4L-LoD.

Panel of Figure 18 — Figure A5 : Normal quality versus pose error. Per-frame median angular error ( ∘ ) against translation error (m). The spread and bin medians are shown for UAVD4L-LoD [ 30 , 36 ] and SemCityLockeD [ 18 ] .

Figure A6 : Representative failure cases. Examples of challenging lighting, structure shifts, and repetitive structures that limit pose recovery.

	Recall (%) $\uparrow$			Median Err. $\downarrow$
Configuration	1m-1 ∘	2m-2 ∘	3m-3 ∘	T.e. (m)	R.e. ( ∘ )
in-Traj.
Baseline Map	78.9	91.3	94.4	0.60	0.33
Missing Small	78.9	93.8	95.7	0.56	0.30
Missing Mid	78.3	90.7	93.8	0.58	0.31
Missing Large	79.5	91.3	95.0	0.58	0.31

Appendix

Table A2: Ablation on map completeness. Localization performance when a specific building is artificially removed from the LoD3 map prior. Results are evaluated on the UAVD4L-LoD dataset [ 36 ] . Recall is in %. T.e./R.e.: median translation (m) and rotation ( ∘ ) error.

Camera Pose Estimation

Camera pose estimation recovers a camera’s position and orientation from images, visual features, scene geometry, and sometimes sensor measurements.

See all 241 papers

Oct 19, 2026

PlanarCityLoc: Breaking Silhouette Ambiguity in LoD Aerial Localization via 3D Planes

Aerial visual localization is crucial for autonomous drone navigation in urban environments where GNSS signals are often degraded. Lightweight textureless Level-of-Detail (LoD) city models provide a geometric map alternative to textured assets. Existing silhouette-based methods rely on image-space alignment, which can leave different camera poses indistinguishable and favor incorrect poses when segmentation errors distort the building outlines. We present PlanarCityLoc, which moves beyond silhouette overlap by leveraging the planar geometry of building facades and roofs. Following geometry-aware initialization, we match image-side planar patches to LoD model planes and use the resulting 3D-to-2D correspondences to refine pose with Perspective-n-Point (PnP). The framework uses off-the-shelf geometry and segmentation models without dataset-specific training or fine-tuning. Experiments on UAVD4L-LoDv2 and Swiss-EPFLv2 achieve state-of-the-art localization recall among the evaluated off-the-shelf LoD-based methods. By grounding image-space alignment in 3D planes, PlanarCityLoc enables more accurate aerial localization with lightweight, textureless city maps.

Oussema Dhaouadi, Shao Jie Hu-Chen, Jakob Gollreiter +10

Technical University of Munich · University of Cambridge · ETH Zürich +5

Robot Localization Map-Based Localization Camera Pose Estimation UAV Navigation

Ask PDF

Jun 25, 2026

SemCityLoc: Aerial 6DoF Localization Using Semantic 3D City Models

Aerial 6DoF localization typically relies on precise GNSS signals or radiometrically rich 3D reconstructions, limiting scalability and on-board deployment. We propose SemCityLoc, a semantic-geometric alignment system that reframes aerial pose estimation as structured surface registration between foundation-model-derived visual priors and standardized LoD-compliant 3D city models. Instead of matching sparse contours or dense texture, our method aligns semantic surfaces and monocular depth with lightweight semantic 3D building models, increasing pose discriminability in repetitive and occluded urban environments. To enable accurate evaluation, we introduce SemCityLockeD, the first real-world benchmark combining centimeter-accurate UAV poses with standardized LoD1--LoD3 semantic city models and challenging low-altitude imagery. Experiments demonstrate substantial improvements over existing map-based approaches, improving recall by up to 36% and reducing mean positional error from 9.89m to 2.62m in challenging urban canyons. Our results indicate that semantically structured geometry provides sufficient and scalable constraints for high-precision aerial localization without radiometric scene reconstructions. The code and data are available at https://albertchen98.github.io/SemCityLoc.

Jingfeng Mao, Xuyang Chen, Qilin Zhang +6

Technical University of Munich, Munich, Germany · CV4DT, University of Cambridge, Cambridge, United Kingdom · Munich Center for Machine Learning, Munich, Germany +3

Map-Based Localization Robot Localization

Ask PDF

Mar 20, 2026

LoD-Loc v3: Generalized Aerial Localization in Dense Cities using Instance Silhouette Alignment

We present LoD-Loc v3, a novel method for generalized aerial visual localization in dense urban environments. While prior work LoD-Loc v2 achieves localization through semantic building silhouette alignment with low-detail city models, it suffers from two key limitations: poor cross-scene generalization and frequent failure in dense building scenes. Our method addresses these challenges through two key innovations. First, we develop a new synthetic data generation pipeline that produces InsLoD-Loc - the largest instance segmentation dataset for aerial imagery to date, comprising 100k images with precise instance building annotations. This enables trained models to exhibit remarkable zero-shot generalization capability. Second, we reformulate the localization paradigm by shifting from semantic to instance silhouette alignment, which significantly reduces pose estimation ambiguity in dense scenes. Extensive experiments demonstrate that LoD-Loc v3 outperforms existing state-of-the-art (SOTA) baselines, achieving superior performance in both cross-scene and dense urban scenarios with a large margin. The project is available at https://nudt-sawlab.github.io/LoD-Locv3/.

Shuaibang Peng, Juelin Zhu, Xia Li +4

National University of Defense Technology · Northwestern Polytechnical University

Robot Localization Map-Based Localization Synthetic Data Generation

Ask PDF

Abstract

Camera Pose Estimation

Explore similar work

PlanarCityLoc: Breaking Silhouette Ambiguity in LoD Aerial Localization via 3D Planes

SemCityLoc: Aerial 6DoF Localization Using Semantic 3D City Models

LoD-Loc v3: Generalized Aerial Localization in Dense Cities using Instance Silhouette Alignment