Aerial visual localization over Level-of-Detail (LoD) 3D Maps has recently gained significant traction due to its global scalability, storage efficiency, and privacy-preserving nature. However, existing methods rely almost exclusively on building elements, which leads to catastrophic localization failures in rural or undeveloped areas where buildings are sparse or non-existent. To address this, we construct a unified Panoptic LoD 3D Map by fusing OpenStreetMap (OSM), Digital Elevation Models (DEM), and LoD assets, reformulating the problem as a panoptic registration task. By parsing the query image into discrete building thing' instances and continuous stuff' regions comprising road, vegetation, and water, we establish a holistic alignment with this map to robustly optimize the camera pose. To enable reliable panoptic perception, we develop a new automated pipeline that constructs AirPan - the largest aerial panoptic segmentation dataset to date, comprising 85k images with pixel-accurate annotations. This empowers our panoptic segmentation model to exhibit remarkable zero-shot generalization capabilities. Extensive experiments demonstrate that LoD-Loc v4 consistently outperforms state-of-the-art (SOTA) baselines by a significant margin. Most notably, it successfully maintains reliable localization in building-less scenarios, completely overcoming the catastrophic failures of prior building-centric paradigms.
Figures & tables
Figure 1: In this paper, we introduce LoD-Loc v4 to extend aerial localization beyond building-centric scenes based on the Panoptic LoD 3D Map. In urban scenes, our method outperforms SOTA baselines. In building-less scenes, LoD-Loc v4 maintains reliable results by utilizing road, vegetation, and water layouts as panoptic cues, whereas previous methods fail entirely.
Figure 2: Overview of LoD-Loc v4. 1. Constructing a Panoptic LoD 3D Map M from DEM, OSM, and LoD models (Section 3.1 ). 2. Extracting building instances Bq and continuous regions Rq from query Iq via a segmentation module (Section 3.2 ). 3. Refining the prior pose ξp to ξ∗ via panoptic alignment between predicted masks and rendered hypotheses (Section 3.3 ).
Figure 3: Overview of the AirPan dataset. This figure visualizes representative RGB imagery alongside corresponding ground-truth masks sampled from geographically diverse locations. Encompassing both densely built urban layouts and extremely building-sparse environments, the visualization further summarizes the overall dataset scale in terms of total image volume and spatial coverage.
Figure 4: Qualitative comparison between LoD-Loc v3 and LoD-Loc v4. The columns from left to right show: query image, prior pose, LoD-Loc v3, LoD-Loc v4 (Ours), and the ground truth. All overlays show the same panoptic map rendered at the corresponding poses.
Metric
All
Thing
Stuff
Vegetation
Water
Road
Background
PQ
80.47
75.71
81.66
90.86
81.30
82.95
71.52
SQ
88.69
87.55
88.98
93.87
93.56
85.63
82.85
RQ
90.67
86.48
91.72
96.80
86.90
96.87
86.32
Table 1: Panoptic segmentation quality.
Method
in-Traj.
out-of-Traj.
2m-2 ∘
3m-3 ∘
5m-5 ∘
T.e./R.e.
2m-2 ∘
3m-3 ∘
5m-5 ∘
T.e./R.e.
Prior
0
0
4.30
6.48/1.63
0
0
0.36
11.10/0.92
OrienterNet [ 45 ]
0
0
0.06
-
0
0
0
-
VecMapLocNet [ 55 ]
0
0
0.06
-
0
0
0
-
LoD-Loc [ 65 ]
49.56
71.82
89.09
3.32/1.48
54.20
75.05
89.51
3.33/1.18
LoD-Loc v2 [ 64 ]
93.70
98.40
99.50
0.72/0.15
97.90
99.80
100.00
0.71/0.14
Table 2: Quantitative comparison results of different methods over UAVD4L-LoDv2 dataset. T.e. and R.e. denote median translation error (m) and median rotation error ( ∘ ), respectively. ‘-’ indicates that the median translation and rotation errors exceed 50m and 10∘ .
Method
in-Place.
out-of-Place.
2m-2 ∘
3m-3 ∘
5m-5 ∘
T.e./R.e.
2m-2 ∘
3m-3 ∘
5m-5 ∘
T.e./R.e.
Prior
0
0
0.56
17.6/3.87
0
0
1.06
17.9/3.94
OrienterNet
0
0
0
-
0
0
0
-
VecMapLocNet
0
0
0
-
0
0
0
-
LoD-Loc
36.79
50.56
69.77
2.87/1.78
14.24
31.39
59.89
8.73/2.78
LoD-Loc v2
54.20
74.60
92.00
1.83/0.85
31.40
58.53
86.30
2.64/0.73
Table 3: Quantitative comparison results of different methods over Swiss-EPFLv2 dataset.
Method
Grid-Traj.
Sequence-Traj.
2m-2 ∘
3m-3 ∘
5m-5 ∘
T.e./R.e.
2m-2 ∘
3m-3 ∘
5m-5 ∘
T.e./R.e.
Prior
0.10
1.70
8.90
8.03/1.78
0.30
0.80
8.80
8.19/1.76
OrienterNet
0
0
0
-
0
0
0
-
VecMapLocNet
0.07
0.15
0.51
-
0
0
0
-
LoD-Loc
10.52
20.95
33.80
8.03/1.78
0
0
5.06
8.19/1.76
LoD-Loc v2
2.70
8.10
22.70
7.86/1.48
2.30
6.80
24.00
8.75/1.52
Table 4: Quantitative comparison results of different methods over Tokyo-LoDv3 dataset.
Method
China-NB
Switzerland-NB
2m-2 ∘
3m-3 ∘
5m-5 ∘
T.e./R.e.
2m-2 ∘
3m-3 ∘
5m-5 ∘
T.e./R.e.
Prior
0
0
6.32
8.29/0.17
0
0.43
3.42
11.42/0.17
OrienterNet
0
0
0
-
0
0
0
-
VecMapLocNet
0
0.37
0.74
-
0
0
0
-
LoD-Loc
0
0
0
-
0
0
0
-
LoD-Loc v2
0
0
0
-
0
0
0
-
Table 5: Quantitative comparison results of different methods over building-less scenes.
Composition
in-Place.
out-of-Place.
2m-2 ∘
3m-3 ∘
5m-5 ∘
T.e./R.e.
2m-2 ∘
3m-3 ∘
5m-5 ∘
T.e./R.e.
no thing
52.20
69.40
81.20
1.87/0.71
37.20
63.10
84.50
2.38/0.66
no stuff
61.00
77.10
90.20
1.51/0.79
49.90
73.40
90.20
2.01/0.73
Full
71.30
84.10
93.10
1.29/0.61
53.80
74.10
91.30
1.89/0.64
Table 6: Ablation study on panoptic composition.
Epoch
PQ
Japan_06
Japan_07
2m-2 ∘
3m-3 ∘
5m-5 ∘
T.e/R.e
2m-2 ∘
3m-3 ∘
5m-5 ∘
T.e/R.e
10
44.38
49.50
78.00
94.90
2.02/0.25
43.20
71.20
92.60
2.17/0.26
20
53.28
53.10
80.20
95.20
1.95/0.25
49.00
73.30
92.30
2.04/0.25
30
63.74
57.00
85.30
97.60
1.81/0.23
50.50
77.30
94.70
1.99/0.25
Table 7: Ablation study on segmentation quality.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Name
Area
Images
Height[ m ]
Coverage[ km2 ]
Size[ pixel ]
View Angle[°]
Netherlands_01
Amsterdam
392
500
18.3879
1600×1200
[0, 30]
Netherlands_02
Amsterdam
1242
200
17.3980
1600×1200
[30, 50]
Netherlands_03
Hague
2048
300
16.9807
1600×1200
[0, 30]
Netherlands_04
Delft
953
200
20.1190
1600×1200
[0, 30]
Netherlands_05
Amsterdam
558
500
16.9844
1600×1200
[30, 50]
Netherlands_06
Tilburg
2874
200
19.7211
1600×1200
[0, 30]
Appendix
Table 8: Detailed characteristics of the regional datasets composing the AirPan dataset.
Figure 5: Visualization of training samples. This figure displays pairs of synthetic RGB images and their corresponding panoptic annotations. The top four rows showcase samples of the Japan, Switzerland, and Netherlands regions from the AirPan dataset, while the bottom two rows feature samples from the integrated auxiliary dataset.
Figure 6: Visualization of samples from the test datasets. Rows show samples from UAVD4L-LoDv2, Swiss-EPFLv2, Tokyo-LoDv3, China-NB, and Switzerland-NB.
Method
Japan_09*
2m-2 ∘
3m-3 ∘
5m-5 ∘
T.e./R.e.
Prior
0.52
0.79
7.35
8.14/1.80
LoD-Loc v2
1.80
3.10
8.70
14.69/0.99
LoD-Loc v3
24.70
55.10
93.70
2.86/0.23
LoD-Loc v4
52.00
81.40
96.90
1.97/0.25
Appendix
Table 9: Quantitative comparison of different methods under adverse-weather condition.
We present LoD-Loc v3, a novel method for generalized aerial visual localization in dense urban environments. While prior work LoD-Loc v2 achieves localization through semantic building silhouette alignment with low-detail city models, it suffers from two key limitations: poor cross-scene generalization and frequent failure in dense building scenes. Our method addresses these challenges through two key innovations. First, we develop a new synthetic data generation pipeline that produces InsLoD-Loc - the largest instance segmentation dataset for aerial imagery to date, comprising 100k images with precise instance building annotations. This enables trained models to exhibit remarkable zero-shot generalization capability. Second, we reformulate the localization paradigm by shifting from semantic to instance silhouette alignment, which significantly reduces pose estimation ambiguity in dense scenes. Extensive experiments demonstrate that LoD-Loc v3 outperforms existing state-of-the-art (SOTA) baselines, achieving superior performance in both cross-scene and dense urban scenarios with a large margin. The project is available at https://nudt-sawlab.github.io/LoD-Locv3/.
Shuaibang Peng, Juelin Zhu, Xia Li +4
National University of Defense Technology · Northwestern Polytechnical University
Aerial visual localization is crucial for autonomous drone navigation in urban environments where GNSS signals are often degraded. Lightweight textureless Level-of-Detail (LoD) city models provide a geometric map alternative to textured assets. Existing silhouette-based methods rely on image-space alignment, which can leave different camera poses indistinguishable and favor incorrect poses when segmentation errors distort the building outlines. We present PlanarCityLoc, which moves beyond silhouette overlap by leveraging the planar geometry of building facades and roofs. Following geometry-aware initialization, we match image-side planar patches to LoD model planes and use the resulting 3D-to-2D correspondences to refine pose with Perspective-n-Point (PnP). The framework uses off-the-shelf geometry and segmentation models without dataset-specific training or fine-tuning. Experiments on UAVD4L-LoDv2 and Swiss-EPFLv2 achieve state-of-the-art localization recall among the evaluated off-the-shelf LoD-based methods. By grounding image-space alignment in 3D planes, PlanarCityLoc enables more accurate aerial localization with lightweight, textureless city maps.
Oussema Dhaouadi, Shao Jie Hu-Chen, Jakob Gollreiter +10
Technical University of Munich · University of Cambridge · ETH Zürich +5
Aerial 6DoF localization typically relies on precise GNSS signals or radiometrically rich 3D reconstructions, limiting scalability and on-board deployment. We propose SemCityLoc, a semantic-geometric alignment system that reframes aerial pose estimation as structured surface registration between foundation-model-derived visual priors and standardized LoD-compliant 3D city models. Instead of matching sparse contours or dense texture, our method aligns semantic surfaces and monocular depth with lightweight semantic 3D building models, increasing pose discriminability in repetitive and occluded urban environments. To enable accurate evaluation, we introduce SemCityLockeD, the first real-world benchmark combining centimeter-accurate UAV poses with standardized LoD1--LoD3 semantic city models and challenging low-altitude imagery. Experiments demonstrate substantial improvements over existing map-based approaches, improving recall by up to 36% and reducing mean positional error from 9.89m to 2.62m in challenging urban canyons. Our results indicate that semantically structured geometry provides sufficient and scalable constraints for high-precision aerial localization without radiometric scene reconstructions. The code and data are available at https://albertchen98.github.io/SemCityLoc.
Jingfeng Mao, Xuyang Chen, Qilin Zhang +6
Technical University of Munich, Munich, Germany · CV4DT, University of Cambridge, Cambridge, United Kingdom · Munich Center for Machine Learning, Munich, Germany +3