Observation Quality in DSM-Assisted Aerial Localization: A Benchmark and Base-Anchored Uncertainty Calibration

Pengyu Yin; Junxiang Tan; Jie Pan; Mengqi Fu; Kun Cheng; Yifan Liu

cs.CVOct 13, 2026

Observation Quality in DSM-Assisted Aerial Localization: A Benchmark and Base-Anchored Uncertainty Calibration

Authors: Pengyu Yin, Junxiang Tan, Jie Pan, Mengqi Fu, Kun Cheng, Yifan Liu

Organizations: Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China · School of Electronic Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China

Abstract

Aerial map matching provides position observations for navigation, but a position error alone does not determine how an observation should be weighted. We define five benchmark tasks: position availability and agreement, grouped error structure, predictive uncertainty, controlled updates, and temporal accumulation. A case study in digital-surface-model-assisted localization evaluates base-anchored uncertainty calibration within this framework. The method combines propagated image and map sensitivities with the second moment of a compact configuration subset relative to the reported position. The reported position remains unchanged, and scale is fitted using separate position references without local surface-height ground truth. Thirteen covariance models are evaluated on two held-out query sets from locations represented in the calibration data. Relative to a constant full-matrix baseline, the method reduces equal-location normalized position-update risk by 7.70% on 728 conditions and 5.46% on a separate 142 conditions under an independent prior, with all localization failures included. On the primary set, equal-location negative log likelihood improves by 2.571 nat. These results show improved reference-based weighting at seen locations. Exploratory base-quality and conditional-addition controls leave the incremental update benefit of configuration disagreement unresolved. Navigation benefit and real temporal accumulation remain untested.

Figures & tables

Fig. 1: Benchmark and method overview; all small plots are schematic. (a) Five tasks characterize position observations and their use. (b) Geometric sensitivity

G

and base-relative moment

A

predict error matrix

R

while preserving position

z_{0}

. Solid arrows carry inference data; the dashed arrow supplies parameters estimated from separate calibration data and fixed during evaluation. Task labels indicate which tasks are evaluated on real data. F-C is the constant full-matrix baseline; F-GA combines geometry and anchoring. Real-data evaluation excludes temporal accumulation and cross-frame group moments.

Fig. 2: Synthetic counterexamples with known reference positions. (a) Equal sample count and MSE, different group structure; each observation is a group. (b) Fixed residual, prior and ellipse area, different update error after rotating the ellipse. (c) Reordering the same errors preserves single-frame RMSE but changes window-mean error.

Labels	Meaning	Unit
EC, OC, CA	Execution coverage, output coverage, conditional availability	1
MD, MedD, P95	Mean, median, and 95th-percentile distance	m
$\mathrm{TR}_{t}$ , $A_{5}$ , $L_{1000}$	Tail rate, 5 m reference-agreement rate, capped mean loss	1, 1, m
MSE, RMSE	Mean squared error and its square root	m 2 , m
OSM, BGM, WGM	Overall squared mean, between-group and within-group moments	m 2
$\mathrm{RMSE}_{\mu}$ , $\mathrm{RMSE}_{\rm mac}$	Micro and macro RMSE	m

TABLE I: Metric glossary. Units refer to per-observation values or the stated aggregate. MV and SM are vector and matrix diagnostics, not scalar rankings. The five benchmark tasks specify evaluation samples, failure treatment and aggregation.

Fig. 3: Synthetic operator examples on identical metre-scale axes. Cross: reference

(0,0)

; square: fixed base; rings: minimum-diameter subset (

k=3

); dashed segments: base-relative offsets. Traces of all-member scatter

D

, subset scatter

C_{S}

(Cs), and anchored moment

A

are computed from the displayed configurations. (a) Deviating variants inflate

D

. (b)

A

retains an isolated base’s offset from a compact subset. (c) Shared displacement remains invisible. Matrices describe raw configuration dispersion rather than calibrated error covariances. The reference is shown for interpretation and is excluded from subset selection and scatter calculations.

Model	$V$	$\alpha$	$\beta$	$\gamma$	Converged/starts	Iteration range
F-G	—	0	2.252799	—	3/3	7–11
F-D	$D$	0.018637	—	11.790664	3/3	15–20
F-A	$A$	0.025794	—	3.250298	3/3	14–17
F-GD	$D$	0.008367	0.289716	5.900141	4/4	22–33
F-GA	$A$	0.013870	0.203725	2.403562	4/4	21–26
F-GS	$S$	0.006267	0.442200	2.484152	4/4	19–26

TABLE II: Fitted model coefficients and convergence. All 25 starts converged within the 500-iteration limit; the solution with the lowest calibration objective was retained. Values are rounded for display.

Study	Calibration data	Evaluation data	Interpretation
Ranking diagnostic	246 outputs, 3 locations	247 outputs, same 3 locations	Within-location development analysis; overall update risk unavailable
Method development	300 opportunities, 24 locations	258 opportunities, disjoint locations	F-GD failure analysis motivating anchoring
Main comparison	Original 300; 248 complete-feature outputs	728 validation and 142 inPlace; 24 and 3 represented locations	Held-out queries; thirteen models fixed before reference labels were accessed
Base-quality control	Original 248 complete-feature outputs	Same 728/142, previously evaluated	Exploratory H-Q comparison with no extra localizer runs
Conditional addition	Nested location holdout over original 300; calibration-only fitting	300 source OOF plus previously evaluated 728/142	Exploratory H-QA/H-QI comparison within the same domain

TABLE III: Experiment stages and evaluation data. Counts denote opportunities unless explicitly marked as outputs. Later exploratory studies reuse the earlier data.

Fig. 4: Real inputs and the uncertainty attached to an unchanged position. Rows show the cases with median F-GA-minus-F-C NLL change among all 629 validation outputs and among the 33 cases with increased NLL, respectively; ties are resolved by query ID. Query images, paired maps and DSM heights are from OrthoLoC. Heights share a color scale and represent the surface, not DSM error; blank map areas are retained. The right column shows dataset-local XY coordinates relative to the reference position. The dot is the shared prediction; both 90% regions share this center. Linked insets share limits and a 0.5 m scale bar. Tighter regions contain the reference in (a) but miss it in (b), despite lower NR25 in both. The examples illustrate individual outcomes alongside the complete-cohort results.

Fig. 5: Calibration, controlled updates and comparison with base-only quality. (a,b) Coverage minus nominal level; zero is ideal. Markers are offset horizontally for legibility. (c,d) Thirteen covariance models relative to F-C: NLL difference and percentage NR25 change. Arrows connect F-C to F-GA; negative values on both axes indicate improvement in both metrics. (e,f) Exploratory comparisons with H-Q on the same evaluation sets: F-GA minus H-Q, with descriptive 95% paired-location bootstrap intervals (10,000 resamples, seed 20261009). Negative favors F-GA; both validation intervals cross zero and inPlace NR25 slightly favors H-Q. All panels use equal-location means: 24 validation and three inPlace locations. PC/NLL include 629/136 outputs with fallbacks; NR25 includes all 728/142 eligible queries with no-output risk one, under

P=25I_{2}

m 2 . Positions and states are unchanged. The intervals describe these locations rather than a broader geographic population.

Cohort	MSE mac	Mean $\\|b_{g}\\|^{2}$	Mean $W_{g}$
Val	1.5649	0.0430	1.5218
InPlace	0.3809	0.0284	0.3525

TABLE IV: Equal-location averages of position-error moments (m 2 ) among successful outputs.

b_{g}

is the location’s mean residual and

W_{g}=\operatorname{mean}_{i\in g}\|d_{i}-b_{g}\|^{2}

. Mean

\|b_{g}\|^{2}

summarizes local offsets, not global OSM. Global BGM also requires a verified common orientation across locations. All thirteen models use the same positions.

Constant and configuration-only baselines
	Validation				inPlace
Method	NLL $\downarrow$	PC90	EA90	NR25 $\downarrow$	NLL $\downarrow$	PC90	EA90	NR25 $\downarrow$
	(nat)	target 0.90	(m 2 )		(nat)	target 0.90	(m 2 )
F-I	2.6583	0.9763	16.6503	0.1549	2.1439	0.9744	16.6503	0.0410
F-C	2.7420	0.9739	14.8253	0.1552	2.1251	0.9744	14.8253	0.0415
Config	0.4404	0.9549	29.8456	0.1501	-0.0748	0.9744	3.1150	0.0399

TABLE V: Comparison of thirteen covariance models using equal-location means, identical position estimates and identical failure states. NLL, PC90 and EA90 use successful outputs; NR25 includes every eligible query, with risk one for missing outputs. Bold cells mark minimum NLL/NR25 or PC90 closest to 0.90, based on unrounded values. EA90 describes region size and is interpreted alongside coverage.

\dagger

denotes an NR25 tie after rounding only.

Fig. 6: Paired location differences, F-GA minus F-C (squares) or F-GD (triangles). Blue negative values indicate improvement; red positive values indicate regression. The 27 cohort–location entries share a fixed order; dashed lines separate validation (V) and inPlace (I). Right: means and descriptive 95% paired-location bootstrap intervals for each cohort and comparator. Resampling is by location rather than query; inPlace has only three locations. The right panels use different x-axis limits from the left panels; scales are shared within each panel. NLL uses outputs with valid covariance estimates; NR25 uses all eligible queries (risk 1 for cases without an output). Position predictions are unchanged.

Fig. 7: Coverage and update risk across prior settings. (a,b) Coverage at nominal 50%, 90%, and 95%; diagonals denote ideal coverage. (c,d) All thirteen models under

P=pI_{2}

p\in\{4,25,100\}\,\mathrm{m}^{2}

(outlined: primary

p=25

). Cells show

100[\mathrm{NR}_{m}(p)/\mathrm{NR}_{\mathrm{F-C}}(p)-1]\%

; negative values indicate lower risk. Heatmaps share a symmetric scale. Equal-location means use 24 validation and three inPlace locations. Coverage uses 629/136 successful outputs; NR includes all 728/142 eligible cases, with no-output risk one. Performance ordering varies with the prior.

Cohort	Model	NLL (nat)	PC90 (%)	EA90 (m 2 )	NR25
Val	F-C	2.7420	97.39	14.825	0.155248
Val	H-Q	0.2722	93.14	6.996	0.143556
Val	F-GA	0.1712	92.84	7.126	0.143292
InPlace	F-C	2.1251	97.44	14.825	0.041488
InPlace	H-Q	-0.4146	88.76	2.776	0.039049
InPlace	F-GA	-0.6941	98.21	1.858	0.039224

TABLE VI: Exploratory comparison with base-only calibration on the same evaluation sets. Equal-location PC90 and EA90 use all outputs including fallbacks; NR25 retains all eligible queries. Results are reported separately for validation and inPlace; the latter contains only three locations.

Cohort	Model	NLL (nat)	PC90 (%)	EA90 (m 2 )	NR25
Source OOF	H-Q	0.8936	90.22	8.775	0.138470
Source OOF	H-QA	0.2544	94.20	13.735	0.135774
Source OOF	H-QI	0.8603	91.21	8.825	0.138469
Source OOF	F-GA	0.1427	90.76	17.020	0.135331
Val	H-Q	0.2722	93.14	6.996	0.143556
Val	H-QA	0.2166	95.33	10.431	0.141186

TABLE VII: Additive-model comparison using equal-location means. Source OOF uses nested location holdout within the calibration data; validation and inPlace reuse the earlier evaluation sets. All cohorts belong to the same domain.

Uncertainty Calibration

Uncertainty calibration aligns model confidence and predictive intervals with observed correctness and outcome frequencies.

See all 265 papers

AeroMap3D: Anchoring Monocular UAV 6-DoF Localization to Visual-Geometric-Semantic Map Priors
Jul 15, 2026Zhiyun Deng, Luis SentisCross-View Geo-Localization Image Matching
The University of Texas at Austin · AIVE AI Systems
SemCityLoc: Aerial 6DoF Localization Using Semantic 3D City Models
Jun 25, 2026Jingfeng Mao, Xuyang Chen, Qilin Zhang +6Map-Based Localization Robot Localization
Technical University of Munich, Munich, Germany · CV4DT, University of Cambridge, Cambridge, United Kingdom · Munich Center for Machine Learning, Munich, Germany +3
DSM-Free UAV Visual Localization via Latent-Depth Variable Projection and Structure-Adaptive Coupling
Oct 10, 2026Yibin Ye, Xichao Teng, Shuo Chen +4Robot Localization Map-Based Localization
College of Aerospace Science and Engineering, National University of Defense Technology, Changsha 410073, China, and Hunan Provincial Key Laboratory of Image Measurement and Vision Navigation, Changsha 410073, China. · College of Electronic Science and Technology, National University of Defense Technology, Changsha 410073, China.

Jul 15, 2026J/K move · Enter open · S save

AeroMap3D: Anchoring Monocular UAV 6-DoF Localization to Visual-Geometric-Semantic Map Priors

Zhiyun Deng, Luis Sentis

The University of Texas at Austin · AIVE AI Systems

Cross-View Geo-Localization Image Matching UAV Navigation GNSS-Denied Localization Map-Based Localization Robot Localization

Open paper AskPDF arXiv

We present AeroMap3D, a monocular 6-DoF UAV localization system that anchors onboard imagery to visual, geometric, and semantic map priors for GNSS-denied navigation. AeroMap3D addresses two fundamental challenges in map-referenced aerial localization: the cross-view discrepancy between UAV imagery and satellite maps, and the structural inconsistency between bare-earth digital elevation models (DEMs) and urban scenes. First, we introduce a lightweight adapter that enables a dense matcher pretrained on internet-scale generic data to perform reliable UAV-to-map registration without finetuning. By estimating the scale ratio and yaw offset between the UAV image and map tile, the adapter removes the dominant geometric misalignment induced by altitude, camera field of view, and heading before dense correspondence estimation. Second, AeroMap3D lifts 2D UAV-map correspondences onto DEM terrain while using OpenStreetMap annotations to reject semantically unreliable matches before RANSAC-PnP pose estimation, thereby reducing errors caused by unmodeled building heights and off-nadir structures. Delayed map-based pose measurements are further fused with relative-motion priors using a delayed-state EKF for continuous trajectory estimation. Without UAV-Terra3D retraining or tuning, AeroMap3D localizes all trajectories across eight Austin sites within 50 m and achieves 5.88 m mean 3D error over 55 km of flight.

Abstract

Uncertainty Calibration

Explore similar work

AeroMap3D: Anchoring Monocular UAV 6-DoF Localization to Visual-Geometric-Semantic Map Priors

SemCityLoc: Aerial 6DoF Localization Using Semantic 3D City Models

DSM-Free UAV Visual Localization via Latent-Depth Variable Projection and Structure-Adaptive Coupling