Sky2Ground: A Benchmark for Site Modeling under Varying Altitude

Zengyan Wang; Sirshapan Mitra; Rajat Modi; Grace Lim; Yogesh Rawat

cs.CVMar 14, 2026

Sky2Ground: A Benchmark for Site Modeling under Varying Altitude

Authors: Zengyan Wang, Sirshapan Mitra, Rajat Modi, Grace Lim, Yogesh Rawat

Organizations: Center for Research In Computer Vision, University of Central Florida

Abstract

We introduce Sky2Ground, a three-view dataset designed for varying altitude camera localization, correspondence learning, and reconstruction. The dataset combines structured synthetic imagery with real, in-the-wild images, providing both controlled multi-view geometry and realistic scene noise. Each of the 51 sites contains thousands of satellite, aerial, and ground images spanning wide altitude ranges and nearly orthogonal viewing angles, enabling rigorous evaluation across global-to-local contexts. We benchmark state of the art pose estimation models, including MASt3R, DUSt3R, Map Anything, and VGGT, and observe that the use of satellite imagery often degrades performance, highlighting the challenges under large altitude variations. We also examine reconstruction methods, highlighting the challenges introduced by sparse geometric overlap, varying perspectives, and the use of real imagery, which often introduces noise and reduces rendering quality. To address some of these challenges, we propose SkyNet, a model which enhances cross-view consistency when incorporating satellite imagery with a curriculum-based training strategy to progressively incorporate more satellite views. SkyNet significantly strengthens multi-view alignment and outperforms existing methods by 9.6% on RRA@5 and 18.1% on RTA@5 in terms of absolute performance. Sky2Ground and SkyNet together establish a comprehensive testbed and baseline for advancing large-scale, multi-altitude 3D perception and generalizable camera localization. Code and models will be released publicly for future research.Project page: https://sky2ground2026.github.io/sky2ground/

Figures & tables

Figure 1: Cross-view examples from the Sky2Ground dataset. Satellite, aerial, and ground-level images for a variety of urban scenes in Sky2Ground, where each column corresponds to a unique site. These examples highlight strong viewpoint and appearance variations across modalities, revealing the challenges of cross-view matching and multi-scale scene understanding. Real images additionally introduce diverse lighting conditions, weather effects, and natural scene noise, further emphasizing the complexity of real-world cross-view perception.

Dataset	Sites	Imgs	Syn A	Syn G	Real A	Real G	Sat
KITTI-360 [ 21 ]	1	300k	✗	✗	✗	✓	✗
nuScenes [ 3 ]	2	1.4M	✗	✗	✗	✓	✗
GauU-Scene [ 47 ]	6	4.7k	✗	✗	✓	✗	✗
UC-GS [ 53 ]	2	1.4k	✓	✓	✗	✗	✗
BungeeNeRF [ 46 ]	12	3.1k	✓	✗	✗	✗	✓
MatrixCity [ 20 ]	1	519k	✓	✓	✗	✗	✗

Table 1: Comparison with existing datasets. A: Aerial, G: Ground, Sat: Satellite, Syn: Synthetic.

Attribute	Sat	Syn A	Real A	Syn G	Real G
Modality	Ortho-Rectified	RGB	RGB	RGB	RGB
#Images per site	120	1080	120	50–250	120
Altitude (AGL)	1–2 km	200–800 m	-	5–80 m	-

Table 2: Sky2Ground dataset statistics. A: Aerial, G: Ground, Sat: Satellite, Syn: Synthetic. -: altitude for real is unknown, due to lack of ground-truth poses.

Figure 2: Overview of the Sky2Ground dataset. The middle trajectory illustrates camera poses from one of our collected sites. Dots indicate ground-truth camera positions for synthetic images, while red frustums represent the estimated camera poses for real images. The surrounding images showcase example satellite, aerial, and ground views—where the real images demonstrate more diverse illumination conditions and realistic noise. The top-right map depicts the geographic distribution of all collected landmarks, highlighting the dataset’s global coverage.

Figure 3: Benchmark splits and modality setups. (a) Image counts per split for synthetic CR - Core, D1 - Dense 1, D2 - Dense 2, D3 - Dense3 and D4 - Dense 4, across ground, aerial, and satellite views. (b) View combinations used in each benchmark setup: Ground (G), Ground+Aerial (GA), Ground+Satellite (GS), and Ground+Aerial+Satellite (GAS).

Figure 4: Comparison of RRA@5 and RTA@5 metrics for four methods (Dust3r, Mast3r, Map Anything, and VGGT).

Figure 5: Comparison of models across view combinations.

Figure 6: Comparison of reconstruction quality across ground/aerial/satellite. : We report PSNR

\uparrow

and DreamSim

\downarrow

(lower is better). All methods benefit from increased camera density. 2DGS consistently achieves the best perceptual quality.

		PSNR $\uparrow$			DreamSim $\downarrow$
Data	GS Type	Aerial	Ground	Satellite	Aerial	Ground	Satellite
Synthetic	2D	12.1	14.9	13.0	0.49	0.26	0.19
	3D	11.8	14.7	13.0	0.52	0.31	0.17
	3D MCMC	12.5	15.0	13.3	0.53	0.29	0.21
Syn+Real	2D	11.1	13.9	12.6	0.61	0.40	0.26
	3D	11.1	13.4	12.6	0.64	0.46	0.28

Table 3: Rendering results across synthetic and real-image setups. We report PSNR and DreamSim for each GS type (2DGS, 3DGS, 3DGS MCMC ) across aerial, street, and satellite viewpoints.

Figure 7: Rendering results across satellite, aerial, and ground viewpoints. Each row shows a different view, with the leftmost column showing the ground-truth (GT) image followed by model renderings (R). From D1 to D4, the input density increases, and real images are incorporated with D2, introducing additional noise and domain mismatch. In particular, the ground-view renderings in D2 + Real contain real-image features that do not exist in the synthetic input distribution, highlighting the challenge of mixing synthetic and real sources.

Method	Data	Ground		Satellite		Aerial		Avg
		RRA@5	RTA@5	RRA@5	RTA@5	RRA@5	RTA@5	RRA	RTA
Dust3r	ZS	28.5 $\pm$ 0.71	21.6 $\pm$ 0.53	90.0 $\pm$ 1.92	39.9 $\pm$ 0.84	58.3 $\pm$ 1.37	44.3 $\pm$ 1.18	58.9 $\pm$ 1.23	35.3 $\pm$ 0.79
Mast3r	ZS	36.4 $\pm$ 0.86	29.6 $\pm$ 0.77	50.0 $\pm$ 1.04	13.3 $\pm$ 0.28	73.2 $\pm$ 1.49	68.9 $\pm$ 1.26	53.2 $\pm$ 1.07	37.3 $\pm$ 0.91
VGGT	ZS	75.1 $\pm$ 1.83	60.9 $\pm$ 1.29	66.6 $\pm$ 1.42	0.0 $\pm$ 0.00	79.2 $\pm$ 1.67	72.6 $\pm$ 1.23	73.6 $\pm$ 1.38	44.5 $\pm$ 0.95
Dust3r	A-MD	25.5 $\pm$ 0.64	23.2 $\pm$ 0.71	83.3 $\pm$ 1.78	29.9 $\pm$ 0.82	83.6 $\pm$ 1.74	64.8 $\pm$ 1.35	64.1 $\pm$ 1.27	39.3 $\pm$ 0.92
Mast3r	A-MD	29.1 $\pm$ 0.73	24.1 $\pm$ 0.61	96.6 $\pm$ 2.31	26.6 $\pm$ 0.68	77.0 $\pm$ 1.59	94.5 $\pm$ 2.18	67.6 $\pm$ 1.33	48.4 $\pm$ 0.97

Table 4: Localization results across Ground+Aerial+Satellite setup. ZS = Zero-Shot, A-MD = Aerial-MegaDepth. RRA@5 / RTA@5 measure rotation / translation accuracy within 5° / 5 m. RRA Avg and RTA Avg denote mean performance across all domains.

Figure 8: SkyNet: An architecture for cross-view camera localization. SkyNet consists of two encoders, 1) a Sat-Encoder processes input satellite images, and a GAS-Encoder processes ground/aerial/satellite views. Our model first patchifies the input images into tokens by DINO, and appends camera tokens for camera prediction. GAS-encoder then alternates between self-attention and Masked-Satellite Attention. Camera/Depth heads collect satellite tokens from Sat-encoder, ground-aerial tokens from GAS-encoder and predict camera-poses and depth-maps respectively. 2) Masked-Satellite Attention restricts aerial/ground from interacting with satellite modality. 3) Progressive-sampling gradually samples far-away cameras as training proceeds.

Table 13

Figure 10: (Top row): synthetic satellite images generated from Google Earth Studio. (Bottom row) : corresponding real satellite images we collected.

Figure 11: (Top row): synthetic satellite images generated from Google Earth Studio. (Bottom row) : corresponding real satellite images we collected.

Figure 12: Qualitative results of SkyNet vs VGGT: Given one satellite image, and two ground views, VGGT fails to localize the pointmaps properly, for eg, the pyramid is formed at wrong location (marked in red) , in contrast, SkyNet correctly localizes the ground views (marked in blue) , even when the views are non-overlapping.

Method	Data	Ground			Satellite			Avg
		RRA	RTA	Avg	RRA	RTA	Avg	Overall Avg	Drop
Synth-Sat
VGGT [ 40 ]	Sky2Ground	34.5	40.2	37.4	83.3	33.3	58.3	47.8	–
SkyNet	Sky2Ground	71.3	64.9	68.1	88.0	36.3	62.2	65.1	–
Real-Sat
VGGT	Sky2Ground	30.1	37.8	33.9	62.4	25.6	44.0	38.9	$-18.6\%$

Table A.1 : Evaluation on real satellite-images, across Ground+Satellite setup RRA@5 / RTA@5 denote rotation and translation accuracy within 5° / 5 m. SkyNet has lower relative-drop in performance than the VGGT-baseline.

Figure 13: (Top row): synthetic satellite images generated from Google Earth Studio. (Bottom row) : corresponding real satellite images collected from aerial sources.

Figure 14: Ferry Building : Visualization of the ground-truth point cloud on our Sky2Ground dataset. The top row shows the reconstructed scene from a satellite view perspective, and the bottom row presents two aerial view renderings for structural details and geometry.

Figure 15: Trafalgar Square : Visualization of the ground-truth point clouds on our Sky2Ground dataset. The top row shows the reconstructed scene from a satellite view perspective, and the bottom row presents two aerial view renderings for structural details and geometry.

Figure 16: Charles Bridge : Visualization of the ground-truth point cloud on our Sky2Ground dataset. The top row shows the reconstructed scene from a satellite view perspective, and the bottom row presents two aerial view renderings for structural details and geometry.

Figure 17: Colosseum : Visualization of the ground-truth point cloud on our Sky2Ground dataset. The top row shows the reconstructed scene from a satellite view perspective, and the bottom row presents two aerial view renderings for structural details and geometry.

Figure 18: Piazza Navona :Visualization of the ground-truth point cloud on our Sky2Ground dataset. The top row shows the reconstructed scene from a satellite view perspective, and the bottom row presents two aerial view renderings for structural details and geometry.

Cross-View Image Matching

See all 17 papers

May 8, 2026

Seeing Across Skies and Streets: Feedforward 3D Reconstruction from Satellite, Drone, and Ground Images

Cross-view localization classically asks: where does this ground image lie on the satellite tile? Existing methods are typically limited to 3-DoF estimates -- an

(x,y)

position and a yaw angle -- because nadir satellite imagery provides no direct cues for roll, pitch, or altitude, forcing a reliance on planar-motion and zero-tilt assumptions. These assumptions break on real terrain with slopes, ramps, and tilted camera mounts. To overcome this, we introduce a single UAV image as an intermediate viewpoint: it reveals the 3D structure invisible from nadir, supplies the cues for roll, pitch, and altitude that the satellite alone cannot provide, and needs only spatial overlap with the ground camera -- no known relative pose is required. Building on this insight, we propose Cross3R, a flexible feed-forward model that ingests a satellite tile together with a UAV image, a ground image, or both, and, in a single forward pass, recovers a cross-view 3D point cloud, the 6-DoF poses of every input camera, and the on-tile

(x,y)

position and yaw of each perspective camera. For training and evaluation, we also construct CrossGeo, a 278K-image tri-view dataset spanning 85 scenes across every continent except Antarctica. On CrossGeo, Cross3R consistently outperforms feed-forward 3D baselines in point-cloud reconstruction, 6-DoF camera-pose estimation, and cross-view localization. On KITTI, it outperforms dedicated cross-view methods trained on KITTI on most metrics, despite having no KITTI training itself.

Qiwei Wang, Zhongyao Tuo, Xianghui Ze +1

ShanghaiTech University Shanghai, China · Nanjing University of Science and Technology Nanjing, China

Relative Pose Estimation Cross-View Geo-Localization 3D Reconstruction Camera Pose Estimation Feed-Forward 3D Reconstruction

Ask PDF

Aug 29, 2026

Ground-to-Satellite Localization in Unconstrained Image Collections for 3D Scene Reconstruction

Ground image localization with respect to satellite imagery is a key enabler for metrically-accurate, geo-localized 3D scene reconstruction from unconstrained image collections. Existing cross-view localization methods have strict requirements such as panoramic imagery or known initial locations, limiting their applicability for in-the-wild reconstruction settings. We propose a robust hierarchical cross-view localization framework that leverages geometric constraints from Structure-from-Motion (SfM) models derived from unconstrained ground image collections. Our method generates coarse-to-fine pose hypotheses through a cross-view matching approach and aggregates noisy predictions across SfM model(s) using Kernel Density Estimation to recover consensus alignments while filtering outliers. Experiments demonstrate reliable localization performance from challenging image collections. Empirically we found satellite-referenced alignment enables accurate metric scale estimation, doppelgänger detection, and merging of disjoint SfM reconstructions, resulting in more complete, geo-localized site models than are possible with SfM alone.

Angel Daruna, Ben Southall, Niluthpol Chowdhury Mithun +6

Center for Vision Technologies, SRI International, Princeton, NJ, USA

Cross-View Geo-Localization 3D Reconstruction

Ask PDF

Mar 30, 2026

TerraSky3D: Multi-View Reconstructions of European Landmarks in 4K

Despite the growing need for data of more and more sophisticated 3D reconstruction pipelines, we can still observe a scarcity of suitable public datasets. Existing 3D datasets are either low resolution, limited to a small amount of scenes, based on images of varying quality because retrieved from the internet, or limited to specific capturing scenarios. Motivated by this lack of suitable 3D datasets, we captured TerraSky3D, a high-resolution large-scale 3D reconstruction dataset comprising 50,000 images divided into 150 ground, aerial, and mixed scenes. The dataset focuses on European landmarks and comes with curated calibration data, camera poses, and depth maps. TerraSky3D tries to answer the need for challenging dataset that can be used to train and evaluate 3D reconstruction-related pipelines.

Mattia D'Urso, Yuxi Hu, Christian Sormann +2

Graz University of Technology · Sony

3D Reconstruction Multi-View 3D Reconstruction Large-Scale 3D Reconstruction

Ask PDF

Abstract

Cross-View Image Matching

Explore similar work

Seeing Across Skies and Streets: Feedforward 3D Reconstruction from Satellite, Drone, and Ground Images

Ground-to-Satellite Localization in Unconstrained Image Collections for 3D Scene Reconstruction

TerraSky3D: Multi-View Reconstructions of European Landmarks in 4K