GroundControl: Satellite-Conditioned Multi-View Diffusion for Reconstruction-Free Novel View Synthesis

Akshay Krishnan; Matias Turkulainen; Jamie Wynn; Zawar Qureshi; Mohamed Sayed; Guillermo Garcia-Hernando; Sara Vicente; James Hays

cs.CVOct 27, 2026

GroundControl: Satellite-Conditioned Multi-View Diffusion for Reconstruction-Free Novel View Synthesis

Authors: Akshay Krishnan, Matias Turkulainen, Jamie Wynn, Zawar Qureshi, Mohamed Sayed, Guillermo Garcia-Hernando, Sara Vicente, James Hays,

Organizations: Niantic Spatial · Georgia Institute of Technology · Aalto University

Abstract

Multi-view or video diffusion models have recently emerged as state-of-the-art approaches for novel-view synthesis from sparse image collections. While they can produce high quality images, they tend to hallucinate unrealistic spatial layouts when synthesizing views outside the input views (scene extrapolation). We propose to address this challenge for outdoor scenes by using satellite images as an additional information source to condition the multi-view generative model. We introduce a new satellite-conditioned multi-view diffusion model that combines conditioning from a satellite image and street-level input images when generating novel-views. Our cross-attention based conditioning allows us to retain pretrained multi-view diffusion priors to generate seamless, realistic, novel views from a handful of source views, while also adhering to the scene layout as seen from satellite images. Our model does not use 3D scene reconstruction, and only relies on posed street and satellite views. Our experiments on outdoor datasets show that our model is better than both satellite-conditioned or street-view conditioned baselines on novel-view synthesis, especially on challenging cases where target views have very little overlap with source views. Project page: https://nianticspatial.github.io/groundcontrol

Figures & tables

Figure 1: GroundControl builds on recent novel-view synthesis (NVS) diffusion models, and enables more accurate synthesis of novel views by additionally conditioning on a top-down satellite image. When compared to the NVS diffusion model SEVA [ 3 ] (left), our model has better fidelity to the scene’s spatial layout.

	Street cond.	Ortho sat.	MV cons.	Public code	Intermediate representation	Output modality
[ 71 ]	N	N	N	N	Mesh	Panorama
[ 72 ]	N	N	Y	N	Mesh	Pose-controlled
[ 19 ]	N	N	Y	N	Splat	Pose-controlled
[ 37 ]	N	N	N	N	Voxels	Panorama
[ 59 ]	N	Y	N	N	Warp	Panorama
[ 50 ]	N	Y	N	Y	Volume	Panorama

Table 1: Compared with other related works, GroundControl enables street-level conditioning and orthorectified satellite views. We control views via pose conditioning, and outputs are multi-view consistent. Abbreviations: Street cond. = street-level conditioning; Ortho sat. = orthorectified satellite; MV cons. = multi-view consistency. * [ 11 ] accepts map input but not satellite images;

\dagger

we will release code and models upon acceptance.

Figure 2: GroundControl incorporates satellite images as an efficient conditioning signal for pose-controllable multi-view diffusion models. (a) Our inputs include a DINOv3 encoded satellite image, known ground-view image(s) $\mathcal{X}^{\text{inp}}$ with associated camera parameters $\mathcal{C}^{\text{inp}}$ , and target camera parameters $\mathcal{C}^{\text{tgt}}$ corresponding to the views to be synthesized. GroundControl then generates output images $\mathcal{X}^{\text{tgt}}$ for each target camera. (b) A key contribution is the integration of satellite information into the SEVA architecture [ 3 ] . (c) Our ray-guided attention sub-block fuses satellite features with noisy latent embeddings via multi-head cross-attention.

Figure 3: Our ray-guided cross-attention attends to tokens of the satellite image that are along the 3D ray of the street-view token.

	nuScenes [ 6 ]							Metropolis [ 44 ]
	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DINO $\downarrow$	SEG $\uparrow$	IoU road $\uparrow$	IoU bdg $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DINO $\downarrow$	SEG $\uparrow$	IoU road $\uparrow$	IoU bdg $\uparrow$
Street-View only
SEVA	11.334	0.308	0.616	6.140	0.492	0.425	0.182	10.884	0.315	0.586	5.654	0.551	0.581	0.291
Satellite only
Sat2Density [ 50 ]	10.679	0.301	0.714	6.657	0.507	–	–	9.074	0.346	0.743	6.775	0.537	–	–
Street-view + Satellite

Table 2: Comparison with baseline methods on test datasets that match our training domain , i.e . driving datasets. Our model improves over the baseline systems. Semantic IoU includes vehicles in both the road and building masks.

	DL3DV [ 41 ]
	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DINO $\downarrow$	SEG $\uparrow$	TSED $\downarrow$	%Consistent $\uparrow$	#Corrs. $\uparrow$
Sat2Density [ 50 ]	10.204	0.232	0.768	7.116	0.451	N/A	0.0	0
ViewCrafter* [ 77 ]	13.350	0.260	0.547	5.230	0.541	3845.57	2.02	332.62
SEVA [ 3 ]	14.912	0.360	0.407	3.534	0.674	22.75	62.99	287
Ours	15.170	0.372	0.414	3.653	0.674	18.35	65.56	290

Table 3: Comparison with baseline methods on DL3DV, a dataset which is out of distribution with respect to our training data , as it has large-viewpoint NVS with sparse inputs. We use the geolocalized dataset split from [ 64 ] . We use a total of 39 sequences with

3

input views and

37

test views per sequence. *ViewCrafter is incompatible with our input-target view split since it only generates a continuous trajectory, and therefore uses a different continuous input-target configuration.

Panel of Figure 4 — Figure 4: Qualitative results on in-distribution dataset: Metropolis Top 4 rows show satellite images along with corresponding input and target views. Final two rows show synthesis of target views using GroundControl and SEVA. GroundControl is significantly better than SEVA at adhering to the layout of the scene.

Panel of Figure 5 — Figure 5: Qualitative results on in-distribution datasets: Metropolis and nuScenes. Each row shows, from left: The satellite view $S$ ; an example ground-level ‘seed’ input view $\mathcal{X}^{\text{inp}}$ ; a ground truth target view $\mathcal{X}^{\text{tgt}}$ ; then predictions from GroundControl, [ 3 ] and [ 50 ] . Satellite images have input camera frustums plotted in red , and the target frustum in green . $\dagger$ We crop Sat2Density results to make pinhole images from their panoramic outputs; see supplementary for their full panoramic output.

Panel of Figure 6 — Figure 6: Qualitative results on the DL3DV benchmark: Despite being far from the distribution of street scenes, we find that GroundControl learns to leverage the satellite image when generating regions without sufficient overlap from sparse input views ( e.g . buildings in the background). $\dagger$ We resample Sat2Density panoramas to make pinhole images; see supplementary for their full panoramic output. Red triangles in satellite view are inputs, green is the shown target view.

	PSNR $\uparrow$	DINO $\downarrow$	SEG $\uparrow$
Satellite view as input view	11.312	5.862	0.558
W/o ray guided cross-attention (RGCA)	11.675	5.583	0.573
W/o Ray and timestep embed. in RGCA query	11.372	5.309	0.579
UNet decoder conditioning	11.291	5.400	0.572
W/o separate CFG	12.706	5.144	0.608
Ours	12.809	5.247	0.605

Table 4: Ablation study on the Metropolis dataset.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Figure I: PSNR performance vs. image overlap measured by the number of correspondences on Metropolis dataset . GroundControl obtains better performance compared to the SEVA baseline across all overlap regimes.

	nuScenes				DL3DV
offset (m,°)	PSNR $\uparrow$	SSIM $\uparrow$	DINO $\downarrow$	Seg $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	DINO $\downarrow$	Seg $\uparrow$
SEVA	11.334	0.308	6.140	0.492	14.912	0.360	3.534	0.674
0,0	11.880	0.331	6.145	0.498	15.170	0.372	3.653	0.674
5,5	11.518	0.311	6.156	0.490	15.131	0.372	3.679	0.670
10,10	11.509	0.311	6.156	0.490	15.124	0.371	3.682	0.669

Appendix

Table I : Robustness to pose misalignment: Performance degradation is mild with alignment offsets.

Views	Model	Straight				Turns
Views	Model	PSNR $\uparrow$	LPIPS $\downarrow$	DINO $\downarrow$	Seg. $\uparrow$	PSNR $\uparrow$	LPIPS $\downarrow$	DINO $\downarrow$	Seg. $\uparrow$
1	RealCamI2V	11.57	0.508	4.482	0.627	10.28	0.599	5.672	0.557
	SEVA	12.35	0.505	4.718	0.611	10.48	0.616	5.968	0.523
	Ours	12.73	0.518	4.687	0.623	11.12	0.612	5.787	0.548
2	Veo 3.1	14.81	0.395	3.245	0.707	11.99	0.538	5.068	0.574
	SEVA	12.99	0.472	4.427	0.633	11.29	0.545	5.196	0.571

Appendix

Table II : Comparison to image-to-video diffusion models on the Metropolis dataset for a video generation task (continuous trajectories). We divide our test set into straight car trajectories vs turns. Our model is slightly better than RealCamI2V in both cases, while being comparable to Veo on turns, with better semantic-feature metrics.

Dataset	p100	p75	p50	p25	p10	p0
nuScenes	2.714	0.850	0.527	0.221	-0.095	-0.817
Metropolis	3.444	2.396	1.962	1.449	0.941	0.258

Appendix

Table III : Distribution (percentile values) of per-scene

(\text{PSNR}_{\text{ours}}-\text{PSNR}_{\text{SEVA}})

. These show that satellite conditioning improves generations in most cases on nuScenes and Metropolis.

	PSNR $\uparrow$	DINO $\downarrow$	SEG $\uparrow$
Flipped satellite	12.017	5.782	0.513
Mismatched satellite	12.052	5.591	0.530
Ours	12.809	5.247	0.605

Appendix

Table IV: Impact of corrupted satellite imagery: using flipped satellite images or mismatched satellite images degrades performance.

Figure II: Multi-view generations on the Metropolis dataset: These are multi-frame generations from a single diffusion process with 7 input views (shown in red on the satellite images). We show target views from 5 consecutive timesteps ( green in the satellite images). Our model’s generations are multi-view consistent, and adheres to the spatial layout information in the satellite image.

Panel of Figure 17 — Figure III: Qualitative results on the Metropolis dataset. We show results of our satellite-conditioned model on sequences of length 21 frames. This includes 7 input frames (from the front camera) and 14 target frames (from the left and right cameras at the corresponding timestamps). The SEVA model without satellite conditioning underperforms in this sparse input-view regime which has little overlap with the target views.

Panel of Figure 18 — Figure IV: Qualitative results on nuScenes dataset These show cases with very little overlap between input views ( red in satellite view) and the target views ( green in satellite view). Our model’s generations adhere more closely to the layout of the scene, since these are conditioned on the satellite view.

Panel of Figure 19 — Figure V: Qualitative results with multiple noise samples : We compare variations in the generations from SEVA to that of our model, when starting the denoising process with different noise samples. In regions that are non-overlapping with the input view, SEVA’s generations vary more, and are inconsistent with the scene layout. Since our generations are conditioned on satellite imagery, they show less variance in the generated layout, while still generating diverse samples.

Figure VI: Varying satellite CFG on a scene from Metropolis Increasing the satellite CFG strengthens the adherence to the satellite image, but higher values trade off against image quality. The input views on the top row correspond to the red frustums in the satellite image, and the predicted views shown in the second row correspond to the green frustum.

Panel of Figure 21 — Figure VII: Qualitative ablation results on NuScenes. Here we compare three alternatives: a model with no satellite input (No sat)̇; a model with satellite input but no ray-based cross-attention (no ray CA), and finally our model. Columns on the left show the satellite view (S) with the input views in red and target view in green , a representative input view (input) and the ground truth (GT) corresponding to this target view. We note that our model tends to have better fidelity to the ground truth layout.

Panel of Figure 22 — Figure VIII: Visualization of Sat2Density results on nuScenes, complementing Fig. 5 . The predicted panorama is 128 $\times$ 512. Each row shows (left) the panorama prediction and below it a highlight showing the region used to generate the cropped image, and (right) the ground-truth target image.

Panel of Figure 23 — Figure IX: Visualization of Satellite attention. Top row: input images; second row: satellite features, satellite ray samples aligned with target image, target image and generated image; third row: attention maps according to the Z value along the ray.

Figure X: Top row: some input views from our 3-view DL3DV split. Bottom row: a test view, showing the ground truth, ours, and LongLRM.

Figure XI: Veo 3.1 qualitative results on the Metropolis dataset with 3 input frames. We generate 2 separate videos with Veo to use the 3 conditioning frames, but GroundControl generates the target views in a single pass. Veo produces highly photorealistic videos, since its model size and training data are larger by orders of magnitude. Unlike our model, it can also handle dynamic objects (including the hood of the car the camera is mounted on). However, its camera trajectory may not necessarily follow the required target poses. It can also frequently hallucinate scene content in incorrect locations (as highlighted in the image).

Dataset	MV	Pano	#Scenes	Contrib	Satellite source
Argoverse 2 [ 67 ]	Yes	No	700	10%	Google + ArcGIS
nuScenes [ 6 ]	Yes	No	700	13%	ArcGIS
Metropolis [ 44 ]	Yes	No	75	7%	Original dataset
VIGOR [ 84 ]	Yes*	Yes	52105 $\dagger$	70%	Azure Maps

Appendix

Table V : Training datasets. * We pair nearby panoramas to create a multiview version of the dataset.

\dagger

Unlike other datasets, the number of scenes in VIGOR corresponds to the number of panoramas. MV = is multi-view? Pano = is panorama? Contrib = training contribution. We train jointly on all datasets.

Novel View Synthesis

Novel View Synthesis creates images of a scene from camera viewpoints not present in the input, using learned geometry and appearance models.

See all 292 papers

VGGT-Diff: Visual Geometry Meets Diffusion for Sparse-View Novel View Synthesis
Sep 27, 2026Kangjie Chen, Xiangyu Li, Dongbin Zhang +12Novel View Synthesis Conditional Diffusion Models
XPeng Motors · The Chinese University of Hong Kong · Tsinghua University
Cross-View Splatter: Feed-Forward View Synthesis with Georeferenced Images
May 19, 2026Matias Turkulainen, Akshay Krishnan, Filippo Aleotti +63D Gaussian Splatting Novel View Synthesis
Aalto University · Niantic Spatial · Georgia Tech +3
BetterScene: 3D Scene Synthesis with Representation-Aligned Generative Model
Feb 26, 2026Yuci Han, Charles Toth, John E. Anderson +2Novel View Synthesis Multi-View Image Synthesis
Dept. of Electrical and Computer Engineering, The Ohio State University · USACE ERDC GRL

Sep 27, 2026J/K move · Enter open · S save

VGGT-Diff: Visual Geometry Meets Diffusion for Sparse-View Novel View Synthesis

Kangjie Chen, Xiangyu Li, Dongbin Zhang, Chaoda Zheng+11

XPeng Motors · The Chinese University of Hong Kong · Tsinghua University

Novel View Synthesis Conditional Diffusion Models Multi-View Consistency

Open paper AskPDF arXiv

We present VGGT-Diff, a geometry-routed multi-view diffusion model for sparse-view novel view synthesis. Existing novel view synthesis (NVS) methods face a fundamental trade-off: reconstruction-based approaches preserve observed geometry but struggle to synthesize unseen regions, while diffusion-based methods provide strong generative priors yet rely on implicit source-to-query correspondence. VGGT-Diff bridges these regimes by routing visual geometry latents from VGGT-Ω into a pretrained video diffusion model. Each visual token is associated with a 3D point and confidence, then transformed into query-aligned latent conditions through a confidence-aware Visual Geometry Router (VGR) that preserves front and back surface evidence. These conditions guide joint target-view denoising, while Point-Track Residual Consistency (PTRC) regularizes predicted-clean residuals along reliable 3D tracks, improving multi-view stability. We further introduce robust geometry conditioning, combining training-time regularization with inference-time guidance for improved robustness. Experiments show competitive or state-of-the-art performance across interpolation and extrapolation under different viewpoint difficulties. Our code is available at https://github.com/chenkangjie1123/VGGT-Diff.

Abstract

Novel View Synthesis

Explore similar work

VGGT-Diff: Visual Geometry Meets Diffusion for Sparse-View Novel View Synthesis

Cross-View Splatter: Feed-Forward View Synthesis with Georeferenced Images

BetterScene: 3D Scene Synthesis with Representation-Aligned Generative Model