Sat2RealCity: Geometry-Aware and Appearance-Controllable 3D Urban Generation from Satellite Imagery

Generating explicit textured 3D city assets from a single satellite image is important for urban simulation and digital twins. Most prior methods, however, learn 3D proxies optimized for rendering street-view images or videos over prescribed viewpoints and trajectories, rather than producing explicit 3D assets. Our previous framework, Sat2City, took a first step toward this goal with task-specific cascaded sparse-voxel latent diffusion conditioned on satellite-derived height maps. However, it relied on synthetic data and did not condition appearance on the satellite input. To address these limitations, we present Sat2City v2, a framework for generating explicit textured 3D city assets directly from real satellite images. We first construct a real-world dataset of 16,027 geographically matched but weakly aligned satellite-image-textured-mesh pairs, comprising 14,651 training pairs and 1,376 held-out test pairs. Building on this dataset, Sat2City v2 uses a native structured-latent 3D prior pretrained on curated assets to anchor geospatial adaptation to its asset manifold. We empirically demonstrate that weakly aligned satellite images and regional textured 3D meshes can support asset-level conditional generation: learned attention lets satellite tokens steer geometry generation without calibrated pixel-to-surface correspondence, while the generated geometry anchors satellite-guided material synthesis. Sat2City v2 ranks first among the evaluated baselines on every reported metric across geometry accuracy, generative mesh quality, and satellite-to-asset feature alignment.

Generating a street-level 3D scene from a single satellite image is a crucial yet challenging task. Current methods present a stark trade-off: geometry-colorization models achieve high geometric fidelity but are typically building-focused and lack semantic diversity. In contrast, proxy-based models use feed-forward image-to-3D frameworks to generate holistic scenes by jointly learning geometry and texture, a process that yields rich content but coarse and unstable geometry. We attribute these geometric failures to the extreme viewpoint gap and sparse, inconsistent supervision inherent in satellite-to-street data. We introduce Sat3DGen to address these fundamental challenges, which embodies a geometry-first methodology. This methodology enhances the feed-forward paradigm by integrating novel geometric constraints with a perspective-view training strategy, explicitly countering the primary sources of geometric error. This geometry-centric strategy yields a dramatic leap in both 3D accuracy and photorealism. For validation, we first constructed a new benchmark by pairing the VIGOR-OOD test set with high-resolution DSM data. On this benchmark, our method improves geometric RMSE from 6.76m to 5.20m. Crucially, this geometric leap also boosts photorealism, reducing the Fréchet Inception Distance (FID) from $\sim$40 to 19 against the leading method, Sat2Density++, despite using no extra tailored image-quality modules. We demonstrate the versatility of our high-quality 3D assets through diverse downstream applications, including semantic-map-to-3D synthesis, multi-camera video generation, large-scale meshing, and unsupervised single-image Digital Surface Model (DSM) estimation. The code has been released on https://github.com/qianmingduowan/Sat3DGen.

Synthesizing large-scale, explorable, and geometrically accurate 3D urban scenes is a challenging yet valuable task for immersive and embodied applications. The challenge lies in the lack of large-scale and high-quality real-world 3D scans for training generalizable generative models. In this paper, we take an alternative route to create large-scale 3D scenes by leveraging readily available satellite imagery for realistic coarse geometry and open-domain diffusion models for high-quality close-up appearance synthesis. We propose Skyfall-GS, a novel hybrid framework that synthesizes immersive city-block scale 3D urban scenes by combining satellite reconstruction with diffusion refinement, eliminating the need for costly 3D annotations, and also featuring real-time, immersive 3D exploration. We tailor a curriculum-driven iterative refinement strategy to progressively enhance geometric completeness and photorealistic texture. Extensive experiments demonstrate that Skyfall-GS provides improved cross-view consistent geometry and more realistic textures compared to state-of-the-art approaches. Project page: https://skyfall-gs.jayinnn.dev/