Urban Building Energy Modeling plays a critical role in achieving the United Nations' Sustainable Development Goals 7 and 11. Although existing studies based on satellite imagery and deep learning have achieved remarkable progress, many challenges exist: most existing studies are inherently predictive, failing to reflect the generative nature of urban planning; although generative AI and diffusion models have seen explosive growth in satellite imagery, they lack the urban functional generation (e.g., energy layer); third, aligned high-quality high-resolution building energy data with satellite imagery is limited and scarce. Here we propose SENSE (Satellite-based ENergy Synthesis for Sustainable Environment), a unified generative UBEM framework that jointly synthesizes realistic urban satellite imagery and aligned high-quality building energy consumption and height maps. By conditioning on road networks and urban density metrics, SENSE, based on a controllable diffusion model, leverages the knowledge learned by large vision models to generate urban building energy consumption and height information (annotations) in the latent space. Experiments across four cities (New York City, Boston, Lyon, Busan) demonstrate that SENSE achieves high visual fidelity and strong physical consistency, satisfying the ASHRAE standard metric. Experiments demonstrate that SENSE can generate enough annotated synthetic data using less than 20% labeled energy data, boosting downstream prediction performance by 10% IoU. Compared to SOTA urban energy prediction methods, SENSE significantly reduced prediction error (reduced 3%-11% NMBE and 1%-9% CVRMSE). This study offers an energy-efficiency urban planning and physical generation solution for urban science, energy science and building science. The dataset and code: https://huggingface.co/datasets/skl24/MUSE and https://github.com/kailaisun/GenAI4Urban-Energy/.
3D urban generation from satellite imagery is an important task for scalable digital twins and real-world simulation environments. Existing approaches primarily rely on scene-level generation paradigms, which often require large-scale 3D city assets and struggle with controllability, geographic alignment, and realistic appearance grounding in real-world urban environments. To address these limitations, we present Sat2RealCity, a grounded urban generation framework that leverages object-level 3D generative priors for scalable city synthesis from satellite imagery. Our framework decomposes cities into geographically grounded building entities, enabling the reuse of pretrained object-level 3D generative priors while preserving real-world spatial structures. Supported by our constructed BuildVerse3D dataset, (1) we introduce an OpenStreetMap (OSM)-guided spatial grounding strategy to inject geospatial constraints into the 3D generation process; (2) we design an appearance-guided controllable generation mechanism for realistic architectural appearance and regional style consistency; and (3) we construct an MLLM-powered semantic pipeline for regional appearance understanding and semantic-aware appearance synthesis. Extensive experiments demonstrate that Sat2RealCity achieves strong geographic alignment, regional stylistic consistency, and plausible urban asset synthesis compared with existing urban generation and 3D asset generation approaches.
Generating explicit textured 3D city assets from a single satellite image is important for urban simulation and digital twins. Most prior methods, however, learn 3D proxies optimized for rendering street-view images or videos over prescribed viewpoints and trajectories, rather than producing explicit 3D assets. Our previous framework, Sat2City, took a first step toward this goal with task-specific cascaded sparse-voxel latent diffusion conditioned on satellite-derived height maps. However, it relied on synthetic data and did not condition appearance on the satellite input. To address these limitations, we present Sat2City v2, a framework for generating explicit textured 3D city assets directly from real satellite images. We first construct a real-world dataset of 16,027 geographically matched but weakly aligned satellite-image-textured-mesh pairs, comprising 14,651 training pairs and 1,376 held-out test pairs. Building on this dataset, Sat2City v2 uses a native structured-latent 3D prior pretrained on curated assets to anchor geospatial adaptation to its asset manifold. We empirically demonstrate that weakly aligned satellite images and regional textured 3D meshes can support asset-level conditional generation: learned attention lets satellite tokens steer geometry generation without calibrated pixel-to-surface correspondence, while the generated geometry anchors satellite-guided material synthesis. Sat2City v2 ranks first among the evaluated baselines on every reported metric across geometry accuracy, generative mesh quality, and satellite-to-asset feature alignment.
Synthesizing large-scale, explorable, and geometrically accurate 3D urban scenes is a challenging yet valuable task for immersive and embodied applications. The challenge lies in the lack of large-scale and high-quality real-world 3D scans for training generalizable generative models. In this paper, we take an alternative route to create large-scale 3D scenes by leveraging readily available satellite imagery for realistic coarse geometry and open-domain diffusion models for high-quality close-up appearance synthesis. We propose Skyfall-GS, a novel hybrid framework that synthesizes immersive city-block scale 3D urban scenes by combining satellite reconstruction with diffusion refinement, eliminating the need for costly 3D annotations, and also featuring real-time, immersive 3D exploration. We tailor a curriculum-driven iterative refinement strategy to progressively enhance geometric completeness and photorealistic texture. Extensive experiments demonstrate that Skyfall-GS provides improved cross-view consistent geometry and more realistic textures compared to state-of-the-art approaches. Project page: https://skyfall-gs.jayinnn.dev/