Envisioning global urban development with satellite imagery and generative AI

Kailai Sun; Yuebing Liang; Mingyi He; Yunhan Zheng; Alok Prakash; Shenhao Wang; Jinhua Zhao; Alex "Sandy'' Pentland

cs.CVMar 27, 2026

Envisioning global urban development with satellite imagery and generative AI

Authors: Kailai Sun, Yuebing Liang, Mingyi He, Yunhan Zheng, Alok Prakash, Shenhao Wang, Jinhua Zhao, Alex "Sandy'' Pentland

Organizations: Singapore–MIT Alliance for Research and Technology Centre (SMART), Singapore · Department of Urban Planning, Tsinghua University, Beijing, China · Department of Urban Studies and Planning, Massachusetts Institute of Technology, Cambridge, MA, USA · College of Urban and Environmental Sciences, Peking University, Beijing, China ·

Abstract

Urban development has been a defining force in human history, shaping cities for centuries. However, past studies mostly analyze such development as predictive tasks, failing to reflect its generative nature. Therefore, this study designs a multimodal generative AI framework to envision sustainable urban development at a global scale. By integrating prompts and geospatial controls, our framework can generate high-fidelity, diverse, and realistic urban satellite imagery across the 500 largest metropolitan areas worldwide. It enables users to specify urban development goals, creating new images that align with them while offering diverse scenarios whose appearance can be controlled with text prompts and geospatial constraints. It also facilitates urban redevelopment practices by learning from the surrounding environment. Beyond visual synthesis, we find that it encodes and interprets latent representations of urban form for global cross-city learning, successfully transferring styles of urban environments across a global spatial network. The latent representations can also enhance downstream prediction tasks such as carbon emission prediction. Further, human expert evaluation confirms that our generated urban images are comparable to real urban images. Overall, this study presents innovative approaches for accelerated urban planning and supports scenario-based planning processes for worldwide cities.

Figures & tables

Figure 1: Spatial heterogeneity of population density and building volume density for global 500 cities. a. City-level global population density. b. Distribution of population density across 500 cities, sorted in ascending order. c. City-level global building volume density. d. Distribution of building volume density across 500 cities, sorted in ascending order. In a and c , the city-level densities are divided into five qualitative levels, where warm (orange and red) and cool colors (green and slight green) refer to high and low densities, respectively. In b and d , each dot corresponds to a city and is color-coded by continent (Europe, Asia, North America, South America, Oceania, and Africa). Representative cities, including Boston, Stockholm, Kigali, Singapore, and Hong Kong, are highlighted for reference.

Figure 2: Multi-modal conditional counterfactual synthesis of urban satellite imagery. a . Generated images respond to counterfactual template texts. The first row shows three generated images from the density metric prompt, and the second row shows three generated images from the land use prompt. b . Generated images respond to free-form texts. c . Urban redevelopment with the image inpaint model. d . Generative fidelity, precision and diversity performance of our framework.

Figure 3: Cross-city learning through generative urban visualization. a . Generated satellite images across cities from the source city (e.g., Hong Kong, Lusaka). b . Performance variation across regions. c . Human evaluation of our framework by urban experts. 42 urban experts (from the United States, Singapore, China, etc.; from both academia and industry) were recruited to rate (1-10 scores) for 136 questions. The generated urban images are comparable to those of real urban images.

Figure 4: Our framework quantifies urban visuals in latent space. a . Our framework has learned a distinguishable visual representation for each city. other city’s results can be found in SI. b . Cross-city transfer via prompt modification from “Stockholm” to “Stockholm from Kigali” yields hybrid satellite imagery, with latent features (black points) located between Kigali and Stockholm clusters. c . Performance comparison of predicted versus true global fossil fuel carbon emission for ResNet and DINOv3, using original data (top) and generative augmentation (bottom).

Extended Data Fig. 1 Modeling diagram of our framework.

Extended Data Fig. 2 Multi-modal (text-image) conditional counterfactual synthesis of urban satellite imagery in Kigali and Stockholm cities. a . Generated images respond to counterfactual texts in Kigali City. b . Generated images respond to counterfactual texts in Stockholm City. The first row shows three generated images from the density metric prompt, and the second row shows three generated images from the land use prompt.

Extended Data Fig. 3 Multi-modal (text-image) conditional counterfactual synthesis of urban satellite imagery in Chicago and Tampa cities. a . Generated images respond to counterfactual texts in Chicago City. b . Generated images respond to counterfactual texts in Tampa City. The first row shows three generated images from the density metric prompt, and the second row shows three generated images from the land use prompt.

Extended Data Fig. 4. Examples of reader study. a . An example question for urban experts to evaluate generated urban image in the urban density dimension. b . An example question for urban experts to evaluate generated urban redevelopment image.

Figure S1 : Relationship between Density Metrics

Figure S2 : Generated urban imagery responds to prompts and constraints. a . Generated images respond to different environmental constraints and text b . Generated imagery is diverse under the same control. RPD: residential population density; RVC: residential volume per capita; BVD: building volume density; BCR: building coverage ratio; Land use ratios: residential, commercial, industrial, etc.

Models	Steps	Params	Fidelity		Diversity		Precision
			FID $\downarrow$	PSNR $\uparrow$	SSIM $\downarrow$	FSIM $\downarrow$	SSIM $\uparrow$	FSIM $\uparrow$	LPIPS $\downarrow$	MS-SSIM $\uparrow$
Baselines
Original SD (No finetuning)	0	1.4B	86.5	$10.56$ $\pm 0.9$	–	–	$0.0927$ $\pm 0.008$	$0.5556$ $\pm 0.024$	$0.7162$ $\pm 0.045$	$0.0978$ $\pm 0.007$
ControlNet	60k	1.4B	76.5	$10.94$ $\pm 2.2$	$0.1847$ $\pm 0.13$	$0.5802$ $\pm 0.06$	$0.1087$ $\pm 0.07$	$0.5603$ $\pm 0.04$	$0.5623$ $\pm 0.038$	$0.1179$ $\pm 0.044$
Few Steps
UCN+CLIP	20k	1.5B	72.5	$10.58$ $\pm 2.3$	$0.1665$ $\pm 0.08$	$0.5560$ $\pm 0.04$	$0.1235$ $\pm 0.06$	$0.5545$ $\pm 0.035$	$0.5808$ $\pm 0.041$	$0.1120$ $\pm 0.048$

Table S1: Quantitative assessment of image fidelity, diversity, and precision metrics

Experiment	BCR		BVD
	MAE $\downarrow$	$R^{2}\uparrow$	MAE $\downarrow$	$R^{2}\uparrow$
UrbanControlNet	171.525	0.619	6.186	0.871

Table S2 : Prediction performance of a pre-trained CNN on

D_{test}

and on synthetic SAIs generated using the models listed in the experiment column

Figure S3 : Text-constraint-conditional counterfactual synthesis of urban satellite imagery . a . Generated images respond to counterfactual density inputs. b . Generated images respond to free-form texts.

Figure S4 : Multi-source pseudo-Siamese network in UrbanControlNet.

Figure S5 : Numeral-aware network in UrbanControlNet.

Figure S6 : Model performance (R 2 ) distribution across more than 100 cities globally.

Model	$R^{2}(\log)\uparrow$	$R^{2}\uparrow$	MAE $\downarrow$	RMSE $\downarrow$
ResNet	0.830	0.610	22.04	60.82
ResNet+TradAug	0.837	0.630	21.74	57.72
ResNet+AutoAug [ 8 ]	0.842	0.626	21.45	58.53
ResNet+SARandAug [ 36 ]	0.834	0.634	21.69	57.79
ResNet+GA	0.879	0.696	18.42	52.86
Swin Transformer	0.874	0.690	19.01	53.73

Table S3 : Performance comparison of different models for global fossil fuel carbon emission prediction.

Image Generation

Image generation studies models that create or edit images from text, examples, layouts, and other structured controls.

See all 463 papers

Sat2RealCity: Geometry-Aware and Appearance-Controllable 3D Urban Generation from Satellite Imagery
Nov 14, 2025Xinliang Wang, Yijie Kang, Zhenyu Wu +13D Asset Generation 3D Scene Generation
KE Holdings Inc., Beijing, China
Designing streetscapes from street-view imagery using diffusion models
May 17, 2026Yuzhou Chen, Yuebing Liang, Lingqian Hu +4Image Generation Diffusion-Based Image Generation
Department of Urban and Regional Planning, University of Florida · Singapore-MIT Alliance for Research and Technology Centre (SMART) · Department of Landscape Architecture and Urban Planning, Texas A&M University +1
SENSE: Satellite-based ENergy Synthesis for Sustainable Environment
May 18, 2026Kailai Sun, Mingyi He, Heye Huang +5Conditional Image Generation Synthetic Data Generation
SMART, Singapore · Massachusetts Institute of Technology Cambridge, MA, USA · University of Florida Gainesville, Florida, USA

Nov 14, 2025J/K move · Enter open · S save

Sat2RealCity: Geometry-Aware and Appearance-Controllable 3D Urban Generation from Satellite Imagery

Xinliang Wang, Yijie Kang, Zhenyu Wu, Yifeng Shi

KE Holdings Inc., Beijing, China

3D Asset Generation 3D Scene Generation

Open paper AskPDF arXiv

3D urban generation from satellite imagery is an important task for scalable digital twins and real-world simulation environments. Existing approaches primarily rely on scene-level generation paradigms, which often require large-scale 3D city assets and struggle with controllability, geographic alignment, and realistic appearance grounding in real-world urban environments. To address these limitations, we present Sat2RealCity, a grounded urban generation framework that leverages object-level 3D generative priors for scalable city synthesis from satellite imagery. Our framework decomposes cities into geographically grounded building entities, enabling the reuse of pretrained object-level 3D generative priors while preserving real-world spatial structures. Supported by our constructed BuildVerse3D dataset, (1) we introduce an OpenStreetMap (OSM)-guided spatial grounding strategy to inject geospatial constraints into the 3D generation process; (2) we design an appearance-guided controllable generation mechanism for realistic architectural appearance and regional style consistency; and (3) we construct an MLLM-powered semantic pipeline for regional appearance understanding and semantic-aware appearance synthesis. Extensive experiments demonstrate that Sat2RealCity achieves strong geographic alignment, regional stylistic consistency, and plausible urban asset synthesis compared with existing urban generation and 3D asset generation approaches.

Abstract

Image Generation

Explore similar work

Sat2RealCity: Geometry-Aware and Appearance-Controllable 3D Urban Generation from Satellite Imagery

Designing streetscapes from street-view imagery using diffusion models

SENSE: Satellite-based ENergy Synthesis for Sustainable Environment