CityPaint: A Free Lunch from 2D Diffusion for City-Scale 3D Texturing
Authors: Yuchen Zou, Lihuang Fang, Siyu Zhu, Xiao Hu
Organizations: Xi’an Jiaotong University Xi’an, China · Southern University of Science and Technology Shenzhen, China · International Digital Economy Academy Shenzhen, China · HKUST (Guangzhou) Guangzhou, China
Generating high-fidelity, city-scale 3D environments plays a crucial role in applications such as virtual reality, gaming, and urban digital twins. However, the scarcity of real-world city-scale 3D training data forces existing methods to either stitch independently generated assets or learn from synthetic urban corpora, often sacrificing realism and global consistency. In contrast, 2D image data and their corresponding generative models are abundant and encode rich visual priors. To bridge this gap, we propose CityPaint, a novel framework that leverages 2D diffusion models to generate globally consistent city-scale 3D textures without relying on city-scale 3D training data. Specifically, CityPaint integrates geometric priors with the generative power of 2D diffusion models, introducing a robust top-K camera baking strategy for iterative texture projection. To address the artifacts of 2D-to-3D projection, we further propose a decoupled refinement mechanism that separately optimizes terrain and building textures. Extensive experiments show that CityPaint produces globally coherent textures with fine local details and achieves favorable results against strong baselines in both visual quality and cross-view consistency.
Figures & tables
Figure 1. Capabilities of CityPaint. Given text prompts, our framework generates high-fidelity, globally consistent 3D city textures. By leveraging the rich visual priors of 2D diffusion models, CityPaint naturally supports diverse architectural styles, climates, and lighting conditions without relying on city-scale 3D training data. Rendered city scenes with varied styles, climates, lighting, and viewpoints.
Method
3D Training Free
3D Dataset Free
Asset Free
Text Control
CityDreamer ( Xie et al., 2024 )
×
×
✓
×
UrbanWorld ( Shang et al., 2024 )
×
×
✓
✓
GaussianCity ( Xie et al., 2025 )
×
×
✓
×
CitySculpt ( Yao et al., 2025b )
×
×
✓
×
MajutsuCity ( Huang et al., 2025b )
×
×
×
✓
CityPaint (Ours)
✓
✓
✓
✓
Table 1. Comparison of existing 3D city generation paradigms. Comparison of six methods across four training and control properties.
Figure 2. Overview of the CityPaint Pipeline (Stages 1 & 2). In Stage 1, we construct a 3D city mesh from OpenStreetMap (OSM) data and establish heightfield-aware camera trajectories for stable observation. In Stage 2, a conditional diffusion model iteratively synthesizes the scene guided by rendered depth and semantic maps. The generated views are accumulated into a global UV atlas using a face-wise top- K camera baking strategy to suppress artifacts from grazing angles. Pipeline showing city mesh construction, adaptive camera sampling, conditional generation, and top-K texture baking.
Figure 3. Texture Refinement (Stages 3 & 4). Top: tile-wise UV denoising restores terrain continuity and removes seams. Bottom: object-centric orbit rendering recovers fine-grained building facade details. Diagram showing terrain UV denoising and building facade refinement.
Figure 4. Visual comparison of reference distributions. Top: Google Earth imagery often suffers from 3D reconstruction artifacts, blurred textures, and compression defects. Bottom: Our proposed CityWeb benchmark provides a cleaner, high-fidelity, and photorealistic reference distribution. Blurred Google Earth examples above and sharper CityWeb photographs below.
Figure 5. Qualitative comparison of different city generation methods (zoom-in recommended for details). Grid comparing city scenes generated by baseline methods and CityPaint.
Method
Venue
CE ↓
DE ↓
Google Earth
CityWeb
NIQE ↓
BRISQUE ↓
FID ↓
KID ↓
FID ↓
KID ↓
Text2Tex ( Chen et al., 2023a )
ICCV’23
0.066
0.158
110.7
0.103
108.7
0.118
8.85
89.12
Paintit ( Youwang et al., 2024 )
CVPR’24
0.075
0.180
121.2
0.110
98.4
0.105
9.54
95.41
CityDreamer ( Xie et al., 2024 )
CVPR’24
0.060
0.147
97.38
0.096
145.7
0.130
8.51
86.70
GaussianCity ( Xie et al., 2025 )
CVPR’25
0.057
0.136
86.94
0.090
125.4
0.137
7.69
75.34
CitySculpt ( Yao et al., 2025b )
ACMMM’25
0.043
0.047
53.96
0.063
-
-
5.13
53.96
Table 2. Quantitative comparison on geometric consistency (CE, DE) and visual fidelity (FID, KID, NIQE, BRISQUE) using Google Earth and CityWeb. Comparison of seven methods on geometric consistency and visual quality metrics.
Model Variations
Geometry
CityWeb
Perceptual
CE ↓
DE ↓
FID ↓
KID ↓
NIQE ↓
BRISQUE ↓
w/o Height-aware Cam.
0.058
0.082
68.53
0.071
4.96
32.41
w/o Top- K Baking
0.042
0.061
75.42
0.083
5.12
38.65
w/o Terrain Refinement
0.038
0.053
60.15
0.065
4.51
28.92
w/o Building Refinement
0.039
0.056
56.78
0.060
4.18
26.54
CityPaint (Full)
0.035
0.051
42.84
0.044
3.28
17.15
Table 3. Ablation study of core components. Ablation results for the four core components of CityPaint.
Figure 6. Effect of top- K camera baking. Without it (left), facade textures are blurred and inconsistent; our method (right) produces sharper, coherent details. Blurred facades without top-K baking on the left and sharper results with it on the right.
Synthesizing large-scale, explorable, and geometrically accurate 3D urban scenes is a challenging yet valuable task for immersive and embodied applications. The challenge lies in the lack of large-scale and high-quality real-world 3D scans for training generalizable generative models. In this paper, we take an alternative route to create large-scale 3D scenes by leveraging readily available satellite imagery for realistic coarse geometry and open-domain diffusion models for high-quality close-up appearance synthesis. We propose Skyfall-GS, a novel hybrid framework that synthesizes immersive city-block scale 3D urban scenes by combining satellite reconstruction with diffusion refinement, eliminating the need for costly 3D annotations, and also featuring real-time, immersive 3D exploration. We tailor a curriculum-driven iterative refinement strategy to progressively enhance geometric completeness and photorealistic texture. Extensive experiments demonstrate that Skyfall-GS provides improved cross-view consistent geometry and more realistic textures compared to state-of-the-art approaches. Project page: https://skyfall-gs.jayinnn.dev/
Jie-Ying Lee, Yi-Ruei Liu, Shr-Ruei Tsai +6
National Yang Ming Chiao Tung University · UIUC · University of Zaragoza +1
We present SynCity 3000, a framework for generating 3D scenes that are globally coherent while enabling fine-grained layout control. Building on the ability of current image-to-3D generators to produce complex 3D assets from a single image, we extend this capability to the scale of entire scenes by adapting the generator to be applicable as a convolutional operator. We achieve this by fine-tuning the model on scene-like data generated by a new synthetic data engine, which we propose to address the scarcity of 3D scene data for training. The convolutional generator is then applied to a dimetric image of the entire scene, generated from the user prompt, resulting in 3D scenes of arbitrary size and complexity. Across diverse prompts and layouts, SynCity 3000 produces large, coherent, and detailed scenes, addressing the shortcomings of prior approaches to 3D scene generation.
Recent 3D generative models can synthesize high-quality geometry but often struggle to reproduce intricate textures from reference images, largely due to the scarcity of large-scale 3D training data with rich surface appearance. In contrast, visual generative models are trained on datasets several orders of magnitude larger and excel at modeling complex visual patterns. Motivated by this gap, we introduce Ink3D, a framework that bridges 3D generation with large-scale video generative models to synthesize extremely complex textures. Ink3D first reconstructs a white-mesh geometry using an off-the-shelf 3D generation model. It then employs OrbitPainter, a conditional video generative model, to produce dense orbit-scan videos capturing object appearance across viewpoints. To convert these views into coherent textures, we introduce TextureOptimizer, a neural baking module that integrates dense multi-view observations while mitigating geometry inconsistencies arising from video generation. By decoupling geometry and texture synthesis and leveraging large-scale pretrained video priors, Ink3D enables significantly richer and more faithful texture generation than prior approaches.
Yue Han, Chong Li, Zhening Liu +5
ZGCA & ZGCI · Zhejiang University · Microsoft Research +1