Generating editable 3D urban scenes from a single image requires detailed building assets with a coherent spatial arrangement. Occlusion and perspective ambiguity make it difficult to infer complete building geometry together with consistent relative scales and positions. We present CityLift, a framework that generates individual textured building meshes from an urban image by coupling scene-level spatial modeling with building-level synthesis. CityLift first learns building shape and placement jointly through coupled scene and instance streams, producing coarse sparse structures in a common coordinate system. We then introduce generative spatial inheritance, which maps each building's coarse occupancy and scene-conditioned 3D feature field into a local volume to guide a flow model that generates finer structure. The same coordinate mapping places the generated meshes back into the scene, preserving their spatial relationship to the scaffold. To address the scarcity of urban data pairing scene images with individual 3D buildings, we construct CityLift-5K, comprising 4,997 synthetic scenes with instance masks, building meshes, and placement annotations. Experiments on CityLift-5K and scene blocks from UrbanScene3D and InstanceBuilding demonstrate improved building geometry and scene composition. The project page is available at https://ranhaogong.github.io/CityLift/.
Figures & tables
Figure 2: Overview of CityLift. Given a single urban image and building instance masks, CityLift generates individually editable building meshes in a shared scene frame. (a) A shared scene–instance generator predicts coarse occupancy in a common scene frame, implicitly modeling building placement, scale, and geometry. (b) A scaffold-derived coordinate map Ti deterministically resamples occupancy and contextual features Fi into each building’s local volume, forming the aligned condition Ai . (c) Conditioned on Ai and building image features ei , a local flow model regenerates finer sparse structures from noise. These structures guide mesh generation, and the resulting buildings are mapped back into the scene using the inherited transforms.
Building level
Scene level
Runtime ↓ (s/scene)
Method
CD ↓
Acc ↓
Comp ↓
F@1 ↑
F@2 ↑
CD ↓
Acc ↓
Comp ↓
F@1 ↑
F@2 ↑
TRELLIS.2
—
—
—
—
—
3.327
3.274
3.380
0.318
0.517
39.2
SAM 3D
2.761
2.565
2.956
0.397
0.595
2.595
2.501
2.689
0.387
0.597
83.7
Pixal3D
2.547
2.206
2.889
0.396
0.603
2.617
2.306
2.928
0.380
0.592
1482.4
3D-Fixer
2.701
2.533
2.870
0.403
0.602
2.832
2.853
2.811
0.385
0.590
310.3
SceneGen
9.352
10.092
8.612
0.126
0.223
4.973
4.795
5.151
0.214
0.368
231.9
Table 1: Geometric evaluation on CityLift-5K. An em dash denotes unavailable building-level output.
Figure 3: Qualitative comparison on blocks extracted from CityLift-5K testset. We slightly adjust the viewing angle to better showcase the generated scenes.
Building level
Scene level
Method
CD ↓
Acc ↓
Comp ↓
F@1 ↑
F@2 ↑
CD ↓
Acc ↓
Comp ↓
F@1 ↑
F@2 ↑
TRELLIS.2
—
—
—
—
—
2.809
2.766
2.851
0.401
0.616
SAM 3D
1.789
1.794
1.785
0.576
0.776
2.112
2.112
2.111
0.466
0.685
Pixal3D
1.490
1.303
1.678
0.600
0.818
2.067
1.666
2.469
0.473
0.693
3D-Fixer
1.797
1.497
2.098
0.489
0.708
2.507
1.973
3.042
0.422
0.641
SceneGen
13.376
13.634
13.118
0.083
0.158
8.017
5.566
10.467
0.147
0.248
Table 2: External evaluation on UrbanScene3D. Quantitative comparison on blocks extracted from a synthetic urban scene, assessing the geometric accuracy of individual buildings and assembled scenes.
Building level
Scene level
Method
CD ↓
Acc ↓
Comp ↓
F@1 ↑
F@2 ↑
CD ↓
Acc ↓
Comp ↓
F@1 ↑
F@2 ↑
TRELLIS.2
—
—
—
—
—
3.667
3.324
4.009
0.200
0.385
SAM 3D
1.995
2.421
1.570
0.366
0.620
2.418
2.887
1.949
0.351
0.594
Pixal3D
1.768
1.720
1.816
0.428
0.711
2.012
2.116
1.909
0.422
0.694
3D-Fixer
1.983
2.186
1.780
0.369
0.637
2.192
2.403
1.982
0.387
0.648
SceneGen
10.321
10.835
9.807
0.098
0.195
5.801
6.034
5.567
0.178
0.325
Table 3: External evaluation on InstanceBuilding. Quantitative comparison on scene blocks from real-world reconstructed meshes, assessing the geometric accuracy of individual buildings and assembled scenes.
Figure 4: Qualitative comparison on blocks extracted from UrbanScene3D and InstanceBuilding. We slightly adjust the viewing angle to better showcase the generated scenes.
Method
L
G
A
M
Overall
TRELLIS.2
84.2
75.2
71.7
74.8
77.6
SAM 3D
80.8
67.9
69.2
68.3
72.8
Pixal3D
79.6
64.2
63.8
64.6
69.5
3D-Fixer
80.8
65.8
58.5
65.6
69.6
SceneGen
44.0
55.8
53.1
54.8
51.0
Ours
87.3
81.2
77.9
81.7
82.8
Table 4: MLLM-as-a-judge visual evaluation on UrbanScene3D. ChatGPT 6 Astra scores the input image and four rendered views of each output. Results are averaged over all blocks shared by all six methods. All values are on a 0–100 scale, with higher values better. L, G, A, and M denote layout and completeness, shape and proportion, appearance and detail, and multi-view structural quality.
Building
Scene
Method
CD ↓
F@2 ↑
CD ↓
F@2 ↑
Coarse decoding
1.15
0.85
1.22
0.84
w/o scene guidance
3.22
0.35
2.59
0.46
w/o aligned injection
2.51
0.47
2.41
0.50
w/o hidden-feature injection
2.21
0.56
2.17
0.58
w/o occupancy injection
1.18
0.85
1.26
0.83
Table 5: Quantitative results of the ablation study on CityLift-5K.
Figure 5: Qualitative results of the ablation study.
Synthesizing large-scale, explorable, and geometrically accurate 3D urban scenes is a challenging yet valuable task for immersive and embodied applications. The challenge lies in the lack of large-scale and high-quality real-world 3D scans for training generalizable generative models. In this paper, we take an alternative route to create large-scale 3D scenes by leveraging readily available satellite imagery for realistic coarse geometry and open-domain diffusion models for high-quality close-up appearance synthesis. We propose Skyfall-GS, a novel hybrid framework that synthesizes immersive city-block scale 3D urban scenes by combining satellite reconstruction with diffusion refinement, eliminating the need for costly 3D annotations, and also featuring real-time, immersive 3D exploration. We tailor a curriculum-driven iterative refinement strategy to progressively enhance geometric completeness and photorealistic texture. Extensive experiments demonstrate that Skyfall-GS provides improved cross-view consistent geometry and more realistic textures compared to state-of-the-art approaches. Project page: https://skyfall-gs.jayinnn.dev/
Jie-Ying Lee, Yi-Ruei Liu, Shr-Ruei Tsai +6
National Yang Ming Chiao Tung University · UIUC · University of Zaragoza +1
We present SynCity 3000, a framework for generating 3D scenes that are globally coherent while enabling fine-grained layout control. Building on the ability of current image-to-3D generators to produce complex 3D assets from a single image, we extend this capability to the scale of entire scenes by adapting the generator to be applicable as a convolutional operator. We achieve this by fine-tuning the model on scene-like data generated by a new synthetic data engine, which we propose to address the scarcity of 3D scene data for training. The convolutional generator is then applied to a dimetric image of the entire scene, generated from the user prompt, resulting in 3D scenes of arbitrary size and complexity. Across diverse prompts and layouts, SynCity 3000 produces large, coherent, and detailed scenes, addressing the shortcomings of prior approaches to 3D scene generation.
Recent advances in single image-to-3D generation have enabled high-quality asset synthesis, yet extending these capabilities to indoor scene generation remains challenging. Existing methods focus on asset-level generation while neglecting the structural layout, which is essential for downstream applications and serves as the spatial anchor for grounding assets. However, a single image with a limited field of view lacks the spatial coverage to recover a coherent global layout. To this end, we use a 360° image represented in equirectangular projection (ERP) and propose InSpace, a structure-aware framework for 3D indoor scene generation. InSpace comprises three stages: (1) estimating partial scene geometry as spatial priors, (2) generating coarse scene structure with view-selective cross-attention, and (3) producing detailed layout and asset geometry with textures through a global-local hybrid attention, using flow matching. We also propose ERP-FRONT, a paired ERP-Image-to-3D indoor scene dataset based on 3D-FRONT. Experiments show that InSpace generates complete 3D indoor scenes with structural layout, along with separate textured assets from a single ERP image, achieving strong performance across 3D and 2D metrics. Project Page: https://kookie12.github.io/InSpace-Project-Page/
Gwanhyeong Koo, Hyunsu Kim, Youngji Kim +5
1KAIST · *Work done during an internship at NAVER LABS. · 2NAVER LABS +1