CityCode: Building Agent-Operable 3D Cities from Sparse Aerial Observations

Remote-sensing systems usually describe urban content with detection boxes, semantic masks, or vector boundaries. Such outputs locate classes and support image-plane scoring, yet they do not by themselves constitute an executable layout that retains object identities, typed relations, topology, and regeneration rules. Code-as-City instead casts urban-layout extraction from a single top-down image as constrained code generation with a multimodal large language model (MLLM). An image model first produces an aligned five-class semantic layout prior. Three ordered MLLM passes use the image and this prior to recover roads, land-cover regions and relations, and buildings. Deterministic normalization converts the accumulated records into a city graph and a restricted layout program. Executing the program creates a renderable 3D city layout and an orthographic semantic projection over shared geometry. The projection admits pixel-level comparison with remote-sensing masks, while named objects, relations, and editing operations remain available for synchronized regeneration of both views. Evaluated on the 100 scenes of CityLayout-100, the complete framework obtains 41.1% mean intersection-over-union and 48.3% global intersection-over-union. This result provides quantitative evidence that visual observations can be translated into inspectable, editable city code with coupled planar and 3D outputs.

Reconstructing large-scale urban scenes from sparse aerial views is a crucial yet challenging task. Due to biased top-down and shallow-oblique camera poses, sparse aerial captures exhibit strong evidence imbalance: roofs and open regions are repeatedly observed, while facades, distant buildings, and occluded structures receive little multi-view support. Existing feed-forward 3D Gaussian Splatting methods directly regress a deterministic representation from sparse inputs, but this often leads to ghosting, melted facades, and stretched textures. Recent pseudo-view and video-based generative reconstruction methods use additional supervision or generative priors. However, they often lack a clear separation between observed geometry and prior-driven content, which can lead to plausible but inconsistent structures. We propose AnyCity, an observation-grounded generative reconstruction framework for sparse aerial urban scenes. AnyCity first predicts an observation-supported geometry latent to anchor reliable structures, and then uses scaffold-conditioned aerial completion tokens to predict a gated residual update for weakly constrained content before Gaussian decoding. During training, dense-to-sparse distillation transfers structural cues from dense-view reconstruction, while an aerial-adapted video diffusion prior provides fine-grained urban appearance cues through gated token conditioning. Observation-preserving objectives keep the refined representation consistent with input-supported geometry. At inference time, AnyCity reconstructs the final 3D Gaussian scene from sparse aerial views in a single feed-forward pass, achieving coherent urban novel-view synthesis with second-level inference. Experiments on synthetic, aerial-domain, UAV-textured, and real-world scenes show consistent improvements over feed-forward baselines.

Generating explicit textured 3D city assets from a single satellite image is important for urban simulation and digital twins. Most prior methods, however, learn 3D proxies optimized for rendering street-view images or videos over prescribed viewpoints and trajectories, rather than producing explicit 3D assets. Our previous framework, Sat2City, took a first step toward this goal with task-specific cascaded sparse-voxel latent diffusion conditioned on satellite-derived height maps. However, it relied on synthetic data and did not condition appearance on the satellite input. To address these limitations, we present Sat2City v2, a framework for generating explicit textured 3D city assets directly from real satellite images. We first construct a real-world dataset of 16,027 geographically matched but weakly aligned satellite-image-textured-mesh pairs, comprising 14,651 training pairs and 1,376 held-out test pairs. Building on this dataset, Sat2City v2 uses a native structured-latent 3D prior pretrained on curated assets to anchor geospatial adaptation to its asset manifold. We empirically demonstrate that weakly aligned satellite images and regional textured 3D meshes can support asset-level conditional generation: learned attention lets satellite tokens steer geometry generation without calibrated pixel-to-surface correspondence, while the generated geometry anchors satellite-guided material synthesis. Sat2City v2 ranks first among the evaluated baselines on every reported metric across geometry accuracy, generative mesh quality, and satellite-to-asset feature alignment.