Greenscapes: Optimized Vegetation Configurations in Cities
Authors: Patryk Bartkowiak, Marcin Marlewski, Paweł Wójcik, Jae Joong Lee, Dominik L. Michels, Bedrich Benes, Sören Pirk, Wojciech Pałubicki
Organizations: Adam Mickiewicz University, Poznań, Poland · Purdue University, United States of America · King Abdullah University of Science and Technology (KAUST), Saudi Arabia · Christian-Albrechts-Universität zu Kiel, Germany
Vegetation plays a central role in cities, serving functional roles such as balancing shade and sky exposure, shaping the microclimate, carbon sequestration, and providing acoustic screening. Furthermore, aesthetic choices in landscape styles (formal, modernist, picturesque) are important for citizens' well-being and should be considered in the design of urban landscapes. In this paper, we present a novel method for placing vegetation, ranging from shrubs to large trees, in 3D urban scenes. Our method uses individual lots as input and employs a conditioned Vision-Language Model (VLM) to generate a top-down rendering of vegetation, placing large trees, small trees, and shrubs on lawn areas. This rendering is then converted into a 3D scene by detecting individual plants and selecting appropriate 3D assets. To refine the proposed configurations, we define an optimization loop that keeps the VLM-generated positions fixed after a deterministic feasibility repair and uses CMA-ES to optimize per-instance scale transformations along with discrete vegetation asset substitutions. To define the objective function for the optimization, we evaluate the generated 3D scenes with efficient surrogate models of shade and path-based acoustic exposure. Furthermore, we render in-scene views at each optimization step to compute view-dependent perceptual descriptors, enabling the generation of both aesthetic and functional vegetation layouts for urban scenes.
Figures & tables
Figure 1. Three complete park designs synthesized from identical Open Street Map site constraints but different style prompts. Left: formal design and regular spacing. Middle: picturesque -asymmetric composition and clustered planting. Right: modernist -clear geometry. All scenes are generated 3D layouts optimized under perceptual and functional objectives using a non-convex approach.
Figure 2. Overview of our pipeline for prompt-driven park vegetation design. Inputs are a user text prompt (a) and OSM-derived site context (b). An image-to-image constrained editor generates multiple top-down layout proposals (c), which are converted to a metric top-down representation (d) and processed by instance extraction (e) to obtain candidate pedestrian viewpoints (f) and vegetation instance positions and coarse types (g). These instances are projected into world coordinates and assembled into a 3D park scene (h). We evaluate candidate scenes by rendering multiple in-scene views to compute perceptual descriptors and biomass-related quantities (i) and by running physics-inspired field models for functional measures such as shade and sound (j). CMA-ES then refines continuous instance parameters and triggers discrete species substitutions under a Gaussian-mixture asset prior using the combined perceptual and functional objectives (k), yielding the final optimized park layout (l).
Figure 3. Feasibility and constraint adherence of the first stages of our pipeline. In (a), we show an example conditioning raster obtained from OSM data. In (b), a constrained-edit proposal of the VLM is depicted for that input raster. In (c), we show the VLM-generated bounding box annotation overlay.
Figure 4. Color-mapped visualization of the ear-height slice of the simulated 3D sound-energy voxel space. A sound source is placed near the upper-right corner (red marker), and a linear tree belt (green outline) separates the source from receiver samples (red markers) below. Increasing the per-vegetation attenuation parameter slows the diffusion update through the tree belt, reducing and delaying energy propagation toward the receivers; the three panels differ only in this attenuation coefficient, while keeping the geometry and discretization fixed.
Figure 5. Example park scenes before and after CMA-ES refinement of the VLM prior. Path shading is visualized as a scalar field over the walkable network (plasma colormap), and vegetation biomass is visualized per instance (viridis colormap). Optimization increases functional shading where needed while reallocating plant mass to meet biomass/coverage targets under the same geometric constraints; the images show the resulting redistribution of canopy placement along paths.
Figure 6. Sky-view exposure before and after CMA-ES refinement from a fixed in-park viewpoint. The fisheye sky segmentation (white = sky) shows increased open-sky visibility after optimization, raising the sky view factor (SVF) to 0.70 while preserving the surrounding layout constraints.
Figure 7. Skyline periodicity matching before (left) and after (right) CMA-ES refinement from a fixed viewpoint. The green curve is the target sinusoidal skyline profile; CMA-ES improves agreement by jointly adjusting per-instance scales and performing discrete species substitutions to introduce canopies with more suitable sizes and crown shapes, yielding a more periodic silhouette while maintaining feasibility constraints.
Figure 8. Visualization of the CMA-ES trajectory in the 6D attribute space after PCA projection to 2D. Elliptical contours indicate projected Gaussian components (species priors) used by the GMM-based substitution rule; the polyline shows successive CMA-ES proposals and the induced discrete component assignments.
Metric (Target)
Real Park (Ref) ∣Δ∣
VLM Prior ∣Δ∣
Optimized ∣Δ∣
SVF ( 0.75 )
0.12
0.06±0.04
0.03±0.01
Shade ( 2.5 )
0.32
0.80±0.20
0.28±0.07
Acoustics ( 3.5 )
5.76
0.50±0.14
0.21±0.06
Biomass ( 75k )
9,736
18,683±6,174
896±190
Skyline Fit ( 10k )
10,213
9,433±1,815
2,986±456
Total Cost
1.30
0.40±0.10
0.05±0.01
Table 1. Quantitative error analysis over N=30 experimental runs reports the mean absolute deviation from design targets ( μ±σ ) to demonstrate robustness. While the VLM Prior exhibits high variance, the Optimized result shows significantly lower standard deviation, indicating that our method reliably converges to the design goals regardless of initialization. Real Park is a fixed reference geometry.
Run
Shade
Biomass
Cost
Target
2.00
56,317
0.0000
Random Init.
1.65
54,473
0.2012
Random+CMA-ES
1.91±0.04
59,447±1,176
0.1205±0.0075
VLM Prior
1.70
56,317
0.0227
VLM+CMA-ES
2.04±0.06
57,581±780
0.0059±0.0030
Table 2. Ablation of the two components of our method: initialization (random vs. VLM-derived prior) and CMA-ES refinement (on vs. off)-reported on Shade, Biomass, and the optimization Cost , the weighted squared distance to the Shade and Biomass targets (lower is better). CMA-ES on the VLM prior achieves the lowest cost ( 0.006 ), reaching the shade target ( 2.04 vs. 2.00 ) while keeping biomass within ∼ 2.3% of baseline; CMA-ES on random initialization falls short of the shade target ( 1.91 ) and overshoots biomass by ∼5.5% , ending ∼20× further from the joint target.
Figure 9. City-scale vegetation placement examples generated from OSM-derived urban context. Panels show the area around (a) Sydney Park, Australia, (b) the Mausoleum Charlottenburg area in Berlin, Germany, and (c) the Sołacz district in Poznań, Poland, with over 4000 placed tree instances per scene. Panel (d) shows an in-scene rendering of a generated layout.
Figure 12. Objective-ablation results for Joyce Kilmer Park. Panel (a) shows the reconstructed reference layout used as the input condition. Panels (b–f) show CMA-ES refinements obtained by assigning one objective term a dominant weight while keeping the remaining terms at low or zero weights: (b) skyline sine-function fit, (c) SVF ratio, (d) minimal path shading, (e) acoustic/noise-reduction objective, and (f) maximum biomass. All runs use the same site boundary, path network, feasibility constraints, asset library, and optimizer settings; the visual differences therefore illustrate the qualitative effect of each dominant objective.
Figure 13. Examples of generated planting layouts for different site lots. For each lot geometry, our method produces distinct plant configurations, demonstrating its ability to adapt planting structure and spatial organization to varying parcel shapes and constraints.
Figure 14. We plot 33 reconstructed reference parks with known architectural styles and 30 generated parks produced from OSM sites and style prompts. Colors indicate the intended style ( formal blue, picturesque green, modernist red). Disks versus crosses distinguish generated from reconstructed references. The resulting clustering shows that the proposed entropy and linearity metrics separate the three strategies and that the generated parks align with the corresponding reference distributions.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
Value
VLM model/version
gemini-3-pro-image-preview
Layout-generation input
OSM-derived top-down conditioning image plus fixed filled prompt
Layout-generation output
Complete edited top-down image with vegetation added only on grass/editable pixels
Image resolution
1024×1024
Aspect ratio
1:1
Accepted candidates
One accepted generated image per prompt instance
Appendix
Table 1. VLM settings used for the reported experiments. “Prompt instance” denotes a fixed filled prompt, i.e., a template together with the corresponding arrangement rules.
Parameter
Setting
VLM model/version
gemini-3-pro-image-preview
VLM output resolution
1024×1024 , aspect ratio 1:1
Prompt protocol
Fixed prompt instances; no interactive prompt tuning during reported runs
Accepted VLM candidates
One accepted image per prompt instance; failed API calls retried up to five times; the image API exposes no random seed
Annotation labels
bigtree , smalltree , shrub
Maximum generated plant count requested in prompt
Approximately 300 plants for park prompts
Appendix
Table 2. Reproducibility table for the reported GreenScapes experiments. Values are fixed unless explicitly stated by a specific experiment. Per-experiment weights, targets, and step sizes are given in the released configuration files ( configs/optimizer/ ).
Main-paper panel
Configuration
Dominant term
Optimization direction
Remaining objective weights
Fig. 12(b)
Skyline sine-function criterion
Jsky
Minimize deviation from the target sinusoidal skyline profile
SVF, path shading, acoustic exposure, and biomass use wlow
Fig. 12(c)
SVF ratio
JSVF
Match the target sky-view factor / increase openness when the target requires higher SVF
Skyline, path shading, acoustic exposure, and biomass use wlow
Fig. 12(d)
Minimal path shading
Jshade
Minimize path-level shading according to the ablation target
SVF, skyline, acoustic exposure, and biomass use wlow
Fig. 12(e)
Acoustic objective
Jacoustic
Reduce acoustic exposure / increase excess attenuation at receiver viewpoints
SVF, skyline, path shading, and biomass use wlow
Fig. 12(f)
Maximum biomass
Jbio
Maximize normalized vegetation biomass under feasibility constraints
SVF, skyline, path shading, and acoustic exposure use wlow
Appendix
Table 3. Objective configurations used for the objective-ablation figure in the main paper (Fig. 12). Each ablation assigns one dominant weight w⋆ to the selected term and low or zero weights wlow to the remaining terms. The exact weight vectors and targets for each panel are provided in the released configuration files ( configs/optimizer/ ). This preserves weak regularization from the other objectives while visualizing the qualitative effect of the dominant criterion.
Figure 1. CMA-ES optimization trace for Joyce Kilmer Park. Normalized target error (0 = at target, 1 = worst) over generations for SVF, skyline fit, shade, biomass, and noise reduction. Most terms drop rapidly in the first ∼10 generations, then refine more slowly with occasional non-monotonic trade-offs. Noise reduction decreases more gradually, indicating a harder objective under the same geometric constraints.
Remote-sensing systems usually describe urban content with detection boxes, semantic masks, or vector boundaries. Such outputs locate classes and support image-plane scoring, yet they do not by themselves constitute an executable layout that retains object identities, typed relations, topology, and regeneration rules. Code-as-City instead casts urban-layout extraction from a single top-down image as constrained code generation with a multimodal large language model (MLLM). An image model first produces an aligned five-class semantic layout prior. Three ordered MLLM passes use the image and this prior to recover roads, land-cover regions and relations, and buildings. Deterministic normalization converts the accumulated records into a city graph and a restricted layout program. Executing the program creates a renderable 3D city layout and an orthographic semantic projection over shared geometry. The projection admits pixel-level comparison with remote-sensing masks, while named objects, relations, and editing operations remain available for synchronized regeneration of both views. Evaluated on the 100 scenes of CityLayout-100, the complete framework obtains 41.1% mean intersection-over-union and 48.3% global intersection-over-union. This result provides quantitative evidence that visual observations can be translated into inspectable, editable city code with coupled planar and 3D outputs.
Zigan Zhou, Kai Li, Yupeng Deng
City University of Hong Kong Hong Kong SAR, China · School of Electronic, Electrical and Communication Engineering University of Chinese Academy of Sciences Beijing, China · Aerospace Information Research Institute Chinese Academy of Sciences Beijing, China
We study open-vocabulary 3D indoor layout generation, which synthesizes diverse and physically plausible scenes from unlabeled 3D assets using free-form language instructions. Recent methods leverage large language models (LLMs) and vision-language models (VLMs) to generate structured scenes from text. However, most model inter-asset relations implicitly or rely on local pairwise constraints and local optimization. These formulations are poorly aligned with the global, highly non-convex layout space, often yielding locally plausible yet globally inconsistent or physically infeasible scenes. We address this problem with a graph-based intermediate representation that separates semantic coherence from physical feasibility, together with a hybrid search-and-refinement strategy. First, Global Semantic Verification (GSV) represents scenes as structured graphs and enforces semantic constraints through rule-based verification. This explicit validation removes contradictory configurations and produces a globally consistent semantic scaffold. Second, Global Physical Feasibility Search (GPFS) combines evolutionary search for global exploration with gradient-based refinement for local exploitation. It reduces dependence on VLM-proposed initialization and improves robustness in non-convex and discontinuous feasible spaces. Together, GSV and GPFS move layout generation beyond local relational modeling and initialization-sensitive optimization toward globally consistent reasoning and search. Experiments show that our method achieves state-of-the-art performance in open-vocabulary 3D indoor layout generation, improving both semantic consistency and physical plausibility.
Jialu Huang, Yingxuan You, Fei Wang +1
National Key Laboratory of Human-Machine Hybrid Augmented Intelligence Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, China · 2 École Polytechnique Fédérale de Lausanne (EPFL), CVLab, Switzerland · School of Electronics and Information, Northwestern Polytechnical University and Shaanxi Key Laboratory of Information Acquisition and Processing, China
Generating editable 3D urban scenes from a single image requires detailed building assets with a coherent spatial arrangement. Occlusion and perspective ambiguity make it difficult to infer complete building geometry together with consistent relative scales and positions. We present CityLift, a framework that generates individual textured building meshes from an urban image by coupling scene-level spatial modeling with building-level synthesis. CityLift first learns building shape and placement jointly through coupled scene and instance streams, producing coarse sparse structures in a common coordinate system. We then introduce generative spatial inheritance, which maps each building's coarse occupancy and scene-conditioned 3D feature field into a local volume to guide a flow model that generates finer structure. The same coordinate mapping places the generated meshes back into the scene, preserving their spatial relationship to the scaffold. To address the scarcity of urban data pairing scene images with individual 3D buildings, we construct CityLift-5K, comprising 4,997 synthetic scenes with instance masks, building meshes, and placement annotations. Experiments on CityLift-5K and scene blocks from UrbanScene3D and InstanceBuilding demonstrate improved building geometry and scene composition. The project page is available at https://ranhaogong.github.io/CityLift/.
Haoran Gong, Jie Feng, Chenxu Wang
School of Software Engineering, Xi’an Jiaotong University · Zhongguancun Academy, Beijing, China