Organizations: School of Computer Science, Peking University · Technical University of Munich · Beijing Jiaotong University · Beijing Digital Native Digital City Research Center · Theta Labs, Inc. · Beijing Normal University
Scene generation has extensive industrial applications, demanding both high realism and precise control over geometry and appearance. Language-driven retrieval methods compose plausible scenes from a large object database, but overlook object-level control and often fail to enforce scene-level style coherence. Graph-based formulations offer higher controllability over objects and inform holistic consistency by explicitly modeling relations, yet existing methods struggle to produce high-fidelity textured results, thereby limiting their practical utility. We present FlowScene, a tri-branch scene generative model conditioned on multimodal graphs that collaboratively generates scene layouts, object shapes, and object textures. At its core lies a tight-coupled rectified flow model that exchanges object information during generation, enabling collaborative reasoning across the graph. This enables fine-grained control of objects' shapes, textures, and relations while enforcing scene-level style coherence across structure and appearance. Extensive experiments show that FlowScene outperforms both language-conditioned and graph-conditioned baselines in terms of generation realism, style consistency, and alignment with human preferences.
Figures & tables
Figure 1. Scene Generation from Diverse Input. The prospective system, powered by FlowScene , supports the generation of style-consistent 3D scenes from multi-source descriptions, including text input, GUI selections, and mixed information. Users can flexibly specify object categories and, if desired, provide their detailed RGB images in arbitrary views, along with their optional inter-object relationships (A). These descriptions are parsed into a multimodal graph by modern LLMs or VLMs as graph constructors (B). Based on this graph, FlowScene generates complete textured scenes, comprising both spatial scene layout and individual posed objects (C).
Figure 2. Multimodal Graph Processing. Given a graph GM (upper left), the graph network (right) first projects multimodal features of nodes VM , followed by message passing and feature aggregation along the edges E connecting VM . Within the denoising process of a generative model, it acts as an InfoExchangeUnit to incorporate temporal denoising data Dt , which is projected and fused with the node features (lower left). In this way, the network yields node-wise conditioning Ct while respecting the global constraints imposed by the graph structure (bottom).
Figure 3. FlowScene Pipeline. Conditioned on GM in Figure 2 , FlowScene stacks three coordinated branches to generate a scene. A. Layout Branch generates the scene layout from noise B1 , where the x-axis (red) points to the right and the z-axis (blue) points to the front. The LayoutExchangeUnit iteratively applies temporal layout constraints to the generation process by conditioning ΘB . B. Shape Branch is parallel to the layout branch. It exchanges shape information among nodes according to GM through the ShapeExchangeUnit . At inference, we sample noise S1 , and generate latent shape codes S via ΘS , and decode them into voxelized shapes X using ΦD . C. Texture Branch is subsequent to the shape branch. It anchors Gaussian noise onto X to form X1e , which is then denoised by ΘT into structured latents Xe . Finally, ΨD decodes them into objects O , which are scaled and placed into the generated layout B to complete the scene.
Method
Bedroom
Living room
Dining room
FID
FIDCLIP
KID
FID
FIDCLIP
KID
FID
FIDCLIP
KID
CommonScenes ∗ ( Zhai et al., 2024c )
49.52
0 5.56
0 2.20
78.19
0 7.59
0 6.12
69.89
0 7.88
0 4.78
EchoScene ∗ ( Zhai et al., 2024b )
44.10
0 4.49
0 0.30
75.35
0 7.37
0 5.59
67.52
0 7.27
0 3.15
MMGDreamer ∗ ( Yang et al., 2025a )
42.38
0 4.48
0 -0.14
74.14
0 7.31
0 5.62
67.97
0 7.39
0 2.93
FlowScene (Ours)
35.01
0 3.14
0 -0.34
70.34
0 7.18
0 5.11
60.26
0 6.72
0 0.07
CommonScenes ( Zhai et al., 2024c )
57.68
0 4.86
0 6.59
80.99
0 7.05
0 6.39
65.71
0 7.04
0 5.47
Table 1. Scene-Level Fidelity. This is evaluated by comparing top-down renderings of generated and real scenes at 2562 resolution using FID, FIDCLIP , and KID ( ×0.001 ), following ( Zhai et al., 2024b ) (lower is better). * denotes their retrieval branches, which generate layouts and retrieve well-designed meshes from 3D-FUTURE ( Fu et al., 2021b ) . The best results are in bold .
Method
Metric
Bed
N.stand
Ward.
Chair
Table
Cabinet
Lamp
Shelf
Sofa
TV stand
CommonScenes ( Zhai et al., 2024c )
MMD ( ↓ )
0.49
0.92
0.54
0.99
1.91
0.96
1.50
2.73
0.57
0.29
EchoScene ( Zhai et al., 2024b )
0.37
0.75
0.39
0.62
1.47
0.83
0.66
2.52
0.48
0.35
MMGDreamer ( Yang et al., 2025a )
0.22
0.41
0.24
0.35
0.55
0.71
0.34
1.58
0.43
0.24
FlowScene (Ours)
0.17
0.23
0.20
0.32
0.64
0.60
0.20
1.53
0.48
0.13
CommonScenes ( Zhai et al., 2024c )
COV ( %,↑ )
24.07
24.17
26.62
26.72
40.52
28.45
36.21
40.00
28.45
33.62
EchoScene ( Zhai et al., 2024b )
39.51
25.59
37.07
17.25
35.05
43.21
33.33
50.00
41.94
40.70
Table 2. Object-Level Fidelity. Quantitative evaluation of generated object quality and diversity using MMD ( ×0.01 ), COV, and 1-NNA metrics across different categories. The best results are highlighted in bold .
Method
CLIPScore ↑
FPVScore ↑
Perceptual Study ↑
Bedroom
Living room
Dining room
PA
LC
VQ
SC
OP
PA
LC
VQ
SC
OP
Echoscene ∗ ( Zhai et al., 2024b )
0.1837
0.1799
0.1553
3.03
3.07
2.82
3.03
3.01
7.21
6.83
6.86
7.12
7.24
MMGDreamer ∗ ( Yang et al., 2025a )
0.1957
0.1877
0.1626
3.12
3.31
3.08
3.25
3.27
7.47
7.12
7.25
7.31
7.42
Holodeck ( Yang et al., 2024c )
0.1912
0.2116
0.1968
2.79
2.73
2.71
2.70
2.70
7.16
6.61
7.43
6.72
6.64
LayoutVLM ( Sun et al., 2025a )
0.2045
0.2056
0.1947
2.36
2.44
2.15
2.23
2.31
6.52
6.33
7.21
6.43
6.32
FlowScene (Ours)
0.2386
0.2246
0.2092
3.35
3.54
3.83
3.85
3.76
7.97
7.65
8.46
8.72
8.57
Table 3. Scene Perception Evaluation. For FPVScore, we report the average ranking across all scenes, where 1.00 is the worst and 5.00 is the best. For the perceptual study, scores range from 1.00 to 10.00. Each score is averaged across all participants and scenes. Both FPVScore and the perceptual study benchmark prompt adherence (PA), layout correctness (LC), visual quality (VQ), style consistency (SC), and overall preference (OP). The best results are highlighted in bold . * denotes the retrieval mode.
Figure 4. Qualitative comparison. The first column shows the input multimodal graph with key scene edges. Red rectangles show inconsistencies and artifacts, while green rectangles highlight consistent regions. † Text-only scene graphs are given.
Figure 5. Qualitative comparison with training-free, language-based methods. The grey boxes display the input textual instructions derived from text-only scene graphs, while FlowScene takes the text-only graphs as input.
Method
Layout
Shape
Texture
Time (s)
EchoScene ( Zhai et al., 2024b )
✓
✓
–
44.27
MMGDreamer ( Yang et al., 2025a )
✓
✓
–
45.34
FlowScene (Ours, w/o TB)
✓
✓
✗
6.83
FlowScene (Ours)
✓
✓
✓
37.38
Table 4. Inference Time Comparison.
LEU
SEU
TEU
FID
FIDCLIP
KID
50.83
4.55
7.32
✓
40.55
3.48
2.58
✓
44.73
4.42
4.97
✓
43.82
3.86
3.68
✓
✓
36.81
2.92
1.31
✓
✓
36.30
2.90
0.77
Table 5. Ablation Study on the InfoExchangeUnit across different branches, including the LayoutExchangeUnit (LEU), ShapeExchangeUnit (SEU), and TextureExchangeUnit (TEU).
Shape
Layout
FID
FIDCLIP
KID
Diffusion
Flow
Diffusion
Flow
✓
✓
35.23
3.26
1.78
✓
✓
34.96
2.90
1.03
✓
✓
33.03
2.94
0.62
✓
✓
32.76
2.86
0.38
Table 6. Ablation studies on different generation backbones.
Modality
FPVScore ↑
Visual
Textual
LC
VQ
SC
OP
10%
90%
3.28
3.37
3.42
3.26
40%
60%
3.45
3.86
3.89
3.67
50%
50%
3.42
3.88
3.91
3.64
60%
40%
3.36
3.88
3.91
3.62
90%
10%
3.28
3.78
3.82
3.59
Table 7. Ablation studies on the ratio of node modalities.
Figure 6. Data Preparation.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Mode
left/ right
front/ behind
smaller/ larger
taller/ shorter
close by
symmetrical
Graph-to-3D ( Dhamo et al., 2021 )
Change
0.91
0.92
0.86
0.89
0.69
0.46
CommonScenes ( Zhai et al., 2024c )
0.91
0.92
0.86
0.91
0.69
0.59
EchoScene ( Zhai et al., 2024b )
0.94
0.96
0.92
0.93
0.74
0.50
MMGDreamer ( Yang et al., 2025a )
0.95
0.96
0.93
0.93
0.71
0.53
FlowScene (Ours)
0.95
0.96
0.93
0.94
0.75
0.58
Graph-to-3D ( Dhamo et al., 2021 )
Addition
0.94
0.95
0.91
0.93
0.63
0.47
Appendix
Table 8. Scene graph constraints. Comparisons are across three modes: Relationship Change, Node Addition, and Generation Only (listed top to bottom). The best results are highlighted in bold .
Method
Bedroom
Living room
Dining room
N.stand
Lamp
Table
Lamp
Chair
Table
CommonScenes ( Zhai et al., 2024c )
2.69
-
11.75
-
1.96
9.04
EchoScene ( Zhai et al., 2024b )
1.68
30.07
3.02
10.06
1.75
1.26
MMGDreamer ( Yang et al., 2025a )
1.33
2.29
2.44
0.18
0.23
0.83
FlowScene (Ours)
0.46
0.19
1.61
0.14
0.21
0.37
Appendix
Table 9. Inter-shape consistency. The Chamfer Distance (CD, ×0.001 ) is computed between object pairs linked by the same as relationship. Lower values indicate a higher degree of shape similarity between paired instances.
Training
Testing
FID
FIDCLIP
KID
Multi
Single
Multi
Single
✓
✓
33.77
2.88
0.73
✓
✓
33.79
2.97
0.99
✓
✓
40.73
4.04
3.79
✓
✓
32.76
2.86
0.38
Appendix
Table 10. Ablation studies of different input views.
Mask Ratio
FID
FIDCLIP
KID
20%
34.40
2.83
0.43
50%
38.94
3.22
0.96
80%
43.54
3.78
2.19
0% ∗
40.89
3.36
2.05
Appendix
Table 11. Ablation Study on the mask ratio of relation. * indicates results from MMGDreamer.
System & Hardware Overview
CPU
AMD EPYC 7702 64-Core
GPU
8 × NVIDIA A100 GPU
Memory
1T RAM
Operating System
Ubuntu 22.04 LTS
CUDA Version
12.8
NVIDIA Driver
570.133.20
Appendix
Table 12. System and Software Configuration.
Figure 7. Qualitative comparison with graph-conditioned generative models. The top row illustrates the input graphs, highlighting key semantic and spatial edges. In the generated scenes, red rectangles indicate generation failures in baselines (e.g., collisions), whereas green rectangles mark consistent regions in our results. * denotes the retrieval mode. † Text-only scene graphs are given.
Figure 8. Qualitative comparison on object generation with other methods.
Figure 9. Failure case. The left panel shows the input multimodal scene graph, while the right panel shows the generated failure case. Red cross marks indicate removed relationships.
Figure 10. Prompt template for evaluating FPVScore ( Huang et al., 2025 ) with GPT-4o ( Achiam et al., 2023 ) .
Figure 11. Perceptual Study Interface: Instructions and Metrics. This introductory page is presented to participants at the start of the perceptual study. It outlines the study objectives, provides detailed instructions for viewing and rating, and clearly defines the five evaluation metrics (PA, LC, VQ, SC, OP).
Figure 12. Perceptual Study Interface: Example Page. Participants are presented with a detailed text prompt (top) and the corresponding generated 3D scenes from five different methods. Users interactively rate each method using sliding scales (1.00-10.00) across the five defined criteria, allowing for a direct and comprehensive comparison of generation quality.
Figure 13. The Interface of Language Description with Images.
We present DistScene, a framework for single-image compositional 3D scene generation by jointly modeling the environment and individual objects. Unlike existing methods that represent scenes primarily as collections of objects, we model the environment as an explicit scene component to provide geometric context for object placement. Specifically, we introduce Scene-Frame Generation, which jointly generates separate environment and object components in a shared coordinate frame, allowing their geometry and relative placement to be learned together. Then we introduce Object-Centric Refinement to refine each object in a local frame with scene context. Finally, we develop Object-to-Scene Distillation to transfer pretrained object-generation priors to scene generation through automatically composed and rendered synthetic scenes. Evaluations on indoor and outdoor benchmarks demonstrate improved scene-level spatial coherence over the evaluated baselines. Project page: https://coolbeam.github.io/DistScene/
Kunming Luo, Hongyu Yan, Ken Deng +6
The Hong Kong University of Science and Technology · TeleAI, China Telecom
Text-driven 3D indoor scene generation has advanced from dataset-bound layout modeling to open-vocabulary synthesis with large language and vision-language models. Yet existing methods remain limited: one-pass generators often yield geometrically invalid layouts, heavy post-hoc optimization is costly and unstable, and prompt-only planners lack reusable layout priors for functional grouping and object relations. We propose ScenePilot, a retrieval-augmented Grow-and-Repair framework that formulates scene generation as prior-guided incremental growth with learned rectification. Given a prompt, the Hierarchical Retrieval-Augmented Planning (HRAP) module retrieves room-, group-, and anchor-level layout priors to support functional group planning. A text-driven base generator then inserts object groups sequentially, while the Reinforcement Multimodal Repair (RMR) module performs lightweight local correction after each insertion and a final global repair after completion. To train this policy, we construct SceneReverse-17k, a repair-trajectory dataset built by perturbing high-quality 3D scenes in position, rotation, and scale, then using inverse operations as executable rectification targets. The policy predicts structured move--rotate--scale actions from rendered views, scene state, retrieved priors, and edit history. By combining HRAP with RMR, ScenePilot offers an efficient alternative to one-shot generation and heavy full-scene optimization, improving physical plausibility, functional coherence, and controllability while preserving diversity.
Jiawei Zhang, Hongsong Wang, Pan Zhou
Southeast University · Singapore Management University
Recent progress in image and video synthesis has inspired their use in advancing 3D scene generation. However, we observe that text-to-image and -video approaches struggle to maintain scene- and object-level consistency beyond a limited environment scale without a persistent, explicit geometric representation. We thus present a geometry-first approach that decouples this complex problem of large-scale 3D scene synthesis into its structural composition, represented as a mesh scaffold, and realistic appearance synthesis, which leverages powerful image synthesis models conditioned on the mesh scaffold. From an input text description, we first construct a mesh capturing the environment's geometry (walls, floors, etc.), and then use image synthesis, segmentation and object reconstruction to populate the mesh structure with objects in realistic layouts. This mesh scaffold is then rendered to condition image synthesis, providing a structural backbone for consistent appearance generation. This enables scalable, arbitrarily-sized 3D scenes of high object richness and diversity, combining robust 3D consistency with photorealistic detail. We believe this marks a significant step toward generating truly environment-scale, immersive 3D worlds.