Generating editable graphic designs requires arranging visual and textual elements into a coherent composition while preserving their content and editability. Although recent MLLM-based approaches employ iterative generation and visual refinement to reflect the real-world design workflows, their high-level plans are predetermined prior to execution, limiting their adaptability as the design evolves. To address this limitation, we propose PlayGround, a progressive layout generation framework that dynamically adapts to the current rendered canvas. PlayGround establishes a progressive design loop comprising a Progressive Planner, a Layout Generator, and a Visual Verifier, guided by a Structure-Aware Retriever. Given an initial set of unordered elements, the Structure-Aware Retriever initially infers a coarse layout structure to retrieve structurally similar reference designs. Subsequently, at each turn, the Progressive Planner examines the current rendered canvas alongside the remaining elements, forms the next functional group, and plans its composition. The Layout Generator then predicts editable layout attributes, while the Visual Verifier evaluates rendered candidates and provides feedback when refinement is necessary. To train the Progressive Planner with step-wise supervision, we construct a two-stage data generation pipeline that extracts functional groups and turn-level planning trajectories from completed designs. Experiments on Crello and LICA demonstrate the effectiveness of PlayGround over existing methods. Our code and weights will be publicly released.
Figures & tables
Figure 1: Prior methods ( Cheng et al., 2025 ; Liu et al., 2026 ) either (a) generate layout in a single forward pass, or (b) refine results iteratively while overall design plan stays fixed outside the feedback loop. In contrast, (c) PlayGround decomposes planning into steps and places them inside the iterative loop. As PlayGround plans and fills the canvas progressively based on the current canvas state, PlayGround is robust to intermediate design errors and produces more coherent final designs.
Figure 2: Data Construction Pipeline. Stage 1 constructs functional groups, execution order, coarse group regions, and cumulative canvas states from completed designs. Stage 2 uses these outputs to derive turn-level planning annotations for each functional group.
Figure 3: Overview of PlayGround .
Dataset
Method
VLM-based metrics ↑
Rule-based metrics ↓
(i)
(ii)
(iii)
(iv)
(v)
(vi)
T.Occ.
Rea.
FlexDM
2.670 ± 0.690
2.156 ± 0.656
3.133 ± 0.705
3.637 ± 0.581
3.200 ± 0.740
3.021 ± 0.992
0.1519
0.0583
LaDeCo
3.103 ± 0.722
2.615 ± 0.809
3.451 ± 0.721
3.979 ± 0.574
3.592 ± 0.743
3.528 ± 1.029
0.0570
0.0432
PosterCopilot
2.980 ± 0.729
2.623 ± 0.810
3.263 ± 0.776
3.809 ± 0.653
3.467 ± 0.784
3.435 ± 1.073
0.0319
0.0374
VFLM
3.335 ± 0.564
2.718 ± 0.645
3.714 ± 0.576
3.969 ± 0.565
3.759 ± 0.642
3.693 ± 0.947
0.0002
0.0360
[][]
PlayGround
3.794 ± 0.583
3.503 ± 0.660
3.899 ± 0.587
4.381 ± 0.542
4.183 ± 0.635
4.248 ± 0.919
0.0041
0.0345
Table 1: Quantitative Results on the Crello and LICA datasets. We report six VLM-based metrics (mean ± standard deviation) and two rule-based metrics, T.Occ. and Rea. Bold and underline mark the best and the second best within each dataset, excluding GT.
Figure 4: Qualitative results on Crello dataset.
Figure 5: Effect of our Progressive Planner. Our Progressive Planner adapts to the rendered canvas and uses available space, while the Static Planner follows an outdated initial plan.
#
Retriever
Planner
K
Verifier
VLM ↑
(I)
Top-5
✓
5
✓
3.958
(II)
✗
✓
5
✓
3.901
(III)
Lowest-5
✓
5
✓
3.867
(IV)
Top-5
✓
3
✓
3.923
(V)
Top-5
✓
1
✓
3.891
(VI)
Top-5
✓
5
✗
3.698
Table 2: Ablation on pipeline components. VLM denotes the mean of six VLM scores on a Crello subset. △ denotes a Static Planner.
Grouping scheme
Per-turn Unit
VLM ↑
(I) None
Element-wise
3.464
(II) None
Full-layout
3.351
(III) LaDeCo-style
Semantic layer
3.532
(IV) Ours
Functional Group
3.578
Table 3: Ablation on per-turn granularity and grouping. VLM denotes the mean of six VLM scores on a Crello subset.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Detailed Data Construction Pipeline. Stage 1 constructs functional groups and their execution order, and Stage 2 derives a turn-level plan for each group.
Figure 7: Statistics of the constructed functional groups.
Figure 8: Group size and modality composition by functional role.
Figure 9: Execution-turn distribution by functional role. Each row shows the distribution of execution turns for the corresponding functional role. T1, T2, … denote the first, second, … group-wise planning turns, and T10+ denotes turn 10 or later.
Figure 10: Detailed example of Progressive Design Generation.
Figure 11: Distribution of reference reuse across test queries. For each reference appearing in the Top-5 retrieval results, we count how many times it is selected across all test queries. The histogram shows the distribution of these selection frequencies across references, with the vertical axis shown on a logarithmic scale.
Figure 12: Qualitative examples of retrieved references. For each query, we show the five highest-ranked references retrieved from the reference pool. The retrieved designs exhibit similar coarse layout structures and canvas geometry.
Figure 13: Qualitative comparison of retrieval ranking. For the same input elements, we compare the Top-5 references with the lowest-ranked five references among candidates sharing the query canvas size, together with the resulting layout instructions and final layouts.
Configuration
Structure-Aware Retriever
Progressive Planner
Layout Generator
Model and adaptation
Base model
Qwen3-VL-8B-Instruct
Qwen3-VL-8B-Instruct
Qwen3-VL-8B-Instruct
LoRA rank / α / dropout
32 / 64 / 0.05
32 / 64 / 0.05
32 / 64 / 0.05
LoRA target
LLM
LLM
LLM
Trainable non-LoRA module
Merger
Merger
Merger
Optimization
Appendix
Table 4: Training configurations. Configurations used to fine-tune the Structure-Aware Retriever, Planner, and Layout Generator.
Dataset
Method
Rule-based metrics
Val ↑
Ove ↓
Ali ↓
IoU ↑
IOPR ↓
ARD ↓
FlexDM
0.8602
0.4699
0.0012
0.2059
–
0.1245
LaDeCo
0.9227
0.2026
0.0012
0.1297
–
0.1749
PosterCopilot
0.9755
0.3323
0.0007
0.1759
0.0871
0.0191
VFLM
–
–
–
–
–
–
PlayGround
0.9309
0.2164
0.0011
0.2253
0.0446
0.0097
Appendix
Table 5: Rule-based Quantitative Comparison. We report the mean of six rule-based metrics, Val, Ove, Ali, IoU, IOPR, and ARD. A dash marks a metric that we do not report. Bold and underline mark the best and the second best within each dataset, excluding GT.
Figure 14: User Study Results. Human ranking distributions across six evaluation criteria.
Figure 15: More qualitative results on Crello dataset.
Figure 16: More qualitative results on Crello dataset.
Figure 17: More qualitative results on LICA dataset.
Figure 18: Qualitative results of PlayGround on the intention-to-design task.
Automating graphic design synthesis from user-provided elements requires both a coherent overall composition and the exact preservation of each asset. Existing methods predict a layout as explicit bounding-box coordinates with a language model and then paste the assets into it, which separates spatial planning from visual synthesis and tends to produce rigid, mis-scaled compositions. We instead ask whether the layout can emerge implicitly inside a pretrained image-editing diffusion transformer. We present Mise-en-Scène, a two-stage framework. In the first stage, a diffusion transformer adapted with a small, knockout-selected LoRA drafts a complete design in which the arrangement of the elements emerges jointly with the rendered canvas. In the second stage, a deterministic match-and-place step moves the original high-resolution assets to the drafted positions, which guarantees exact asset fidelity and yields an editable, layered design that a designer can keep refining rather than a flat image. Notably, a minimal adaptation of the pretrained transformer already suffices, without the specialized conditioning machinery commonly introduced for multi-element generation. On the large-scale PrismLayersPlus benchmark, the designs produced by Mise-en-Scène are the closest to the ground truth in perceived quality among all compared methods, by a wide margin over both an LLM layout planner and a specialized layout transformer, while our match-and-place stage bridges the remaining fidelity gap to the ground-truth composites.
Designing functional and aesthetically coherent floor plans requires exploring a vast space of possible room arrangements, a task that quickly becomes overwhelming for human designers. In this paper, we propose GRE-Diff, a controllable and interactive diffusion-based framework that automates the creation and editing of apartment floor plans under user-specified constraints. By combining AI-generated suggestions with real-time, human-in-the-loop editing, the system enables users to specify room types, room counts, boundary shapes, and editing operations through LLM-parsed instructions or GUI-based interaction. It then generates a diverse set of plausible and well-structured designs for refinement. At the core of our approach is Gaussian Room Embedding (GRE), a continuous latent representation that models each room as a spatial Gaussian distribution capturing its location and extent. Extensive experiments on the RPLAN dataset show that GRE-Diff produces high-quality, constraint-aware, and editable polygonal layouts, offering a practical step toward bridging AI-driven automation and human creativity in spatial design.
Jing Wang, Haoran Xiong, Zihao Yan +2
Guangdong Provincial Key Laboratory of Visual Media and Multidimensional Intelligence, CSSE, Shenzhen University, China · School of Computer Science, University of Guelph, Canada
Recent advances in Multimodal Large Language Models (MLLMs) have enabled automated generation of structured layouts from natural language descriptions. Existing methods typically follow a code-only paradigm that generates code to represent layouts, which are then rendered by graphic engines to produce final images. However, they are blind to the rendered visual outcome, making it difficult to guarantee readability and aesthetics. In this paper, we identify visual feedback as a critical factor in layout generation and propose Visual Feedback Layout Model (VFLM), a self-improving framework that leverages visual feedback iterative refinement. VFLM is capable of performing adaptive reflective generation, which leverages visual information to reflect on previous issues and iteratively generates outputs until satisfactory quality is achieved. It is achieved through reinforcement learning with a visually grounded reward model that incorporates OCR accuracy. By rewarding only the final generated outcome, we can effectively stimulate the model's iterative and reflective generative capabilities. Experiments across multiple benchmarks show that VFLM consistently outperforms advanced MLLMs, existing layout models, and code-only baselines, establishing visual feedback as critical for design-oriented MLLMs. Our code and data are available at https://github.com/FolSpark/VFLM.
Junrong Guo, Shancheng Fang, Yadong Qu +1
University of Science and Technology of China · Shenzhen University