Seeing is Improving: Visual Feedback for Iterative Text Layout Refinement

Junrong Guo; Shancheng Fang; Yadong Qu; Hongtao Xie

cs.CVMar 23, 2026

Seeing is Improving: Visual Feedback for Iterative Text Layout Refinement

Authors: Junrong Guo, Shancheng Fang, Yadong Qu, Hongtao Xie

Organizations: University of Science and Technology of China · Shenzhen University

Abstract

Recent advances in Multimodal Large Language Models (MLLMs) have enabled automated generation of structured layouts from natural language descriptions. Existing methods typically follow a code-only paradigm that generates code to represent layouts, which are then rendered by graphic engines to produce final images. However, they are blind to the rendered visual outcome, making it difficult to guarantee readability and aesthetics. In this paper, we identify visual feedback as a critical factor in layout generation and propose Visual Feedback Layout Model (VFLM), a self-improving framework that leverages visual feedback iterative refinement. VFLM is capable of performing adaptive reflective generation, which leverages visual information to reflect on previous issues and iteratively generates outputs until satisfactory quality is achieved. It is achieved through reinforcement learning with a visually grounded reward model that incorporates OCR accuracy. By rewarding only the final generated outcome, we can effectively stimulate the model's iterative and reflective generative capabilities. Experiments across multiple benchmarks show that VFLM consistently outperforms advanced MLLMs, existing layout models, and code-only baselines, establishing visual feedback as critical for design-oriented MLLMs. Our code and data are available at https://github.com/FolSpark/VFLM.

Figures & tables

Figure 1: Method comparison. Traditional methods only generate through text code and have no perception of the generated results; our Visual Feedback method can iteratively reflect on the generated result and optimize the layout effect.

Figure 2: Pipeline of Visual Feedback Method, including Data Construction for Cold-Start SFT and RL with Modified Advantages.

Method	Model	OCR	Graphic				GPT-4o
Method	Model	Char-F $\uparrow$	$R_{ali}\downarrow$	$R_{ove}\downarrow$	$R_{com}\downarrow$	RM Score $\uparrow$	Text	Harmony	Quality	Meaning	Overall
TextLayout
MLLM	GPT-4o	0.8258	0.0046	0 .0033	18.8443	0.3561	8.5165	8.0341	7.2826	7.6553	7.8721
	Claude3.7	0.8672	0.0053	0.0383	16.4401	0.5295	8.8058	8.4026	7.7691	8.2651	8.3106
	Doubao-Seed-1.6	0.8484	0.0058	0.0216	18.5823	0.4063	8.5663	8.2304	7.4463	7.8917	8.0337
	Qwen2.5-VL-72B	0.7178	0 .0031	0.0358	19.9399	0.1989	7.8633	7.6094	6.5402	6.5582	7.1428

Table 1: The Graphic quality metrics and OCR metrics on the TextLayout, Crello and DESIGNERINTENTION test set, where VFLM-step1 and VFLM respectively represent the metrics of our results in the first output and the final result output after iterative reflection.

Figure 3: In comparison with existing methods, we selected one model from each category of methods as a representative. For a more comprehensive comparison, please refer to the supplementary materials.

level	OCR	Graphic			RM Score $\uparrow$
level	Char-F $\uparrow$	$R_{ali}\downarrow$	$R_{ove}\downarrow$	$R_{com}\downarrow$	RM Score $\uparrow$
Level-I	0.9474	0.0089	0.0038	6.6795	1.0594
Level-II	0.9049	0.0112	0.0134	12.2626	0.6343
Level-III	0.5657	0.0354	0.0444	21.2628	-0.2345
Level-IV	0.5324	0.0536	0.0375	17.5270	-1.3457

Table 2: Metrics and RM score across four quality-level datasets.

Model	OCR	Graphic			RM Score $\uparrow$
Model	Char-F $\uparrow$	$R_{ali}\downarrow$	$R_{ove}\downarrow$	$R_{com}\downarrow$	RM Score $\uparrow$
VFLM-step1	0.9071	0.0035	0.0059	15.4583	0 .5415
VFLM	0.9376	0.0039	0.0009	11.8678	0.6018
Cold-Start-step1	0.7913	0.0078	0.0297	19.0560	0.2572
Cold-Start	0.7980	0.0081	0.0298	19.0577	0.2608
Single-Round RL	0.8792	0 .0024	0.0053	18.9428	0.4063

Table 3: Ablation experiments on Graphic quality metrics and OCR metrics on TextLayout test set.

Panel of Figure 4 — Figure 4: Comparison of training processes: simple outcom rewards vs. complex process rewards.

	OCR	Graphic
level	Char-F $\uparrow$	$R_{ali}\downarrow$	$R_{ove}\downarrow$	$R_{com}\downarrow$	RM Score $\uparrow$
Outcome RL-step1	0.9071	0.0035	0.0059	1 5.4583	0 .5415
Outcome RL	0.9376	0 .0039	0.0009	11.8678	0.6018
$\Delta$ (vs step1)	+0.0305	-0.0004	+0.005	+3.5905	+0.0603
Process RL-step1	0.9032	0.0053	0.0030	16.6528	0.4936
Process RL	0 .9239	0.0052	0 .0027	16.4220	0.5241

Table 4: Comparison of simple outcome rewards and complex process rewards on our test set.

Figure 5: Data example, consisting of a background image, target text, SVG layout code, and the corresponding rendered image.

Figure 6: Qualitative comparison of four levels of data in the training data of the reward model

Figure 7: A detailed GPT4o evaluation output.

Figure 8: Training processes: reflection times of VFLM, constrained between 1 and 4.

Figure 9: Reflection Turns vs. Input Complexity. The positive trend (red line) demonstrates that VFLM autonomously increases reflection depth for more complex tasks.

Figure 11: Comparison of inference time and latency with OpenCOLE and IGD.

Model	Char-F $\uparrow$	$R_{ali}\downarrow$	$R_{ove}\downarrow$	$R_{com}\downarrow$	RM Score $\uparrow$
OpenCOLE*	0.7671	0.7114	0.0143	21.8279	0.3620
VFLM	0.9071	0.0035	0.0059	15.4583	0.5415
Qwen2.5-VL-7B-feedback	0.6830 $\blacktriangle$	0.0025 $\blacktriangle$	0.0188 $\blacktriangle$	28.8221 $\blacktriangledown$	0.0910 $\blacktriangledown$
GPT-4o-feedback	0.8494 $\blacktriangle$	0.0039 $\blacktriangle$	0.0028 $\blacktriangle$	20.8852 $\blacktriangledown$	0.3662 $\blacktriangle$

Table 5: Metrics and RM score on OpenCOLE* and MLLM feedback baseline.

Model	OCR	Graphic			RM Score $\uparrow$
Model	Char-F $\uparrow$	$R_{ali}\downarrow$	$R_{ove}\downarrow$	$R_{com}\downarrow$	RM Score $\uparrow$
VFLM-step1	0.8774	0.0046	0.0061	19.5917	0.4392
VFLM	0.9256	0.0025	0 .0022	14.8063	0.5548
Cold-Start-step1	0.7904	0.0118	0.0410	24.3143	0.2721
Cold-Start	0.7928	0.0121	0.0402	24.0888	0.2774
Single-Round RL	0.8829	0.0010	0.0094	24.4845	0.4007

Table 6: Ablation experiments on Graphic quality metrics and OCR metrics on Crello test set.

Model	OCR	Graphic			RM Score $\uparrow$
Model	Char-F $\uparrow$	$R_{ali}\downarrow$	$R_{ove}\downarrow$	$R_{com}\downarrow$	RM Score $\uparrow$
VFLM-step1	0.9415	0.0033	0.0023	14.1230	0.5285
VFLM	0.9663	0.0024	0 .0008	12.1167	0.5688
Cold-Start-step1	0.8860	0.0130	0.0264	17.5229	0.3784
Cold-Start	0.8881	0.0127	0.0257	16.9280	0.3850
Single-Round RL	0.9382	0 .0006	0.0033	16.7466	0.4768

Table 7: Ablation experiments on Graphic quality metrics and OCR metrics on DESIGNERINTENTION test set.

Model	OCR	Graphic			RM Score $\uparrow$
Model	Char-F $\uparrow$	$R_{ali}\downarrow$	$R_{ove}\downarrow$	$R_{com}\downarrow$	RM Score $\uparrow$
Single-Round RL	0.8792	0.0024	0.0053	18.9428	0.4063
Single-Round RL-w/o OCR	0.8696	0.0024	0.0054	20.0621	0.4153
Direct Output	0.9237	0.0027	0.0021	17.0654	0.4964
Direct Output-w/vo OCR	0.9146	0.0026	0.0059	16.6961	0.5039

Table 8: Ablation experiments on the reward function on the TextLayout test set.

Figure 13: Another detailed VFLM output case.

Figure 15: In comparison with all existing methods(Part1).

Figure 16: In comparison with all existing methods(Part2).

Multimodal Generation

Multimodal generation creates coordinated text, images, audio, video, 3D, and other outputs from mixed inputs while preserving meaning and consistency.

See all 243 papers

PLAYGROUND: Progressive Layout Generation with Render-Grounded Planning
Oct 13, 2026Jaeho Lee, Junhwan Heo, Junghyun Park +5LLM Planning Visually Grounded Reasoning
KAIST AI
Autoregressive Mosaics: Probing 2D Spatial Reasoning in Text-Only Language Models
Aug 31, 2026Ashwin Nedungadi, Stefan Oehmcke, Stefan LüdtkeSpatial Reasoning Benchmarks Spatial Reasoning in Language Models
Institute for Visual & Analytic Computing (VAC), University of Rostock
Render-in-the-Loop: Vector Graphics Generation via Visual Self-Feedback
Apr 22, 2026Guotao Liang, Zhangcheng Wang, Juncheng Hu +5Vector Graphics Generation Text-to-SVG Generation
School of Software, Beihang University, China · 3 4Paradigm, China · Department of Computer Science, The University of Hong Kong, China

Oct 13, 2026J/K move · Enter open · S save

PLAYGROUND: Progressive Layout Generation with Render-Grounded Planning

Jaeho Lee, Junhwan Heo, Junghyun Park, WonJun Moon+4

KAIST AI

LLM Planning Visually Grounded Reasoning

Open paper AskPDF arXiv

Generating editable graphic designs requires arranging visual and textual elements into a coherent composition while preserving their content and editability. Although recent MLLM-based approaches employ iterative generation and visual refinement to reflect the real-world design workflows, their high-level plans are predetermined prior to execution, limiting their adaptability as the design evolves. To address this limitation, we propose PlayGround, a progressive layout generation framework that dynamically adapts to the current rendered canvas. PlayGround establishes a progressive design loop comprising a Progressive Planner, a Layout Generator, and a Visual Verifier, guided by a Structure-Aware Retriever. Given an initial set of unordered elements, the Structure-Aware Retriever initially infers a coarse layout structure to retrieve structurally similar reference designs. Subsequently, at each turn, the Progressive Planner examines the current rendered canvas alongside the remaining elements, forms the next functional group, and plans its composition. The Layout Generator then predicts editable layout attributes, while the Visual Verifier evaluates rendered candidates and provides feedback when refinement is necessary. To train the Progressive Planner with step-wise supervision, we construct a two-stage data generation pipeline that extracts functional groups and turn-level planning trajectories from completed designs. Experiments on Crello and LICA demonstrate the effectiveness of PlayGround over existing methods. Our code and weights will be publicly released.

Abstract

Multimodal Generation

Explore similar work

PLAYGROUND: Progressive Layout Generation with Render-Grounded Planning

Autoregressive Mosaics: Probing 2D Spatial Reasoning in Text-Only Language Models

Render-in-the-Loop: Vector Graphics Generation via Visual Self-Feedback