Planning and Rendering in Concert: DeepFusion of Autoregressive Layouts and Diffusion for Visual Text Generation

Guanqiao Chen; Jingru Tan; Dongxing Mao; Catherine Chen; Zijian Du; Libo Qin; Hu Jian Guo; Alex Jinpeng Wang

cs.CVSep 19, 2026

Planning and Rendering in Concert: DeepFusion of Autoregressive Layouts and Diffusion for Visual Text Generation

Authors: Guanqiao Chen, Jingru Tan, Dongxing Mao, Catherine Chen, Zijian Du, Libo Qin, Hu Jian Guo, Alex Jinpeng Wang

Organizations: University of Science and Technology of China · Central South University · Louisiana State University · Arizona State University · Sun Yat-sen University

Abstract

Generating text-rich images from prompts requires both textual fidelity and the coherent integration of text into the surrounding image. An explicit layout can provide structured guidance about what text should appear and where, but a well-formed plan alone does not guarantee that the renderer will realize it faithfully. Existing layout-based AR-diffusion systems typically optimize planning and rendering separately, preventing the planner's representations from being adapted jointly with image synthesis. We introduce DuetGen, an autonomous visual text generator built on DeepFusion, which jointly learns autoregressive planning and continuous diffusion rendering. DeepFusion conditions a diffusion transformer on the planner's prompt and bbox-content hidden states, allowing rendering supervision to shape the representations connecting textual plans with visual outputs. Its joint objective combines autoregressive plan supervision, text-region-weighted diffusion learning, and auxiliary coordinate supervision to maintain structured planning, emphasize text-bearing regions, and improve the spatial precision of planner representations. During inference, Phase-Aware Attention Modulation strengthens the correspondence between image regions and their matched coordinate and content states, facilitating region-specific execution of the generated plan. With a 2B planner and a 4B single-stream DiT, DuetGen achieves 0.8293 word accuracy on CVTG-2K and 0.938 accuracy on LongText-Bench, closely matching the substantially larger Qwen-Image on both benchmarks. These results demonstrate the value of jointly learned planning representations and region-specific rendering for autonomous visual text generation.

Figures & tables

Figure 1: Paradigms for autonomous layout-aware visual text generation. (a) Separated AR–diffusion systems separately optimize planning and continuous rendering in two stages. (b) Fully autoregressive systems jointly model layout and image tokens, but rely on discrete visual representations. (c) DeepFusion jointly optimizes AR planning and continuous diffusion rendering through shared planner states and region-specific execution.

Figure 2: Overview of D u e t G e n . (a) DeepFusion jointly optimizes planning and rendering through prompt-and-plan states, autoregressive plan supervision, text-region-weighted diffusion learning, and auxiliary coordinate supervision. (b) During inference, Phase-Aware Attention Modulation strengthens attention from in-region patches to their matched bbox and content states in middle DiT layers and denoising steps.

Method	Word Accuracy $\uparrow$					NED $\uparrow$	CLIPScore $\uparrow$
Method	2	3	4	5	Avg.	NED $\uparrow$	CLIPScore $\uparrow$
SD3.5 Large ( Esser et al., 2024 )	0.7293	0.6825	0.6574	0.5940	0.6548	0.8470	0.7797
FLUX.1 dev ( BlackForest, 2024 )	0.6089	0.5531	0.4661	0.4316	0.4965	0.6879	0.7401
AnyText ( Tuo et al., 2023 )	0.0513	0.1739	0.1948	0.2249	0.1804	0.4675	0.7432
TextDiffuser-2 ( Chen et al., 2024a )	0.5322	0.3255	0.1787	0.0809	0.2326	0.4353	0.6765
RAG-Diffusion ( Chen et al., 2024c )	0.4388	0.3316	0.2116	0.1910	0.2648	0.4498	0.7797

Table 1: Quantitative results on CVTG-2K ( Du et al., 2025 ) . Word accuracy is grouped by the number of text regions. Best and second-best results are bold and underlined .

Method	Acc. $\uparrow$	Method	Acc. $\uparrow$
BLIP3-o ( Chen et al., 2025b )	0.021	OmniGen2 ( Wu et al., 2025b )	0.561
Show-o2 ( Xie et al., 2025a )	0.119	FLUX.1 ( BlackForest, 2024 )	0.607
Janus-Pro ( Chen et al., 2025f )	0.019	TextDiffuser-2 ( Chen et al., 2024a )	0.371
Kolors 2.0 ( team, 2025 )	0.543	HiDream-I1-Full ( Cai et al., 2025 )	0.543
RAG-Diffusion ( Chen et al., 2024c )	0.321	TextCrafter ( Du et al., 2025 )	0.683
Qwen-Image ( Wu et al., 2025a )	0.943	D u e t G e n	0.938

Table 2: Text accuracy on LongText-Bench ( Geng et al., 2025 ) . Best and second-best results are bold and underlined .

Figure 3: Qualitative comparison on prompts containing multiple text regions and extended text.

Table 7

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Source	Pairs	Share	Primary coverage
CC12M	4.0M	26.7%	Real-world images and long-tail concepts
LAION (high-quality subset)	3.5M	23.3%	Visual concepts, styles, and scenes
COYO (high-quality subset)	2.5M	16.7%	Web scenes and contemporary imagery
RedCaps	2.0M	13.3%	Natural user-authored descriptions
JourneyDB	2.0M	13.3%	Aesthetic synthetic images and detailed prompts
WIT	1.0M	6.7%	Entity, cultural, and limited multilingual coverage

Appendix

Table 6: Composition of the 15M-pair generic text-to-image pretraining mixture. The high-quality LAION and COYO subsets are sampled before pretraining.

Method	Vis. Aes. $\uparrow$	Text Qual. $\uparrow$	Spatial Harm. $\uparrow$
AnyText	7.12	7.85	7.43
TextDiffuser-2	7.45	8.21	7.92
FLUX.1 dev	8.92	8.64	8.15
D u e t G e n	8.75	9.12	8.95

Appendix

Table 7: Human evaluation on 50 randomly sampled CVTG-2K prompts. Scores range from 1 to 10; higher is better.

Model	Sng. Obj.	Tw. Obj.	Cntng	Clrs	Pstn	Clr Attr.	Overall
D u e t G e n	0.99	0.95	0.88	0.90	0.92	0.79	0.91

Appendix

Table 8: GenEval ( Ghosh et al., 2023 ) results. Sng. Obj.: single object; Tw. Obj.: two objects; Cntng: counting; Clrs: colors; Pstn: position; Clr Attr.: color attribution.

Condition	Target shift $r$	Target IoU $\uparrow$	Center Error $\downarrow$	Word Acc. $\uparrow$	CLIPScore $\uparrow$
Clean self-generated plan	0.00	0.9176	0.068	0.8309	0.7925
Forced valid-shift plan	0.35	0.9042	0.074	0.8295	0.7941

Appendix

Table 9: Counterfactual bbox-following results on the four-region CVTG-2K subset. The forced-shift plan preserves box size, stays in bounds, and contains no overlapping pair; only coordinate-span condition tokens differ from the clean run.

Figure 9: Qualitative examples on Korean (top) and Japanese (bottom) prompts, which are underrepresented in the current training distribution. D u e t G e n preserves the overall composition and script-like visual appearance, but may produce inaccurate strokes or character identities.

Explore similar work

Large Language Models are Universal Reasoners for Visual Generation
May 5, 2026Sucheng Ren, Chen Chen, Zhenbang Wang +5Diffusion Model Guidance Image Generation
1Johns Hopkins University · 2Apple
VoT: Vision-of-Thought for Unified Multimodal Representation Alignment
Sep 7, 2026Jingxiang Sun, Chao Liao, Zhengxiong Luo +6T2I Generation Compositional T2I Generation
ByteDance Seed
Injecting Image Guidance into Text-Conditioned Diffusion Models at Inference
May 24, 2026Agata Żywot, Iason Skylitsis, Thijmen Nijdam +4Diffusion Model Guidance Test-Time Optimization
University of Amsterdam, Netherlands

May 5, 2026J/K move · Enter open · S save

Large Language Models are Universal Reasoners for Visual Generation

Sucheng Ren, Chen Chen, Zhenbang Wang, Liangchen Song+4

1Johns Hopkins University · 2Apple

Diffusion Model Guidance Image Generation Compositional T2I Generation VLM Reasoning

Open paper AskPDF arXiv

Text-to-image generation has advanced rapidly with diffusion models, progressing from CLIP and T5 conditioning to unified systems where a single LLM backbone handles both visual understanding and generation. Despite the architectural unification, these systems frequently fail to faithfully align complex prompts during synthesis, even though they remain highly accurate at verifying whether an image satisfies those same prompts. We formalize this as the understanding-generation gap and propose UniReasoner, a framework that leverages the LLM as a universal reasoner to convert its understanding strength into direct generation guidance. Given a prompt, the LLM first produces a coarse visual draft composed of discrete vision tokens. It then performs a self-critique by evaluating the draft for prompt consistency, producing a grounded textual evaluation that pinpoints what needs to be corrected. Finally, a diffusion model is conditioned jointly on the prompt, the visual draft, and the evaluation, ensuring that generation is guided by explicit corrective signals. Each signal addresses a limitation of the other: the draft provides a concrete, scene-level anchor that reduces under-specification in text-only conditioning, while the evaluation turns verification into grounded, actionable constraints that correct omissions, hallucinations, and relational errors. Experiments show that UniReasoner improves compositional alignment and semantic faithfulness under the same diffusion backbone while maintaining image quality, demonstrating a practical way to exploit LLM reasoning to close the understanding-generation gap.

Abstract

Explore similar work

Large Language Models are Universal Reasoners for Visual Generation

VoT: Vision-of-Thought for Unified Multimodal Representation Alignment

Injecting Image Guidance into Text-Conditioned Diffusion Models at Inference