Synergizing Understanding and Generation with Interleaved Analyzing-Drafting Thinking

Shengqiong Wu; Bobo Li; Xinkai Wang; Xiangtai Li; Lei Cui; Furu Wei; Shuicheng Yan; Hao Fei

cs.CVFeb 24, 2026

Synergizing Understanding and Generation with Interleaved Analyzing-Drafting Thinking

Authors: Shengqiong Wu, Bobo Li, Xinkai Wang, Xiangtai Li, Lei Cui, Furu Wei, Shuicheng Yan, Hao Fei,

Organizations: National University of Singapore · University of Oxford · Nanyang Technological University · Microsoft Research

Abstract

Unified Vision-Language Models (UVLMs) aim to advance multimodal learning by supporting both understanding and generation within a single framework. However, existing approaches largely focus on architectural unification while overlooking the need for explicit interaction between the two capabilities during task solving. As a result, current models treat understanding and generation as parallel skills rather than synergistic processes. To achieve real synergy, we introduce the interleaved Analyzing-Drafting problem-solving loop (AD-Loop), a new think paradigm that dynamically alternates between analytic and drafting operations. By interleaving textual thoughts with visual thoughts, AD-Loop enables models to iteratively refine both comprehension and outputs, fostering genuine synergy. To train this mechanism, we design a two-stage strategy: supervised learning on interleaved thought data to initialize alternation, followed by reinforcement learning to promote adaptive and autonomous control. Extensive experiments demonstrate that AD-Loop consistently improves performance across standard benchmarks for both understanding and generation, with strong transferability to various UVLMs architectures. Visual analyses further validate the effectiveness of implicit visual thoughts. These results highlight AD-Loop as a principled and broadly applicable strategy for synergizing comprehension and creation. The project page is at https://sqwu.top/AD-Loop.

Figures & tables

Figure 1: Illustration of the interleaved analytic–drafting problem-solving loop, where understanding and generation interact synergistically to yield accurate solutions.

Figure 2: Pipeline of our training framework. Stage-1 : train the UVLM to emit interleaved thinking traces. Stage-2 : apply GRPO for hybrid reasoning. The policy samples multiple traces with/without the interleaved AD-Loop for each input. A reward model scores outcomes, and then group-normalized advantages are applied to update the policy, teaching the model when AD-Loop helps.

Model	#Params	POPE $\uparrow$	MME-P $\uparrow$	MMB $\uparrow$	SEED $\uparrow$	GQA $\uparrow$	MMMU $\uparrow$	MM-Vet $\uparrow$
$\bullet$ Und. Only
LLaVA-v1.5 ( Liu et al., 2024a )	7B	85.9	1510.7	64.3	58.6	62.0	35.4	31.1
Qwen-VL-Chat ( Bai et al., 2023 )	7B	-	1487.5	60.6	58.2	57.5	–	–
IDEFICS ( Laurençon et al., 2023 )	8B	-	-	48.2	-	38.4	-	-
InstructBLIP ( Dai et al., 2023 )	13B	78.9	1212.8	-	-	49.5	–	25.6
$\bullet$ Und. and Gen.

Table 1: Comparison with baselines on multimodal understanding benchmarks. “Und.” and “Gen.” denote understanding and generation , respectively.

Model	Single Obj. $\uparrow$	Two Obj. $\uparrow$	Counting $\uparrow$	Colors $\uparrow$	Position $\uparrow$	Attri. $\uparrow$	Overall $\uparrow$
$\bullet$ Gen. Only
Emu3-Gen ( Wang et al., 2024 )	0.98	0.71	0.34	0.81	0.17	0.21	0.54
SDXL ( Podell et al., 2023 )	0.98	0.74	0.39	0.85	0.15	0.23	0.55
FLUX.1-dev ( Yang et al., 2024 )	0.99	0.81	0.79	0.74	0.20	0.47	0.67
SD3-Medium ( AI, 2025 )	0.99	0.94	0.72	0.89	0.33	0.60	0.74
$\bullet$ Und. and Gen.

Table 2: Comprehensive generation comparison on GenEval ( Ghosh et al., 2023 ) benchmark. “Und.” and “Gen.” denote understanding and generation , respectively.

Think Strategy	Understanding			Generation (WISE)
Think Strategy	MathVista $\uparrow$	LogicVista $\uparrow$	SAT $\uparrow$	Cultural $\uparrow$	Space $\uparrow$	Biology $\uparrow$
Isolated thinking	61.5	40.2	0.63	0.44	0.68	0.44
$T$	68.3	44.1	0.74	0.67	0.69	0.56
$T+I$ ( explicit )	72.9	46.6	0.81	0.73	0.74	0.64
$T+\widetilde{I}$ ( implicit )	73.6	47.2	0.84	0.75	0.77	0.65
$T$ / $T+\widetilde{I}$	75.8	49.5	0.89	0.79	0.78	0.68

Table 3: Comparison with different thinking strategies. Isolated thinking : understanding or generation performed in isolation;

T

: analyzing-only thinking;

T+I

(explicit) : supervised textual–visual (explicit) interleaving;

T+\widetilde{I}

(implicit) : learned textual–visual (implicit) interleaving;

T

T+\widetilde{I}

: adaptive interleaved thinking that automatically selects with or without interleaved thinking.

Figure 3: Qualitative comparison: original prompt (left), self-think mode, interleaved thoughts, and text-only thoughts filtered from the interleaved thoughts (right). $[$ V-T $]$ means latent visual thoughts.

Figure 4: Absolute performance gain after applying AD-Loop, comparing Janus-Pro with discrete tokenization and BAGEL with continuous embedding for visual thoughts learning.

	MMstar	MathVista	LogicVista	GenEval	WISE (cultural)	WISE (Biology)
From Gen. Encoder	54.9	75.8	47.5	0.86	0.79	0.68
From Und. Encoder	51.6	70.9	44.3	0.84	0.71	0.61

Table 4: Comparison of latent visual representations learn by the generation (Gen.) vs. understanding (Und.) encoder.

Figure 5: Examples of latent visual thoughts. Each case shows the original image (left) and the corresponding visual thoughts (right), capturing abstract visual structures.

Figure 6: Performance across skills and capabilities on the LogicVista dataset, comparing models with and without visual thoughts, alongside the proportion of visual-thought usage.

Figure 7: Comparison of unified MLLMs on T2I generation. Existing models often fail on prompts requiring deeper reasoning, whereas our approach yields more faithful outcomes.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Comparison of existing mechanisms for synergizing understanding and generation, including isolated learning where the two abilities are trained independently, dual learning which leverages cross-modal reconstruction for mutual supervision, and co-learning which jointly optimizes both tasks with paired samples.

Figure 9: Visualization of the constructed dataset. Given a user query, we establish an analyzing–drafting problem-solving loop in which textual and visual thoughts alternate dynamically, ultimately yielding the final answer.

Task	Dataset	Avg.	Stage-1	Stage-2	Total
Understanding	CMoT ( Cheng et al., 2025b )	3	1K	1K	2K
	Visual-CoT ( Shao et al., 2024 )	2	12K	1K	13K
	CoF ( Zhang et al., 2025b )	3	4K	1K	5K
	VSP ( Yang et al., 2025b )	3	3K	1K	4K
Generation	GoT-T2I ( Fang et al., 2025 )	4	12K	3K	15K
Generation	Inter-T2I	4	10K	2K	12K

Appendix

Table 5: The statistics of the constructed dataset for two-stage learning. ‘Avg.’ indicates the average number of latent visual thoughts within the datasets.

Type	Model	Level 3	Level 6
Explicit	Anole ( Chern et al., 2025 )	0.02	0.00
Explicit	MVoT ( Li et al., 2025 )	0.21	0.03
Implicit	Mirage ( Yang et al., 2025b )	0.75	0.39
Implicit	Ours	0.81	0.47

Appendix

Table 6: Performance comparison of explicit and latent visual thoughts on the spatial visual planning task at levels 3 and 6.

Figure 10: Qualitative comparison across baseline unified MLLMs on multimodal understanding. Correct answers are marked in green , errors in red , and $[$ V-T $]$ denotes latent visual thoughts.

Figure 11: Understanding Comparison across UVLMs on the math question answering task. Correct answers are marked in green , errors in red , and $[$ V-T $]$ denotes latent visual thoughts.

Figure 12: Understanding Comparison across UVLMs on multimodal content reasoning task. Correct answers are marked in green , errors in red , and $[$ V-T $]$ denotes latent visual thoughts.

Figure 13: Generation Comparison across UVLMs.

Figure 14: Generation Comparison on MindOmni ( Xiao et al., 2025b ) and BAGEL ( Deng et al., 2025 ) with/without Think process.Our proposed method achieves more accurate generation results.

Figure 15: Editing Comparison on BAGEL ( Deng et al., 2025 ) with/without think process.

Figure 16: Failure cases illustrating over-drafting on simple tasks, as well as challenges on problems that require strictly logical mathematical reasoning.

Figure 17: Failure cases showing that the proposed method still struggles with rendering scene text, as well as fine-grained body details and numerically precise content.

VLM Reasoning

Vision-language model reasoning studies how multimodal systems use visual evidence and language to answer questions, plan, and explain.

See all 853 papers

Reversing the Flow: Generation-to-Understanding Synergy in Large Multimodal Models
May 15, 2026Yujun Tong, Dongliang Chang, Zijin Yin +3Unified Multimodal Models Multimodal Reasoning
Beijing University of Posts and Telecommunications · Beijing Key Laboratory of Multimodal Data Intelligent Perception and Governance
Uncovering Understanding-Generation Synergy in Native Unified Multimodal Models: From Representation, Task to System
Sep 1, 2026Penghao Wu, Haiwen Diao, Weichen Fan +3Unified Multimodal Models Multimodal Generation
1S-Lab, Nanyang Technological University · 2SenseTime Research
Towards Unified Multimodal Interleaved Generation via Group Relative Policy Optimization
Mar 10, 2026Ming Nie, Chunwei Wang, Jianhua Han +2Multimodal Generation Unified Multimodal Models
School of Data Science, Fudan University · Noah’s Ark Lab, Huawei · Yinwang Intelligent Technology Co., Ltd.

May 15, 2026J/K move · Enter open · S save

Reversing the Flow: Generation-to-Understanding Synergy in Large Multimodal Models

Yujun Tong, Dongliang Chang, Zijin Yin, Xintong Liu+2

Beijing University of Posts and Telecommunications · Beijing Key Laboratory of Multimodal Data Intelligent Perception and Governance

Unified Multimodal Models Multimodal Reasoning Multimodal Generation Multimodal Foundation Models Multimodal Learning Multimodal Understanding

Open paper AskPDF arXiv

The long-standing goal of multimodal AI is to build unified models in which visual understanding and visual generation mutually enhance one another. Despite recent works such as BAGEL, BLIP3o achieves remarkable progress; In practice, however, this unification remains one-directional: understanding routinely guides generation, yet how and why generation can support understanding is rarely investigated. We revisit this asymmetry and propose Generation-to-Understanding (G2U) synergy, where visual generation becomes an explicit intermediate reasoning step. Our framework enables a model to perform controlled generative acts, such as detail enhancement, context expansion or structural visualisation, to produce self-generated visual thoughts, which are then fed back into the model to refine perception without retraining or external tools. Through a comprehensive evaluation on twelve benchmarks, this reversed information flow consistently improves multimodal understanding. We show that generative fidelity bounds perceptual gain and that distinct families of edit prompts govern transfer efficiency. We further analyse whether models can decide what to imagine. While they can produce plausible edits, these self-generated visual thoughts lack stable task alignment, revealing that current large multimodal models fall short of true self-reflection. This work exposes a missing mechanism in unified cognition and suggests that imagination is not the end of understanding but its beginning.

Abstract

VLM Reasoning

Explore similar work

Reversing the Flow: Generation-to-Understanding Synergy in Large Multimodal Models

Uncovering Understanding-Generation Synergy in Native Unified Multimodal Models: From Representation, Task to System

Towards Unified Multimodal Interleaved Generation via Group Relative Policy Optimization