cs.CVMar 26, 2026
SaveWan-Weaver: Interleaved Multi-modal Generation via Decoupled Training
Organizations: Tongyi Lab · Tsinghua University
Abstract
Recent unified models have made unprecedented progress in both understanding and generation. However, while most of them accept multi-modal inputs, they typically produce only single-modality outputs. This challenge of producing interleaved content is mainly due to training data scarcity and the difficulty of modeling long-range cross-modal context. To address this issue, we decompose interleaved generation into textual planning and visual consistency modeling, and introduce a framework consisting of a planner and a visualizer. The planner produces dense textual descriptions for visual content, while the visualizer synthesizes images accordingly. Under this guidance, we construct large-scale textual-proxy interleaved data (where visual content is represented in text) to train the planner, and curate reference-guided image data to train the visualizer. These designs give rise to Wan-Weaver, which exhibits emergent interleaved generation ability with long-range textual coherence and visual consistency. Meanwhile, the integration of diverse understanding and generation data into planner training enables Wan-Weaver to achieve robust task reasoning and generation proficiency. To assess the model's capability in interleaved generation, we further construct a benchmark that spans a wide range of use cases across multiple dimensions. Extensive experiments demonstrate that, even without access to any real interleaved data, Wan-Weaver achieves superior performance over existing methods.
Figures & tables
Figure 2: Overview of the inference process of Wan-Weaver . Given a prompt, the planner expert autoregressively generates plain text and dense prompts as visualization cues. Through causal multi-modal self-attention, the visualizer interacts with the planner, enabling it to synthesize images conditioned on the dense prompt context and visual references. The resulting text–image outputs are appended to the history and fed back into the planner, enabling an iterative interleaved generation process that maintains long-range contextual coherence.
Figure 3: Illustration of our decoupled training strategy.
| Method | Completeness | Quality | Richness | Correctness | Human Alignment | IT Coherency | Multi-step Consistency | Overall |
|---|---|---|---|---|---|---|---|---|
| NExT-GPT | 3.89 | 4.25 | 3.35 | 3.61 | 5.35 | 3.32 | 3.85 | 3.95 |
| MiniGPT-5 | 3.91 | 4.50 | 3.61 | 3.63 | 5.51 | 3.56 | 4.10 | 4.12 |
| Orthus | 4.43 | 4.30 | 3.71 | 4.15 | 4.80 | 3.51 | 4.20 | 4.16 |
| Show-o | 4.37 | 4.79 | 3.83 | 3.76 | 5.78 | 4.04 | 4.33 | 4.41 |
| VILA-U | 5.60 | 5.14 | 4.68 | 4.78 | 5.69 | 4.74 | 4.79 | 5.06 |
| SEED-LLaMA | 5.59 | 5.50 | 4.61 | 4.59 | 6.50 | 4.43 | 5.13 | 5.19 |
Table 1: Quantitative comparison with existing state-of-the-art methods on interleaved generation benchmark OpenING [ 92 ] .
Figure 4: Statistics of WeaverBench. (a) Topic distribution across 14 everyday categories. (b) Prompt length distribution. (c) Distribution of the number of images requested per prompt.
Figure 5: Qualitative comparison with the state-of-the-art commercial system Nano Banana on interleaved text–image generation.
| Method | PA | NC | CC | IC | CP | Overall |
|---|---|---|---|---|---|---|
| Orthus | 2.47 | 1.88 | 1.69 | 1.51 | 1.91 | 1.89 |
| Anole | 4.14 | 3.76 | 3.77 | 3.42 | 3.64 | 3.74 |
| Emu3.5 | 7.65 | 7.55 | 7.56 | 7.50 | 7.41 | 7.53 |
| Nano Banana | 8.53 | 8.19 | 8.53 | 8.38 | 8.29 | 8.38 |
| Wan-Weaver (Ours) | 8.71 | 8.33 | 8.50 | 8.13 | 8.46 | 8.43 |
Table 2: Quantitative comparison on WeaverBench. PA: Prompt Adherence, NC: Narrative Coordination, CC: Content Consistency, IC: Image Consistency, CP: Completeness.
| Model | Understanding | Image Generation | Image Editing | |||
|---|---|---|---|---|---|---|
| MMMU | MathVista | GenEval | DPG | ImgEdit | GEdit–EN | |
| InternVL3-38B | 69.7 | 76.3 | – | – | – | – |
| Ovis2-34B | 66.7 | 76.1 | – | – | – | – |
| Qwen2.5-VL-32B † | 75.1 | 84.7 | – | – | – | – |
| FLUX.1-dev | – | – | 0.66 | 84.0 | – | – |
| Step1X-Edit | – | – | – | – | 3.06 | 6.70 |
Table 3: Comparison across single-modality generation tasks (understanding, image generation, and editing). † : Our in-house base model with thinking mode (enabled only for understanding).
Figure 6: Loss curves of different training strategies.
Figure 7: Impact of training on different types of data on our planner. (left) Performance on understanding metrics. (right) Token prediction statistics per task.
Figure 8: Visual comparison of the results generated by different variants of our method.
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
| Category | Encyclopedic Knowledge | News Media | Travel Guide | Daily Life | Food Cooking |
|---|---|---|---|---|---|
| #Data Samples | 35 (16+19) | 35 (16+19) | 36 (17+19) | 35 (16+19) | 35 (16+19) |
| Category | Education Tutorial | Art Creation | Fashion Beauty | Media Entertainment | Product Marketing |
| #Data Samples | 36 (17+19) | 35 (16+19) | 36 (17+19) | 34 (16+18) | 35 (16+19) |
| Category | Academic Research | Document Guides | Social Community | Reasoning | Multi-language |
| #Data Samples | 35 (17+18) | 36 (17+19) | 29 (15+14) | 32 (16+16) | 28 (28+0) |
Table 4: Distribution of data samples across the 15 categories in the WeaverBench benchmark. ( The number before ‘+’ represents the amount for text-only input, and the number after ‘+’ represents the amount for image-text input.)
| Method | PA | NC | CC | IC | CP | Overall | Acc | |||
|---|---|---|---|---|---|---|---|---|---|---|
| Intra-output | Input-to-image | Input-to-text | Entity | Style | ||||||
| Orthus | 2.47 | 1.88 | 1.42 | 1.30 | 2.35 | 1.19 | 1.82 | 1.91 | 1.89 | 3.28% |
| Anole | 4.14 | 3.76 | 3.56 | 3.31 | 4.44 | 3.00 | 3.83 | 3.64 | 3.74 | 6.56% |
| Emu3.5 | 7.65 | 7.55 | 7.42 | 7.41 | 7.86 | 7.22 | 7.78 | 7.41 | 7.53 | 48.36% |
| Nano Banana | 8.53 | 8.19 | 8.31 | 8.57 | 8.70 | 8.09 | 8.67 | 8.29 | 8.38 | 66.39 % |
| Wan-Weaver (Ours) | 8.71 | 8.33 | 8.26 | 8.41 | 8.83 | 7.85 | 8.41 | 8.46 | 8.43 | 93.44 % |
Table 5: Quantitative comparison on WeaverBench. PA: Prompt Adherence, NC: Narrative Coordination, CC: Content Consistency, IC: Image Consistency, CP: Completeness, Acc: Accuracy of the number of the generated images.
| Model | Understanding | Image Generation | Image Editing | |||
| MMMU | MathVista | GenEval | DPG | ImgEdit | GEdit–EN | |
| Visual Understanding Models | ||||||
| LLaVA-1.5 [ 44 ] | 67.8 | – | – | – | – | – |
| LLaVA-NeXT [ 43 ] | 51.1 | 39.6 | – | – | – | – |
| InternVL3-38B [ 93 ] | 69.7 | 76.3 | – | – | – | – |
| Ovis2-34B [ 47 ] | 66.7 | 76.1 | – | – | – | – |
Table 6: Comparison across single-modality generation tasks (understanding, image generation, and editing). † : Our in-house base model with thinking mode (enabled only for understanding). ‡ : Methods are using LLM rewriter.
Figure 9: Impact of training on different types of data on our planner. (left) Performance on understanding metrics. (right) Token prediction statistics per task.
Figure 10: Visual examples of failure cases.
Figure 11: Showcase of Wan-Weaver in general text-to-image generation.
Figure 12: Showcase of Wan-Weaver in general image-to-image generation, including object addition, removal, replacement, element extraction, pose manipulation, text editing, style/texture transfer, novel view synthesis, object detection/segmentation, and more.
Figure 13: Showcase of Wan-Weaver in interleaved text-image generation.
Figure 14: Examples of test samples from WeaverBench. For each category, we show one text-only user prompt and one mixed text–image user prompt (Part I).
Figure 15: Examples of test samples from WeaverBench. For each category, we show one text-only user prompt and one mixed text–image user prompt (Part II).
Figure 16: Examples of test samples from WeaverBench. For each category, we show one text-only user prompt and one mixed text–image user prompt (Part III).
| Method | GPT Evaluation | IntJudge Evaluation | ||||||
|---|---|---|---|---|---|---|---|---|
| FDT | w/o Tie | w/ Tie (0) | w/ Tie (.5) | FDT | w/o Tie | w/ Tie (0) | w/ Tie (.5) | |
| NExT-GPT | 15.70% | 15.60% | 15.36% | 16.13% | 44.86% | 32.61% | 2.57% | 48.63% |
| GILL | 28.27% | 28.29% | 27.75% | 28.71% | 34.73% | 6.52% | 0.52% | 46.51% |
| MiniGPT-5 | 30.20% | 29.60% | 28.84% | 30.12% | 31.97% | 9.52% | 1.03% | 45.64% |
| Show-o | 32.55% | 32.10% | 31.46% | 32.46% | 39.42% | 13.16% | 0.90% | 47.47% |
| Orthus | 34.63% | 34.20% | 33.45% | 34.54% | 43.05% | 17.50% | 1.19% | 47.80% |
Table 7: Quantitative comparison of model win rates evaluated by GPT-4o and IntJudge. FDT: Force Dividing Tie metric. w/o Tie: Non-Tie case. w/ Tie (0) and w/ Tie Tie(.5): Count a tie as 0 and 0.5 wins for a model in a battle, respectively.
Figure 17: Examples of short, medium, and long user prompts in WeaverBench.
Table 8: The system prompts for obtaining detailed scores from GPT-based evaluators (Part I).
Table 9: The system prompts for obtaining detailed scores from GPT-based evaluators (Part II).
Table 10: The system prompts for obtaining detailed scores from GPT-based evaluators (Part III).
Table 11: The system prompts for obtaining detailed scores from GPT-based evaluators (Part IV).