We present "Narrative Weaver", a novel framework that addresses a fundamental challenge in generative AI: achieving multi-modal controllable, long-range, and consistent visual content generation. While existing models excel at generating high-fidelity short-form visual content, they struggle to maintain narrative coherence and visual consistency across extended sequences - a critical limitation for real-world applications such as filmmaking and e-commerce advertising. Narrative Weaver introduces the first holistic solution that seamlessly integrates three essential capabilities: fine-grained control, automatic narrative planning, and long-range coherence. Our architecture combines a Multimodal Large Language Model (MLLM) for high-level narrative planning with a novel fine-grained control module featuring a dynamic Memory Bank that prevents visual drift. To enable practical deployment, we develop a progressive, multi-stage training strategy that efficiently leverages existing pre-trained models, achieving state-of-the-art performance even with limited training data. Recognizing the absence of suitable evaluation benchmarks, we construct and release the E-commerce Advertising Video Storyboard Dataset (EAVSD) - the first comprehensive dataset for this task, containing over 330K high-quality images with rich narrative annotations. Through extensive experiments across three distinct scenarios (controllable multi-scene generation, autonomous storytelling, and e-commerce advertising), we demonstrate our method's superiority while opening new possibilities for AI-driven content creation.
Figures & tables
Figure 1: Narrative Weaver Overview. (a) Narrative Weaver Framework: This system utilizes a hybrid design that integrates Autoregressive (AR) and Diffusion models. The bottom panel illustrates a Multimodal Large Language Model (MLLM) acting as the AR model, responsible for generating narrative plans in textual form and encoding historical information into learnable queries. During the diffusion generation stage, a dynamic memory bank encodes initial conditions and prior outputs to prevent visual content drift. (b) Memory Bank: We employ a series-based decay of prior visual feature length to ensure a bounded total memory length. (c) Attention Mask: A specially designed Attention Mask ensures efficient training, where gray areas are ignored during processing.
Figure 2: Qualitative results of consistent visual generation. (a) Narrative Weaver produces visually coherent frames that preserve both stylistic and semantic alignment with the given prompts, while effectively advancing the cinematic story progression. (b) Our model maintains environmental consistency conditioned on the input image and achieves more natural visual transitions compared to other methods.
Method
Capability
GPT-4o Score ↑
Text.
Ctrl.
ITC
RGC
MSSC
MSCC
IMQ
TALC [ 3 ]
✗
✗
2.87
1.86
6.94
5.81
3.20
StoryDiffusion [ 78 ]
✗
✗
6.54
5.86
7.48
6.00
6.80
IP-Adapter [ 73 ]
✗
✓
7.11
6.10
8.57
7.57
6.65
AnimeShooter [ 49 ]
✗
✓
2.80
2.39
4.98
4.19
4.24
Flux.1-kontext [ 4 ]
✗
✓
7.06
9.41
8.11
7.28
6.94
Table 1: GPT-4o Evaluation of Consistent Visual Generation.
Figure 3: Flux.1-Kontext tend to exhibit “copy–paste” behavior when failing to interpret instructions, resulting in a misleading appearance of high consistency.
Figure 4: User Study Results: Model Preference Distribution. The results were aggregated from over 180 responses, each representing user’s selection of the most preferred output.
Figure 5: Qualitative results of autonomous narrative planning. Narrative Weaver demonstrates a dual capability: maintaining robust visual consistency while also employing fundamental cinematic techniques. The figure showcases examples where the model autonomously plans and generates contextually appropriate subsequent shots that adhere to standard conventions, including cut-ins for detail, cross-cuts for parallel action, and so on.
Method
Question-based Generation
Style
Entity
Trend
CPL.
ImgQ
IRS
MiniGPT-5 [ 76 ]
6.62
5.96
5.83
6.26
5.67
2.73
SEED-Llama-8B [ 16 ]
7.35
5.00
3.85
5.07
5.21
2.57
SEED-Llama-14B [ 16 ]
7.00
6.28
5.71
6.02
5.85
3.28
Qwen2.5-VL + Flux.1-Dev
5.39
4.89
4.69
4.79
4.73
2.98
Narrative Weaver (Ours)
7.10
6.49
6.67
7.14
6.54
2.77
Table 3: GPT-4o evaluation on the autonomous narrative planning capability of Narrative Weaver (Q2).
Figure 6: Visual Scenery Planning for E-commerce Video Ads. Visualization results demonstrate Narrative Weaver’s capability in generating consistent keyframe sequences with precise scene composition for e-commerce scenarios.
stage2
stage3
ITC
RGC
MSSC
MSCC
IMQ
Avg.
✗
✗
5.99
6.09
8.19
7.22
8.12
7.12
✓
✗
6.05
6.78
8.59
7.73
8.20
7.47
✗
✓
6.19
8.53
8.84
8.28
8.18
8.00
✓
✓
6.39
8.68
8.97
8.34
8.14
8.10
Table 4: Ablation study validating the contributions of Stage 2 (semantic coherence) and Stage 3 (fine-grained control).
Figure 7: The prompt for Qwen3-30B-A3B to generate text instructions for reference image generation.
Figure 8: Prompt Template for Qwen3-30B-A3B in Generating Keyframe Scenario Recommendations.
Figure 9: The prompt for Qwen3-30B-A3B to generate text instructions for Flux.1-kontext image generation.
Figure 10: Sample sequences from our EAVSD dataset. The figure showcases the dataset’s diversity across multiple e-commerce categories. Each row displays a sequence where a consistent subject and product are placed in various scenes, guided by descriptive text prompts. The dataset is designed to train models on tasks requiring high visual consistency while allowing for controlled narrative changes in action and setting, which is critical for advertising applications.
Figure 11: The prompt template provided to GPT-4o for our consistency evaluation. It requires the model to score instruction following, consistency, and image quality, and to provide a rationale for each score.
Hyperparameters
Stage-1
Stage-2.1
Stage-2.2
Stage-3
Learning rate
1×10−5
5×10−5
1×10−5
1×10−6
LR scheduler
Constant
Cosine
Constant
Constant
Weight decay
0.0
0.0
0.0
0.0
Gradient norm clip
1.0
1.0
1.0
1.0
Optimizer
AdamW ( β1=0.9,β2=0.99,ϵ=1×10−8 )
Loss type
CE
MSE
MSE
MSE
Table 5: Training recipe of Narrative Weaver.
Figure 12: Additional qualitative results of multi-frame narrative generation by Narrative Weaver. Each row showcases a complete generation sequence. The leftmost column presents the user’s initial input (a reference image and its description). The subsequent columns display the multi-frame visual narrative autonomously generated by our model, including both the synthesized images and their corresponding textual descriptions. These diverse examples highlight Narrative Weaver’s proficiency in maintaining high cross-frame consistency—preserving subject identity, apparel, and key background elements—while coherently evolving the narrative through subtle changes in pose, expression, and camera perspective.
Figure 13: Comparison with leading image editing models on multi-frame narrative generation. While all models exhibit strong adherence to the text prompts and maintain high subject consistency, Narrative Weaver uniquely generates outputs with a more cinematic and aesthetically pleasing quality . Note our model’s superior handling of lighting, color, and composition, which contributes to a more authentic, film-like visual narrative compared to the often more literal or digitally rendered feel of the baseline results.
Figure 14: Qualitative examples of autonomous story continuation by Narrative Weaver. Given only the first frame and text as input for each sequence, Narrative Weaver autonomously plans and generates a coherent multi-frame continuation. The diverse examples—from procedural tasks like cooking to dynamic events like sports—showcase the model’s robust planning capabilities. Its ability to logically advance a narrative by continuing actions, introducing new elements, or shifting focus highlights its understanding of storytelling beyond simple image editing.
Figure 15: Comparison with a leading editing model, Qwen-Image-Edit. Narrative Weaver not only demonstrates superior stylistic consistency with the conditional image but also excels in preserving key semantic details required by the task. In contrast, Qwen-Image-Edit exhibits noticeable failure modes: it struggles with inconsistent color tones between frames (upper example) and introduces a warm color cast that deviates from the style of the reference image (lower example).
Figure 16: Ablation Study Results. Comparison between Narrative Weaver and its variant without Stage 3 training demonstrates that: (1) the first two stages establish fundamental semantic alignment; (2) Stage 3 significantly enhances inter-frame consistency; and (3) Stage 3 is crucial for imparting fine-grained control, as the variant without it produces images that deviate significantly from the reference.
Implementation
Keyframe Num
1
2
3
4
5
6
7
8
9
10
11
12
16
20
Vanilla
TFLOPs
82
230
450
744
1112
1553
2068
2656
3318
4053
4862
5744
10008
15449
Ours
82
165
248
331
441
497
580
663
746
829
912
995
1077
1160
Table 6: Computational cost (TFLOPs) as a function of the number of generated keyframes. Our approach demonstrates significantly better scalability compared to a vanilla self-attention baseline. The computational cost of our method grows linearly with the sequence length, unlike the vanilla implementation, whose cost grows quadratically, making our method far more efficient for longer sequences.
Long-form visual storytelling requires maintaining continuity across shots, including consistent characters, stable environments, and smooth scene transitions. While existing generative models can produce strong individual frames, they fail to preserve such continuity, leading to appearance changes, inconsistent backgrounds, and abrupt scene shifts. We introduce CANVAS (Continuity-Aware Narratives via Visual Agentic Storyboarding), a multi-agent framework that explicitly plans visual continuity in multi-shot narratives. CANVAS enforces coherence through character continuity, persistent background anchors, and location-aware scene planning for smooth transitions within the same setting We evaluate CANVAS on two storyboard generation benchmarks ST-BENCH and ViStoryBench and introduce a new challenging benchmark HardContinuityBench for long-range narrative consistency. CANVAS consistently outperforms the best-performing baseline, improving background continuity by 21.6%, character consistency by 9.6% and props consistency by 7.6%.
Recent unified models have made unprecedented progress in both understanding and generation. However, while most of them accept multi-modal inputs, they typically produce only single-modality outputs. This challenge of producing interleaved content is mainly due to training data scarcity and the difficulty of modeling long-range cross-modal context. To address this issue, we decompose interleaved generation into textual planning and visual consistency modeling, and introduce a framework consisting of a planner and a visualizer. The planner produces dense textual descriptions for visual content, while the visualizer synthesizes images accordingly. Under this guidance, we construct large-scale textual-proxy interleaved data (where visual content is represented in text) to train the planner, and curate reference-guided image data to train the visualizer. These designs give rise to Wan-Weaver, which exhibits emergent interleaved generation ability with long-range textual coherence and visual consistency. Meanwhile, the integration of diverse understanding and generation data into planner training enables Wan-Weaver to achieve robust task reasoning and generation proficiency. To assess the model's capability in interleaved generation, we further construct a benchmark that spans a wide range of use cases across multiple dimensions. Extensive experiments demonstrate that, even without access to any real interleaved data, Wan-Weaver achieves superior performance over existing methods.
Unified multimodal models hold the promise of generating extensive, interleaved narratives, weaving text and imagery into coherent long-form stories. However, current systems suffer from a critical reliability gap: as sequences grow, generation quality rapidly collapses. In this work, we investigate the mechanism behind this failure and argue that it is distinct from standard long-context challenges. We reveal that in generation, accumulated visual history acts as a source of active pollution, a decay governed specifically by the number of image events rather than raw token count. We identify a structural vulnerability where dense visual tokens overwhelm the attention mechanism, creating noise that distorts future synthesis. Guided by these mechanistic insights, we propose UniLongGen, a training-free inference strategy that prioritizes safe conditioning over total recall. Instead of retaining all history, UniLongGen dynamically curates the model's memory, identifying and discarding interfering visual signals based on the model's own internal relevance rankings. Extensive experiments demonstrate that this active forgetting approach is essential for stability: UniLongGen significantly outperforms baselines in long-horizon fidelity and consistency, while simultaneously reducing memory footprint and inference time.
Haoyu Chen, Qing Liu, Yuqian Zhou +7
The Hong Kong University of Science and Technology (Guangzhou) · Adobe Research · Huazhong University of Science and Technology +1