cs.CVMar 4, 2026
SaveNarrative Weaver: Towards Controllable Long-Range Visual Consistency with Multi-Modal Conditioning
Organizations: Peking University · Kuaishou Technology
Abstract
We present "Narrative Weaver", a novel framework that addresses a fundamental challenge in generative AI: achieving multi-modal controllable, long-range, and consistent visual content generation. While existing models excel at generating high-fidelity short-form visual content, they struggle to maintain narrative coherence and visual consistency across extended sequences - a critical limitation for real-world applications such as filmmaking and e-commerce advertising. Narrative Weaver introduces the first holistic solution that seamlessly integrates three essential capabilities: fine-grained control, automatic narrative planning, and long-range coherence. Our architecture combines a Multimodal Large Language Model (MLLM) for high-level narrative planning with a novel fine-grained control module featuring a dynamic Memory Bank that prevents visual drift. To enable practical deployment, we develop a progressive, multi-stage training strategy that efficiently leverages existing pre-trained models, achieving state-of-the-art performance even with limited training data. Recognizing the absence of suitable evaluation benchmarks, we construct and release the E-commerce Advertising Video Storyboard Dataset (EAVSD) - the first comprehensive dataset for this task, containing over 330K high-quality images with rich narrative annotations. Through extensive experiments across three distinct scenarios (controllable multi-scene generation, autonomous storytelling, and e-commerce advertising), we demonstrate our method's superiority while opening new possibilities for AI-driven content creation.
Figures & tables
Figure 1: Narrative Weaver Overview. (a) Narrative Weaver Framework: This system utilizes a hybrid design that integrates Autoregressive (AR) and Diffusion models. The bottom panel illustrates a Multimodal Large Language Model (MLLM) acting as the AR model, responsible for generating narrative plans in textual form and encoding historical information into learnable queries. During the diffusion generation stage, a dynamic memory bank encodes initial conditions and prior outputs to prevent visual content drift. (b) Memory Bank: We employ a series-based decay of prior visual feature length to ensure a bounded total memory length. (c) Attention Mask: A specially designed Attention Mask ensures efficient training, where gray areas are ignored during processing.
Figure 2: Qualitative results of consistent visual generation. (a) Narrative Weaver produces visually coherent frames that preserve both stylistic and semantic alignment with the given prompts, while effectively advancing the cinematic story progression. (b) Our model maintains environmental consistency conditioned on the input image and achieves more natural visual transitions compared to other methods.
| Method | Capability | GPT-4o Score | |||||
| Text. | Ctrl. | ITC | RGC | MSSC | MSCC | IMQ | |
| TALC [ 3 ] | ✗ | ✗ | 2.87 | 1.86 | 6.94 | 5.81 | 3.20 |
| StoryDiffusion [ 78 ] | ✗ | ✗ | 6.54 | 5.86 | 7.48 | 6.00 | 6.80 |
| IP-Adapter [ 73 ] | ✗ | ✓ | 7.11 | 6.10 | 8.57 | 7.57 | 6.65 |
| AnimeShooter [ 49 ] | ✗ | ✓ | 2.80 | 2.39 | 4.98 | 4.19 | 4.24 |
| Flux.1-kontext [ 4 ] | ✗ | ✓ | 7.06 | 9.41 | 8.11 | 7.28 | 6.94 |
Table 1: GPT-4o Evaluation of Consistent Visual Generation.
Figure 3: Flux.1-Kontext tend to exhibit “copy–paste” behavior when failing to interpret instructions, resulting in a misleading appearance of high consistency.
Figure 4: User Study Results: Model Preference Distribution. The results were aggregated from over 180 responses, each representing user’s selection of the most preferred output.
| Method | Shot-level | Story-level | |||
| shot-1 | shot-2 | shot-3 | shot-4 | Avg. | |
| DreamSim | |||||
| TALC [ 3 ] | 83.60 | 84.03 | 88.32 | 89.38 | 83.87 |
| StoryDiffusion [ 78 ] | 54.63 | 57.35 | 59.06 | 58.90 | 56.33 |
| IP-Adapter [ 73 ] | 33.03 | 33.57 | 34.07 | 34.83 | 33.30 |
| AnimeShooter [ 49 ] | 75.89 | 71.49 | 71.87 | 70.29 | 73.14 |
Table 2: Automated Evaluation (CLIP Score / DreamSim) of Consistent Keyframe Generation (Q1).
Figure 5: Qualitative results of autonomous narrative planning. Narrative Weaver demonstrates a dual capability: maintaining robust visual consistency while also employing fundamental cinematic techniques. The figure showcases examples where the model autonomously plans and generates contextually appropriate subsequent shots that adhere to standard conventions, including cut-ins for detail, cross-cuts for parallel action, and so on.
| Method | Question-based Generation | |||||
| Style | Entity | Trend | CPL. | ImgQ | IRS | |
| MiniGPT-5 [ 76 ] | 6.62 | 5.96 | 5.83 | 6.26 | 5.67 | 2.73 |
| SEED-Llama-8B [ 16 ] | 7.35 | 5.00 | 3.85 | 5.07 | 5.21 | 2.57 |
| SEED-Llama-14B [ 16 ] | 7.00 | 6.28 | 5.71 | 6.02 | 5.85 | 3.28 |
| Qwen2.5-VL + Flux.1-Dev | 5.39 | 4.89 | 4.69 | 4.79 | 4.73 | 2.98 |
| Narrative Weaver (Ours) | 7.10 | 6.49 | 6.67 | 7.14 | 6.54 | 2.77 |
Table 3: GPT-4o evaluation on the autonomous narrative planning capability of Narrative Weaver (Q2).
Figure 6: Visual Scenery Planning for E-commerce Video Ads. Visualization results demonstrate Narrative Weaver’s capability in generating consistent keyframe sequences with precise scene composition for e-commerce scenarios.
| stage2 | stage3 | ITC | RGC | MSSC | MSCC | IMQ | Avg. |
| ✗ | ✗ | 5.99 | 6.09 | 8.19 | 7.22 | 8.12 | 7.12 |
| ✓ | ✗ | 6.05 | 6.78 | 8.59 | 7.73 | 8.20 | 7.47 |
| ✗ | ✓ | 6.19 | 8.53 | 8.84 | 8.28 | 8.18 | 8.00 |
| ✓ | ✓ | 6.39 | 8.68 | 8.97 | 8.34 | 8.14 | 8.10 |
Table 4: Ablation study validating the contributions of Stage 2 (semantic coherence) and Stage 3 (fine-grained control).
Figure 7: The prompt for Qwen3-30B-A3B to generate text instructions for reference image generation.
Figure 8: Prompt Template for Qwen3-30B-A3B in Generating Keyframe Scenario Recommendations.
Figure 9: The prompt for Qwen3-30B-A3B to generate text instructions for Flux.1-kontext image generation.
Figure 10: Sample sequences from our EAVSD dataset. The figure showcases the dataset’s diversity across multiple e-commerce categories. Each row displays a sequence where a consistent subject and product are placed in various scenes, guided by descriptive text prompts. The dataset is designed to train models on tasks requiring high visual consistency while allowing for controlled narrative changes in action and setting, which is critical for advertising applications.
Figure 11: The prompt template provided to GPT-4o for our consistency evaluation. It requires the model to score instruction following, consistency, and image quality, and to provide a rationale for each score.
| Hyperparameters | Stage-1 | Stage-2.1 | Stage-2.2 | Stage-3 |
| Learning rate | ||||
| LR scheduler | Constant | Cosine | Constant | Constant |
| Weight decay | 0.0 | 0.0 | 0.0 | 0.0 |
| Gradient norm clip | 1.0 | 1.0 | 1.0 | 1.0 |
| Optimizer | AdamW ( ) | |||
| Loss type | CE | MSE | MSE | MSE |
Table 5: Training recipe of Narrative Weaver.
Figure 12: Additional qualitative results of multi-frame narrative generation by Narrative Weaver. Each row showcases a complete generation sequence. The leftmost column presents the user’s initial input (a reference image and its description). The subsequent columns display the multi-frame visual narrative autonomously generated by our model, including both the synthesized images and their corresponding textual descriptions. These diverse examples highlight Narrative Weaver’s proficiency in maintaining high cross-frame consistency—preserving subject identity, apparel, and key background elements—while coherently evolving the narrative through subtle changes in pose, expression, and camera perspective.
Figure 13: Comparison with leading image editing models on multi-frame narrative generation. While all models exhibit strong adherence to the text prompts and maintain high subject consistency, Narrative Weaver uniquely generates outputs with a more cinematic and aesthetically pleasing quality . Note our model’s superior handling of lighting, color, and composition, which contributes to a more authentic, film-like visual narrative compared to the often more literal or digitally rendered feel of the baseline results.
Figure 14: Qualitative examples of autonomous story continuation by Narrative Weaver. Given only the first frame and text as input for each sequence, Narrative Weaver autonomously plans and generates a coherent multi-frame continuation. The diverse examples—from procedural tasks like cooking to dynamic events like sports—showcase the model’s robust planning capabilities. Its ability to logically advance a narrative by continuing actions, introducing new elements, or shifting focus highlights its understanding of storytelling beyond simple image editing.
Figure 15: Comparison with a leading editing model, Qwen-Image-Edit. Narrative Weaver not only demonstrates superior stylistic consistency with the conditional image but also excels in preserving key semantic details required by the task. In contrast, Qwen-Image-Edit exhibits noticeable failure modes: it struggles with inconsistent color tones between frames (upper example) and introduces a warm color cast that deviates from the style of the reference image (lower example).
Figure 16: Ablation Study Results. Comparison between Narrative Weaver and its variant without Stage 3 training demonstrates that: (1) the first two stages establish fundamental semantic alignment; (2) Stage 3 significantly enhances inter-frame consistency; and (3) Stage 3 is crucial for imparting fine-grained control, as the variant without it produces images that deviate significantly from the reference.
| Implementation | Keyframe Num | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 16 | 20 |
| Vanilla | TFLOPs | 82 | 230 | 450 | 744 | 1112 | 1553 | 2068 | 2656 | 3318 | 4053 | 4862 | 5744 | 10008 | 15449 |
| Ours | 82 | 165 | 248 | 331 | 441 | 497 | 580 | 663 | 746 | 829 | 912 | 995 | 1077 | 1160 |
Table 6: Computational cost (TFLOPs) as a function of the number of generated keyframes. Our approach demonstrates significantly better scalability compared to a vanilla self-attention baseline. The computational cost of our method grows linearly with the sequence length, unlike the vanilla implementation, whose cost grows quadratically, making our method far more efficient for longer sequences.
More in this topic
Controllable Video Generation
Controllable video generation steers how videos look, move, and change over time using user-provided instructions and structured controls.
See all 321 papers