Organizations: Adobe Research · KAUST · Independent Researcher · University of Oregon · University of Memphis · Johns Hopkins University · Meta AI · Texas A&M University · Dolby Labs · Virginia Tech · Cisco · University of Maryland, College Park
Generating long-form storytelling videos with consistent visual narratives remains a significant challenge in video synthesis. We present a novel framework, dataset, and a model that address three critical limitations: background consistency across shots, seamless multi-subject shot-to-shot transitions, and scalability to hour-long narratives. Our approach introduces a background-consistent generation pipeline that maintains visual coherence across scenes while preserving character identity and spatial relationships. We further propose a transition-aware video synthesis module that generates smooth shot transitions for complex scenarios involving multiple subjects entering or exiting frames, going beyond the single-subject limitations of prior work. To support this, we contribute with a synthetic dataset of 10,000 multi-subject transition sequences covering underrepresented dynamic scene compositions. On VBench, InfinityStory achieves the highest Background Consistency (88.94), highest Subject Consistency (82.11), and the best overall average rank (2.80), showing improved stability, smoother transitions, and better temporal coherence.
Figures & tables
Figure 2: Overview of the proposed storytelling video generation pipeline. Green shapes: are the output of the agentic pipeline. Purple Shapes: Narrative odd shots generate keyframe images which are used to generate video shots using I2V. Red shapes: While the transition in-between (even) shots take the next keyframe and the last frame from the generated I2V shot to generate a First-Last-Frame-to-Video (FLF2V) which smoothly bridges consecutive narrative shots. The output video would be stitched together to form one coherent video, i.e., shot-1 (I2V) → shot-2 (FLF2V) → shot-3 (I2V) → shot-4 (FLF2V) → .. and so on.
Figure 3: This novel framework is designed to create a large dataset for smooth transitions to train the First-Last-Frame-to-Video (FLF2V). We use 4 agentic pipelines to generate the prompt that generates the transition video. Then we use VLM to filter out bad videos which does not have the correct number of characters and to generate another prompt for the transition video. We use the combination of prompts from the video prompter and VLM along with the generated video, first-frame and last-frame of the video to train the FLF2V.
Figure 4: Our pipeline results in smooth transitions
Model
Img Quality ↑
Subject Cons. ↑
Bg Cons. ↑
Aesthetic ↑
Motion Smth. ↑
Rank Avg. ↓
StableDiffusion [ 1 ] + CogVideo [ 8 ]
75.52
78.05
85.27
59.61
97.63
4.80
StableDiffusion [ 1 ] + Wan2.1 [ 27 ]
76.93
78.54
87.43
69.75
96.67
3.40
StoryAdapter [ 13 ] + CogVideo [ 8 ]
76.17
72.17
88.03
63.55
98.16
4.00
StoryAdapter [ 13 ] + Wan [ 27 ]
75.96
75.04
88.64
73.38
97.02
3.60
MovieAgent [ 31 ]
72.09
68.61
79.84
55.40
99.01
5.80
VoT [ 37 ]
63.85
75.11
85.78
74.91
99.25
3.60
Table 2: Main results on TinyStories [ 3 ] , presenting scores for Image Quality (Img Quality), Subject Consistency (Subject Cons.), Background Consistency (Bg Cons.), Aesthetic Quality (Aesthetic), Motion Smoothness (Motion Smth.), and Average Rank (Rank Avg. - the average ranking position across all models). InfinityStory achieves the best ranking average.
Model
Img Quality ↑
Subject Cons. ↑
Bg Cons. ↑
Aesthetic ↑
Motion Smth. ↑
InfinityStory
73.64
82.11
88.94
64.47
98.58
w/o Background Injection
72.61
78.51
87.32
62.74
98.58
w/o Multi-Character Transition
72.36
81.31
88.64
63.00
97.63
Table 3: Ablation studies: InfinityStory trained with multi-subject transitions and background injection achieves the best metrics compared to variants without these components.
Figure 5: Website we developed to collect human evaluations with Video A, B, and C altering between the three methods.
Figure 6: Results show that we outperform other methods on human studies.
Metric
InfinityStory (Ours)
MovieAgent
Video-Gen-of-Thought
Background Consistency
59.32
32.20
8.47
Character Consistency
42.37
38.98
18.64
Image Quality & Aesthetic
50.85
33.90
15.25
Motion Smoothness
55.93
30.51
13.56
Smooth Transitions
45.76
33.90
20.34
Table 4: Human Study Results
Figure S1 : Agentic story planning creating the story in the form of chapters
Figure S2 : Location-based background prompt for background consistency
Figure S3 : Scene planning prompt utilizing the story chapter planning along with the reusable background locations.
Figure S4 : Narrative shot planning based on the scenes for the main even shots.
Figure S5 : Transition shot planning based on the scenes for the odd shots.
Figure S6 : Failed transition dataset generation as the character appears partially in the first frame. Character in first frame highlighted in red. Transition from zero characters to one character. These failed videos are filtered by our VLM.
Figure S7 : Successful transition dataset generation as no character appears in full nor partially in the first frame. First row is a transition from zero characters to one character. Second row is a transition from 1 character to 3 characters.
Storyboard synthesis plays a crucial role in visual storytelling, aiming to generate coherent shot sequences that visually narrate cinematic events with consistent characters, scenes, and transitions. However, existing approaches are mostly adapted from text-to-image diffusion models, which struggle to maintain long-range temporal coherence, consistent character identities, and narrative flow across multiple shots. In this paper, we introduce DreamShot, a video generative model based storyboard framework that fully exploits powerful video diffusion priors for controllable multi-shot synthesis. DreamShot supports both Text-to-Shot and Reference-to-Shot generation, as well as story continuation conditioned on previous frames, enabling flexible and context-aware storyboard generation. By leveraging the spatial-temporal consistency inherent in video generative models, DreamShot produces visually and semantically coherent sequences with improved narrative fidelity and character continuity. Furthermore, DreamShot incorporates a multi-reference role conditioning module that accepts multiple character reference images and enforces identity alignment via a Role-Attention Consistency Loss, explicitly constraining attention between reference and generated roles. Extensive experiments demonstrate that DreamShot achieves superior scene coherence, role consistency, and generation efficiency compared to state-of-the-art text-to-image storyboard models, establishing a new direction toward controllable video model-driven visual storytelling.
Junjia Huang, Binbin Yang, Pengxiang Yan +6
1Sun Yat-sen University · 2ByteDance Intelligent Creation
Long-form visual storytelling requires maintaining continuity across shots, including consistent characters, stable environments, and smooth scene transitions. While existing generative models can produce strong individual frames, they fail to preserve such continuity, leading to appearance changes, inconsistent backgrounds, and abrupt scene shifts. We introduce CANVAS (Continuity-Aware Narratives via Visual Agentic Storyboarding), a multi-agent framework that explicitly plans visual continuity in multi-shot narratives. CANVAS enforces coherence through character continuity, persistent background anchors, and location-aware scene planning for smooth transitions within the same setting We evaluate CANVAS on two storyboard generation benchmarks ST-BENCH and ViStoryBench and introduce a new challenging benchmark HardContinuityBench for long-range narrative consistency. CANVAS consistently outperforms the best-performing baseline, improving background continuity by 21.6%, character consistency by 9.6% and props consistency by 7.6%.
Long-form video generation requires recurring subjects to remain consistent across various shots, viewpoints, motions, and scene transitions. Existing temporal decomposition methods improve scalability by generating videos shot by shot. However, they mainly focus on optimizing plausible next-shot continuations without verifying whether the historical memory preserves identity-critical subject evidence. Consequently, as generation proceeds, recurring subjects may be diluted, overwritten, or forgotten. In this paper, we propose Memento, a subject-reconstruction-guided framework that treats subject preservation as an explicit identity grounding problem, based on the premise that a memory bank faithfully preserving a subject should support reconstructing that subject from memory alone. Specifically, Memento jointly trains autoregressive next-shot generation with memory-based subject reconstruction, recovering target appearances using historical memory and global story captions. To disentangle long-range subject evidence from short-range cues, Memento introduces a dual-query memory mechanism, where one query retrieves identity-relevant memory and the other selects short-context keyframes for coherent continuation. Additionally, a subject-aware cinematic data pipeline provides precise reconstruction supervision via consistent, pronoun-free subject descriptions. Experiments demonstrate that Memento achieves state-of-the-art performance in long-term subject consistency, cross-shot coherence, and visual quality.