Multi-shot video generation is crucial for long narrative storytelling, yet current bidirectional architectures suffer from limited interactivity and high latency. We propose ShotStream, a novel causal multi-shot architecture that enables interactive storytelling and efficient on-the-fly frame generation. By reformulating the task as next-shot generation conditioned on historical context, ShotStream allows users to dynamically instruct ongoing narratives via streaming prompts. We achieve this by first fine-tuning a text-to-video model into a bidirectional next-shot generator, which is then distilled into a causal student via Distribution Matching Distillation. To overcome the challenges of inter-shot consistency and error accumulation inherent in autoregressive generation, we introduce two key innovations. First, a dual-cache memory mechanism preserves visual coherence: a global context cache retains conditional frames for inter-shot consistency, while a local context cache holds generated frames within the current shot for intra-shot consistency. And a RoPE discontinuity indicator is employed to explicitly distinguish the two caches to eliminate ambiguity. Second, to mitigate error accumulation, we propose a two-stage distillation strategy. This begins with intra-shot self-forcing conditioned on ground-truth historical shots and progressively extends to inter-shot self-forcing using self-generated histories, effectively bridging the train-test gap. Extensive experiments demonstrate that ShotStream generates coherent multi-shot videos with sub-second latency, achieving 16 FPS on a single GPU. It matches or exceeds the quality of slower bidirectional models, paving the way for real-time interactive storytelling. Training and inference code, as well as the models, are available on our
Figures & tables
Figure 2: Overview of the ShotStream workflow, which enables real-time, long, multi-shot video generation from streaming prompts.
Figure 3: Architecture of the Bidirectional Next-Shot Teacher Model. To realize ShotStream, we first fine-tune a text-to-video model into a bidirectional next-shot model, which generates subsequent shots conditioned on sparse context frames from preceding shots. These conditional context frames are encoded into latents via a 3D VAE and injected by concatenating them with noise latents along the temporal dimension. Notably, only the 3D spatial-temporal attention layers within the DiT Blocks are optimized during fine-tuning. A 4-shot example is shown here for illustration.
Figure 4: Causal Architecture and Two-Stage Distillation Pipeline. We distill a slow, multi-step bidirectional teacher into an efficient, few-step causal generator. To maintain visual coherence, we propose a novel dual-cache memory mechanism: a global context cache stores conditional frames to ensure inter-shot consistency, while a local context cache retains generated frames within the target shot to guarantee intra-shot consistency. To prevent error accumulation, we employ a progressive two-stage distillation strategy. In the first stage, intra-shot self-forcing distillation (Step 2.1), the model is conditioned on ground-truth historical shots to causally generate the current shot chunk-by-chunk. In the second stage, inter-shot self-forcing distillation (Step 2.2), the model is conditioned on its own previously generated shots, rolling out the video shot-by-shot while iteratively generating the frames of each individual shot chunk-by-chunk.
Method
Architecture
FPS
Intra-shot Cons.
Inter-shot Cons.
Trans. Control ↑
Text Align. ↑
Aesthetic Quality ↑
Dynamic Degrees ↑
Sub. ↑
Bg. ↑
Semantic ↑
Sub. ↑
Bg. ↑
Mask2DiT [ 27 ]
Bidirectional
0.149
0.646
0.679
0.711
0.612
0.534
0.513
0.184
0.520
48.91
EchoShot [ 35 ]
Bidirectional
0.643
0.772
0.739
0.596
0.392
0.396
0.664
0.186
0.543
65.92
CineTrans [ 41 ]
Bidirectional
0.413
0.776
0.797
0.459
0.412
0.459
0.572
0.170
0.513
59.47
Self Forcing [ 12 ]
Causal
16.36
0.737
0.707
0.738
0.542
0.445
0.633
0.214
0.512
55.45
LongLive [ 43 ]
Causal
16.55
0.758
0.792
0.722
0.594
0.565
0.693
0.216
0.565
58.45
Table 1: Quantitative results for multi-shot video generation. The best results are highlighted in boldface , while the second best are underlined . Here, Sub., Bg., Cons., and Align. denote Subject, Background, Consistency, and Alignment, respectively.
Figure 5: Qualitative Comparison. We present the initial frames of each shot generated by all compared methods. Our approach not only adheres strictly to the prompts and maintains high visual coherence, but also produces natural transitions between shots.
Method
Mask2DiT
EchoShot
CineTrans
Self Forcing
LongLive
Rolling Forcing
Infinity-RoPE
Ours
Visual Consistency
3.08 %
12.31 %
6.21 %
1.54 %
12.31 %
15.38 %
16.92 %
87.69 %
Prompt Following
0.83 %
3.08 %
1.54 %
10.77 %
16.15 %
16.15 %
14.62 %
76.15 %
Visual Quality
7.69 %
18.46 %
16.92 %
10.77 %
18.46 %
23.08 %
15.38 %
83.08 %
Table 2: User Preference Rate. Participants select their preferred video from a randomized set of results generated by all methods. Multiple selections are allowed.
Method
Intra-shot Cons.
Inter-shot Cons.
Trans. Control ↑
Text Align. ↑
Aesthetic Quality ↑
Dynamic Degrees ↑
Sub. ↑
Bg. ↑
Semantic ↑
Sub. ↑
Bg. ↑
Context Frames Sampling Strategy
First Only
0.789
0.793
0.671
0.618
0.612
0.956
0.212
0.502
61.55
First & Last
0.809
0.827
0.709
0.629
0.638
0.969
0.223
0.528
62.08
Dynamic (Ours)
0.825
0.819
0.762
0.654
0.645
0.978
0.234
0.571
63.06
Condition Frames Captioning Strategy
Table 3: Ablation Study on the Bidirectional Next-Shot Teacher Model Design.
Method
Intra-shot Cons.
Inter-shot Cons.
Trans. Control ↑
Text Align. ↑
Aesthetic Quality ↑
Dynamic Degrees ↑
Sub. ↑
Bg. ↑
Semantic ↑
Sub. ↑
Bg. ↑
Dual-Cache Distinction Strategy
w/o Indicator
0.813
0.816
0.728
0.507
0.465
0.967
0.203
0.549
63.09
Learnable Emb.
0.811
0.809
0.737
0.518
0.588
0.972
0.204
0.523
61.19
RoPE Offset (Ours)
0.825
0.819
0.762
0.654
0.645
0.978
0.234
0.571
63.06
Causal Distillation Training Strategy
Table 4: Ablation Study on the Causal Student Model Design and Training Strategy.
Figure 6: Qualitative Ablation Results for the Causal Student Model. Please refer to project page for video comparisons.
Multi-shot video generation creates sequences of distinct shots that follow a shared story while preserving subjects, settings, and temporal continuity.
Autoregressive video generation aims at real-time, open-ended synthesis. Yet, cinematic storytelling is not merely the endless extension of a single scene; it requires progressing through evolving events, viewpoint shifts, and discrete shot boundaries. Existing autoregressive models often struggle in this setting. Trained primarily for short-horizon continuation, they treat long sequences as extended single shots, inevitably suffering from motion stagnation and semantic drift during long rollouts. To bridge this gap, we introduce CausalCine, an interactive autoregressive framework that transforms multi-shot video generation into an online directing process. CausalCine generates causally across shot changes, accepts dynamic prompts on the fly, and reuses context without regenerating previous shots. To achieve this, we first train a causal base model on native multi-shot sequences to learn complex shot transitions prior to acceleration. We then propose Content-Aware Memory Routing (CAMR), which dynamically retrieves historical KV entries according to attention-based relevance scores rather than temporal proximity, preserving cross-shot coherence under bounded active memory. Finally, we distill the causal base model into a few-step generator for real-time interactive generation. Extensive experiments demonstrate that CausalCine significantly outperforms autoregressive baselines and approaches the capability of bidirectional models while unlocking the streaming interactivity of causal generation. Demo available at https://yihao-meng.github.io/CausalCine/
Current video generation models achieve impressive results in single-shot generation, yet remain limited in cinematic video generation, where coherent narratives and effective multi-shot composition require explicit shot planning. To address this challenge, we propose ShotPlan, a framework for explicit multi-shot cinematic video generation built upon a video diffusion foundation model. Our method introduces learnable planning tokens that capture shot-level transition cues and can be seamlessly integrated with the original video generation tokens to control transition timestamps. Unlike standard video generation tokens, the proposed planning tokens are equipped with Fractional Temporal Rotary Position Embedding (FRoPE), enabling shot transitions to be modeled at the frame level. Experiments demonstrate that ShotPlan significantly outperforms existing cinematic video generation methods, offering more flexible shot management and stronger inter-shot consistency.
Su Guo, Guangce Liu, Haosen Yang +7
Institute of Artificial Intelligence (TeleAI), China Telecom · Harbin Institute of Technology
Storyboard synthesis plays a crucial role in visual storytelling, aiming to generate coherent shot sequences that visually narrate cinematic events with consistent characters, scenes, and transitions. However, existing approaches are mostly adapted from text-to-image diffusion models, which struggle to maintain long-range temporal coherence, consistent character identities, and narrative flow across multiple shots. In this paper, we introduce DreamShot, a video generative model based storyboard framework that fully exploits powerful video diffusion priors for controllable multi-shot synthesis. DreamShot supports both Text-to-Shot and Reference-to-Shot generation, as well as story continuation conditioned on previous frames, enabling flexible and context-aware storyboard generation. By leveraging the spatial-temporal consistency inherent in video generative models, DreamShot produces visually and semantically coherent sequences with improved narrative fidelity and character continuity. Furthermore, DreamShot incorporates a multi-reference role conditioning module that accepts multiple character reference images and enforces identity alignment via a Role-Attention Consistency Loss, explicitly constraining attention between reference and generated roles. Extensive experiments demonstrate that DreamShot achieves superior scene coherence, role consistency, and generation efficiency compared to state-of-the-art text-to-image storyboard models, establishing a new direction toward controllable video model-driven visual storytelling.
Junjia Huang, Binbin Yang, Pengxiang Yan +6
1Sun Yat-sen University · 2ByteDance Intelligent Creation