Injecting new subjects naturally during streaming video generation requires integrating newly supplied visual references into the existing video while preserving established subjects and the scene. Encoding references independently of the current scene provides little guidance for this integration, while keeping their features in persistent memory can interfere with established context. We propose ReferStream, a framework that requires no additional training and combines reference construction using the current scene with transient cache injection. Joint Latent places reference latents in target regions of the latest scene latent, allowing reference features to be extracted in context. Cache Restoration uses these features to condition the injection chunk, then restores the overwritten global cache entries while retaining the newly generated history. This procedure incorporates new visual conditions without retraining the generator or revising emitted frames. We further introduce ReferStream Bench, a new benchmark with scheduled reference arrivals, evolving instructions, and explicit annotations of subject presence for InShot and MultiShot generation. Evaluation follows the event schedule and measures both the fidelity of new subjects and the preservation of established subjects, alongside video quality, text adherence, and count accuracy. Experiments on ReferStream Bench show higher scores for text adherence, subject similarity, and count accuracy than the compared baselines, including offline methods given all references in advance. Project page: https://xushuolin.github.io/ReferStream/.
Figures & tables
Figure 1: Interactive subject injection during streaming generation. A new cat reference is supplied after the umbrella scene is established. The cat enters while the umbrella remains and previously emitted frames stay fixed.
Figure 2: Overview of ReferStream, illustrated for an InShot event. Top: reference B arrives after subject A is established and conditions the next chunk. Middle: (A) Joint Latent composes the reference with the latest scene latent; a clean DiT pass extracts Rj . (B) Cache Restoration saves global K/V, injects Rj , denoises and commits the event chunk, then restores the saved entries. Generated local history L′ remains. S′ denotes the shot sink after the update; S′=S when no refresh is scheduled for this chunk. Bottom: schematic cache access, not measured attention weights. The photographs illustrate a benchmark example; subsequent generation is schematic.
Figure 3: ReferStream Bench statistics. (a) Instances of reference subjects across the frozen collection; plants are included in objects. (b) Approximate durations of archived videos. (c) Scripted behavior frequencies, counted once per entry per label; labels can overlap. (d) Vocabulary of reference subjects, including recorded appearance descriptors. The figure describes benchmark content, not generation success or unique identities.
Method
Aesth. ↑
Motion ↑
GME-T ↑
New-ID † ↑
Exist-ID † ↑
Count † ↑
LongLive-2.0
53.08
99.39
60.62
39.99
42.29
73.90
ReferStream (ours)
57.29
99.36
62.41
52.86
73.18
82.33
VACE-14B
57.57
99.23
57.83
43.33
65.97
51.22
Phantom-14B
48.37
97.10
50.90
29.23
62.72
31.07
HOMIE-14B
55.55
99.53
57.41
22.10
64.57
26.34
Table 1: Overall comparison on ReferStream Bench using six computed metrics. Scores are raw values ×100 ( ↑ ), averaged with equal weight over eligible cases. Thin rules separate online event conditioning (above) from offline conditioning on all references (below). Bold marks the best point estimate in each column; the light blue row indicates our method. † : automatic estimates that depend on detections.
Figure 4: Qualitative comparisons on InShot: umbrella and black cat (top), cobalt bottle and raven (bottom). Reference A establishes the scene; B is revealed at t=5.21 s. ReferStream introduces B into the ongoing shot while retaining A in the subsequent frames. Orange labels denote aligned benchmark time in seconds. LongLive2 receives online text events without visual references; VACE, Phantom, and HOMIE receive all references and the full script in advance. The selected frames illustrate reference fidelity, subject persistence, and count differences under these respective information settings.
Variant
New-ID ↑
Existing-ID ↑
New Dup. ↓
Existing Dup. ↓
w/o Joint Latent
63.10
70.56
16.67
24.13
w/o Cache Restoration
52.94
69.36
20.49
29.17
ReferStream (Full)
51.31
72.17
14.41
23.26
Table 3: Component ablations on the common InShot set (seed 0). Bold marks the best point estimate in each column; Full is shaded blue. Both components improve preservation and duplication point estimates at a cost in New-ID.
Figure 5: Effect of Joint Latent on established subjects. Full and w/o Joint Latent are compared on the kettle and woman example (top) and the burner and crane example (bottom). Without joint construction, the kettle and burner change shape, and an extra vessel appears beside the burner. Reference images are shown on the left; f denotes the source frame index.
Variant
New-ID ↑
Existing-ID ↑
New Dup. ↓
Existing Dup. ↓
w/o Joint Latent
64.23
74.55
13.68
21.79
ReferStream (Full)
52.22
75.30
12.82
20.09
Table 4: Joint Latent under initial reference conditioning (seed 0). References are injected at generation start. Full and w/o Joint Latent use the same cache policy; only reference construction differs. Scores use the subset of short sequences, with metrics matching Table 3 . Bold marks the best point estimate; Full is shaded blue.
Figure 6: Effect of Cache Restoration after subject injection. Full and w/o Cache Restoration are compared on the bag and dog example (top) and the mixer and duck example (bottom). Without restoration, star patterns progressively cover the first scene and an extra duck appears in the second. Reference images are shown on the left; f denotes the source frame index.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Role
Checkpoint / implementation
Source
Aesthetic features
CLIP ViT-L/14
Radford et al. [25]
Aesthetic head
LAION V1 ViT-L/14 linear head
LAION-AI [18]
Motion interpolation
AMT-S
Li et al. [20]
Text adherence
gme-Qwen2-VL-7B-Instruct
Zhang et al. [36]
Object detection
Grounding DINO Tiny
Liu et al. [22]
Foreground masking
SAM 2.1 Hiera Large
Ravi et al. [26]
Appendix
Table 5: Evaluation components and pretrained checkpoints. Linked names point to the checkpoint provider or official implementation.
Variant
New-ID ↑
Existing-ID ↑
New Dup. ↓
Existing Dup. ↓
w/o Cache Restoration
53.67
46.44
16.67
46.30
ReferStream (Full)
47.38
58.60
21.30
37.04
Appendix
Table 6: Cache Restoration on the multistage subset (seed 0). Full restores the overwritten global entries; w/o Cache Restoration retains raw reference K/V. Bold marks the best point estimate; Full is shaded blue.
Figure 7: InShot injection of a white hen beside a bronze burner. Reference B arrives at t=5.21 s; A should remain in the same shot. Orange labels indicate aligned benchmark time in seconds.
Figure 8: MultiShot subject composition in a furniture design discussion. The requested composition is A , B , B , A+B , A+B across the displayed columns. Orange labels indicate aligned benchmark time in seconds.