Injecting new subjects naturally during streaming video generation requires integrating newly supplied visual references into the existing video while preserving established subjects and the scene. Encoding references independently of the current scene provides little guidance for this integration, while keeping their features in persistent memory can interfere with established context. We propose ReferStream, a framework that requires no additional training and combines reference construction using the current scene with transient cache injection. Joint Latent places reference latents in target regions of the latest scene latent, allowing reference features to be extracted in context. Cache Restoration uses these features to condition the injection chunk, then restores the overwritten global cache entries while retaining the newly generated history. This procedure incorporates new visual conditions without retraining the generator or revising emitted frames. We further introduce ReferStream Bench, a new benchmark with scheduled reference arrivals, evolving instructions, and explicit annotations of subject presence for InShot and MultiShot generation. Evaluation follows the event schedule and measures both the fidelity of new subjects and the preservation of established subjects, alongside video quality, text adherence, and count accuracy. Experiments on ReferStream Bench show higher scores for text adherence, subject similarity, and count accuracy than the compared baselines, including offline methods given all references in advance. Project page: https://xushuolin.github.io/ReferStream/.
Figures & tables
Figure 1: Interactive subject injection during streaming generation. A new cat reference is supplied after the umbrella scene is established. The cat enters while the umbrella remains and previously emitted frames stay fixed.
Figure 2: Overview of ReferStream, illustrated for an InShot event. Top: reference B arrives after subject A is established and conditions the next chunk. Middle: (A) Joint Latent composes the reference with the latest scene latent; a clean DiT pass extracts Rj . (B) Cache Restoration saves global K/V, injects Rj , denoises and commits the event chunk, then restores the saved entries. Generated local history L′ remains. S′ denotes the shot sink after the update; S′=S when no refresh is scheduled for this chunk. Bottom: schematic cache access, not measured attention weights. The photographs illustrate a benchmark example; subsequent generation is schematic.
Figure 3: ReferStream Bench statistics. (a) Instances of reference subjects across the frozen collection; plants are included in objects. (b) Approximate durations of archived videos. (c) Scripted behavior frequencies, counted once per entry per label; labels can overlap. (d) Vocabulary of reference subjects, including recorded appearance descriptors. The figure describes benchmark content, not generation success or unique identities.
Method
Aesth. ↑
Motion ↑
GME-T ↑
New-ID † ↑
Exist-ID † ↑
Count † ↑
LongLive-2.0
53.08
99.39
60.62
39.99
42.29
73.90
ReferStream (ours)
57.29
99.36
62.41
52.86
73.18
82.33
VACE-14B
57.57
99.23
57.83
43.33
65.97
51.22
Phantom-14B
48.37
97.10
50.90
29.23
62.72
31.07
HOMIE-14B
55.55
99.53
57.41
22.10
64.57
26.34
Table 1: Overall comparison on ReferStream Bench using six computed metrics. Scores are raw values ×100 ( ↑ ), averaged with equal weight over eligible cases. Thin rules separate online event conditioning (above) from offline conditioning on all references (below). Bold marks the best point estimate in each column; the light blue row indicates our method. † : automatic estimates that depend on detections.
Figure 4: Qualitative comparisons on InShot: umbrella and black cat (top), cobalt bottle and raven (bottom). Reference A establishes the scene; B is revealed at t=5.21 s. ReferStream introduces B into the ongoing shot while retaining A in the subsequent frames. Orange labels denote aligned benchmark time in seconds. LongLive2 receives online text events without visual references; VACE, Phantom, and HOMIE receive all references and the full script in advance. The selected frames illustrate reference fidelity, subject persistence, and count differences under these respective information settings.
Variant
New-ID ↑
Existing-ID ↑
New Dup. ↓
Existing Dup. ↓
w/o Joint Latent
63.10
70.56
16.67
24.13
w/o Cache Restoration
52.94
69.36
20.49
29.17
ReferStream (Full)
51.31
72.17
14.41
23.26
Table 3: Component ablations on the common InShot set (seed 0). Bold marks the best point estimate in each column; Full is shaded blue. Both components improve preservation and duplication point estimates at a cost in New-ID.
Figure 5: Effect of Joint Latent on established subjects. Full and w/o Joint Latent are compared on the kettle and woman example (top) and the burner and crane example (bottom). Without joint construction, the kettle and burner change shape, and an extra vessel appears beside the burner. Reference images are shown on the left; f denotes the source frame index.
Variant
New-ID ↑
Existing-ID ↑
New Dup. ↓
Existing Dup. ↓
w/o Joint Latent
64.23
74.55
13.68
21.79
ReferStream (Full)
52.22
75.30
12.82
20.09
Table 4: Joint Latent under initial reference conditioning (seed 0). References are injected at generation start. Full and w/o Joint Latent use the same cache policy; only reference construction differs. Scores use the subset of short sequences, with metrics matching Table 3 . Bold marks the best point estimate; Full is shaded blue.
Figure 6: Effect of Cache Restoration after subject injection. Full and w/o Cache Restoration are compared on the bag and dog example (top) and the mixer and duck example (bottom). Without restoration, star patterns progressively cover the first scene and an extra duck appears in the second. Reference images are shown on the left; f denotes the source frame index.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Role
Checkpoint / implementation
Source
Aesthetic features
CLIP ViT-L/14
Radford et al. [25]
Aesthetic head
LAION V1 ViT-L/14 linear head
LAION-AI [18]
Motion interpolation
AMT-S
Li et al. [20]
Text adherence
gme-Qwen2-VL-7B-Instruct
Zhang et al. [36]
Object detection
Grounding DINO Tiny
Liu et al. [22]
Foreground masking
SAM 2.1 Hiera Large
Ravi et al. [26]
Appendix
Table 5: Evaluation components and pretrained checkpoints. Linked names point to the checkpoint provider or official implementation.
Variant
New-ID ↑
Existing-ID ↑
New Dup. ↓
Existing Dup. ↓
w/o Cache Restoration
53.67
46.44
16.67
46.30
ReferStream (Full)
47.38
58.60
21.30
37.04
Appendix
Table 6: Cache Restoration on the multistage subset (seed 0). Full restores the overwritten global entries; w/o Cache Restoration retains raw reference K/V. Bold marks the best point estimate; Full is shaded blue.
Figure 7: InShot injection of a white hen beside a bronze burner. Reference B arrives at t=5.21 s; A should remain in the same shot. Orange labels indicate aligned benchmark time in seconds.
Figure 8: MultiShot subject composition in a furniture design discussion. The requested composition is A , B , B , A+B , A+B across the displayed columns. Orange labels indicate aligned benchmark time in seconds.
Streaming autoregressive video models generate long videos chunk by chunk, using historical memory to maintain consistency. Existing methods typically expose subject and scene queries to history through similar policies. This stabilizes the subject, but can also lock backgrounds, viewpoints, and scene structure to previously generated states even when local motion continues. We call this failure memory-anchored scene under-progression; consistency and motion metrics alone can miss it. We introduce TetherMem, a training-free, query-aware spatiotemporal memory router for frozen video generators. TetherMem separates subject and scene queries and modulates historical access with region- and age-conditioned priors: subject queries retain identity-bearing history, while scene queries reduce reliance on subject history and stale backgrounds. Across 2,400 blinded pairwise judgments from 10 annotators, TetherMem achieves the highest estimated expected preference among eight streaming long-video baselines for overall quality (0.780) and scene progression (0.769). On complete 30-second videos, it sustains changes in background, viewpoint, and scene state while preserving subject recognizability and temporal continuity.
Chen Li, Peng Zhang, Hanyu Zhou +5
Huazhong University of Science and Technology · MiLM Plus, Xiaomi Inc.
Streaming video generation holds strong potential for world modeling, where future frames must be inferred online sequentially to form a continuous video stream. However, streaming video diffusion models introduce a fundamental train-inference mismatch: inference follows a specialized denoising order, whereas advanced training strategies typically require diverse noise-level configurations. To address this trade-off between train-inference consistency and training coverage, we reformulate the video diffusion sampling as a frame-indexed stochastic process over noise levels. Within this stochastic process space, we construct a continuous training trajectory along which the sampling schedule progressively evolves from independent sampling to inference-consistent sampling. We further introduce a joint calibration algorithm and a temporal correlative sampling algorithm to ensure trajectory smoothness and cross-frame correlation. Building on these designs, we propose Stream Forcing, a unified training framework for streaming video generation that balances training sufficiency and inference efficiency. Extensive experiments demonstrate that Stream Forcing significantly improves generation quality with a 36.6% FVD improvement on the UCF-101 benchmark. Furthermore, our method facilitates robust zero-shot extrapolation to long-horizon video generation with a 27.9% FVD improvement on the UCF-101 benchmark.
Yueting Zhu, Yuehao Song, Kaicheng Zhang +5
Huazhong University of Science & Technology · Anyverse Dynamics · Horizon Robotics
Conditioning a video generator on multiple images requires preserving appearance while associating each reference with its intended role. We present MSR (Multiple Subject Reference), a slot-aware conditioning scheme for LTX-based video generation. Each reference image is independently encoded as a static clip and represented by a separate latent-token group. A compact Fourier-feature multilayer perceptron adds a numeric slot embedding, while slot-dependent temporal offsets modify the group's rotary coordinates. The reference groups are prepended to noisy target tokens and serve as clean context during target-only flow-matching training. We implement this scheme through low-rank adaptation and release the resulting weights and inference workflows. Qualitative examples demonstrate compositions containing distinct characters and referenced environments in realistic and stylized scenes. Development observations suggest reduced reference confusion relative to an earlier continuous-reference baseline, while similar clothing, complex garments, and viewpoint changes remain challenging. We describe the conditioning mechanism, the retained training configuration, and the observed strengths and limitations of the released system. A supplementary audio-reference experiment adds voice conditioning while keeping the visual parameters frozen.