Interactive video generation commonly relies on small bidirectional chunks to reduce response latency. Yet our preliminary experiments show that larger chunks improve video quality and better utilize GPU parallelism, allowing more latents to be sampled with only a modest increase in sampling time. Motivated by these observations, we propose Speculative Sampling, a strategy that combines the modeling power of large bidirectional windows with the low-latency interaction of small ones. It expands the modeling window by appending noise latents as padding and caches the resulting predictions for reuse when the condition remains unchanged. Predictions are delivered incrementally, while the uncommitted future remains replaceable. Our latency analysis shows that this removes the waiting-time lower bound imposed by chunk boundaries, shifting the optimization focus from reducing chunk size to accelerating per-chunk sampling. Bidirectional window size thus becomes a decision variable for jointly optimizing visual quality and sampling time, rather than a necessary constraint on low-latency interaction. This opens a path to serving large bidirectional models in interactive streaming systems without complex streaming post-training.
Figures & tables
Figure 1: Fine-grained interaction without shrinking generation chunks. Our method preserves large-chunk joint sampling while allowing updates beyond the protected prefix, rather than only at chunk boundaries. Gray/green denote old/new conditions; the dashed line marks playback position at condition arrival, and thick green edges mark updated output. Positions represent video content, not computation time.
Figure 2: Generation speed alone does not determine response latency. In this periodic reference schedule, the highlighted update waits R for generation, takes C to produce deliverable output, and waits Q for playback. Each chunk is prepared for the next playback period; V0 is prebuffered.
Figure 3: Speculative Sampling serving system and control. (a) Joint generation populates a prediction cache for fine-grained delivery and playback; playback progress guides refill. (b) The finite-state machine coordinates generation in Produce and prediction reuse in Consume, illustrated for q=1 . Condition updates trigger feasible replanning without interrupting active generation; playback continues in both states.
Figure 4: At fixed T , direct generation scales as C/T , while residual wait scales as E[R]/T=(C/T)2/2 under the stated periodic model.
Figure 5: Chunk size exposes a quality–efficiency tradeoff across models. (a) Wan four-step VBench totals at two resolutions. (b) Wan H200 forward latency across USP sizes at 480p. Quality uses separately trained configurations; timings use a shared reference checkpoint and synthetic fixed-length history. LTX quality and cached-forward profiles are reported in Table 2 .
Resolution
Steps
K
Total
Quality
Semantic
Dynamic
VisionReward
Instruction
480p
1
1
79.11
81.80
68.34
4.00
6.50
90.00
1
3
80.38
83.40
68.29
58.00
4.81
94.00
1
7
81.35
83.97
70.86
48.00
5.79
100.00
1
21
81.68
84.83
69.08
85.00
9.93
94.00
2
1
80.07
82.76
69.34
14.00
9.31
98.00
2
3
81.62
84.63
69.57
57.00
9.82
100.00
Table 1: Wan1.3B quality measurements. Total is the reported VBench aggregate, with its quality and semantic components; the table also includes dynamic degree, VisionReward, and instruction following.
Model
Resolution
Configuration
Low-frequency updates
High-frequency updates
L50
L95
GPU
Stall
L50
L95
GPU
Stall
Wan1.3B
480p
Short-Baseline
1.13
1.47
49.67
0.000
1.24
1.65
52.19
0.000
Long-Baseline
3.23
4.64
16.97
0.000
3.11
4.66
17.29
0.000
Short–Speculative Sampling
1.01
1.14
50.45
0.000
1.05
1.35
52.25
0.000
Long–Speculative Sampling
1.32
1.91
17.28
0.000
1.35
1.63
19.92
0.000
LTX-2.3
480p
Short-Baseline
2.12
2.58
22.02
19.669
2.01
2.64
22.69
20.339
Table 3: Online serving results at 480p class for Wan1.3B, LTX-2.3, and Sol-H3. Baseline denotes chunk-level commitment; Speculative Sampling denotes our method. LTX’s parenthesized CUDA Graph configurations apply graph capture and replay to the DiT; Long versus Short isolates chunk size among eager configurations, while parenthesized versus unparenthesized Short isolates this execution mode. L50/L95 are seconds; GPU kernel activity and stall ratio are percentages. GPU is the time-weighted mean over the monitored deployment GPUs, with model-specific scope described in the text.
Few-step autoregressive video generators enable streaming generation, yet they run every token through the final denoising step, regardless of how much refinement it needs. We find this uniform computation is often unnecessary: many tokens already closely match their eventual full-step outputs in the initial steps, while others require refinement until the end. We introduce Speculative Forcing, a framework that treats intermediate clean estimates as drafts and predicts when individual tokens can be safely accepted early. Once accepted, a token is frozen as its final output and undergoes no further denoising, while its cached keys and values remain available as context for the remaining tokens. We explore two alternative verifier designs: a training-free one based on similarity to clean content and stability across steps, and a learned one that reads the frozen generator's intermediate features to estimate each token's remaining gap to its full-step output. Across short-video, long-video, and world generation, Speculative Forcing preserves the quality of full-step sampling while substantially reducing computation. On LongLive, it matches the four-step generator with ~1.7 denoising steps per token, running over 1.5x faster at 41.7 and 37.8 fps. On Lingbot-World 2.0, where camera motion continually changes the visible content, it maintains quality with ~2.5 denoising steps per token and runs 1.35x faster at 24.5 fps.
Video Large Language Models (Video-LLMs) excel in video understanding but suffer from high inference latency during autoregressive generation. Speculative Decoding (SD) mitigates this by applying a draft-and-verify paradigm, yet existing methods are constrained by rigid exact-match rules, severely limiting the acceleration potential. To bridge this gap, we propose LVSpec, the first training-free loosely SD framework tailored for Video-LLMs. Grounded in the insight that generation is governed by sparse visual-relevant anchors (mandating strictness) amidst abundant visual-irrelevant fillers (permitting loose verification), LVSpec employs a lightweight visual-relevant token identification scheme to accurately pinpoint the former. To further maximize acceptance, we augment this with a position-shift tolerant mechanism that effectively salvages positionally mismatched but semantically equivalent tokens. Experiments demonstrate that LVSpec achieves high fidelity and speed: it preserves >99.8 of target performance while accelerating Qwen2.5-VL-32B by 2.70x and LLaVA-OneVision-72B by 2.94x. Notably, it boosts the mean accepted length and speedup ratio by 136% and 35% compared to SOTA training-free SD methods for Video-LLMs.
We present Wan-Streamer v0.2, a latency-preserving upgrade of the native-streaming, end-to-end audio-visual interaction model. v0.2 keeps the v0.1 modeling formulation, but raises the interactive output stream from 192x336 to 640x368 while preserving approximately 200 ms model-side signal-to-signal latency at 25 FPS. The higher-resolution stream supports scene-grounded mid-shot agents whose posture, gaze, hands, nearby objects, and local scene layout remain legible during real-time conversation. To support the larger visual stream without adding user-visible delay, v0.2 keeps the thinker as a single-GPU low-latency path for streaming perception, the short language/state Transformer pass that builds the generation cache, and final decoding. The performer becomes a multi-GPU Ulysses-style context-parallel group for the expensive next-unit latent generation. Each performer rank writes incoming K/V into a pre-sharded local cache. The long high-resolution latent video sequence is split across ranks for denoising and gathered through Ulysses communication, while the much shorter audio latent sequence is generated without sequence sharding. In this split, the thinker's language/state computation reaches the performer only as K/V conditioning, so no separate language sequence has to be communicated inside the performer group. This concentrates additional hardware on visual generation while preserving the compact thinker-performer boundary, keeping total remote interaction latency at approximately 550 ms when a 350 ms bidirectional network budget is included.