Streaming video understanding (SVU) requires video-LLMs to process incoming video continuously while answering user queries with low latency. Recent methods reduce query-time cost by prefilling frames into KV caches during idle time and reusing the cached KVs at query time. To bound per-frame computation over unbounded video streams, these methods restrict the context available during prefill. This creates a fundamental context mismatch: the restricted context may omit information needed to properly contextualize video tokens, yet, still contain information irrelevant to the eventual query, which is unknown during prefill. We propose FrameBlend, a simple yet effective plug-and-play method to address this mismatch through selective KV recomputation, without requiring any additional training. At query time, FrameBlend re-prefills selected video tokens using query-relevant context and replaces their cached KVs with recomputed ones. Crucially, this recomputation enables shorter attention windows during idle-time prefill, substantially reducing prefill cost while improving accuracy. Experiments across three SVU frameworks and five streaming and offline benchmarks demonstrate accuracy gains of up to 3.9 pp. On RVS-Ego, FrameBlend reduces per-frame prefill FLOPs by over 20% and total processing time by 14-59% for retrieval-based frameworks, including the cost of query-time recomputation. These results demonstrate that a simple change in how contextualization is distributed between idle time and query time can improve the accuracy-efficiency trade-off for SVU.
Figures & tables
Figure 1: Motivation and performance of FrameBlend. (a) Sliding-window attention can omit inter-token context needed at query time. (b) KV compression can retain unnecessary context from evicted tokens. (c) FrameBlend improves SVU framework accuracy while reducing total latency.
Figure 2: Overview of FrameBlend. FrameBlend uses a short attention window during idle-time prefill and selectively recomputes the KVs of query-relevant tokens at query time.
Figure 3: Effect of window size. Accuracy of ReKV with FrameBlend on OVO-Bench.
RVS-Ego
RVS-Movie
OVO-Bench
Method
Acc.
Score
Acc.
Score
Real-time
Backward
Avg.
LLaVA-OV-7B
ReKV
59.3
3.93
49.2
3.45
61.5
43.2
52.3
ReKV + FrameBlend
62.4
3.99
50.1
3.41
63.0
44.6
53.8
LiveVLM
60.1
3.95
51.5
3.50
63.6
43.5
53.5
Table 1: FrameBlend on streaming benchmarks. We compare the accuracy of various SVU frameworks with and without FrameBlend on streaming VQA benchmarks.
Video-MME
MLVU
EgoSchema
LLaVA-OV-7B
Short
Medium
Long
Total
Holistic
Single- Detail
Multi- Detail
Total
Total
ReKV
71.2
55.3
46.9
57.8
83.2
73.1
41.1
68.4
60.0
ReKV + FrameBlend
70.7
57.0
50.7
59.4
83.6
73.4
41.3
68.7
60.3
LiveVLM
68.4
56.2
48.4
57.7
82.7
70.8
41.1
67.0
60.6
LiveVLM + FrameBlend
69.6
56.1
48.8
58.1
83.4
71.9
42.8
68.1
60.5
Table 2: FrameBlend on offline benchmarks. We compare the accuracy of various SVU frameworks with and without FrameBlend on offline VQA benchmarks.
Prefill (idle time)
Answering (query time)
Total
LLaVA-OV-7B
TFLOPs/f
ms/f
Recomp. (s)
TTFT (s)
TFLOPs
Latency (s)
ReKV
4.38
81.1
–
0.17
4647.8
90.6
ReKV + FrameBlend
3.24
32.0
0.77
0.93
3567.4
37.4
LiveVLM
3.78
32.6
–
0.04
4007.2
37.6
LiveVLM + FrameBlend
2.94
28.5
0.77
0.84
3247.6
32.3
Table 3: Efficiency metrics. We compare SVU methods with and without FrameBlend on RVS-Ego.
Figure 4: Efficiency at varying numbers of input frames. We analyze ReKV with FrameBlend under varying video lengths, and observe that the efficiency gain increases as the video gets longer.
Figure 5: TTFT–accuracy trade-off. We compare ReKV with and without FrameBlend, and the offline LLaVA-OV-7B using 64 uniformly sampled frames.
OVO-Bench
RVS-Movie
RVS-Ego
LLaVA-OV-7B
Acc.
Added TTFT (s)
Acc.
Added TTFT (s)
Acc.
Added TTFT (s)
ReKV
52.3
–
49.2
–
59.3
–
ReKV + CacheBlend
52.7
0.90
48.0
1.51
61.4
2.26
ReKV + FrameBlend
53.8
0.67
50.1
0.75
62.4
0.76
Table 4: Comparison against CacheBlend. Accuracy and TTFT when applied on ReKV.
RVS-Ego
RVS-Movie
OVO-Bench
LLaVA-OV-7B
Acc.
Score
Acc.
Score
Real-time
Backward
Avg.
ReKV
59.3
3.93
49.2
3.45
61.5
43.2
52.3
+ short window
60.7
3.94
46.8
3.30
63.3
43.0
53.1
+ recomputation
58.7
3.94
48.9
3.46
61.7
44.8
53.2
+ both
62.4
3.99
50.1
3.41
63.0
44.6
53.8
Table 5: Ablation studies. We ablate two key components of FrameBlend.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Streaming benchmarks
Offline benchmarks
Method
OVO-Bench
RVS-Ego
RVS-Movie
Video-MME
MLVU
EgoSchema
LLaVA-OV-7B
ReKV
2048
196
196
8192
8192
384
HERMES
1024
1024
512
512
512
1024
LiveVLM
196
1024
4096
2048
4096
512
Appendix
Table 6: Prefill window sizes used by FrameBlend. Values denote the number of video tokens.
Figure 6: Prefill window size sweep with full recomputation. We evaluate the accuracy of ReKV with FrameBlend across different prefill window sizes under full recomputation.
Figure 7: Efficiency at varying numbers of input frames. We analyze LiveVLM and HERMES with FrameBlend under varying video lengths and observe that the efficiency gain increases as the video becomes longer.
Streaming video understanding studies how AI systems interpret continuous video in real time while retaining useful history and responding at the right moment.
Streaming video understanding requires models to robustly encode, store, and retrieve information from a continuous video stream to support accurate video question answering (VQA). Existing state-of-the-art approaches rely on key-value caching to accumulate frame-level information over time, but use a limited number of tokens per frame, leading to the loss of fine-grained visual details. In this work, we propose scaling the token budget to enable more granular spatiotemporal understanding and reasoning. First, we find that current methods are ill-equipped to handle dense streams: their feature encoding causes query-frame similarity scores to increase over time, biasing retrieval toward later frames. To address this, we introduce an adaptive selection strategy that reduces token redundancy while preserving local spatiotemporal information. We further propose a training-free retrieval mixture-of-experts that leverages external models to better identify relevant frames. Our method, MemStream, achieves +8.0% on CG-Bench, +8.5% on LVBench, and +2.4% on VideoMME (Long) over ReKV with Qwen2.5-VL-7B.
Streaming video understanding requires processing unbounded video streams with limited memory and computation, posing two key challenges. First, continuously constructing new and evicting old key-value(KV) caches is required for unbounded streams. Secondly, due to the high cost of collecting and training on unbounded streams, models must learn from short sequences while generalizing to long streams. Existing streaming VideoVLLMs fail to scale to unbounded video streams or focus on cache reuse strategies, leaving the impact of cache construction underexplored. In this paper, we propose Decoupled Streaming Cache(DSCache), a training-free cache construction mechanism that adapts pretrained offline models to streaming settings. DSCache maintains a cumulative past KV cache while constructing a separate instant cache on-demand, decoupled from past caches to preserve the informativeness of recent inputs. To enable position extrapolation beyond the training length, DSCache further incorporates a position-agnostic encoding strategy, ensuring KV caches to support unseen positions and preventing position overflow. Experiments on Streaming Video QA benchmarks demonstrate DSCache's state-of-the-art performance, with an average 2.5% accuracy gains over prior methods.
Streaming video understanding (SVU) must answer queries that arrive asynchronously while visual tokens stream continuously under strict GPU-memory and query-time latency budgets. A key challenge is delayed query: decisive cues may appear briefly, yet many subsequent updates occur before the query arrives, increasing the risk that those cues are evicted or diluted under bounded memory. We propose ProtoKV, a constant-footprint SVU memory that represents far history as a fixed-capacity summary state rather than retaining token instances. ProtoKV keeps an exact near-window KV cache and aggregates older content into a semantic-spatial prototype bank with residual statistics. At query time, each prototype is exposed through a bounded pseudo-token interface that is drop-in compatible with standard attention. Under matched budgets and comparable query-time cost, ProtoKV improves accuracy by up to 12.5 points over token-retention baselines on SVU benchmarks in the long-delay regime, with gains that grow as query delay increases.
Le Tu Ngoc Minh, Jinyeong Lim, Dongsu Han
KAIST · School of Electrical Engineering, KAIST, Daejeon, Republic of Korea · Kim Jaechul Graduate School of AI, KAIST, Daejeon, Republic of Korea