Streaming video understanding (SVU) requires video-LLMs to process incoming video continuously while answering user queries with low latency. Recent methods reduce query-time cost by prefilling frames into KV caches during idle time and reusing the cached KVs at query time. To bound per-frame computation over unbounded video streams, these methods restrict the context available during prefill. This creates a fundamental context mismatch: the restricted context may omit information needed to properly contextualize video tokens, yet, still contain information irrelevant to the eventual query, which is unknown during prefill. We propose FrameBlend, a simple yet effective plug-and-play method to address this mismatch through selective KV recomputation, without requiring any additional training. At query time, FrameBlend re-prefills selected video tokens using query-relevant context and replaces their cached KVs with recomputed ones. Crucially, this recomputation enables shorter attention windows during idle-time prefill, substantially reducing prefill cost while improving accuracy. Experiments across three SVU frameworks and five streaming and offline benchmarks demonstrate accuracy gains of up to 3.9 pp. On RVS-Ego, FrameBlend reduces per-frame prefill FLOPs by over 20% and total processing time by 14-59% for retrieval-based frameworks, including the cost of query-time recomputation. These results demonstrate that a simple change in how contextualization is distributed between idle time and query time can improve the accuracy-efficiency trade-off for SVU.
Figures & tables
Figure 1: Motivation and performance of FrameBlend. (a) Sliding-window attention can omit inter-token context needed at query time. (b) KV compression can retain unnecessary context from evicted tokens. (c) FrameBlend improves SVU framework accuracy while reducing total latency.
Figure 2: Overview of FrameBlend. FrameBlend uses a short attention window during idle-time prefill and selectively recomputes the KVs of query-relevant tokens at query time.
Figure 3: Effect of window size. Accuracy of ReKV with FrameBlend on OVO-Bench.
RVS-Ego
RVS-Movie
OVO-Bench
Method
Acc.
Score
Acc.
Score
Real-time
Backward
Avg.
LLaVA-OV-7B
ReKV
59.3
3.93
49.2
3.45
61.5
43.2
52.3
ReKV + FrameBlend
62.4
3.99
50.1
3.41
63.0
44.6
53.8
LiveVLM
60.1
3.95
51.5
3.50
63.6
43.5
53.5
Table 1: FrameBlend on streaming benchmarks. We compare the accuracy of various SVU frameworks with and without FrameBlend on streaming VQA benchmarks.
Video-MME
MLVU
EgoSchema
LLaVA-OV-7B
Short
Medium
Long
Total
Holistic
Single- Detail
Multi- Detail
Total
Total
ReKV
71.2
55.3
46.9
57.8
83.2
73.1
41.1
68.4
60.0
ReKV + FrameBlend
70.7
57.0
50.7
59.4
83.6
73.4
41.3
68.7
60.3
LiveVLM
68.4
56.2
48.4
57.7
82.7
70.8
41.1
67.0
60.6
LiveVLM + FrameBlend
69.6
56.1
48.8
58.1
83.4
71.9
42.8
68.1
60.5
Table 2: FrameBlend on offline benchmarks. We compare the accuracy of various SVU frameworks with and without FrameBlend on offline VQA benchmarks.
Prefill (idle time)
Answering (query time)
Total
LLaVA-OV-7B
TFLOPs/f
ms/f
Recomp. (s)
TTFT (s)
TFLOPs
Latency (s)
ReKV
4.38
81.1
–
0.17
4647.8
90.6
ReKV + FrameBlend
3.24
32.0
0.77
0.93
3567.4
37.4
LiveVLM
3.78
32.6
–
0.04
4007.2
37.6
LiveVLM + FrameBlend
2.94
28.5
0.77
0.84
3247.6
32.3
Table 3: Efficiency metrics. We compare SVU methods with and without FrameBlend on RVS-Ego.
Figure 4: Efficiency at varying numbers of input frames. We analyze ReKV with FrameBlend under varying video lengths, and observe that the efficiency gain increases as the video gets longer.
Figure 5: TTFT–accuracy trade-off. We compare ReKV with and without FrameBlend, and the offline LLaVA-OV-7B using 64 uniformly sampled frames.
OVO-Bench
RVS-Movie
RVS-Ego
LLaVA-OV-7B
Acc.
Added TTFT (s)
Acc.
Added TTFT (s)
Acc.
Added TTFT (s)
ReKV
52.3
–
49.2
–
59.3
–
ReKV + CacheBlend
52.7
0.90
48.0
1.51
61.4
2.26
ReKV + FrameBlend
53.8
0.67
50.1
0.75
62.4
0.76
Table 4: Comparison against CacheBlend. Accuracy and TTFT when applied on ReKV.
RVS-Ego
RVS-Movie
OVO-Bench
LLaVA-OV-7B
Acc.
Score
Acc.
Score
Real-time
Backward
Avg.
ReKV
59.3
3.93
49.2
3.45
61.5
43.2
52.3
+ short window
60.7
3.94
46.8
3.30
63.3
43.0
53.1
+ recomputation
58.7
3.94
48.9
3.46
61.7
44.8
53.2
+ both
62.4
3.99
50.1
3.41
63.0
44.6
53.8
Table 5: Ablation studies. We ablate two key components of FrameBlend.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Streaming benchmarks
Offline benchmarks
Method
OVO-Bench
RVS-Ego
RVS-Movie
Video-MME
MLVU
EgoSchema
LLaVA-OV-7B
ReKV
2048
196
196
8192
8192
384
HERMES
1024
1024
512
512
512
1024
LiveVLM
196
1024
4096
2048
4096
512
Appendix
Table 6: Prefill window sizes used by FrameBlend. Values denote the number of video tokens.
Figure 6: Prefill window size sweep with full recomputation. We evaluate the accuracy of ReKV with FrameBlend across different prefill window sizes under full recomputation.
Figure 7: Efficiency at varying numbers of input frames. We analyze LiveVLM and HERMES with FrameBlend under varying video lengths and observe that the efficiency gain increases as the video becomes longer.
Streaming video understanding studies how AI systems interpret continuous video in real time while retaining useful history and responding at the right moment.