Online Video Large Language Models (VideoLLMs) play a critical role in supporting responsive, real-time interaction. Existing methods focus on streaming perception, lacking a synchronized logical reasoning stream. However, directly applying test-time scaling methods incurs unacceptable response latency. To address this trade-off, we propose Video Streaming Thinking (VST), a novel paradigm for streaming video understanding. It supports a thinking while watching mechanism, which activates reasoning over incoming video clips during streaming. This design improves timely comprehension and coherent cognition while preserving real-time responsiveness by amortizing LLM reasoning latency over video playback. Furthermore, we introduce a comprehensive post-training pipeline that integrates VST-SFT, which structurally adapts the offline VideoLLM to causal streaming reasoning, and VST-RL, which provides end-to-end improvement through self-exploration in a multi-turn video interaction environment. Additionally, we devise an automated training-data synthesis pipeline that uses video knowledge graphs to generate high-quality streaming QA pairs, with an entity-relation grounded streaming Chain-of-Thought to enforce multi-evidence reasoning and sustained attention to the video stream. Extensive evaluations show that VST-7B performs strongly on online benchmarks, e.g. 79.5% on StreamingBench and 59.3% on OVO-Bench. Meanwhile, VST remains competitive on offline long-form or reasoning benchmarks. Compared with Video-R1, VST responds 15.7 times faster and achieves +5.4% improvement on VideoHolmes, demonstrating higher efficiency and strong generalization across diverse video understanding tasks. Code, data, and models will be released at https://github.com/1ranGuan/VST.
Figures & tables
Figure 1: Benchmark results and paradigm comparison. (a) VST-7B delivers strong performance on online and offline video understanding benchmarks while maintaining low QA latency. (b) Existing streaming VideoLLMs focus on efficient streaming processing, but lack explicit analytical reasoning. (c) VideoLLM with CoT performs heavy post-query step-by-step reasoning to improve performance, but incurs high QA latency. (d) Our method introduces proactive pre-query reasoning, interleaving it with video consumption to achieve both strong performance and efficient responsiveness.
Figure 2: Illustration of the Video Streaming Thinking pipeline. The model employs a streaming thought mechanism to compress visual dynamics into a long-term textual memory. Combined with the short-term visual buffer, this enables efficient reasoning over indefinite video streams with fixed memory budgets.
Figure 3: Overview of the training pipeline. (a) VST-SFT applies a streaming attention mask to enforce temporal causality, restricting attention to the current visual buffer and history textual context. (b) VST-RL performs on-policy optimization via an agentic loop, improving the quality of streaming thoughts through verifiable rewards computed solely from the final answer.
Figure 4: Stream-Thought QA data curation pipeline. We incrementally extract video entities and relations to build a knowledge graph, sample multi-hop evidence chains, and use Gemini to generate streaming QA pairs with grounded streaming thoughts, followed by automatic filtering.
Model
Venue
OP
CR
CS
ATP
EU
TR
PR
SU
ACP
CT
Overall
Proprietary Models
Gemini 1.5 pro [ 43 ]
-
79.0
80.5
83.5
79.7
80.0
84.7
77.8
64.2
72.0
48.7
75.7
GPT-4o [ 36 ]
-
77.1
80.5
83.9
76.5
70.2
83.8
66.7
62.2
69.1
49.2
73.3
Open-source Offline Models
VILA-1.5-8B [ 30 ]
CVPR’24
53.7
49.2
71.0
56.9
53.4
53.9
54.6
48.8
50.1
17.6
52.3
LongVA-7B [ 66 ]
TMLR’25
70.0
63.3
61.2
70.9
62.7
59.5
61.1
53.7
54.7
34.7
60.0
Table 1: Comparison of offline and online VideoLLMs on StreamingBench Real-Time understanding tasks.
Model
Venue
Real-Time
Backward
Forward
Overall
OCR
ACR
ATR
STU
FPD
OJR
Avg.
EPM
ASI
HLD
Avg.
REC
SSR
CRR
Avg.
Avg.
Proprietary Models
Gemini 1.5 pro [ 43 ]
-
85.9
67.0
79.3
58.4
63.4
62.0
69.3
58.6
76.4
52.6
62.5
35.5
74.2
61.7
57.2
63.0
GPT-4o [ 36 ]
-
69.8
64.2
71.6
51.1
70.3
59.8
64.5
57.9
75.7
48.7
60.8
27.6
73.2
59.4
53.4
59.5
Open-source Offline Models
Qwen2-VL-72B [ 47 ]
-
65.8
60.6
69.8
51.7
69.3
54.4
61.9
52.5
60.8
57.5
57.0
38.8
64.1
45.0
49.3
56.3
Table 2: Comparison of offline and online VideoLLMs on OVO-Bench.
Model
Venue
VideoMME w/o sub.
LongVideoBench
VideoHolmes
Long
Overall
Proprietary Models
Gemini 1.5 pro [ 43 ]
-
67.4
75.0
64.0
45.7
GPT-4o [ 36 ]
-
65.3
71.9
66.7
42.0
Open-source Offline Models
LongVA-7B [ 66 ]
TMLR’25
47.6
54.3
56.3
-
Table 3: Comparison of offline and online VideoLLMs on VideoMME (without subtitles), LongVideoBench, and VideoHolmes.
Model & Config
OVO-Bench
VideoMME
Backward
Forward
Overall
w/o sub. Overall
Qwen2.5-VL-7B (Base model)
47.5
41.9
50.5
62.9
Ablation on VST-SFT training data
+LLava-Vid (50K)
49.9
42.4
52.3
61.8
+LLava-Vid (30K) & VST (20K)
52.0
50.1
56.8
62.5
+LLava-Vid (20K) & VST (30K)
53.3
50.0
57.1
63.1
Table 4: Ablation study on VST training schedule.
Figure 5: Ablation study on max thinking times.
Size
Model
OVOB. Overall
StreamingB. Realtime
V-MME w/o sub. Overall
LongVideoB.
VideoHolmes
3B
Qwen2.5-VL
53.1
67.8
57.9
53.3
30.7
VST
56.2
75.5
59.5
54.1
36.1
7B
Qwen2.5-VL
55.0
71.7
62.3
54.7
32.9
VST
59.3
79.5
64.9
58.0
41.9
32B
Qwen2.5-VL
60.1
71.5
65.8
59.8
40.1
VST
63.5
80.7
67.2
60.7
45.1
Table 5: Ablation study on the scale of the base offline VideoLLM. All results in this table are obtained with the VST inference pipeline, where Qwen is evaluated as a zero-shot inference baseline.
Online / Offline Method
QA Latency
Qwen2.5-VL-7B
0.54s
Qwen2.5-VL-7B w/CoT
5.30s
Video-R1 w/CoT
8.80s
VideoLLM-online-8B
0.38s
Dispider-7B
1.10s
VST-3B (Ours)
0.53s
Table 6: Inference Latency.
Figure 6: Case Study from VideoHolmes. We compare VST-7B with Video-R1-7B. VST-7B processes the video stream and performs streaming thinking before the query, then answers directly once the query arrives. In contrast, Video-R1-7B generates CoT after the query, resulting in higher QA latency. VST-7B achieves better performance with lower QA latency in this example.
Figure 7: The streaming inference pipeline of VST. By generating stream thoughts for incoming video clips before a user query arrives, VST effectively hides reasoning latency and enables rapid QA responses.
Real-time understanding of continuous video streams is essential for interactive assistants and multimodal agents operating in dynamic environments. However, most existing video reasoning approaches follow a batch paradigm that defers reasoning until the full video context is observed, resulting in high latency and growing computational cost that are incompatible with streaming scenarios. In this paper, we introduce ThinkStream, a framework for streaming video reasoning based on a Watch--Think--Speak paradigm that enables models to incrementally update their understanding as new video observations arrive. At each step, the model performs a short reasoning update and decides whether sufficient evidence has accumulated to produce a response. To support long-horizon streaming, we propose Reasoning-Compressed Streaming Memory (RCSM), which treats intermediate reasoning traces as compact semantic memory that replaces outdated visual tokens while preserving essential context. We further train the model using a Streaming Reinforcement Learning with Verifiable Rewards scheme that aligns incremental reasoning and response timing with the requirements of streaming interaction. Experiments on multiple streaming video benchmarks show that ThinkStream significantly outperforms existing online video models while maintaining low latency and memory usage. Code, models and data will be released at https://github.com/johncaged/ThinkStream
Zikang Liu, Longteng Guo, Handong Li +7
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · OPPO AI Center, OPPO Inc.
Large Vision Language Models (LVLMs) exhibit strong Chain-of-Thought (CoT) capabilities, yet most existing paradigms assume full-video availability before inference, a batch-style process misaligned with real-world video streams where information arrives sequentially. Motivated by the streaming nature of video data, we investigate two streaming reasoning paradigms for LVLMs. The first, an interleaved paradigm, alternates between receiving frames and producing partial reasoning but remains constrained by strictly ordered cache updates. To better match streaming inputs, we propose Think-as-You-See (TaYS), a unified framework enabling true concurrent reasoning. TaYS integrates parallelized CoT generation, stream-constrained training, and stream-parallel inference. It further employs temporally aligned reasoning units, streaming attention masks and positional encodings, and a dual KV-cache that decouples visual encoding from textual reasoning. We evaluate all paradigms on the Qwen2.5-VL family across representative video CoT tasks, including event dynamics analysis, causal reasoning, and thematic understanding. Experiments show that TaYS consistently outperforms both batch and interleaved baselines, improving reasoning performance while substantially reducing time-to-first-token (TTFT) and overall reasoning delay. These results demonstrate the effectiveness of data-aligned streaming reasoning in enabling efficient and responsive video understanding for LVLMs. We release our code at https://github.com/EIT-NLP/StreamingLLM/tree/main/TaYS
Jialiang Zhang, Junlong Tong, Junyan Lin +4
Institute of Digital Twin, Eastern Institute of Technology, Ningbo · Ocean University of China · Shanghai Jiao Tong University +3
Multimodal large language models (MLLMs) have shown strong performance on offline video understanding, but most are limited to offline inference or have weak online reasoning, making multi-turn interaction over continuously arriving video streams difficult. Existing streaming methods typically use an interleaved perception-generation paradigm, which prevents concurrent perception and generation and leads to early memory decay as streams grow, hurting long-range dependency modeling. We propose Think While Watching, a memory-anchored streaming video reasoning framework that preserves continuous segment-level memory during multi-turn interaction. We build a three-stage, multi-round chain-of-thought dataset and adopt a stage-matched training strategy, while enforcing strict causality through a segment-level streaming causal mask and streaming positional encoding. During inference, we introduce an efficient pipeline that overlaps watching and thinking and adaptively selects the best attention backend. Under both single-round and multi-round streaming input protocols, our method achieves strong results. Built on Qwen3-VL, it improves single-round accuracy by 2.6% on StreamingBench and by 3.79% on OVO-Bench. In the multi-round setting, it maintains performance while reducing output tokens by 56%. Code is available at: https://github.com/wl666hhh/Think_While_Watching/
Lu Wang, Zhuoran Jin, Yupu Hao +4
The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · Beijing Academy of Artificial Intelligence · Beijing Academy of Artificial Intelligence (BAAI), Beijing, China