Thinking in Streaming Video

Zikang Liu; Longteng Guo; Handong Li; Ru Zhen; Xingjian He; Ruyi Ji; Xiaoming Ren; Yanhao Zhang

cs.CVMar 13, 2026

Thinking in Streaming Video

Authors: Zikang Liu, Longteng Guo, Handong Li, Ru Zhen, Xingjian He, Ruyi Ji, Xiaoming Ren, Yanhao Zhang, +2 more

Organizations: Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · OPPO AI Center, OPPO Inc.

Abstract

Real-time understanding of continuous video streams is essential for interactive assistants and multimodal agents operating in dynamic environments. However, most existing video reasoning approaches follow a batch paradigm that defers reasoning until the full video context is observed, resulting in high latency and growing computational cost that are incompatible with streaming scenarios. In this paper, we introduce ThinkStream, a framework for streaming video reasoning based on a Watch--Think--Speak paradigm that enables models to incrementally update their understanding as new video observations arrive. At each step, the model performs a short reasoning update and decides whether sufficient evidence has accumulated to produce a response. To support long-horizon streaming, we propose Reasoning-Compressed Streaming Memory (RCSM), which treats intermediate reasoning traces as compact semantic memory that replaces outdated visual tokens while preserving essential context. We further train the model using a Streaming Reinforcement Learning with Verifiable Rewards scheme that aligns incremental reasoning and response timing with the requirements of streaming interaction. Experiments on multiple streaming video benchmarks show that ThinkStream significantly outperforms existing online video models while maintaining low latency and memory usage. Code, models and data will be released at https://github.com/johncaged/ThinkStream

Figures & tables

Figure 1: Illustration of the Streaming Watch–Think–Speak paradigm. As video chunks arrive sequentially, the model continuously updates its understanding through incremental reasoning steps ( <think> ). Each update integrates newly observed evidence with accumulated context. Based on this evolving interpretation, the model decides whether sufficient evidence has been gathered to produce a response ( <response> ) or whether it should remain silent ( <silent> ) and continue observing the stream.

Figure 2: Overview of the ThinkStream framework. (a) Streaming Watch-Think-Speak Paradigm & RLVR: The model undergoes streaming rollouts and policy updates driven by format, latency, and accuracy rewards. (b) Reasoning-Compressed Streaming Memory: Outdated dense video tokens are dynamically evicted from the KV cache, while highly compressed reasoning and response tokens are retained as long-term semantic anchors. (c) Streaming Inference: A custom backend utilizes Eager Prefill for variable tokens and replayable CUDA Graphs for both the Decode and Evict Kernels, enabling an efficient chunk-by-chunk streaming loop with in-place memory shifting.

Model	Real-Time Visual Perception							Backward Tracing				Avg.
Model	OCR	ACR	ATR	STU	FPD	OJR	Avg.	EPM	ASI	HLD	Avg.	Avg.
Open-source Offline Models
LLaVA-Video-7B [ 46 ]	69.13	58.72	68.83	49.44	74.26	59.78	63.52	56.23	57.43	7.53	40.4	51.96
Qwen2-VL-7B [ 29 ]	60.4	50.46	56.03	47.19	66.34	55.43	55.98	47.81	35.48	56.08	46.46	51.22
LongVU-7B [ 24 ]	53.69	53.21	62.93	47.75	68.32	59.78	57.61	40.74	59.46	4.84	35.01	46.31
Qwen2.5-VL-3B [ 2 ]	76.51	44.03	67.24	42.13	68.31	61.96	60.03	50.50	53.38	22.04	41.98	51.00

Table 1: Performance comparison of ThinkStream and existing open-source offline and online models on the OVO-Bench streaming video benchmark. ThinkStream-3B achieves a strong average score, significantly surpassing its base model and competing online models.

Proprietary MLLMs
Model	OP	CR	CS	ATP	EU	TR	PR	SU	ACP	CT	Avg.
Human	89.47	92.00	93.60	91.47	95.65	92.52	88.00	88.75	89.74	91.30	91.46
Gemini 1.5 pro [ 11 ]	79.02	80.47	83.54	79.67	80.00	84.74	77.78	64.23	71.95	48.70	75.69
GPT-4o [ 13 ]	77.11	80.47	83.91	76.47	70.19	83.80	66.67	62.19	69.12	49.22	73.28
Claude 3.5 Sonnet	73.33	80.47	84.09	82.02	75.39	79.53	61.11	61.79	69.32	43.09	72.44
Open-source Offline MLLMs

Table 2: Performance comparison of various MLLMs on the StreamingBench Real-Time benchmark. ThinkStream-3B demonstrates highly competitive performance against proprietary models such as GPT-4o and vastly exceeds other open-source online MLLMs.

Model	OVO Real-Time	OVO Backward	VideoMME	LongVideoBench	Avg.
Flash-VStream-7B	28.4	27.4	61.2	-	-
VideoLLM-online-8B	20.8	17.7	26.9	-	-
Dispider-7B	54.6	36.1	57.2	-	-
Streamo-3B	61.5	41.8	61.8	56.2	55.3
Qwen2.5-VL-3B	60.0	41.9	61.5	54.2	54.4
ThinkStream-3B (Ours)	67.0 ( $\uparrow$ 7.0 )	52.3 ( $\uparrow$ 10.4 )	61.9 ( $\uparrow$ 0.8 )	56.4 ( $\uparrow$ 1.8 )	59.4 ( $\uparrow$ 5.0 )

Table 3: Model performance comparison on standard offline video benchmarks, including VideoMME and Long VideoBench. Despite aggressively evicting visual tokens, ThinkStream-3B achieves highly competitive performance, demonstrating that it effectively preserves understanding capabilities on offline tasks.

Table 6

Memory Variant	Streaming.	OVO-BW	OVO-RT	Avg.
Qwen2.5-VL-3B	68.9	41.9	60.0	56.9
Discrete caption as memory	59.0	36.9	50.1	48.7
No memory	69.6	41.8	59.2	56.9
Cold-start CoT memory	70.6	47.6	63.3	60.5
RLVR-optimized CoT memory	75.0	52.3	67.0	64.8

Table 6: Ablation on Stream Memory Representation and RLVR Optimization. Post-training with RLVR demonstrates that reasoning tokens act as a highly compressed long-term memory representation.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 5 — Figure 5: Statistics of our ThinkStream Dataset. We show (a) the hierarchical composition of data and (b) the distribution of thinking process lengths.

Model	Real-Time Visual Perception							Backward Tracing				Avg.
Model	OCR	ACR	ATR	STU	FPD	OJR	Avg.	EPM	ASI	HLD	Avg.	Avg.
Open-source Offline Models
LLaVA-Video-7B [ 46 ]	69.13	58.72	68.83	49.44	74.26	59.78	63.52	56.23	57.43	7.53	40.4	51.96
Qwen2-VL-7B [ 29 ]	60.4	50.46	56.03	47.19	66.34	55.43	55.98	47.81	35.48	56.08	46.46	51.22
LongVU-7B [ 24 ]	53.69	53.21	62.93	47.75	68.32	59.78	57.61	40.74	59.46	4.84	35.01	46.31
Qwen2.5-VL-3B [ 2 ]	76.51	44.03	67.24	42.13	68.31	61.96	60.03	50.50	53.38	22.04	41.98	51.00

Appendix

Table 7: Performance comparison of ThinkStream and existing open-source offline and online models on the OVO-Bench streaming video benchmark.

Explore similar work

Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously
Mar 12, 2026Yiran Guan, Liang Yin, Dingkang Liang +5Streaming Video Understanding VLM Reasoning
Huazhong University of Science and Technology, Wuhan, China · MiLM Plus, Xiaomi Inc., Beijing, China
Watch-Think-Interact: Bootstrapping Long-Horizon Multi-Turn Streaming Video Reasoning with Reinforcement Learning
Sep 29, 2026Ziheng Huang, Yicheng Bao, Xueheng Li +8Streaming Video Understanding Video QA
East China Normal University · University of Science and Technology of China · Xiamen University
Think-as-You-See: Streaming Chain-of-Thought Reasoning for Large Vision-Language Models
Mar 3, 2026Jialiang Zhang, Junlong Tong, Junyan Lin +4VLM Reasoning Streaming Video Understanding
Institute of Digital Twin, Eastern Institute of Technology, Ningbo · Ocean University of China · Shanghai Jiao Tong University +3

Mar 12, 2026J/K move · Enter open · S save

Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously

Yiran Guan, Liang Yin, Dingkang Liang, Jianzhong Ju+4

Huazhong University of Science and Technology, Wuhan, China · MiLM Plus, Xiaomi Inc., Beijing, China

Streaming Video Understanding VLM Reasoning

Open paper AskPDF arXiv

Online Video Large Language Models (VideoLLMs) play a critical role in supporting responsive, real-time interaction. Existing methods focus on streaming perception, lacking a synchronized logical reasoning stream. However, directly applying test-time scaling methods incurs unacceptable response latency. To address this trade-off, we propose Video Streaming Thinking (VST), a novel paradigm for streaming video understanding. It supports a thinking while watching mechanism, which activates reasoning over incoming video clips during streaming. This design improves timely comprehension and coherent cognition while preserving real-time responsiveness by amortizing LLM reasoning latency over video playback. Furthermore, we introduce a comprehensive post-training pipeline that integrates VST-SFT, which structurally adapts the offline VideoLLM to causal streaming reasoning, and VST-RL, which provides end-to-end improvement through self-exploration in a multi-turn video interaction environment. Additionally, we devise an automated training-data synthesis pipeline that uses video knowledge graphs to generate high-quality streaming QA pairs, with an entity-relation grounded streaming Chain-of-Thought to enforce multi-evidence reasoning and sustained attention to the video stream. Extensive evaluations show that VST-7B performs strongly on online benchmarks, e.g. 79.5% on StreamingBench and 59.3% on OVO-Bench. Meanwhile, VST remains competitive on offline long-form or reasoning benchmarks. Compared with Video-R1, VST responds 15.7 times faster and achieves +5.4% improvement on VideoHolmes, demonstrating higher efficiency and strong generalization across diverse video understanding tasks. Code, data, and models will be released at https://github.com/1ranGuan/VST.

Abstract

Explore similar work

Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously

Watch-Think-Interact: Bootstrapping Long-Horizon Multi-Turn Streaming Video Reasoning with Reinforcement Learning

Think-as-You-See: Streaming Chain-of-Thought Reasoning for Large Vision-Language Models