Think-as-You-See: Streaming Chain-of-Thought Reasoning for Large Vision-Language Models

Jialiang Zhang; Junlong Tong; Junyan Lin; Hao Wu; Yirong Sun; Yunpu Ma; Xiaoyu Shen

cs.CVMar 3, 2026

Think-as-You-See: Streaming Chain-of-Thought Reasoning for Large Vision-Language Models

Authors: Jialiang Zhang, Junlong Tong, Junyan Lin, Hao Wu, Yirong Sun, Yunpu Ma, Xiaoyu Shen

Organizations: Institute of Digital Twin, Eastern Institute of Technology, Ningbo · Ocean University of China · Shanghai Jiao Tong University · The Hong Kong Polytechnic University · Munich Center for Machine Learning, LMU Munich · Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative

Abstract

Large Vision Language Models (LVLMs) exhibit strong Chain-of-Thought (CoT) capabilities, yet most existing paradigms assume full-video availability before inference, a batch-style process misaligned with real-world video streams where information arrives sequentially. Motivated by the streaming nature of video data, we investigate two streaming reasoning paradigms for LVLMs. The first, an interleaved paradigm, alternates between receiving frames and producing partial reasoning but remains constrained by strictly ordered cache updates. To better match streaming inputs, we propose Think-as-You-See (TaYS), a unified framework enabling true concurrent reasoning. TaYS integrates parallelized CoT generation, stream-constrained training, and stream-parallel inference. It further employs temporally aligned reasoning units, streaming attention masks and positional encodings, and a dual KV-cache that decouples visual encoding from textual reasoning. We evaluate all paradigms on the Qwen2.5-VL family across representative video CoT tasks, including event dynamics analysis, causal reasoning, and thematic understanding. Experiments show that TaYS consistently outperforms both batch and interleaved baselines, improving reasoning performance while substantially reducing time-to-first-token (TTFT) and overall reasoning delay. These results demonstrate the effectiveness of data-aligned streaming reasoning in enabling efficient and responsive video understanding for LVLMs. We release our code at https://github.com/EIT-NLP/StreamingLLM/tree/main/TaYS

Figures & tables

Figure 1: Conventional LVLM reasoning adheres to the batch thinking paradigm, deferring inference until the entire input is received. This approach often leads to high latency and uneven attention allocation across inputs. In contrast, our proposed streaming thinking paradigm enables LVLMs to reason concurrently with input reception, thereby reducing latency and ensuring consistency between attention and input order.

Figure 2: Overview of the two-step process for generating Streaming Video CoT. Step 1 Adjust the frame ID while maintaining frame caption alignment. Step 2 Generate a progressive frame aware trajectory using the original annotations.

Figure 3: Overview of the streaming reasoning framework. (a) Parallel video reasoning KV caches enable concurrent visual encoding and reasoning generation via dynamic merge and split operations. (b) The streaming attention mask enforces causal alignment between frames and reasoning steps. (c) During inference, parallel information flow reduces attention path length and alleviates sequential blocking compared with interleaved paradigms.

Model	Narr.	Event	Ingr.	Caus.	Theme	Cont.	Infl.	Role	Inter.	Behav.	Emot.	Cook.	Traff.	Situa.	Acc. $\uparrow$
Qwen2.5-VL-3B-Instruct
Batch w/o thinking	39.39	31.51	25.00	18.92	37.50	24.32	43.90	20.69	29.73	16.67	42.11	38.89	30.77	10.00	27.99
Batch with thinking	39.39	26.03	28.57	15.68	37.50	40.54	17.07	48.28	29.73	8.33	47.37	61.11	38.46	20.00	28.16
Batch SFT	48.48	34.25	17.86	17.84	43.75	32.43	39.02	27.59	24.32	8.33	39.47	50.00	46.15	20.00	29.18
Interleaved SFT	48.48	38.36	25.00	28.65	37.50	29.73	36.59	31.03	35.14	16.67	36.84	55.56	46.15	30.00	33.96
TaYS	51.52	36.99	39.29	24.86	46.88	21.62	31.71	20.69	37.84	33.33	44.74	50.00	53.85	20.00	33.45

Table 1: Comparison of reasoning accuracy on the extended VideoEspresso benchmark. TaYS consistently achieves competitive or superior performance while maintaining low latency, demonstrating the effectiveness of the streaming reasoning paradigm. In the table, bold numbers denote the best results, and underlined numbers indicate the second-best results for each task category.

Figure 4: Case study comparing TaYS with the Interleaved paradigm. TaYS produces temporally aligned reasoning, whereas the Interleaved model generates less accurate, fragmented descriptions.

Method	Metric	FPS=1	FPS=2	FPS=3	FPS=4	FPS=5
Batch	TTFT $\downarrow$	10.36	10.48	10.62	10.77	10.93
	Delay $\downarrow$	12.05	13.90	12.93	13.08	13.12
	Acc $\uparrow$	28.33	29.18	31.23	30.03	31.91
Interleaved	TTFT $\downarrow$	0.0303	0.0295	0.0296	0.0301	0.0298
	Delay $\downarrow$	12.94	14.19	16.15	18.03	20.13
	Acc $\uparrow$	33.95	33.96	33.11	31.91	30.55

Table 2: Latency and accuracy comparison across different FPS. TaYS achieves the lowest TTFT and delay, demonstrating superior real-time efficiency.

Figure 5: (a) Latency comparison across paradigms. (b) Latency breakdown of TaYS. Parallel KV Cache design enables the lowest TTFT and stable delay.

Figure 6: Temporal distance

\Delta t

distribution. TaYS aligns reasoning more closely with keyframes, achieving higher precision than the interleaved baseline.

Figure 7: Semantic similarity between consecutive reasoning steps. TaYS maintains a smoother distribution, whereas the interleaved model exhibits repetitive peaks (high similarity), indicating redundancy.

Figure 8: Task distribution in the dataset.

Figure 9: Distribution of keyframe counts per sample.

Task	Train Set	Test Set
Causal Analysis	52,566	208
Event Dynamic Analysis	18,675	82
Preparation Steps / Ingredient Analysis	2,252	74
Theme Analysis	6,206	33
Interaction Analysis	4,208	38
Influence Analysis	4,406	45

Table 3: Distribution of task categories in training and test sets.

Figure 10: Prompt template for QA construction.

Figure 11: Prompt template for CoT inference.

Figure 12: Prompt template for subjective evaluation.

Config	Value
input resolution	variable (pixel-constrained)
max token length	8192
vision encoder	frozen
trainable modules	LLM + MLP projector
precision	bfloat16
optimizer	AdamW

Table 4: Training hyperparameters for TaYS.

Explore similar work

Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously
Mar 12, 2026Yiran Guan, Liang Yin, Dingkang Liang +5Streaming Video Understanding VLM Reasoning
Huazhong University of Science and Technology, Wuhan, China · MiLM Plus, Xiaomi Inc., Beijing, China
Thinking in Streaming Video
Mar 13, 2026Zikang Liu, Longteng Guo, Handong Li +7Streaming Video Understanding Memory-Augmented Video Understanding
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · OPPO AI Center, OPPO Inc.
Think While Watching: Online Streaming Segment-Level Memory for Multi-Turn Video Reasoning in Multimodal Large Language Models
Mar 12, 2026Lu Wang, Zhuoran Jin, Yupu Hao +4Streaming Video Understanding Video Reasoning
The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · Beijing Academy of Artificial Intelligence · Beijing Academy of Artificial Intelligence (BAAI), Beijing, China

Mar 12, 2026J/K move · Enter open · S save

Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously

Yiran Guan, Liang Yin, Dingkang Liang, Jianzhong Ju+4

Huazhong University of Science and Technology, Wuhan, China · MiLM Plus, Xiaomi Inc., Beijing, China

Streaming Video Understanding VLM Reasoning

Open paper AskPDF arXiv

Online Video Large Language Models (VideoLLMs) play a critical role in supporting responsive, real-time interaction. Existing methods focus on streaming perception, lacking a synchronized logical reasoning stream. However, directly applying test-time scaling methods incurs unacceptable response latency. To address this trade-off, we propose Video Streaming Thinking (VST), a novel paradigm for streaming video understanding. It supports a thinking while watching mechanism, which activates reasoning over incoming video clips during streaming. This design improves timely comprehension and coherent cognition while preserving real-time responsiveness by amortizing LLM reasoning latency over video playback. Furthermore, we introduce a comprehensive post-training pipeline that integrates VST-SFT, which structurally adapts the offline VideoLLM to causal streaming reasoning, and VST-RL, which provides end-to-end improvement through self-exploration in a multi-turn video interaction environment. Additionally, we devise an automated training-data synthesis pipeline that uses video knowledge graphs to generate high-quality streaming QA pairs, with an entity-relation grounded streaming Chain-of-Thought to enforce multi-evidence reasoning and sustained attention to the video stream. Extensive evaluations show that VST-7B performs strongly on online benchmarks, e.g. 79.5% on StreamingBench and 59.3% on OVO-Bench. Meanwhile, VST remains competitive on offline long-form or reasoning benchmarks. Compared with Video-R1, VST responds 15.7 times faster and achieves +5.4% improvement on VideoHolmes, demonstrating higher efficiency and strong generalization across diverse video understanding tasks. Code, data, and models will be released at https://github.com/1ranGuan/VST.

Abstract

Explore similar work

Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously

Thinking in Streaming Video

Think While Watching: Online Streaming Segment-Level Memory for Multi-Turn Video Reasoning in Multimodal Large Language Models