Organizations: State Key Laboratory of AI Safety, Institute of Computing Technology, CAS · University of Chinese Academy of Sciences · Qwen Business Unit of Alibaba
Proactive streaming video understanding requires assistants to track unfolding events and respond when a user's requested condition is met. To make these decisions, assistants must accumulate relevant evidence and reassess response conditions as new observations arrive. Existing work offers complementary capabilities for this process: proactive video assistants learn when to respond, while streaming video reasoning methods generate scheduled intermediate text to organize observations and support subsequent answers. Together, these capabilities provide a foundation for proactive reasoning, but how to coordinate ongoing reasoning with timely responses remains underexplored. In a direct adaptation, fixed reasoning intervals can trigger updates when nothing relevant has changed or postpone useful updates until the next scheduled step. Sharing one generation path also couples two distinct requirements: preserving evidence for future decisions and producing only the response requested by the user. Moreover, responses generated after reasoning must wait for it to finish, even when sufficient evidence is already available. To address these challenges, we introduce SyncPro, a synchronous dual-stream framework with specialized generation paths for evidence tracking and user-facing responses. Its Inner and Response branches share causal history and activate independently, allowing relevant evidence changes to trigger internal updates and responses to proceed concurrently with ongoing reasoning. We construct 189K dual-stream examples and train SyncPro through supervised fine-tuning followed by trajectory-level reinforcement learning. Experiments across three proactive benchmarks show improved response timing and more accurate answers to user requests.
Figures & tables
Figure 1: Requirements for proactive streaming reasoning. (a) Adaptive reasoning: internal updates follow query-relevant evidence rather than a fixed schedule. (b) Specialized roles: reasoning supports event monitoring, while responses deliver the requested output. (c) Decoupled activation: responses supported by prior monitoring and current evidence need not await further reasoning.
Figure 2: Dual-stream modeling in SyncPro . At each micro-turn, Inner and Response select actions independently and decode from the same multimodal prefix through separate LoRA routes. Only Response is user-visible. Both outputs update the context for the next micro-turn before the visual tokens are discarded to limit persistent cache growth.
Figure 3: Construction of SyncPro -189K. Two routes produce causal dual-stream trajectories. A synthesized trajectory is retained only when its accumulated Inner entries suffice to reconstruct every later response from restricted visual context. The resulting 189,403 trajectories span three interaction functions, eight subtypes, and 12 video sources.
Method
StreamingBench
ProactiveVideoQA
PO
WEB
EGO
VAD
Streaming models
VideoLLM-online ( Chen et al., 2024 )
4.00
25.90
25.00
25.00
MMDuet ( Wang et al., 2025b )
29.44
38.90
46.00
27.40
MMDuet2 ( Wang et al., 2026b )
34.60
53.30
33.60
28.90
Em-Garde ( Zheng et al., 2026 )
38.00
44.30
52.30
27.40
Table 1: Results on StreamingBench and ProactiveVideoQA. StreamingBench’s Proactive Output (PO) uses Acc@2s with a two-second tolerance. ProactiveVideoQA subsets use PAUC ( ω=0.5 ) to evaluate timing and content. Values are percentages.
Method
CRR (P/R/F1)
SSR (P/R/F1)
REC (P/R/F1)
Avg. F1
Streaming models
VideoLLM-online ( Chen et al., 2024 )
2.08/2.08/2.08
6.88/11.07/8.49
16.46/7.43/10.24
6.93
Flash-VStream ( Zhang et al., 2025 )
5.21/6.25/5.68
9.52/2.46/3.91
7.49/3.46/4.73
4.77
StreamForest ( Zeng et al., 2025 )
2.08/2.08/2.08
13.69/8.02/10.11
28.90/30.49/29.67
13.95
MMDuet2 ( Wang et al., 2026b )
8.47/47.92/14.40
–
42.72 /19.33/26.62
20.51
Em-Garde ( Zheng et al., 2026 )
18.22/47.92/26.40
14.87 / 54.88 / 23.40
39.66 /47.33/43.16
30.99
Table 2: OVO-Bench Forward Active Responding. Precision, recall, and F1 use a 2 s tolerance. Avg. F1 is the task mean. Values are percentages. Best/second-best values are bold / underlined .
Method
Content Acc.
Exact Content Acc.
Content Acc. @Response
Exact Content Acc. @Response
Joint @1s
Joint @2s
SyncPro-RL
44.0
25.6
79.14
46.04
23.2
33.6
SyncPro-SFT
40.0
22.8
79.37
45.24
20.8
30.8
Interleaved
39.6
6.0
59.28
8.98
10.0
17.2
Response-only
24.0
10.0
50.85
21.19
6.4
10.4
Table 3: Formulation comparisons on StreamingBench-PO. Content metrics use containment or exact matching. @Response evaluates samples with a response. Values are percentages.
Figure 4: A streaming rollout with scheduled Inner activation.
Table 8
Figure 5: SFT learning dynamics. Left: validation content-token accuracy over training. Right: mean action-event F1 on 50 evaluated rollouts at five checkpoints.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Table 7: State-merge weights. (a) StreamingBench-PO Acc@2s. (b) OVO-FAR precision, recall, and F1 within two seconds. Fixed uses γ=0.5 . Learned per branch optimizes separate coefficients for Inner and Response . Values are percentages.
Figure 6: Long-stream memory and latency of dual-branch inference. We profile a 480-s stream at 1 fps, excluding the first two compilation-affected steps. Fixed-capacity cache allocation keeps persistent memory stable, while the growing retained prefix increases transient peak memory and decoding cost. Panel (d) fixes branch output lengths to isolate how retained KV context length affects decoding latency for each branch activation pattern.
Method
Size
Frames
OP
CR
CS
ATP
EU
TR
PR
SU
ACP
CT
Overall
Human
–
–
89.50
92.00
93.60
91.50
95.70
92.50
88.00
88.80
89.70
91.30
91.50
Proprietary models
Gemini 1.5 Pro
–
1 fps
79.00
80.50
83.50
79.70
80.00
84.70
77.80
64.20
72.00
48.70
75.70
GPT-4o
–
64
77.10
80.50
83.90
76.50
70.20
83.80
66.70
62.20
69.10
49.20
73.30
Claude 3.5 Sonnet
–
20
73.30
80.50
84.10
82.00
75.40
79.50
61.10
61.80
69.30
43.10
72.40
Offline models
Appendix
Table 8: StreamingBench Real-Time Visual Understanding subset. Bold and underlined values mark the best and second-best results among open-source online models.
Method
Size
Frames
Real-Time Visual Perception
Backward Tracing
OCR
ACR
ATR
STU
FPD
OJR
Avg.
EPM
ASI
HLD
Avg.
Proprietary models
Gemini 1.5 Pro
–
–
85.90
67.00
79.30
58.40
63.40
62.00
69.30
58.60
76.40
52.60
62.50
GPT-4o
–
–
69.80
64.20
71.60
51.10
70.30
59.80
64.50
57.90
75.70
48.70
60.80
Offline models
LLaVA-Video
7B
64
69.80
59.60
66.40
50.60
72.30
61.40
63.50
51.20
64.20
9.70
41.70
Appendix
Table 9: OVO-Bench task accuracy. Bold and underlined values mark the best and second-best results among open-source online models.
Step
Tolerance
Activations
Matches
Precision
Recall
F1
100
0
101
34
33.7
43.0
37.8
200
0
75
28
37.3
35.4
36.4
500
0
113
53
46.9
67.1
55.2
1000
0
93
56
60.2
70.9
65.1
1000
1
93
68
73.1
86.1
79.1
1000
2
93
69
74.2
87.3
80.2
Appendix
Table 10: Temporal alignment of Inner updates on 50 rollouts. All settings use the same 79 reference updates. Precision, recall, and F1 pool event counts across cases and are percentages. Tolerance is measured in micro-turns.
Figure 7: Case studies. Inner updates are purple and Response outputs are pink.
Streaming video understanding studies how AI systems interpret continuous video in real time while retaining useful history and responding at the right moment.