Organizations: Fudan University · Shanghai Innovation Institute · Shanghai Artificial Intelligence Laboratory · Stanford University · Peking University · The Hong Kong University of Science and Technology · The University of Tokyo · The University of Hong Kong · The Chinese University of Hong Kong
Streaming video understanding requires models to repeatedly decide whether to respond or continue observing. Recently, Group Relative Policy Optimization (GRPO) has been applied to this setting, with existing approaches emphasizing reward design while retaining a single trajectory-level advantage across turns. Such shared credit can reinforce redundant replies after earlier successes and discourage useful follow-ups after earlier failures. We propose TurnWise, a turn-level GRPO framework that aligns learning with individual streaming decisions. TurnWise addresses the mismatch between trajectory-level feedback and turn-level interaction, promoting informative responses and reducing redundancy. Specifically, turn-level credit assignment uses silence-completed prefix scoring to construct group-relative advantages from each decision's remaining return, separating its learning signal from earlier rewards. Turn-mean policy optimization averages token losses within each turn and then across turns, giving response and waiting decisions equal weight regardless of output length. These complementary designs improve streaming policy learning without a learned critic or additional rollouts. Extensive experiments show our method outperforms prior RL approaches for streaming video understanding.
Figures & tables
Figure 1: Shared advantages can reinforce redundant replies. GRPO assigns one trajectory-level advantage to all turns, so later repetitions may inherit positive credit from earlier successes. TurnWise subtracts group-relative accrued reward before each decision, allowing later turns to receive credit according to their remaining return. The dialogue is illustrative. The advantage plot uses recorded TurnWise advantages and the corresponding GRPO counterfactual.
Figure 2: Overview of TurnWise. Given streaming video input x , user queries, dialogue history, and an observation plan, the policy samples a group of trajectories {τi}i=1G . TurnWise converts trajectory rewards Ri into group-relative remaining-return advantages A^i,t through silence-completed prefix scoring. Turn-mean aggregation then gives every response-or-wait decision equal weight in the trajectory loss Li .
Method
ProactiveVQA
OVO-Timing
StreamingBench
PAUC ↑
Timing F1 ↑
RealTime Acc. ↑
Proactive Acc. ↑
Avg. ↑
Ego
TV
VAD
Web
Overall
CRR
SSR
REC
Avg.
Qwen2.5-VL-3B
Baseline
9.32
27.85
8.98
42.45
22.15
0.00
0.53
3.46
1.33
69.42
22.40
45.91
+ SFT
26.05
27.60
26.30
35.00
28.74
0.00
0.95
0.76
0.57
74.68
22.80
48.74
+ MMDuet2
44.10
39.70
28.70
45.90
39.60
13.86
17.56
21.76
17.72
74.88
20.40
47.64
Table 1: TurnWise achieves the best aggregate scores on ProactiveVQA, OVO-Timing, and StreamingBench with both backbones. Within each backbone, the best and second-best results are bold and underlined , respectively. Δ denotes TurnWise minus the strongest non-TurnWise result in each aggregate column.
Figure 3: TurnWise achieves higher late-stage reward with fewer redundant replies relative to credit-only optimization. Panel a shows reward, panel b shows reply-normalized repetition penalty magnitude, and panel c shows non-silent decision fraction. Faint traces are batch means. Solid curves are trailing 20-update averages. Gray shading marks credit warmup.
Table 5
Figure 4: Turn-level credit assigns negative advantages to 59.83% of repetitive replies with positive global advantages (432 of 722). Both allocations use the same sampled groups from Qwen2.5-VL-3B training, using complete trajectories with full credit correction at α=1 . Panel a shows advantage density over 84,184 decisions. Turn advantages are clipped to ±5 as in training. The dashed diagonal marks unchanged allocation. Panel b shows advantage signs for all 2,972 Judge-labeled repetitive replies in this cohort.
Streaming video understanding studies how AI systems interpret continuous video in real time while retaining useful history and responding at the right moment.