cs.LGOct 27, 2026
SaveLive-OPSD: On-Policy Self-Distillation for Proactive Streaming Large Video Models
Organizations: Qwen Business Unit of Alibaba
Abstract
Streaming multimodal assistants must make response decisions while events are still unfolding. They need to recognize when the available evidence warrants a response, produce an appropriate continuation, and remain silent otherwise. Supervised fine-tuning learns from recorded assistant turns, but does not directly train on the response prefixes produced by the evolving policy itself. We introduce Live-OPSD, an on-policy self-distillation method that uses privileged hints available only during training to supervise the policy on its own outputs. The policy first generates a continuation from the streaming context alone. The same model then re-evaluates that continuation with a hint describing the relevant evidence, and the resulting distributions guide the unhinted policy. On Qwen3-Omni-30B-A3B, Live-OPSD attains an average score of 59.64 on LiveProBench, 5.26 points above the strongest continued-SFT baseline, and stays competitive on OVO-Bench (65.84). These results indicate that guiding the policy on its own outputs with privileged hints improves both when and how a streaming assistant responds.
Figures & tables
Figure 1: Live-OPSD training. The unhinted policy generates a continuation, while the same current model uses a privileged hint to score its prefixes. DAS selects scored decision points for a reverse-KL update. The updated policy then supplies the next rollout and teacher.
Figure 2: Supervision at streaming decision points. Each interleaved chunk or user turn carries a decision and a privileged hint . Streaming SFT learns the recorded action, whereas Live-OPSD supplies the hint only to the teacher.
Figure 3: Decision-Aware Selection. (a) Sampled decisions are grouped as the two inconsistency directions, consistent responses, or consistent silence. (b) DAS fills a budget of scored points from an inconsistency quota, then consistent responses and silence. The decision label controls selection, not gradient weighting.
| Method or model | Event Sensitivity | OR | WR | AA | SS | DC | Avg. | ||
|---|---|---|---|---|---|---|---|---|---|
| Resp. | Sil. | Geo. | |||||||
| Open-source proactive models | |||||||||
| ROMA | 32.87 | 61.79 | 45.07 | 19.27 | 6.25 | 9.24 | 2.17 | 2.99 | 14.16 |
| MOSS-VL-Realtime | 82.87 | 50.41 | 64.63 | 38.07 | 35.00 | 24.79 | 19.57 | 7.46 | 31.59 |
| AURA | 34.26 | 99.19 | 58.29 | 62.84 | 60.00 | 37.82 | 25.00 | 0.00 | 40.66 |
| JoyAI-VL-Interaction | 47.22 | 78.86 | 61.02 | 52.29 | 47.50 | 18.91 | 30.43 | 35.82 | 41.00 |
Table 1: LiveProBench results (%). The average is computed from the unrounded Event Sensitivity, Onset Response, Window Response, Anomaly Alert, Sustained Silence, and Duplicate Counting scores. Boldface marks the best result within each block, excluding Live-SFT-Zero.
| Model | PA |
|---|---|
| VideoLLM-Online | 0.50 |
| MMDuet | 22.00 |
| M4-a | 25.50 |
| AURA | 37.50 |
| ROMA | 37.50 |
| Live-SFT-Zero-7B | 31.50 |
Table 2: OmniMMI Proactive Alerting accuracy (%).
| Model | All |
|---|---|
| StreamAgent | 49.40 |
| Streamo-7B | 57.90 |
| ViSpeak | 61.10 |
| MiniCPM-o-4.5 | 59.70 |
| AURA | 65.30 |
| Qwen2.5-Omni-7B | 50.30 |
Table 3: OVO-Bench overall accuracy (%) across released systems.
Figure 4: LiveProBench timing outcomes for Qwen3-Omni.
Figure 5: Held-out ablation dynamics over 110 steps on 1,458 fixed decision points; arrows indicate the improving direction.
| Variant | ES | OR | WR | AA | SS | DC | Avg. |
|---|---|---|---|---|---|---|---|
| Reverse KL | 63.34 | 61.93 | 70.00 | 68.49 | 59.78 | 34.33 | 59.64 |
| Forward KL | 43.03 | 31.19 | 58.75 | 11.34 | 54.35 | 17.91 | 36.10 |
| JSD | 67.08 | 67.43 | 66.25 | 70.17 | 61.96 | 22.39 | 59.21 |
| w/o DAS | 59.58 | 55.50 | 51.25 | 54.20 | 44.57 | 0.00 | 44.18 |
Table 4: LiveProBench ablations on Qwen3-Omni (%). The main Live-OPSD result uses reverse KL. The w/o DAS control retains JSD and replaces adaptive selection with a fixed-position rule.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
| Setting | Value |
|---|---|
| Learning rate | |
| Selected-point budget | 128 |
| Teacher support size | 8 |
| Log-probability floor / loss clipping | / |
| Context length | 65,536 |
| Chunk duration / visual sampling rate | 1 s / 2 fps |
Table 5: Verified configuration of the Qwen3-Omni-30B-A3B reverse-KL run used in the main result and divergence ablation.
| RTVU | OSU | CU | ||||||||||||||||||||
| Model | OP | CR | CS | ATP | EU | TR | PR | SU | ACP | CT | Avg | ER | SCU | SD | MA | Avg | MCU | ACU | SQA | PO | Avg | All |
| Gemini-1.5-Pro | 79.0 | 80.5 | 83.5 | 79.7 | 80.0 | 84.7 | 77.8 | 64.2 | 72.0 | 48.7 | 75.7 | 46.8 | 39.6 | 74.9 | 80.0 | 60.2 | 51.4 | 40.7 | 54.8 | 45.1 | 48.7 | 67.10 |
| GPT-4o | 77.1 | 80.5 | 83.9 | 76.5 | 70.2 | 83.8 | 66.7 | 62.2 | 69.1 | 49.2 | 73.3 | 41.2 | 37.2 | 43.6 | 56.0 | 44.5 | 41.2 | 38.4 | 32.8 | 56.9 | 38.7 | 60.20 |
| StreamAgent | 79.6 | 78.3 | 79.3 | 75.9 | 74.7 | 76.9 | 82.9 | 66.3 | 73.7 | 55.4 | 74.3 | 35.9 | 26.3 | 38.9 | 44.0 | 36.3 | 39.7 | 30.3 | 39.6 | 28.9 | 34.6 | 57.00 |
| ViSpeak | 79.8 | 71.1 | 81.4 | 78.8 | 74.5 | 70.1 | 63.9 | 64.2 | 71.4 | 28.0 | 70.4 | 47.2 | 56.4 | 61.6 | 81.2 | 61.6 | 49.2 | 36.4 | 39.2 | 50.8 | 43.9 | 62.60 |
| MiniCPM-o-4.5 | 80.9 | 88.3 | 82.0 | 85.0 | 78.9 | 85.4 | 78.7 | 67.1 | 75.6 | 56.0 | 78.2 | 44.4 | 28.0 | 41.6 | 54.4 | 42.1 | 35.2 | 44.0 | 51.2 | 47.6 | 44.5 | 62.70 |
Table 6: StreamingBench system-level performance (%) on all 18 subtasks. Subtask abbreviations follow the benchmark definition ( Lin et al., 2024 ) ; external comparison results follow the AURA evaluation ( Lu et al., 2026 ) . Our Live-OPSD variants are evaluated locally with audio. For external systems, group and overall scores follow the source aggregation and need not equal unweighted means of the displayed subtasks. “All” is not the mean of the three group averages. Boldface marks the best group or overall score.
| RTVP | BT | FAR | ||||||||||||||
| Model | OCR | ACR | ATR | STU | FPD | OJR | Avg | EPM | ASI | HLD | Avg | REC | SSR | CRR | Avg | All |
| Gemini-1.5-Pro | 85.9 | 67.0 | 79.3 | 58.4 | 63.4 | 62.0 | 69.3 | 58.6 | 76.4 | 52.6 | 62.5 | 35.5 | 74.2 | 61.7 | 57.2 | 63.00 |
| GPT-4o | 69.8 | 64.2 | 71.6 | 51.1 | 70.3 | 59.8 | 64.5 | 57.9 | 75.7 | 48.7 | 60.8 | 73.2 | 27.6 | 59.4 | 53.4 | 59.50 |
| StreamAgent | 71.2 | 53.2 | 63.6 | 53.9 | 67.3 | 58.7 | 61.3 | 54.8 | 58.1 | 25.8 | 41.7 | 35.9 | 48.4 | 52.0 | 45.4 | 49.40 |
| Streamo-7B | 77.2 | 66.1 | 76.7 | 45.5 | 66.3 | 72.8 | 67.4 | 55.6 | 58.1 | 33.9 | 49.2 | 30.8 | 57.6 | 82.5 | 57.0 | 57.90 |
| ViSpeak | 75.2 | 58.7 | 71.6 | 51.1 | 74.3 | 66.9 | 66.3 | 59.9 | 48.7 | 64.0 | 57.5 | 33.8 | 68.5 | 60.4 | 54.3 | 61.10 |
Table 7: OVO-Bench system-level performance (%) on all 12 subtasks. Subtask abbreviations follow the benchmark definition ( Li et al., 2025 ) ; external comparison results follow the AURA evaluation ( Lu et al., 2026 ) . Our Qwen-Omni backbones and their Live-OPSD variants are evaluated locally, video only. “All” is the macro-average of the RTVP, BT, and FAR group averages. Boldface marks the best group or overall score.
Figure 6: Representative LiveProBench task formats from the released annotations. ES illustrates response–silence balance; WR represents the first-response-window format shared by OR, WR, AA, and SS; and DC illustrates final-total counting across repeated events. ES and WR each show six complete video frames; DC uses ten frames spanning two complete action cycles. Frame labels identify silence, valid response moments, and action phases; green borders mark valid ES/WR responses. No model predictions are shown.
Figure 7: Case comparison of Live-OPSD, Vanilla-SFT, CB-SFT, and SSB-SFT on released LiveProBench streams. Each panel shows the sampled video frames, the annotated valid response window, and each method’s first response, colored by verdict: in-window (correct), early (premature), or missed.
More in this topic
On-Policy Distillation
On-policy distillation trains a model on its own outputs using teacher guidance, aligning supervision with the states it encounters at deployment.
See all 517 papers