Watching Itself Watch: Self-Auditing Visual Reliance for Video Reasoning
Organizations: The University of Texas at Austin
Abstract
Video reasoning hinges on a small number of visually decisive tokens embedded in otherwise largely language-driven chains of thought. Yet prevailing post-training methods such as GRPO overlook this structure, assigning coarse trajectory-level rewards while requiring multiple rollouts and often costly external judges. We introduce Visually-Aware On-Policy Self-Distillation (VIOS), a post-training framework that identifies where a model under-utilizes visual evidence and concentrates learning at those positions. Our key insight is that a model can audit its own visual reliance: the divergence between its next-token distributions with and without the video reveals how strongly each prediction depends on the visual input. Measuring this reliance for both a student and an answer-privileged teacher exposes a per-token reliance gap, identifying positions where the teacher exploits visual evidence that the student overlooks. VIOS translates this signal into dense, targeted supervision through three complementary components: an exponential moving average (EMA) teacher that co-evolves with the student, a contrastive objective that encourages predictions to depend on the video, and a reliance-gap gate that focuses learning on visually under-utilized tokens. Crucially, VIOS requires only a single on-policy rollout from a single model, without external rewards, judges, or larger teachers. Across 12 benchmarks spanning general, temporal, spatial, knowledge-intensive, and long-video understanding, VIOS consistently improves three model backbones and outperforms contemporary video-reasoning systems, while training in over 9x fewer steps, 60x fewer FLOPs, and 111x fewer generated tokens. Project page: https://vision.cs.utexas.edu/projects/VIOS/
Figures & tables
| General | Knowledge | Temporal | Long Video | Spatial | |||||||||
| Model | MVBench | Video-MME | MINERVA | MMVU | Video-MMMU | VideoMathQA | SciVideoBench | TempCompass | TVBench | LongVideoBench | MLVU | VSI-Bench | Average |
| Video-R1 [ 7 ] | 64.42 | 58.37 | 30.76 | 63.04 | 49.22 | 23.57 | 24.50 | 71.58 | 50.81 | 55.42 | 63.11 | 35.25 | 49.17 |
| VideoRFT [ 38 ] | 62.25 | 58.26 | 28.26 | 64.96 | 47.44 | 24.76 | 24.10 | 72.65 | 53.35 | 54.08 | 61.82 | 35.10 | 48.92 |
| VideoChat-R1 [ 23 ] | 62.40 | 57.07 | 29.47 | 63.68 | 47.67 | 23.33 | 23.30 | 72.43 | 52.16 | 53.85 | 57.82 | 33.08 | 48.02 |
| VideoChat-R1.5 [ 44 ] | 62.17 | 56.89 | 28.56 | 63.20 | 49.22 | 25.00 | 23.20 | 72.28 | 48.40 | 54.82 | 58.05 | 31.95 | 47.81 |
| Video-R2 [ 27 ] | 62.08 | 56.04 | 27.42 | 64.80 | 48.33 | 20.00 | 25.40 | 72.64 | 51.68 | 50.19 | 58.00 | 32.56 | 47.43 |
| General | Knowledge | Temporal | Long Video | Spatial | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Model | Steps | MVBench | Video-MME | MINERVA | MMVU | Video-MMMU | VideoMathQA | SciVideoBench | TempCompass | TVBench | LongVideoBench | MLVU | VSI-Bench | Average |
| Video-R1 | 1000 | 64.42 | 58.37 | 30.76 | 63.04 | 49.22 | 23.57 | 24.50 | 71.58 | 50.81 | 55.42 | 63.11 | 35.25 | 49.17 |
| VIOS (Ours) | 110 | 64.45 | 58.63 | 31.67 | 65.12 | 46.33 | 23.60 | 25.24 | 73.25 | 52.00 | 54.00 | 61.68 | 37.05 | 49.42 |
| General | Knowledge | Temporal | Long Video | Spatial | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Variant | MVBench | Video-MME | MINERVA | MMVU | Video-MMMU | VideoMathQA | SciVideoBench | TempCompass | TVBench | LongVideoBench | MLVU | VSI-Bench | Average |
| Qwen2.5-VL-SFT | 60.72 | 55.15 | 29.55 | 59.36 | 47.22 | 23.10 | 22.40 | 67.25 | 49.82 | 52.51 | 55.70 | 33.44 | 46.35 |
| Frozen Teacher | 65.07 | 58.15 | 29.92 | 65.60 | 47.78 | 23.81 | 24.10 | 72.92 | 52.71 | 54.90 | 61.50 | 34.90 | 49.28 |
| EMA Teacher (Ours) | 65.50 | 59.59 | 31.36 | 65.76 | 47.44 | 25.48 | 24.90 | 72.77 | 52.24 | 55.87 | 62.74 | 35.73 | 49.95 |
| General | Knowledge | Temporal | Long Video | Spatial | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Variant | MVBench | Video-MME | MINERVA | MMVU | Video-MMMU | VideoMathQA | SciVideoBench | TempCompass | TVBench | LongVideoBench | MLVU | VSI-Bench | Average |
| Qwen2.5-VL-SFT | 60.72 | 55.15 | 29.55 | 59.36 | 47.22 | 23.10 | 22.40 | 67.25 | 49.82 | 52.51 | 55.70 | 33.44 | 46.35 |
| No Gate | 64.60 | 58.11 | 30.45 | 64.96 | 47.00 | 27.86 | 25.10 | 72.19 | 52.28 | 55.80 | 60.95 | 33.13 | 49.37 |
| Gate (Ours) | 65.50 | 59.59 | 31.36 | 65.76 | 47.44 | 25.48 | 24.90 | 72.77 | 52.24 | 55.87 | 62.74 | 35.73 | 49.95 |
| General | Knowledge | Temporal | Long Video | Spatial | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Variant | MVBench | Video-MME | MINERVA | MMVU | Video-MMMU | VideoMathQA | SciVideoBench | TempCompass | TVBench | LongVideoBench | MLVU | VSI-Bench | Average |
| Qwen2.5-VL-SFT | 60.72 | 55.15 | 29.55 | 59.36 | 47.22 | 23.10 | 22.40 | 67.25 | 49.82 | 52.51 | 55.70 | 33.44 | 46.35 |
| Distill. | 64.23 | 56.96 | 31.06 | 62.56 | 48.00 | 24.29 | 24.50 | 69.88 | 49.98 | 53.25 | 59.57 | 34.23 | 48.21 |
| Distill. + Gate | 63.32 | 57.74 | 30.15 | 64.00 | 46.11 | 23.10 | 24.70 | 70.93 | 52.55 | 54.23 | 61.09 | 36.41 | 48.69 |
| Distill. + Gate + Contra. (Ours) | 65.50 | 59.59 | 31.36 | 65.76 | 47.44 | 25.48 | 24.90 | 72.77 | 52.24 | 55.87 | 62.74 | 35.73 | 49.95 |
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
| Pass | VIOS | Vanilla GRPO | Video-R1 |
|---|---|---|---|
| Rollout prefill | Prompt and video | Shared prompt and video | Shared policy prefill; 4 temporal-reward prefills |
| Rollout decode | 1 trajectory | 8 trajectories | 8 policy + 4 temporal-reward trajectories |
| Policy update | 1 trajectory | 8 trajectories | 8 policy trajectories |
| Additional forwards | EMA teacher with/without video; student no-video contrastive pass | Frozen reference model for KL on 8 trajectories | Frozen reference model for KL on 8 policy trajectories |
| Hyperparameter | Value |
|---|---|
| Sampling temperature | |
| Top- | |
| Top- | |
| Training seed | |
| Per-device batch size | |
| Learning rate |
| General | Knowledge | Temporal | Long Video | Spatial | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Variant | MVBench | Video-MME | MINERVA | MMVU | Video-MMMU | VideoMathQA | SciVideoBench | TempCompass | TVBench | LongVideoBench | MLVU | VSI-Bench | Average |
| Qwen2.5-VL-SFT | 60.72 | 55.15 | 29.55 | 59.36 | 47.22 | 23.10 | 22.40 | 67.25 | 49.82 | 52.51 | 55.70 | 33.44 | 46.35 |
| 63.32 | 57.74 | 30.15 | 64.00 | 46.11 | 23.10 | 24.70 | 70.93 | 52.55 | 54.23 | 61.09 | 36.41 | 48.69 | |
| 65.57 | 58.85 | 30.98 | 66.24 | 48.00 | 23.10 | 25.30 | 72.32 | 51.88 | 55.20 | 61.96 | 35.71 | 49.59 | |
| (Ours) | 65.50 | 59.59 | 31.36 | 65.76 | 47.44 | 25.48 | 24.90 | 72.77 | 52.24 | 55.87 | 62.74 | 35.73 | 49.95 |
| 65.72 | 58.15 | 30.98 | 62.40 | 49.89 | 23.10 | 23.40 | 73.82 | 52.16 | 56.54 | 62.65 | 36.23 | 49.59 | |
| General | Knowledge | Temporal | Long Video | Spatial | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Variant | MVBench | Video-MME | MINERVA | MMVU | Video-MMMU | VideoMathQA | SciVideoBench | TempCompass | TVBench | LongVideoBench | MLVU | VSI-Bench | Average |
| Qwen2.5-VL-SFT | 60.72 | 55.15 | 29.55 | 59.36 | 47.22 | 23.10 | 22.40 | 67.25 | 49.82 | 52.51 | 55.70 | 33.44 | 46.35 |
| 65.62 | 58.56 | 30.23 | 64.00 | 48.11 | 21.67 | 26.10 | 73.33 | 52.75 | 56.32 | 62.65 | 35.22 | 49.55 | |
| (Ours) | 65.50 | 59.59 | 31.36 | 65.76 | 47.44 | 25.48 | 24.90 | 72.77 | 52.24 | 55.87 | 62.74 | 35.73 | 49.95 |
| 65.50 | 59.00 | 31.29 | 64.64 | 47.56 | 22.86 | 22.90 | 73.16 | 52.71 | 55.95 | 61.96 | 33.64 | 49.26 | |
| 65.75 | 59.04 | 30.08 | 65.28 | 48.78 | 26.43 | 25.70 | 73.17 | 52.51 | 56.47 | 61.22 | 34.43 | 49.91 | |
| Model | ChartQA | BLINK | CV-Bench | MMMU | Average |
|---|---|---|---|---|---|
| Qwen2.5-VL-SFT | 84.64 | 51.18 | 71.76 | 51.78 | 64.84 |
| + VIOS (Ours) | 84.56 | 55.92 | 76.23 | 53.22 | 67.48 |
| Seed 43 | Seed 44 | |||
|---|---|---|---|---|
| Statistic | Start | End | Start | End |
| Mean | 1.000 | 1.000 | 1.000 | 1.000 |
| Std. dev. | 0.148 | 0.148 | 0.150 | 0.148 |
| 0.774 | 0.742 | 0.772 | 0.741 | |
| 0.992 | 0.995 | 0.993 | 0.996 | |
| 1.025 | 1.034 | 1.025 | 1.034 | |
| Method | Seed | Mean | Std. | p95 | p99 | Max. | Clipped (%) |
|---|---|---|---|---|---|---|---|
| GRPO | 42 | 2.45 | 3.90 | 4.76 | 10.8 | 112.4 | 4.4 |
| GRPO | 43 | 2.44 | 2.84 | 4.81 | 16.1 | 44.3 | 4.5 |
| GRPO | 44 | 2.46 | 2.85 | 4.84 | 10.9 | 57.2 | 4.7 |
| VIOS | 42 | 0.298 | 0.218 | 0.69 | 1.36 | 1.46 | 0.0 |
| VIOS | 43 | 0.309 | 0.306 | 0.61 | 1.06 | 2.89 | 0.0 |
| VIOS | 44 | 0.292 | 0.305 | 0.55 | 0.87 | 3.01 | 0.0 |
| Normalized token entropy | ||||||
|---|---|---|---|---|---|---|
| Method | Seed | KL to base | Start | End | Min. | |
| GRPO | 42 | 0.0735 | – | – | – | – |
| GRPO | 43 | 0.0609 | 0.714 | 0.928 | 0.359 | |
| GRPO | 44 | 0.0681 | 0.711 | 0.914 | 0.397 | |
| VIOS | 42 | 0.0354 | – | – | – | – |
| VIOS | 43 | 0.0349 | 0.721 | 0.694 | 0.620 | |
| Completion length (tokens) | Homog. | ||||
|---|---|---|---|---|---|
| Method | Seed | Start | End | group frac. | |
| GRPO | 42 | 290.6 | 376.5 | 0.299 | |
| GRPO | 43 | 275.5 | 404.1 | 0.319 | |
| GRPO | 44 | 282.9 | 384.3 | 0.292 | |
| VIOS | 42 | 294.1 | 297.2 | – | |
| VIOS | 43 | 295.1 | 293.9 | – | |
Video Reasoning
Video reasoning studies how AI systems use video evidence to infer events, actions, temporal relations, causes, and future states.
See all 220 papers