OWLET: On-Policy Weighted Learning via Evidence Transfer for Video Large Language Model Post-Training

Qitan Lv; Ke Fan; Hong Wang; Wen Wu; Yilin Wang; Bowen Zhou; Feng Wu; Chao Zhang

cs.CVOct 9, 2026

OWLET: On-Policy Weighted Learning via Evidence Transfer for Video Large Language Model Post-Training

Authors: Qitan Lv, Ke Fan, Hong Wang, Wen Wu, Yilin Wang, Bowen Zhou, Feng Wu, Chao Zhang

Organizations: University of Science and Technology of China · Shanghai AI Laboratory · Tsinghua University

Abstract

Post-training has become a key stage for eliciting reasoning capabilities in Video Large Language Models (Video-LLMs). Existing post-training methods mainly follow two paradigms: reinforcement learning (RL) and on-policy distillation (OPD). However, they do not fully exploit visual evidence for effective training. RL supervises the policy with a single trajectory-level reward, which is too coarse-grained for video reasoning: the same response-level advantage is applied to every generated token, disregarding that the tokens of a video response are grounded in the visual stream to very different degrees. OPD provides token-level supervision through output-distribution matching, but lacks direct supervision of the spatiotemporal representations that produce those outputs. We observe across extensive experiments that the same model reasons more accurately when its video view combines global temporal context with question-relevant details. Comparing the same student-generated response under this view and uniform sampling reveals changes in token likelihoods and internal representations, providing a training signal for evidence use. Building on this observation, we propose OWLET, an evidence-privileged RL method that derives training signals from a richer teacher view. This view couples uniform global context with question-conditioned local windows, providing visual evidence without requiring a larger model. To better exploit this privileged visual evidence, OWLET introduces (i) evidence-aware credit redistribution, which compares token likelihoods across the two views and redistributes positive response-level advantages according to their relative evidence support; and (ii) spatiotemporal state alignment, which aligns the two views' hidden states on verifier-accepted responses.

Figures & tables

Figure 1: Illustration of how OWLET improves video reasoning through evidence-guided post-training. Uniform frame sampling misses the temporal transitions needed to answer the question, whereas OWLET learns from a correct evidence-privileged teacher.

Figure 2: (a, b) Evidence-only views improve overall accuracy but lose global-temporal accuracy; only pairing global context with local windows improves both. (c) Half of the teacher–student log-probability gap concentrates on 8.5% of response tokens, so spreading training credit uniformly dilutes the evidence signal. (d) Output-only distillation collapses the output KL, yet the hidden divergence plateaus: output supervision alone does not effectively align spatiotemporal representations.

Figure 3: Overview of OWLET . The student samples responses from uniformly sampled frames, while the teacher scores the same responses using a view that combines global temporal coverage with question-conditioned local windows (red box). OWLET consists of two components: (1) Evidence-aware credit redistribution uses teacher–student token log-probability contrasts to redistribute positive advantages within each response; non-positive advantages remain unchanged; (2) Spatiotemporal state alignment aligns hidden states through cosine similarity on verifier-accepted responses.

Backbone	Method	Video-MME	MVBench	LongVideoBench	Video-MME 2	MLVU	Avg.
Qwen3.5-0.8B	Vanilla	49.4	51.8	48.9	15.8	41.9	41.6
	SFT	49.5	52.2	49.5	15.7	42.1	41.8
	GRPO	50.8	53.9	50.4	18.5	43.3	43.4
	OPSD	50.9	53.9	51.4	17.2	44.1	43.5
	OWLET	54.9	58.6	55.9	19.9	45.2	46.9
Qwen3.5-4B	Vanilla	64.7	67.4	58.0	24.7	50.3	53.0

Table 1: Accuracy (%) results on five video-understanding benchmarks for Qwen3.5 at 64 input frames. We bold the best result for each backbone and benchmark.

Method	Qwen3.5-0.8B			Qwen3.5-4B			Qwen3.5-9B			Qwen3.5-27B
Method	16	32	64	16	32	64	16	32	64	16	32	64
Vanilla	37.8	40.1	41.6	48.0	49.8	53.0	49.0	51.3	54.4	54.3	56.8	59.5
SFT	38.6	40.9	41.8	48.5	50.7	53.6	49.8	52.2	55.2	54.8	56.1	59.8
GRPO	39.6	41.9	43.4	49.8	51.8	54.1	50.5	52.9	55.5	55.7	57.9	60.2
OPSD	39.1	41.5	43.5	49.1	51.0	53.9	50.3	52.6	55.5	55.7	58.1	60.2
OWLET	42.9	45.2	46.9	53.3	54.6	58.2	54.2	57.1	59.8	57.9	60.4	64.5

Table 2: Macro-average accuracy (%) over the five benchmarks for Qwen3.5, at

16

32

, and

64

input frames. We bold the best result for each backbone and frame budget.

Panel of Figure 4 — Figure 4: Ablation results for Qwen3.5-9B across different frame budgets.

Figure 5: Training dynamics. Qwen3.5-9B with 64 student frames. (a) Hidden divergence, measured by response-end

1-\mathrm{CKA}

, for OPSD and OWLET. (b) Output KL between student and teacher views for OPSD and OWLET. (c) Rollout accuracy on the training set for OPSD and OWLET. (d) Cumulative absolute log-probability gap at the top 10% of response tokens for OPSD, GRPO, and OWLET; the dotted line marks the uniform baseline. Diagnostic samples are spaced 10 steps apart and accuracy samples 50 steps apart. Thin dashed lines connect sampled trajectories; thick lines show rolling means.

Appendix figures & tables47 assets

Supplementary material from the paper’s appendix.

Appendix

16-frame budget
View	Uniform frames	Ranked frames	Window size	Total
Vanilla	16	—	—	16
Key frames	—	16	—	16
Key windows	—	4	4	16
Global with local	8	2	4	16
32-frame budget

Appendix

Table 3: Composition of the four input views defined in Section 3.1 at total budgets of 16, 32, and 64 frames. Ranked frames are selected by question-conditioned relevance and used directly in Key frames or as centers of disjoint windows in Key windows and Global with local. All four views use the same total number of frames within each budget.

Figure 6: Teacher-view comparison on the Qwen3-VL model family. (a, b) Non-thinking variants. (c, d) Thinking variants. The left column reports accuracy on all questions; the right column reports accuracy on the global-temporal subset. Each panel compares the four input views defined in Table 3 on the 2B, 4B, and 8B models. Bars are labeled with accuracy in percent, and all four panels use the same vertical scale. In both modes and at all three model sizes, evidence-only views improve overall accuracy while reducing global-temporal accuracy relative to vanilla; global context with local windows improves both metrics.

Stage	Criterion	Excluded / reserved	Remaining
Source aggregation	four public sources	—	104,257
Field validation	non-empty question and answer	812	103,445
Option validation	unique labels, single correct option	1,134	102,311
Video validation	file exists and decodes end to end	1,313	100,998
Temporal validation	sufficient evidence candidates	4,293	96,705
Held-out diagnostics	evaluation split of Section 3	1,000	95,705

Appendix

Table 4: Construction pipeline of Video-Evidence-96K. Validation rows report excluded records and the number remaining after each check. The final row reserves

1{,}000

valid records for diagnostic evaluation, leaving 95,705 training records.

Source	Records	Share
LLaVA-Video-178K	70,468	73.63%
NExT-QA	7,542	7.88%
Perception	6,345	6.63%
STAR	11,350	11.86%
Total	95,705	100.00%

Appendix

Table 5: Composition of the Video-Evidence-96K training split.

		Global-temporal		Local-evidence
Split	Total	Records	Percentage	Records	Percentage
Training	95,705	20,137	21.04%	75,568	78.96%
Held-out diagnostics	1,000	500	50%	500	50%

Appendix

Table 6: Composition of Video-Evidence-96K by question type, following Appendix D.3 . Counts are question–video records, and each percentage uses the total of its own split as the denominator.

Optimization (shared by GRPO, OPSD, and OWLET )
Setting	Value
Learning rate	$1\times 10^{-6}$ , constant, no warm-up
Prompts per step / rollouts per prompt $G$	$16$ / $8$
Mini-batch size	$16$
Training steps	$1,000$
Clipping range $\epsilon$	$0.2$

Appendix

Table 7: Training and evaluation hyper-parameters.

Backbone Model	Method	Video-MME	MVBench	LongVideoBench	Video-MME 2			MLVU	Avg.
					Logic	Rel.	All
Qwen3.5-0.8B	Vanilla	45.1	49.4	44.9	17.8	14.4	15.6	34.2	37.8
	SFT	45.5	50.8	45.9	18.5	14.8	16.1	34.7	38.6
	GRPO	46.8	50.7	47.0	19.6	16.3	17.5	35.8	39.6
	OPSD	46.2	50.4	47.5	18.2	14.9	16.1	35.1	39.1
	OWLET	50.3	56.4	50.6	20.5	17.6	18.6	38.9	42.9

Appendix

Table 8: Accuracy (%) for Qwen3.5 in non-thinking mode at 16 input frames, with the two Video-MME 2 question groups reported separately. Logic and Rel. are the group types and All combines them by question count. We bold the best result for each backbone and benchmark.

Backbone Model	Method	Video-MME	MVBench	LongVideoBench	Video-MME 2			MLVU	Avg.
					Logic	Rel.	All
Qwen3.5-0.8B	Vanilla	48.2	51.8	46.4	17.7	14.7	15.8	38.1	40.1
	SFT	48.8	52.4	46.9	19.5	15.5	16.9	39.4	40.9
	GRPO	50.1	54.8	48.2	19.5	16.4	17.5	38.9	41.9
	OPSD	49.6	53.4	48.0	19.5	15.5	16.9	39.7	41.5
	OWLET	53.9	57.8	53.0	22.2	18.0	19.4	42.0	45.2

Appendix

Table 9: Accuracy (%) for Qwen3.5 in non-thinking mode at 32 input frames, with the two Video-MME 2 question groups reported separately. Logic and Rel. are the group types and All combines them by question count. We bold the best result for each backbone and benchmark.

Backbone Model	Method	Video-MME	MVBench	LongVideoBench	Video-MME 2			MLVU	Avg.
					Logic	Rel.	All
Qwen3.5-0.8B	Vanilla	49.4	51.8	48.9	17.4	14.9	15.8	41.9	41.6
	SFT	49.5	52.2	49.5	17.2	14.9	15.7	42.1	41.8
	GRPO	50.8	53.9	50.4	19.2	18.1	18.5	43.3	43.4
	OPSD	50.9	53.9	51.4	18.7	16.4	17.2	44.1	43.5
	OWLET	54.9	58.6	55.9	22.6	18.4	19.9	45.2	46.9

Appendix

Table 10: Accuracy (%) for Qwen3.5 in non-thinking mode at 64 input frames, with the two Video-MME 2 question groups reported separately. Logic and Rel. are the group types and All combines them by question count. We bold the best result for each backbone and benchmark.

Frames	Method	Short	Medium	Long	All
16	Vanilla	57.1	41.5	36.7	45.1
	SFT	57.4	41.3	37.9	45.5
	GRPO	59.5	42.7	38.2	46.8
	OPSD	57.9	42.5	38.3	46.2
	OWLET	61.8	47.9	41.3	50.3
32	Vanilla	58.7	46.7	39.2	48.2

Appendix

Table 11: Video-MME accuracy (%) by video duration (Qwen3.5-0.8B). All averages the three splits. We bold the best result for each frame budget and duration.

Frames	Method	Short	Medium	Long	All
16	Vanilla	69.1	54.6	48.1	57.2
	SFT	70.3	55.9	48.6	58.3
	GRPO	69.8	56.2	50.5	58.8
	OPSD	70.3	56.1	50.6	59.0
	OWLET	73.7	59.9	55.0	62.8
32	Vanilla	73.4	58.8	52.1	61.4

Appendix

Table 12: Video-MME accuracy (%) by video duration (Qwen3.5-4B). All averages the three splits. We bold the best result for each frame budget and duration.

Frames	Method	Short	Medium	Long	All
16	Vanilla	69.1	54.8	50.7	58.2
	SFT	70.2	54.0	52.4	58.9
	GRPO	70.4	58.0	52.1	60.2
	OPSD	70.7	56.7	52.6	60.0
	OWLET	71.2	60.8	54.6	62.2
32	Vanilla	75.1	58.8	52.1	62.0

Appendix

Table 13: Video-MME accuracy (%) by video duration (Qwen3.5-9B). All averages the three splits. We bold the best result for each frame budget and duration.

Frames	Method	Short	Medium	Long	All
16	Vanilla	74.5	63.5	56.5	64.8
	SFT	74.3	62.8	57.1	64.7
	GRPO	75.6	64.5	57.9	66.0
	OPSD	75.4	64.1	58.0	65.8
	OWLET	77.9	67.7	59.0	68.2
32	Vanilla	79.7	67.5	59.6	68.9

Appendix

Table 14: Video-MME accuracy (%) by video duration (Qwen3.5-27B). All averages the three splits. We bold the best result for each frame budget and duration.

Backbone Model	Method	Video-MME	MVBench	LongVideoBench	Video-MME 2			MLVU	Avg.
					Logic	Rel.	All
Qwen3-VL-2B	Vanilla	45.2	53.1	42.9	14.4	14.3	14.3	34.8	38.1
	SFT	45.8	53.1	43.4	15.4	14.9	15.1	35.8	38.6
	GRPO	47.0	55.4	44.2	15.8	15.9	15.9	37.2	39.9
	OPSD	46.5	53.2	45.3	15.9	15.5	15.6	36.7	39.5
	OWLET	52.4	61.2	50.7	19.9	18.1	18.7	40.6	44.7

Appendix

Table 15: Accuracy (%) for Qwen3-VL in non-thinking mode at 16 input frames, with the two Video-MME 2 question groups reported separately. Logic and Rel. are the group types and All combines them by question count. We bold the best result for each backbone and benchmark.

Backbone Model	Method	Video-MME	MVBench	LongVideoBench	Video-MME 2			MLVU	Avg.
					Logic	Rel.	All
Qwen3-VL-2B	Vanilla	49.4	53.8	44.3	16.1	15.3	15.6	37.1	40.0
	SFT	50.3	54.8	46.9	17.1	16.4	16.6	38.8	41.5
	GRPO	51.8	55.0	45.4	18.2	16.6	17.2	40.2	41.9
	OPSD	51.0	55.3	46.4	18.1	16.4	17.0	38.1	41.6
	OWLET	56.2	60.9	54.9	22.7	18.1	19.7	45.3	47.4

Appendix

Table 16: Accuracy (%) for Qwen3-VL in non-thinking mode at 32 input frames, with the two Video-MME 2 question groups reported separately. Logic and Rel. are the group types and All combines them by question count. We bold the best result for each backbone and benchmark.

Backbone Model	Method	Video-MME	MVBench	LongVideoBench	Video-MME 2			MLVU	Avg.
					Logic	Rel.	All
Qwen3-VL-2B	Vanilla	49.2	53.9	45.1	16.8	17.3	17.1	40.3	41.1
	SFT	50.1	54.6	46.8	18.1	18.0	18.0	41.8	42.3
	GRPO	51.4	55.0	47.6	17.9	19.7	19.1	41.7	42.9
	OPSD	50.7	54.0	47.1	18.1	18.2	18.2	42.1	42.4
	OWLET	58.3	62.2	56.0	22.8	20.2	21.1	48.2	49.2

Appendix

Table 17: Accuracy (%) for Qwen3-VL in non-thinking mode at 64 input frames, with the two Video-MME 2 question groups reported separately. Logic and Rel. are the group types and All combines them by question count. We bold the best result for each backbone and benchmark.

Frames	Method	Short	Medium	Long	All
16	Vanilla	58.4	43.1	34.2	45.2
	SFT	59.0	43.4	34.9	45.8
	GRPO	59.8	44.9	36.3	47.0
	OPSD	59.9	44.6	35.1	46.5
	OWLET	64.4	51.4	41.4	52.4
32	Vanilla	65.1	48.0	35.0	49.4

Appendix

Table 18: Video-MME accuracy (%) by video duration for Qwen3-VL in non-thinking mode (Qwen3-VL-2B). All averages the three splits. We bold the best result for each frame budget and duration.

Frames	Method	Short	Medium	Long	All
16	Vanilla	62.2	45.9	37.9	48.7
	SFT	64.2	47.7	38.4	50.1
	GRPO	64.7	46.8	39.8	50.4
	OPSD	64.4	47.0	39.2	50.2
	OWLET	69.1	58.4	49.2	58.9
32	Vanilla	68.9	50.4	43.8	54.3

Appendix

Table 19: Video-MME accuracy (%) by video duration for Qwen3-VL in non-thinking mode (Qwen3-VL-4B). All averages the three splits. We bold the best result for each frame budget and duration.

Frames	Method	Short	Medium	Long	All
16	Vanilla	68.4	55.1	50.0	57.9
	SFT	69.6	57.2	50.9	59.2
	GRPO	70.9	54.9	51.9	59.2
	OPSD	69.5	56.4	51.4	59.1
	OWLET	72.2	57.8	52.2	60.7
32	Vanilla	74.0	58.0	54.1	62.1

Appendix

Table 20: Video-MME accuracy (%) by video duration for Qwen3-VL in non-thinking mode (Qwen3-VL-8B). All averages the three splits. We bold the best result for each frame budget and duration.

Backbone Model	Method	Video-MME	MVBench	LongVideoBench	Video-MME 2			MLVU	Avg.
					Logic	Rel.	All
Qwen3-VL-2B	Base model	49.3	60.5	45.1	17.8	15.3	16.2	35.1	41.3
	SFT	51.0	61.5	46.0	19.4	16.8	17.7	37.1	42.7
	GRPO	50.9	61.8	46.0	18.3	15.2	16.3	36.3	42.3
	OPSD	50.0	61.3	46.6	19.9	16.6	17.8	35.2	42.2
	OWLET	54.2	65.9	52.8	22.1	17.4	19.1	43.0	47.0

Appendix

Table 21: Accuracy (%) for Qwen3-VL in thinking mode at 16 input frames, with the two Video-MME 2 question groups reported separately. Logic and Rel. are the group types and All combines them by question count. We bold the best result for each backbone and benchmark.

Backbone Model	Method	Video-MME	MVBench	LongVideoBench	Video-MME 2			MLVU	Avg.
					Logic	Rel.	All
Qwen3-VL-2B	Base model	52.2	61.0	51.0	20.1	16.1	17.5	38.0	44.0
	SFT	53.3	62.1	52.3	20.3	16.7	18.0	38.1	44.8
	GRPO	53.5	61.2	51.9	21.9	17.4	19.0	39.1	44.9
	OPSD	53.1	62.1	52.9	21.3	17.4	18.8	39.1	45.2
	OWLET	57.4	67.2	55.0	25.5	19.4	21.5	46.0	49.4

Appendix

Table 22: Accuracy (%) for Qwen3-VL in thinking mode at 32 input frames, with the two Video-MME 2 question groups reported separately. Logic and Rel. are the group types and All combines them by question count. We bold the best result for each backbone and benchmark.

Backbone Model	Method	Video-MME	MVBench	LongVideoBench	Video-MME 2			MLVU	Avg.
					Logic	Rel.	All
Qwen3-VL-2B	Base model	55.5	63.1	51.4	20.1	18.1	18.8	46.0	47.0
	SFT	56.4	65.2	52.1	20.6	19.5	19.9	46.2	48.0
	GRPO	56.7	64.0	52.4	21.2	19.2	19.9	47.3	48.1
	OPSD	56.5	64.7	52.0	20.9	18.8	19.5	47.1	48.0
	OWLET	60.3	66.8	55.6	25.6	21.3	22.8	49.6	51.0

Appendix

Table 23: Accuracy (%) for Qwen3-VL in thinking mode at 64 input frames, with the two Video-MME 2 question groups reported separately. Logic and Rel. are the group types and All combines them by question count. We bold the best result for each backbone and benchmark.

Frames	Method	Short	Medium	Long	All
16	Base model	61.5	46.3	40.1	49.3
	SFT	63.5	48.5	41.1	51.0
	GRPO	63.0	47.5	42.2	50.9
	OPSD	62.6	46.9	40.5	50.0
	OWLET	66.0	52.8	43.8	54.2
32	Base model	65.2	50.4	41.0	52.2

Appendix

Table 24: Video-MME accuracy (%) by video duration for Qwen3-VL in thinking mode (Qwen3-VL-2B). All averages the three splits. We bold the best result for each frame budget and duration.

Frames	Method	Short	Medium	Long	All
16	Base model	65.4	52.1	46.0	54.5
	SFT	67.1	53.4	47.5	56.0
	GRPO	66.9	52.9	46.7	55.5
	OPSD	66.2	53.0	47.4	55.5
	OWLET	70.3	56.5	50.6	59.1
32	Base model	72.4	56.1	48.2	58.9

Appendix

Table 25: Video-MME accuracy (%) by video duration for Qwen3-VL in thinking mode (Qwen3-VL-4B). All averages the three splits. We bold the best result for each frame budget and duration.

Frames	Method	Short	Medium	Long	All
16	Base model	66.1	52.6	47.2	55.3
	SFT	67.0	52.8	47.7	55.8
	GRPO	68.4	54.4	48.2	57.0
	OPSD	67.8	53.5	47.8	56.4
	OWLET	71.4	58.8	51.2	60.5
32	Base model	73.2	57.2	53.2	61.2

Appendix

Table 26: Video-MME accuracy (%) by video duration for Qwen3-VL in thinking mode (Qwen3-VL-8B). All averages the three splits. We bold the best result for each frame budget and duration.

Panel of Figure 7 — Figure 7: Additional motivation diagnostics across four configurations. Each subfigure contains the same two diagnostics: the left panel reports the cumulative absolute teacher–student log-probability gap, with the marked point giving the response-token fraction that carries $50\%$ of the gap; the right panel reports output KL and hidden divergence during 300 steps of output-only distillation.

Panel of Figure 8 — Figure 8: Ablation results for Qwen3.5-0.8B. Panels (a)–(c) use $16$ , $32$ , and $64$ student frames, respectively. The five axes report accuracy (%) on Video-MME, MVBench, LongVideoBench, Video-MME 2, and MLVU. The legend defines the key-window, w/o ECR, and w/o SSA variants described in the ablation text; evaluation uses uniformly sampled student frames.

Panel of Figure 9 — Figure 9: Ablation results for Qwen3.5-4B. Panels (a)–(c) correspond to $16$ , $32$ , and $64$ uniformly sampled student frames.

Panel of Figure 10 — Figure 10: Ablation results for Qwen3.5-27B. Panels (a)–(c) correspond to $16$ , $32$ , and $64$ uniformly sampled student frames.

Panel of Figure 11 — Figure 11: Ablation results for Qwen3-VL-2B in non-thinking mode. Panels (a)–(c) correspond to $16$ , $32$ , and $64$ uniformly sampled student frames.

Panel of Figure 12 — Figure 12: Ablation results for Qwen3-VL-2B in thinking mode. Panels (a)–(c) correspond to $16$ , $32$ , and $64$ uniformly sampled student frames.

Panel of Figure 13 — Figure 13: Ablation results for Qwen3-VL-4B in non-thinking mode. Panels (a)–(c) correspond to $16$ , $32$ , and $64$ uniformly sampled student frames.

Panel of Figure 14 — Figure 14: Ablation results for Qwen3-VL-4B in thinking mode. Panels (a)–(c) correspond to $16$ , $32$ , and $64$ uniformly sampled student frames.

Panel of Figure 15 — Figure 15: Ablation results for Qwen3-VL-8B in non-thinking mode. Panels (a)–(c) correspond to $16$ , $32$ , and $64$ uniformly sampled student frames.

Panel of Figure 16 — Figure 16: Ablation results for Qwen3-VL-8B in thinking mode. Panels (a)–(c) correspond to $16$ , $32$ , and $64$ uniformly sampled student frames.

Method	Video-MME	MVBench	LongVideoBench	Video-MME 2	MLVU	Avg.
OPSD	66.1	68.7	62.9	26.7	53.3	55.5
OPSD-JSD	65.3	68.2	61.5	25.9	53.7	54.9
OPSD+GRPO	66.3	67.8	63.3	27.0	52.7	55.4
OPSD-Anneal	67.1	68.5	63.4	27.1	55.3	56.3
OWLET	70.7	72.8	65.2	28.9	61.6	59.8

Appendix

Table 27: Accuracy (%) of on-policy distillation variants and OWLET on Qwen3.5-9B at 64 student frames. All methods use the same teacher view, training data, and student rollout budget. Avg. is the unweighted mean over the five benchmarks. The best result in each column is shown in bold.

Teacher $\bar{\theta}$	Video-MME	MVBench	LongVideoBench	Video-MME 2	MLVU	Avg.
EMA (decay $0.99$ )	69.3	70.5	64.7	27.9	60.8	58.6
Fixed (default)	70.7	72.8	65.2	28.9	61.6	59.8

Appendix

Table 28: Effect of the teacher parameterization on Qwen3.5-9B at 64 input frames. The fixed teacher retains the initial checkpoint throughout training, while the EMA teacher averages student parameters with decay

0.99

under the same teacher view and training configuration. Scores are accuracies (%), and Avg. is the unweighted mean over the five benchmarks.

Advantage	Steps 0–250	250–500	500–750	750–1,000
$A^{i}>0$	28.5	21.8	10.6	4.7
$A^{i}<0$	7.2	5.8	3.8	2.2
$A^{i}=0$	64.3	72.4	85.6	93.1

Appendix

Table 29: Fractions (%) of rollouts with positive, negative, and zero advantages across training phases on Qwen3.5-9B at 64 input frames.

Category	Top decile	Bottom decile	Examples
Temporal markers	17.3	6.2	after, then, at the end
Event verbs and nouns	20.7	5.2	pours, jumps, collision
Objects, places, attributes	21.0	7.5	kitchen, red container, three
Answer-committing tokens	15.3	5.9	option letters, final numbers
Function words, connectives, and auxiliaries	4.8	23.1	the, is, and, therefore
Punctuation and formatting	3.1	25.4	commas, line breaks

Appendix

Table 30: Composition (%) of tokens in the top and bottom weight deciles among positively advantaged responses. Each response token is assigned to one of seven mutually exclusive categories; tokens that do not match the predefined rules are assigned to Others .

$\lambda_{\mathrm{SSA}}$	Video-MME	MVBench	LongVideoBench	Video-MME 2	MLVU	Avg.
$0$ (w/o SSA)	69.6	71.1	64.6	28.4	59.9	58.7
$0.01$	70.5	71.1	64.8	27.9	59.5	58.8
$0.02$	70.3	71.5	65.3	28.9	59.9	59.2
$0.05$ (default)	70.7	72.8	65.2	28.9	61.6	59.8
$0.1$	70.9	71.3	64.8	28.3	61.9	59.4

Appendix

Table 31: Sensitivity to the SSA weight on Qwen3.5-9B in non-thinking mode with 64 uniformly sampled student frames. Scores are accuracies (%), and Avg. is the unweighted mean over the five benchmarks. All other training settings are held constant;

\lambda_{\mathrm{SSA}}=0

disables SSA while retaining ECR, and

0.05

is the default weight.

Figure 17: Global-temporal case. Responses from Vanilla and checkpoints trained with SFT, GRPO, OPSD, and OWLET to the same video input and question. Vanilla and GRPO select D, and SFT and OPSD select B; OWLET recovers the correct camping–queuing–cheering order and selects C.

Figure 18: Local-evidence case. Responses to the same video input and question about the object visible at a specified cooking step. Vanilla, SFT, and GRPO select C (green onion), and OPSD selects A (knife); OWLET identifies the pot and selects the correct answer B.

Method	Rollout (s/step)	Update (s/step)	Total (s/step)	Total (h)
GRPO	406.99	507.30	914.29	254.0
OPSD	410.89	437.11	848.00	235.6
OWLET	434.31	514.24	948.55	263.5

Appendix

Table 32: On-policy training cost on eight H200 GPUs for Qwen3.5-9B at 64 student frames, averaged over the first

1{,}000

updates. Update covers the non-rollout portion of each timed step. Total (s/step) is the sum of Rollout and Update; Total (h) gives the accumulated step time.

Figure 19: Response length during the first

1{,}000

training steps of OWLET with 64 student frames: (a) Qwen3.5-9B in non-thinking mode, (b) Qwen3-VL-8B-Instruct in non-thinking mode, and (c) Qwen3-VL-8B-Thinking. Thin, light curves show the mean number of non-padding generated tokens across all 128 responses at each step; thick curves show 25-step trailing moving averages, using the available history for the first 24 steps. Thinking lengths include the generated reasoning trace and final answer. The panels use separate vertical scales, and all per-step means are shown.

Model	Mode	Steps 1–100	Steps 901–1,000	Overall	Change (%)
Qwen3.5-9B	Non-thinking	42.4	33.9	40.4	$-20.0$
Qwen3-VL-8B	Non-thinking	77.5	72.0	75.2	$-7.1$
Qwen3-VL-8B	Thinking	145.6	123.7	135.2	$-15.0$

Appendix

Table 33: Mean response length in generated tokens for the three runs in Figure 19 . Overall averages all

1{,}000

steps. Change is the percentage difference between the final and first 100-step means, computed before rounding.

Method	Overall	Global-temporal	Local-evidence
Vanilla	50.5	51.8	49.2
Key Windows	57.8	50.2	65.4
GRPO	55.8	53.4	58.2
OPSD	54.5	52.4	56.6
OWLET	59.2	58.6	59.8

Appendix

Table 34: Accuracy (%) on the held-out diagnostic split before and after post-training, overall and by question type, for Qwen3.5-9B at 64 input frames. The Vanilla and Key Windows rows use the same initial checkpoint with uniformly sampled and question-conditioned Key Windows views, respectively; GRPO, OPSD, and OWLET are evaluated at their final checkpoints under the uniformly sampled student view.

Training steps	Video-MME	MVBench	LongVideoBench	Video-MME 2	MLVU	Avg.
1,000 (main text)	70.7	72.8	65.2	28.9	61.6	59.8
1,200	70.5	73.1	65.5	28.4	60.8	59.7
1,500	71.0	73.6	65.5	28.6	60.5	59.8
1,800	72.3	73.0	64.8	29.5	61.0	60.1
2,000	72.0	73.6	66.1	29.2	62.7	60.7
2,200	70.9	73.3	66.9	29.1	62.9	60.6

Appendix

Table 35: Per-benchmark accuracy (%) of OWLET checkpoints under an extended training schedule, for Qwen3.5-9B at 64 input frames. Checkpoints are saved at nine steps between the main-text budget of

1{,}000

and

3{,}000

, spaced

200

–

300

steps apart, and each is evaluated with the protocol of Table 1 on the same five benchmarks; the

1{,}000

-step row reproduces the corresponding OWLET entry there. Avg. is the macro-average over the five benchmarks, computed before rounding. The last row gives the change from

1{,}000

3{,}000

steps.

Explore similar work

Reinforce to Learn, Elect to Reason: A Dual Paradigm for Video Reasoning
Apr 6, 2026Songyuan Yang, Weijiang Yu, Jilin Ma +5Video Reasoning VLM Inference
National University of Defense Technology · Sun Yat-sen University
Video-OPSD: Exploiting Privileged Visual Evidence for On-Policy Self-Distillation in Video Large Language Models
Aug 27, 2026Ziyue Wang, Shiqi Huang, Weiwen Xu +2Video-Language Models On-Policy Self-Distillation
School of Electrical and Electronic Engineering, Nanyang Technological University · The Chinese University of Hong Kong
When to Retrieve, When to Stay: Uncertainty-Aware Temporal Evidence Allocation for Streaming Video-LLMs
Sep 29, 2026Xiang Hu, Jiazuo Yu, Lu Zhang +2Streaming Video Understanding Evidence Selection
IIAU Lab, Dalian University of Technology

Apr 6, 2026J/K move · Enter open · S save

Reinforce to Learn, Elect to Reason: A Dual Paradigm for Video Reasoning

Songyuan Yang, Weijiang Yu, Jilin Ma, Ziyu Liu+4

National University of Defense Technology · Sun Yat-sen University

Video Reasoning VLM Inference RL for VLMs

Open paper AskPDF arXiv

Video reasoning has advanced with large multimodal models (LMMs), yet their inference is often a single pass that returns an answer without verifying whether the reasoning is evidence-aligned. We introduce Reinforce to Learn, Elect to Reason (RLER), a dual paradigm that decouples learning to produce evidence from obtaining a reliable answer. In RLER-Training, we optimize the policy with group-relative reinforcement learning (RL) and 3 novel task-driven rewards: Frame-sensitive reward grounds reasoning on explicit key frames, Think-transparency reward shapes readable and parsable reasoning traces, and Anti-repetition reward boosts information density. These signals teach the model to emit structured, machine-checkable evidence and potentiate reasoning capabilities. In RLER-Inference, we apply a train-free orchestrator that generates a small set of diverse candidates, parses their answers and cited frames, scores them by evidence consistency, confidence, transparency, and non-redundancy, and then performs a robust evidence-weighted election. This closes the loop between producing and using evidence, improving reliability and interpretability without enlarging the model. We comprehensively evaluate RLER against various open-source and RL-based LMMs on 8 representative benchmarks. RLER achieves state of the art across all benchmarks and delivers an average improvement of 6.3% over base models, while using on average 3.1 candidates per question, indicating a favorable balance between compute and quality. The results support a simple thesis: making evidence explicit during learning and electing by evidence during inference is a robust path to trustworthy video reasoning.

Abstract

Explore similar work

Reinforce to Learn, Elect to Reason: A Dual Paradigm for Video Reasoning

Video-OPSD: Exploiting Privileged Visual Evidence for On-Policy Self-Distillation in Video Large Language Models

When to Retrieve, When to Stay: Uncertainty-Aware Temporal Evidence Allocation for Streaming Video-LLMs