MeSD: Multi-Evidence Self-Distillation for VideoLLM

Weijie Zhu; Han Fang; Hanyu Fu; Yuzhe Zhang; Xin Wei; Zhaoyan Pan; Feiran Liu; Xunjie Jin

cs.CVOct 5, 2026

MeSD: Multi-Evidence Self-Distillation for VideoLLM

Authors: Weijie Zhu, Han Fang, Hanyu Fu, Yuzhe Zhang, Xin Wei, Zhaoyan Pan, Feiran Liu, Xunjie Jin, +8 more

Organizations: UCAS · Xingchen AGI Lab, China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd · PKU · ZJU · BUAA

Abstract

While reinforcement learning with verifiable rewards provides reliable outcome supervision for VideoLLMs, sequence-level rewards offer limited token-level guidance. On-policy self-distillation addresses this limitation by conditioning a self-teacher on privileged information to provide dense token-level supervision. However, aggregating heterogeneous evidence within a single teacher context obscures cross-evidence agreement and conflict. A further challenge lies in determining whether teacher guidance should refine reward-based updates or provide corrective supervision for failed trajectories. To address these issues, we propose MeSD, a multi-evidence self-distillation framework for VideoLLMs. MeSD constructs three evidence-conditioned teachers with shared parameters, using the ground-truth answer as a common semantic context while separately incorporating temporal and spatial evidence. Given the same student-generated prefixes, MeSD evaluates evidence-specific preferences relative to the Answer Teacher and fuses teacher-common preferences with gated teacher-specific residuals. Furthermore, MeSD introduces Verification-Guided Optimization to classify trajectories as Success, Failure, or Indeterminate. For Success and Indeterminate trajectories, MeSD refines token-level advantage magnitudes while preserving reward-derived signs. For verified failure trajectories that contain the required evidence, MeSD applies failure-conditioned distillation, using reverse-KL correction toward the fused distribution. Experiments on multiple video benchmarks demonstrate consistent gains over reinforcement learning and self-distillation baselines.

Figures & tables

Figure 1: (a) Temporal and spatial evidence yield consistent or conflicting token preferences. (b) MeSD combines reward-anchored credit refinement with corrective distillation on verified failures. (c) MeSD leads on selected metrics across six benchmarks. (d) Comparison of training rewards.

Figure 2: Overview of MeSD. Teachers sharing EMA parameters use answer, temporal, and spatial annotations as privileged information to evaluate student-generated trajectories. Using the Answer Teacher as an anchor, teacher distribution fusion aggregates evidence-specific preferences through a gated consensus–residual decomposition. Trajectory verification determines how the student learns. For failed trajectories, the student matches the fused teacher distribution through reverse-KL distillation. For other trajectories, teacher guidance adjusts the strength of token-level updates, and rewards determine whether tokens are reinforced or suppressed.

Figure 3: Training ablation curves. MeSD achieves higher training rewards than (a) credit refinement alone (w/o FCD) and (b) multi-evidence OPSD (w/o VGO). (c) Varying the teacher cutoff step

H

and

\gamma

yields similar reward trajectories.

Figure 4: Sensitivity of MeSD to

\gamma

and the teacher cutoff step

H

across five benchmarks. Filled bars indicate the default settings (

\gamma=0.8

H=1000

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Task type	Samples	Answer	Temporal	Spatial
General video QA (MCQ)	13,000	Option	–	–
General video QA (free-form)	2,000	Text	–	–
Temporal QA	2,280	–	Time intervals	–
Temporal QA (MCQ)	2,904	Option	Time intervals	–
Visual QA	5,000	Object name	–	Object boxes
Joint spatio-temporal QA	12,047	Text	Keyframe timestamps	Object boxes

Appendix

Table 5: Task composition and annotation types of STGR-RL. Answer denotes the semantic answer used in the answer context.

Training and rollout		Self-distillation
Parameter	Value	Parameter	Value
Training epochs	1	EMA update rate	0.01
Optimizer	AdamW	Initial strength $\lambda_{0}$	0.5
Learning rate	$10^{-6}$	Annealing steps	1,000
LR schedule	Cosine	Token-weight clipping	$[0.8,\,1.2]$
Prompt batch size	16	Correction strength $\gamma$	0.8

Appendix

Table 6: Training configuration of MeSD.

Figure 5: Answer teacher prompt with privileged semantic answers.

Figure 6: Answer+Temporal teacher prompt with semantic answers and keyframe timestamps.

Figure 7: Answer+Spatial teacher prompt with semantic answers and object-level bounding boxes.

Task	Success	Failure	RKL eligibility
Video MCQ	Correct option	Incorrect option	Incorrect option
Temporal localization	$\mathrm{tIoU}\geq 0.5$	$\mathrm{tIoU}=0$	$\mathrm{tIoU}=0$
Temporal MCQ	Correct option and $\mathrm{tIoU}\geq 0.5$	Incorrect option or $\mathrm{tIoU}=0$	Incorrect option and $\mathrm{tIoU}=0$
Spatial localization	Not assigned	Not assigned	Not eligible
Other composite or free-form tasks	Not assigned	Not assigned	Not eligible

Appendix

Table 7: Task-specific outcome verification and RKL eligibility under the strict setting.

Method	Steps	Time (h)	A800 GPU-hours
VISD no-feedback	2,327	61.36	981.70
MeSD	2,327	20.96	335.38

Appendix

Table 12: Training efficiency comparison at the same number of training steps.

Model	What	When (tIoU)		Where (sIoU)		Overall
Model	Acc.	Chain1	Chain2	Chain1	Chain2	mAM	mLGM
Gemini-2-Flash [ Google DeepMind, 2025 ]	53.0	24.5	23.8	4.6	2.2	26.9	35.6
GPT-4o [ OpenAI, 2024 ]	60.8	16.7	12.8	6.5	3.0	26.8	38.2
Oryx-1.5-7B [ Liu et al., 2025 ]	20.5	13.5	14.8	10.1	3.5	15.1	13.8
InternVL-2.5-8B [ Chen et al., 2024 ]	44.2	8.7	7.8	0.7	0.1	17.6	24.9
Qwen2.5-VL-7B [ Bai et al., 2025b ]	33.5	15.4	13.8	17.0	2.5	19.3	22.4

Appendix

Table 13: Detailed V-STAR comparison.

Model	WorldSense
Model	Overall	Recognition	Understanding	Reasoning
GPT-4o	42.6	–	–	–
Qwen2.5-VL-7B	35.5	31.8	38.4	38.7
Open-o3-Video-7B	38.9	36.8	39.7	40.5
VISD-7B	39.9	39.4	40.4	40.5
MeSD-7B (Ours)	41.6	38.9	46.0	42.3

Appendix

Table 14: Detailed comparison on WorldSense, VideoMMMU, LRR, and TVGBench.

Component Variants
Model	What	When (tIoU)		Where (sIoU)		Overall
Model	Acc.	Chain1	Chain2	Chain1	Chain2	mAM	mLGM
w/o FCD	56.4	22.0	20.9	23.7	5.0	30.7	41.0
w/o ES	56.5	22.1	21.7	23.4	5.2	30.9	41.3
w/o T&S	56.4	22.3	21.4	23.5	5.1	30.8	41.2
w/o GPA	56.5	22.3	21.7	23.6	5.1	30.9	41.4

Appendix

Table 15: Detailed V-STAR ablations.

Component Variants
Model	WorldSense
Model	Overall	Recognition	Understanding	Reasoning
w/o FCD	38.6	38.2	38.7	39.0
w/o ES	38.3	38.4	38.4	38.1
w/o T&S	37.9	37.3	39.3	37.3
w/o GPA	39.5	37.9	39.5	38.6

Appendix

Table 16: Detailed ablations on WorldSense, VideoMMMU, LRR, and TVGBench.

Figure 8: Training reward for Qwen3-VL models at 4B, 8B, and 32B.

Figure 9: Training reward for MeSD and the w/o ES, w/o T&S, and w/o GPA component variants.

Explore similar work

VISD: Enhancing Video Reasoning via Structured Self-Distillation
May 7, 2026Hao Lin, Kunyang Lv, Xu Jiang +5Latent Visual Reasoning Process-Level Supervision
1HUST · 2Wuhan University · 3Peking University +1
EvoVid: Temporal-Centric Self-Evolution for Video Large Language Models
May 21, 2026Shiqi Huang, Ziyue Wang, Zhongrong Zuo +3Self-Evolution
School of Electrical and Electronic Engineering, Nanyang Technological University · ByteDance
Frame Differential On-Policy Self-Distillation for Video Reasoning
Sep 30, 2026Haiying He, Xin Zheng, Shaoli Hu +4Unsupervised On-Policy Self-Distillation Offline Reinforcement Learning
HKUST · NKU · SEU +1

May 7, 2026, cs.CVJ/K move · Enter open · S save

VISD: Enhancing Video Reasoning via Structured Self-Distillation

Hao Lin, Kunyang Lv, Xu Jiang, Jingqi Tian+4

1HUST · 2Wuhan University · 3Peking University+1

Latent Visual Reasoning Process-Level Supervision

Open paper AskPDF arXiv

Training VideoLLMs for complex reasoning remains challenging due to sparse sequence level rewards and the lack of fine grained credit assignment over long, temporally grounded reasoning trajectories. While reinforcement learning with verifiable rewards (RLVR) provides reliable supervision, it fails to capture token level contributions, leading to inefficient learning. Conversely, existing self distillation methods offer dense supervision but lack structure and diagnostic specificity, and often interact unstably with reinforcement learning. In this work, we propose VISD, a structured self distillation framework that introduces diagnostically meaningful privileged information for video reasoning. VISD employs a video aware judge model to decompose reasoning quality into multiple dimensions, including answer correctness, logical consistency, and spatio-temporal grounding, and uses this structured feedback to guide a teacher policy for token level supervision. To stably integrate dense supervision with RL, we introduce a direction magnitude decoupling mechanism, where rollout level advantages computed from rewards determine update direction, while structured privileged signals modulate token level update magnitudes. This design enables semantically aligned and fine grained credit assignment, improving both reasoning faithfulness and training efficiency. Additionally, VISD incorporates curriculum scheduling and EMA based teacher stabilization to support robust optimization over long video sequences. Experiments on diverse benchmarks show that VISD consistently outperforms strong baselines, improving answer accuracy and spatio temporal grounding quality. Notably, VISD reaches these gains with nearly 2x faster convergence in optimization steps, highlighting the effectiveness of structured self supervision in improving both performance and sample efficiency for VideoLLMs.

Abstract

Explore similar work

VISD: Enhancing Video Reasoning via Structured Self-Distillation

EvoVid: Temporal-Centric Self-Evolution for Video Large Language Models

Frame Differential On-Policy Self-Distillation for Video Reasoning