Authors: Yijiang Li, Yijun Liang, Yunjie Tian, Bingyang Wang, Ke Zhang, Zhenfei Yin, Di Fu, Philip Torr, Nuno Vasconcelos
Organizations: UC San Diego · University of Maryland, College Park · Independent Researcher · Georgia Institute of Technology · Johns Hopkins University · University of Oxford
Visual on-policy distillation relies heavily on an informative teacher-student asymmetry, through either a larger, stronger teacher or privileged supervision, such as reference answers or ground-truth regions of interest. This raises a fundamental question: where can informative asymmetry come from when nothing privileged is available? We answer this by inverting where the asymmetry comes from. Rather than adding privileged information to the teacher, we subtract information from the student. This asymmetry creates the same effective learning signal for free as a teacher with access to information unavailable to the student, without ground-truth annotations, rewards, or a separate stronger teacher model. Building on this principle, we introduce Self-Supervised Visual On-Policy Distillation (S2VOPD), a simple yet effective method that constructs on-policy learning signals from asymmetric augmented views. S2VOPD distills the teacher's distribution conditioned on the original image on-policy into the student distribution conditioned on a strongly augmented view of the same image. We systematically explore a broad design space of visual augmentations and uncover that (1) asymmetry matters: all four augmentation families improve performance, while symmetric self-distillation degrades it; (2) strength matters: performance peaks at a moderate strength; and (3) the gap must remain task-consistent: augmentations that completely remove the question-relevant evidence can induce large but uninformative discrepancies. Across six fine-grained perception benchmarks, S2VOPD improves Qwen3.5-4B from 70.7% to 77.4%, above all open-source models compared, up to Qwen3-VL at 235B, and surpasses GPT-5.4. While holding training data the same, it recovers 96% of the improvement achieved by methods with privileged information. Website is at https://williamium3000.github.io/s2vopd
Figures & tables
Figure 1: Overview of S 2 VOPD. The student generates rollouts from a corrupted view of the image; an EMA teacher scores the same generated prefixes under the clean view; a top- k generalized JSD transfers the teacher’s better-informed token distributions to the student. The augmentation applied to the student view is the sole source of supervision and the object of our study.
Operator
Effects
Strengths Range
Information reduction
Downscale
fewer pixels and visual tokens (no resize-back)
s∼U(0.3,0.6) ; 0.4 – 0.7 ; 0.25 – 0.5
Downscale, 3-tier
one of three bands drawn per sample
[0.2,0.35] , [0.35,0.5] , [0.5,0.75]
Gaussian blur
soft low-pass
radius U(0.5,1.5) ; (1.5,3.0) ; (3.0,6.0)
Spectral band-stop
removes an annulus of the 2D spectrum
inner radius 0.15 – 0.35 , width 0.15 – 0.3
Gaussian noise
additive; lowers signal-to-noise ratio
DDPM step 200 ( σ≈0.11 ), ρ=0.5
Table 1: Overview of augmentation operators on the student policy.
Method
Size
V*
Zoom
HR-4K
HR-8K
MME-RW
MME-RW-CN
Avg
Open-source models
MiniCPM-V-4.5
–
70.68
42.60
69.63
61.50
62.65
61.64
61.45
Qwen2.5-VL
7B
78.53
42.49
71.62
67.88
60.80
58.30
63.27
MiMo-VL-RL
7B
83.25
45.68
73.50
69.38
62.73
55.89
65.07
Qwen3-VL
4B
80.10
40.24
78.25
72.88
63.47
63.63
66.43
Thyme
7B
82.20
45.09
77.00
72.00
64.80
64.59
67.61
Table 2: Performance of S 2 VOPD trained on a 12k subset of FineVision ( Wiedmann et al., 2026 ) , compared with open-source, proprietary, and methods that use privileged information on fine-grained perception benchmarks (accuracy, %; Avg is the average over the 6 benchmarks).
Perception
Math reasoning
Method
V*
Zoom
HR-4K
HR-8K
MME-RW
MME-RW-CN
MathVista
MathVerse
MathVision
Avg
Qwen3.5-4B
Qwen3.5-4B
84.29
47.69
84.38
80.13
63.86
63.70
75.80
67.56
65.26
70.30
w/ privileged info.
ZwZ-4B ( Wei et al., 2026 )
92.67
55.74
81.75
79.50
68.52
68.09
71.80
40.48
52.17
67.86
Opsd ( Zhao et al., 2026a )
85.86
59.88
84.25
76.75
74.70
73.40
72.80
66.17
55.92
72.19
Table 3: Comparison with prior methods trained on the Vision-OPD training data, across six fine-grained perception and three mathematical-reasoning benchmarks at different model scales. Bold and underline mark the best and the second-best entry in each column within each base model; § indicates that best instead of last checkpoint performance is reported.
Figure 2: (a) Performance of each augmentation family applied alone; (b) Accuracy vs. the teacher–student predictive gap induced by the augmentation. We measure the teacher-student gap as the token-level JS Divergence between the teacher and the student distributions, averaged over the first ten steps of training.
Method
V*
Zoom
HR-4K
HR-8K
MME-RW
MME-RW-CN
Avg
S 2 VOPD
87.43
57.99
84.88
83.62
72.87
71.29
76.35
w/o Aug.
79.06
50.18
86.50
80.62
62.85
63.90
70.52
w/o EMA (fixed teacher)
88.48
54.08
86.75
83.25
72.15
70.98
75.95
Table 4: Component ablation on S 2 VOPD-4B. w/o aug removes the view asymmetry by showing the student the same clean image as the teacher; w/o EMA keeps the augmentation but freezes the teacher at the base model.
On-policy self-distillation (OPSD) removes the external teacher required by on-policy distillation (OPD), but still needs asymmetric information between teacher and student so that the self-teacher provides a stronger learning signal. Existing methods create this asymmetry through privileged answers or visual evidence. We ask whether both can be removed, yielding a simpler form of OPSD driven purely by input conditioning. We propose Visual Contrastive Self-Distillation, namely VCSD, which converts image-content removal into an on-policy self-distillation signal. At each student-generated response prefix, the EMA teacher produces two next-token distributions under the same prompt and prefix: one conditioned on the original image and the other on a content-erased control. Their token-wise log-probability difference highlights candidates whose likelihood is specifically increased by the instance-level visual content. We use this contrast to sharpen the teacher's original-image distribution within its plausible support, and distill the resulting full-distribution target into the student. On ViRL39K, VCSD consistently outperforms matched OPSD across Qwen3-VL and Qwen3.5 models. It improves the seven-benchmark aggregate from 72.51% to 76.26% on Qwen3-VL-8B and from 74.97% to 79.24% on Qwen3.5-9B, with gains at all six evaluated scales. VCSD requires no external teacher, privileged answers, visual evidence signals, reasoning traces, or additional inference-time cost.
Yijun Liang, Yunjie Tian, Yijiang Li +4
University of Maryland, College Park · University of California, San Diego · Duke University +1
On-policy knowledge distillation has proven effective for language models, yet its application to vision-language models (VLMs) remains underexplored. We observe that standard on-policy distillation can improve a student's output quality while failing to strengthen its reliance on visual input: on vision-critical tokens, the student's predictions remain largely unchanged whether or not fine-grained visual detail is present, even though the teacher's predictions depend heavily on it.To make this difference observable, we introduce visual advantage (VA), the token-level log-probability difference when the teacher scores a student-generated rollout with versus without access to fine-grained visual detail. VA is concentrated in a small minority of tokens, and these high-VA tokens are the ones that actually carry the visual supervision signal. This motivates a distillation objective that treats them differently from language scaffolding, so their contribution is not diluted by the abundant surrounding language tokens.We propose Visual-Advantage On-Policy Distillation (VA-OPD), which uses VA at two granularities: rollout-level reweighting by trajectory-averaged VA, and token-level KL averaged within high-VA and low-VA groups separately. We train on two math datasets (Geometry3K and ViRL39K) and evaluate on eight benchmarks covering both mathematical reasoning and visual understanding, across three teacher sizes (4B, 8B, and 32B) on the Qwen3-VL family. VA-OPD improves over standard on-policy distillation on every benchmark, with the gain growing monotonically along both the teacher-size and data-scale axes, suggesting that these factors compound consistently.
Ruiqi Liu, Xiaolei Lv, Gengsheng Li +8
Institute of Automation, CAS · School of Advanced Interdisciplinary Sciences, UCAS · 3Hello +1
On-policy distillation (OPD) improves reasoning by providing token-level supervision from a teacher on a student's own trajectories. Existing methods primarily focus on enhancing this teacher-side guidance (e.g., by enriching teacher inputs and refining teacher feedback), yet we find that limited student perception is another critical bottleneck in multimodal OPD. By providing oracle visual facts, the performance of OPD-trained students can still be substantially improved for both weak and strong teachers. To address this bottleneck, we propose S-OPD, a simple multimodal on-policy distillation framework that explicitly strengthens student perceptual learning through two objectives. Specifically, Teacher-calibrated Policy Contrast separates student policies under original and masked images with teacher-based token-level gating, strengthening the student's reliance on visual evidence during reasoning. Policy Agreement aligns student policies under original and noise-perturbed images, further improving perceptual robustness to visual noise. Notably, our method can be seamlessly plugged into existing OPD frameworks, requiring no additional data annotations, model parameters or inference operations. Extensive experiments on eight benchmarks across student scales and distillation paradigms demonstrate consistent performance improvements, with gains of up to 4.25 points on LogicVista. When combined with existing teacher-side supervision methods, our method can yield further gains. Code is available at https://github.com/Sirilaw/S-OPD.
Siyuan Liu, Kanghui Tian, Yue Duan +4
Nanjing University · Fudan University · Nanjing University of Posts and Telecommunications