Self-Supervised Visual On-Policy Distillation

Yijiang Li; Yijun Liang; Yunjie Tian; Bingyang Wang; Ke Zhang; Zhenfei Yin; Di Fu; Philip Torr

cs.CVAug 14, 2026

Self-Supervised Visual On-Policy Distillation

Authors: Yijiang Li, Yijun Liang, Yunjie Tian, Bingyang Wang, Ke Zhang, Zhenfei Yin, Di Fu, Philip Torr,

Organizations: UC San Diego · University of Maryland, College Park · Independent Researcher · Georgia Institute of Technology ·

Abstract

Visual on-policy distillation relies heavily on an informative teacher-student asymmetry, through either a larger, stronger teacher or privileged supervision, such as reference answers or ground-truth regions of interest. This raises a fundamental question: where can informative asymmetry come from when nothing privileged is available? We answer this by inverting where the asymmetry comes from. Rather than adding privileged information to the teacher, we subtract information from the student. This asymmetry creates the same effective learning signal for free as a teacher with access to information unavailable to the student, without ground-truth annotations, rewards, or a separate stronger teacher model. Building on this principle, we introduce Self-Supervised Visual On-Policy Distillation (S $^2$ VOPD), a simple yet effective method that constructs on-policy learning signals from asymmetric augmented views. S $^2$ VOPD distills the teacher's distribution conditioned on the original image on-policy into the student distribution conditioned on a strongly augmented view of the same image. We systematically explore a broad design space of visual augmentations and uncover that (1) asymmetry matters: all four augmentation families improve performance, while symmetric self-distillation degrades it; (2) strength matters: performance peaks at a moderate strength; and (3) the gap must remain task-consistent: augmentations that completely remove the question-relevant evidence can induce large but uninformative discrepancies. Across six fine-grained perception benchmarks, S $^2$ VOPD improves Qwen3.5-4B from 70.7% to 77.4%, above all open-source models compared, up to Qwen3-VL at 235B, and surpasses GPT-5.4. While holding training data the same, it recovers 96% of the improvement achieved by methods with privileged information. Website is at https://williamium3000.github.io/s2vopd

Figures & tables

Figure 1: Overview of S 2 VOPD. The student generates rollouts from a corrupted view of the image; an EMA teacher scores the same generated prefixes under the clean view; a top- $k$ generalized JSD transfers the teacher’s better-informed token distributions to the student. The augmentation applied to the student view is the sole source of supervision and the object of our study.

Operator	Effects	Strengths Range
Information reduction
Downscale	fewer pixels and visual tokens (no resize-back)	$s\,{\sim}\,\mathcal{U}(0.3,0.6)$ ; $0.4$ – $0.7$ ; $0.25$ – $0.5$
Downscale, 3-tier	one of three bands drawn per sample	$[0.2,0.35]$ , $[0.35,0.5]$ , $[0.5,0.75]$
Gaussian blur	soft low-pass	radius $\mathcal{U}(0.5,1.5)$ ; $(1.5,3.0)$ ; $(3.0,6.0)$
Spectral band-stop	removes an annulus of the 2D spectrum	inner radius $0.15$ – $0.35$ , width $0.15$ – $0.3$
Gaussian noise	additive; lowers signal-to-noise ratio	DDPM step $200$ ( $\sigma{\approx}0.11$ ), $\rho{=}0.5$

Table 1: Overview of augmentation operators on the student policy.

Method	Size	V*	Zoom	HR-4K	HR-8K	MME-RW	MME-RW-CN	Avg
Open-source models
MiniCPM-V-4.5	–	70.68	42.60	69.63	61.50	62.65	61.64	61.45
Qwen2.5-VL	7B	78.53	42.49	71.62	67.88	60.80	58.30	63.27
MiMo-VL-RL	7B	83.25	45.68	73.50	69.38	62.73	55.89	65.07
Qwen3-VL	4B	80.10	40.24	78.25	72.88	63.47	63.63	66.43
Thyme	7B	82.20	45.09	77.00	72.00	64.80	64.59	67.61

Table 2: Performance of S 2 VOPD trained on a 12k subset of FineVision ( Wiedmann et al., 2026 ) , compared with open-source, proprietary, and methods that use privileged information on fine-grained perception benchmarks (accuracy, %; Avg is the average over the 6 benchmarks).

	Perception						Math reasoning
Method	V*	Zoom	HR-4K	HR-8K	MME-RW	MME-RW-CN	MathVista	MathVerse	MathVision	Avg
Qwen3.5-4B
Qwen3.5-4B	84.29	47.69	84.38	80.13	63.86	63.70	75.80	67.56	65.26	70.30
w/ privileged info.
ZwZ-4B ( Wei et al., 2026 )	92.67	55.74	81.75	79.50	68.52	68.09	71.80	40.48	52.17	67.86
Opsd ( Zhao et al., 2026a )	85.86	59.88	84.25	76.75	74.70	73.40	72.80	66.17	55.92	72.19

Table 3: Comparison with prior methods trained on the Vision-OPD training data, across six fine-grained perception and three mathematical-reasoning benchmarks at different model scales. Bold and underline mark the best and the second-best entry in each column within each base model; § indicates that best instead of last checkpoint performance is reported.

Figure 2: (a) Performance of each augmentation family applied alone; (b) Accuracy vs. the teacher–student predictive gap induced by the augmentation. We measure the teacher-student gap as the token-level JS Divergence between the teacher and the student distributions, averaged over the first ten steps of training.

Method	V*	Zoom	HR-4K	HR-8K	MME-RW	MME-RW-CN	Avg
S 2 VOPD	87.43	57.99	84.88	83.62	72.87	71.29	76.35
w/o Aug.	79.06	50.18	86.50	80.62	62.85	63.90	70.52
w/o EMA (fixed teacher)	88.48	54.08	86.75	83.25	72.15	70.98	75.95

Table 4: Component ablation on S 2 VOPD-4B. w/o aug removes the view asymmetry by showing the student the same clean image as the teacher; w/o EMA keeps the augmentation but freezes the teacher at the base model.

Table 7

On-Policy Distillation

On-policy distillation trains a model on its own outputs using teacher guidance, aligning supervision with the states it encounters at deployment.

See all 511 papers

Visual Contrastive Self-Distillation
Jul 23, 2026Yijun Liang, Yunjie Tian, Yijiang Li +4On-Policy Self-Distillation VLM Distillation
University of Maryland, College Park · University of California, San Diego · Duke University +1
Visual-Advantage On-Policy Distillation for Vision-Language Models
May 21, 2026Ruiqi Liu, Xiaolei Lv, Gengsheng Li +8Vision-Language Models VLM Distillation
Institute of Automation, CAS · School of Advanced Interdisciplinary Sciences, UCAS · 3Hello +1
Is Better Teacher Supervision Enough? Unlocking Student-side Learning in Multimodal On-Policy Distillation
Sep 30, 2026Siyuan Liu, Kanghui Tian, Yue Duan +4Vision-Language Models Visual Reasoning
Nanjing University · Fudan University · Nanjing University of Posts and Telecommunications

Jul 23, 2026J/K move · Enter open · S save

Visual Contrastive Self-Distillation

Yijun Liang, Yunjie Tian, Yijiang Li, Yuqi Jia+3

University of Maryland, College Park · University of California, San Diego · Duke University+1

On-Policy Self-Distillation VLM Distillation Knowledge Distillation

Open paper AskPDF arXiv

On-policy self-distillation (OPSD) removes the external teacher required by on-policy distillation (OPD), but still needs asymmetric information between teacher and student so that the self-teacher provides a stronger learning signal. Existing methods create this asymmetry through privileged answers or visual evidence. We ask whether both can be removed, yielding a simpler form of OPSD driven purely by input conditioning. We propose Visual Contrastive Self-Distillation, namely VCSD, which converts image-content removal into an on-policy self-distillation signal. At each student-generated response prefix, the EMA teacher produces two next-token distributions under the same prompt and prefix: one conditioned on the original image and the other on a content-erased control. Their token-wise log-probability difference highlights candidates whose likelihood is specifically increased by the instance-level visual content. We use this contrast to sharpen the teacher's original-image distribution within its plausible support, and distill the resulting full-distribution target into the student. On ViRL39K, VCSD consistently outperforms matched OPSD across Qwen3-VL and Qwen3.5 models. It improves the seven-benchmark aggregate from 72.51% to 76.26% on Qwen3-VL-8B and from 74.97% to 79.24% on Qwen3.5-9B, with gains at all six evaluated scales. VCSD requires no external teacher, privileged answers, visual evidence signals, reasoning traces, or additional inference-time cost.

Abstract

On-Policy Distillation

Explore similar work

Visual Contrastive Self-Distillation

Visual-Advantage On-Policy Distillation for Vision-Language Models

Is Better Teacher Supervision Enough? Unlocking Student-side Learning in Multimodal On-Policy Distillation