On-policy distillation (OPD) has emerged as an effective post-training approach to improving fine-grained perception in multimodal large language models by amortizing costly inference-time zooming operations into a single-pass recipe. However, vanilla OPD is inherently susceptible to linguistic momentum: the teacher can follow the student's erroneous prefixes, thus weakening corrective supervision. To address this, we propose Blink On-Policy Distillation (BlinkOPD), in which the teacher "blinks" at every token: it reads the same prefix once with its eyes open and once with its eyes closed. A first-glance gain further strengthens the correction on rollouts that conflict with the teacher. By extracting more effective supervision from each on-policy trajectory, BlinkOPD achieves state-of-the-art average performance across six fine-grained benchmarks using only 500 training images (12x less data) than existing OPD methods. Beyond fine-grained perception, BlinkOPD also demonstrates strong zero-shot improvements across ten held-out benchmarks spanning general visual perception, visual hallucination, and multimodal reasoning abilities. Code: https://github.com/Supltz/BlinkOPD
Figures & tables
Figure 1: Comparison of existing OPSD methods on four visual abilities across 16 benchmarks using Qwen3.5-4B as the base model. With 12 × fewer training images, BlinkOPD outperforms competing methods on four visual abilities across 16 benchmarks.
Figure 2: Comparison of the distillation targets of Vision-OPD, VAD, CV-OPD, OPD-V, OPD-Aha and our proposed BlinkOPD on linguistic momentum and privileged evidence survival. (a) As the teacher reads more of the wrong prefix 1 1 1 We also provide a control experiment that replaces the prefix with another question’s answer of the same length, which confirms that this rise comes from the content of the prefix rather than its length ( Appendix B ). , the follow rate tends to rise: Vision-OPD rises from 23% to 100%, VAD, CV-OPD and OPD-V from about 50% to 100%, and OPD-Aha still ends at 64%, whereas BlinkOPD stays below 40% throughout. (b) As the teacher reads more of the wrong prefix, the survival score tends to decrease: Vision-OPD, VAD, CV-OPD and OPD-V turn negative within the first quarter of the prefix and never recover, and OPD-Aha also falls below zero by the end, whereas BlinkOPD remains positive and outperforms the others throughout.
Figure 3: Comparison of contrastive references on residual drift and false belief. (a) Image-based references leave 20–23% of the drift uncancelled, whereas the text-only and eyes-closed references leave about 10%. (b) All other references raise the probability of “not present” false belief to 0.39–0.63. Only our proposed eyes-closed reference is low on both measures.
Figure 4: Contrastive reference construction for OPSD. (a) Image-based references replace the crop with gray, black, noisy, or mismatched images [ 23 , 16 , 2 , 14 , 30 ] . (b) The text-only reference removes the image from the conversation [ 35 ] . (c) Our eyes-closed reference removes only the image-content tokens while preserving vision markers and original positional indices.
Figure 5: Overview of BlinkOPD. The student generates rollouts from the full image I and question q , together denoted by xs . The frozen privileged teacher θ0 reads each student prefix twice: once with its eyes open on the zoomed crop ( pt+ ), and once with its eyes closed ( pt∅ , Sec. 3.1 ). Their log-ratio rt keeps the privileged visual evidence, while the prefix-induced linguistic drift shared by both cancels. A prefix-free first glance of the same teacher flags rollouts whose answer conflicts with it, and the gain is raised by a factor γ on these rollouts ( Sec. 3.2 ). The student is trained toward the resulting distillation target ( Sec. 3.3 ). At test time, the student runs alone in a single pass.
Fine-Grained
High-Resolution
Real-World
Method
Size
Teacher
#Img
V ∗
ZoomBench
HR-4K
HR-8K
MME-RW EN
MME-RW CN
Avg.
Δ
Proprietary models (reported by [ 36 , 12 ] ; also evaluated in [ 37 , 30 , 23 ] )
GPT-5.2 [ 19 ]
–
–
–
79.1
50.9
81.1
78.4
72.6
68.8
71.8
GPT-5.4 [ 21 ]
–
–
–
77.0
52.7
84.0
77.9
74.2
70.9
72.8
Gemini 3.1 Pro [ 9 ]
–
–
–
88.0
61.2
89.6
86.9
76.5
73.3
79.3
Gemini 3.5 Flash [ 13 ]
–
–
–
89.0
61.4
89.1
86.6
75.3
74.0
79.2
Table 1: Fine-grained perception. Accuracy (%) on six benchmarks. Gray : results reported by the original papers or reproduced by others; listed for reference only. Agentic models and ZwZ are built on older backbones ( ‡ Qwen2.5-VL-7B, § Qwen3-VL-8B). Bold and underline mark the best and second best within each re-evaluated block. Teacher: Self is a privileged self-teacher (a copy of the student on the zoomed crop); V-Zero instead distills from a Qwen3.5-27B teacher. #Img: unique training images. Δ : gain over the base model.
General Perception
Hallucination
Multimodal Reasoning
Average
Method
MMStar
CV-Bench
MMVP
POPE
HallB.
MathVista
MathVerse
MATH-V
We-Math
DynaMath
Perc.
Hall.
Reas.
All
Δ
Qwen3.5-4B
Base model [ 24 ]
68.3
87.0
58.7
89.5
76.1
82.0
74.2
63.9
58.8
57.0
71.3
82.8
67.2
71.0
–
Vision-OPD [ 36 ]
71.5
85.6
59.3
89.4
71.7
81.2
71.8
60.9
63.3
63.9
72.1
80.6
68.2
73.1
+2.1
OPD-V [ 2 ]
71.4
85.8
58.7
89.5
73.3
80.5
72.3
61.6
64.9
61.1
72.0
81.4
68.1
73.2
+2.2
CV-OPD [ 30 ]
71.5
86.6
56.7
89.3
71.8
81.0
72.4
63.5
67.1
57.8
71.6
80.6
68.4
73.4
+2.4
Table 2: Zero-shot transfer to ten held-out benchmarks. Perc., Hall. and Reas.: average accuracy on general perception, hallucination mitigation and multimodal reasoning. All: average over all 16 benchmarks, including the six fine-grained perception benchmarks of Tab. 1 . Δ : change of All relative to the base model. Bold and underline : best and second best within each model size.
Figure 6: Corrective supervision and data efficiency with Qwen3.5-4B. (a) As the teacher reads more of the wrong prefix, the corrective density tends to decrease: Vision-OPD drops from 68% to 0%, VAD, CV-OPD and OPD-V from about 40% to 0%, and OPD-Aha declines to 28%, whereas BlinkOPD stays above 40% throughout. (b) Mean accuracy across the six fine-grained benchmarks at different training scales. BlinkOPD leads at every scale and reaches 79.6% with 6.2K images.
Variant with Qwen3.5-4B as the base model
Avg.
Δ
BlinkOPD (0.5K)
79.1
–
Ablation
w/o first-glance gain (same β for all rollouts)
78.5
−0.6
→ gray-image reference
77.4
−1.7
→ black-image reference
75.4
−3.7
→ Gaussian-noise-image reference
75.6
−3.5
Table 3: Ablation&sensitivity analysis of BlinkOPD (0.5K) on 4B.
Figure 7: Follow rate of the raw teacher (Vision-OPD) on the 1,732 student-wrong rollouts when it reads the student’s own prefix or a prefix of another question with the same length. The thin gray line marks the follow rate before reading any prefix (23.2%).
Figure 8: Residual drift δ (%, Eq. 5 ) when the teacher and the reference read (a) 50%, (b) 75% or (c) all of the student prefix ( Fig. 3 a).
Figure 9: Trade-off of a global gain on the probe of Sec. 2.2 . Each point is the share of rollouts on which the target picks the correct option, averaged over the read-out points τ=25 , 50, 75 and 100%. Raising the global gain β raises the corrective density C on student-wrong rollouts but lowers the retention R on student-correct ones.
Figure 10: First-glance gain ( γ=2 ) against the global gain on the probe of Sec. 2.2 : C against R , each averaged over the four read-out points, as β is swept from 0 to 12 for the global gain and from 0 to 6 for the first-glance gain, so that both reach a largest gain of 12. The first-glance gain lies above the global gain.
Method
Teacher
Reference
Role of the reference
Gain
Granularity
Labels
Vision-OPD [ 36 ]
EMA
none
–
–
–
no
OPD-V [ 2 ]
EMA
crop with a random black patch
on/off weight on each token’s loss
–
per token
no
CV-OPD [ 30 ]
EMA
another region of the same image
hinge term that keeps the student closer to the teacher
fixed weight
per token
yes ∗
VAD [ 37 ]
EMA
pixelated crop
projects the teacher–student shift onto the evidence direction
Table 7: Comparison with the reproduced OPSD baselines. Granularity: the level at which the strength of the reference varies. Labels: whether ground-truth answers are used during training ( ∗ CV-OPD skips the questions on which no rollout is correct).
Figure 11: Qualitative examples of fine-grained perception. Left: the test image; the red box marks the region that the question refers to and is drawn only for this figure, since all models see the original image. Middle: the boxed region, enlarged. Right: the question, its options and the ground truth. Bottom: the answer of each model ( ✓ correct, ✗ wrong).
Figure 12: Qualitative examples of general perception. Left: the test image. Right: the question, its options and the ground truth. Bottom: the answer of each model ( ✓ correct, ✗ wrong).
Figure 13: Qualitative examples of hallucination. The layout is the same as in Fig. 12 .
Figure 14: Qualitative examples of multimodal reasoning. The layout is the same as in Fig. 12 .
Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Xiaohongshu Inc.