Authors: Haowei Wang, Hao Lin, Qi Zhu, Jiayi Ji, Deyi Ji, Liujuan Cao, Rongrong Ji
Organizations: Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University · Moxin Technology
On-policy distillation (OPD) reveals versatility and convenience during VLM post-training, providing dense supervision for student from teacher. However, we observed an anomaly that the distilled VLM could still answer the questions accurately even when the related visual regions were masked or replaced with noise. Building on this observation, our quantitative and qualitative experiments demonstrate that OPD, which uses the answer as a hint, has failed to teach students to genuinely seek out and rely on visual evidence. Our theoretical analysis suggests that this failure arises from a shortcut enabled by the lack of visual constraints during distillation. To address the issue, we introduce Contrastive Vision On-policy Distillation (CV-OPD), which compares teacher distributions induced by positive and negative crops while conditioning on the same student prefix and augments optimization with a margin-based contrastive constraint, which is modulated with discrepancy gating across token and sample level. Specifically, CV-OPD first introduces a negative teacher unrelated to the answer and designs a Contrastive Divergence Loss (CDL) that constrains the feature distribution of student by contrasting the positive and negative teachers, correcting how the student exploits visual regions. The CV-OPD introduces a Discrepancy Aware Gate (DAG) that uses token-level and sample-level discrepancies to select visually relevant tokens for optimization, directing the student's attention toward visual content. Extensive experiments across a broad range of fine-grained visual benchmarks demonstrate that CV-OPD substantially alleviates shortcut and consistently improves the visual reasoning capabilities of existing VLMs.
Figures & tables
Figure 1: (a) The failure mode of Vision-OPD. The distilled VLM can generate correct answer even given wrong visual inputs. (b) Our CV-OPD shows less error with higher accuracy, which is even competitive while compared to closed-source VLMs.
Figure 2: Diagnostic evaluation of visual shortcut. (a) We evaluate accuracy under both black masking and Gaussian noise, where answer-relevant visual regions are replaced. (b) We further visualize the attention ( red rectangle ) of Vision-OPD to find out whether the correct visual information is utilized.
Figure 3: The pipeline of Contrastive Vision On-policy Distillation. CV-OPD introduces a negative teacher that provides unrelated visual information to fix the reliance of visual contents. Samples with at least one correct rollout are retained for training.
Model
Param.
V ⋆
Zoom
HR-Bench [-1pt]4K
HR-Bench [-1pt]8K
MME-RW [-1pt]EN
MME-RW [-1pt]CN
Avg6
Tool-Using and Region-Aware Models
DeepEyes ( Zheng et al., 2026 )
7B
82.63
46.15
75.50
70.00
63.95
62.41
66.77
SenseNova-MARS ( Chng et al., 2025 )
8B
92.15
47.81
83.13
78.38
67.90
68.90
73.05
Vision-RL 2 ( Shi et al., 2026 )
7B
91.62
59.88
78.12
75.00
62.41
58.73
70.96
Closed-Source Models
Gemini 3 Flash ( Google DeepMind, 2025 )
–
85.39
61.49
87.25
85.00
73.15
71.66
77.32
Table 1: Overall comparison on six fine-grained visual benchmarks. We report the models trained on the same data within the Qwen3.5-Based Models. Values are accuracies (%), and Avg6 is the unweighted mean. Within the Qwen3.5-based block, best results are bold and second-best results are underlined; the cyan-green row denotes CV-OPD.
Method
Evaluation
CDL
DAG
V ⋆
Zoom
HR-Bench [-1pt]4K
HR-Bench [-1pt]8K
MME-RW [-1pt]EN
MME-RW [-1pt]CN
Avg6
✗
✗
89.53
59.41
81.75
80.12
74.51
70.17
75.92
✗
✔
90.05
60.71
82.38
82.63
74.67
70.41
76.81
✔
✗
90.58
60.71
83.00
80.88
75.40
70.71
76.88
✔
✔
91.62
62.01
84.38
83.38
76.99
70.93
78.22
Table 2: Ablation of CDL and DAG in the CV-OPD pipeline.
Model
Param.
V ⋆
Zoom
HR-Bench [-1pt]4K
HR-Bench [-1pt]8K
MME-RW [-1pt]EN
MME-RW [-1pt]CN
Avg6
Base
4B
82.20
48.40
86.38
82.00
63.94
63.80
71.12
All Black
4B
91.62
59.76
81.88
81.62
74.94
70.73
76.76
Random Crop
4B
89.53
62.01
84.38
81.50
75.06
70.73
77.20
Neighbor Crop
4B
90.05
59.76
81.62
81.00
75.40
70.54
76.40
CV-OPD
4B
91.62
62.01
84.38
83.38
76.99
70.93
78.22
Table 3: Ablation of negative region selection strategies
DAG
Threshold
V ⋆
Zoom
HR-Bench [-1pt]4K
HR-Bench [-1pt]8K
MME-RW [-1pt]EN
MME-RW [-1pt]CN
Avg6
✗
–
90.58
60.71
83.00
80.88
75.40
70.71
76.88
✔
rollout
90.05
61.54
83.25
82.13
75.05
70.47
77.08
✔
batch
91.62
62.01
84.38
83.38
76.99
70.93
78.22
Table 4: Ablation of DAG threshold settings.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
Value
Role
GPU allocation
8 A100 GPUs
Training worker allocation
Backbone
Qwen3.5-4B
Student and EMA teacher model
Training samples
6,241
Number of visual reasoning examples
Training batch size
96
Number of samples per training batch
Actor update mini-batch size
96
Mini-batch size for actor updates
Rollouts per prompt
8
On-policy responses generated per sample
Appendix
Table 5: Training hyperparameters for CV-OPD.
Figure 4: Qualitative comparison of visual attention and model responses between Vision-OPD and CV-OPD. Vision-OPD relies on the dominant color of the mailbox and predicts red, whereas CV-OPD focuses on the lock itself and identifies its metallic silver color.
Figure 5: Attention visualization and model responses under the black-mask setting. After the answer-relevant region is replaced by black pixels, Vision-OPD still produces the expected answer despite failing to localize the relevant visual evidence, indicating shortcut-based prediction and hallucination. In contrast, CV-OPD explicitly describes the observable scene and its ambiguity, showing stronger reliance on available visual evidence.
Figure 6: CV-OPD response under the black-mask setting. When the umbrella is not clearly visible, CV-OPD explicitly acknowledges the lack of reliable color information and limits its description to the remaining visual evidence, although its forced-choice prediction is incorrect.
Figure 7: CV-OPD response under Gaussian noise. With the glove’s material obscured by visual corruption, CV-OPD recognizes that the available evidence is insufficient for reliable identification, despite producing an incorrect final choice.