cs.CVNov 12, 2026

Teach with Blinks: Breaking Linguistic Momentum in On-Policy Distillation for Fine-Grained Multimodal Perception

Authors: Tangzheng Lian, Hang Zhang, Guanyu Hu, Renjiu Hu, Zhensong Zhang, Jiankang Deng, Jifei Song

Organizations: King’s College London · Cornell University · Aalto University · Huawei Noah’s Ark Lab · 

Abstract

On-policy distillation (OPD) has emerged as an effective post-training approach to improving fine-grained perception in multimodal large language models by amortizing costly inference-time zooming operations into a single-pass recipe. However, vanilla OPD is inherently susceptible to linguistic momentum: the teacher can follow the student's erroneous prefixes, thus weakening corrective supervision. To address this, we propose Blink On-Policy Distillation (BlinkOPD), in which the teacher "blinks" at every token: it reads the same prefix once with its eyes open and once with its eyes closed. A first-glance gain further strengthens the correction on rollouts that conflict with the teacher. By extracting more effective supervision from each on-policy trajectory, BlinkOPD achieves state-of-the-art average performance across six fine-grained benchmarks using only 500 training images (12x less data) than existing OPD methods. Beyond fine-grained perception, BlinkOPD also demonstrates strong zero-shot improvements across ten held-out benchmarks spanning general visual perception, visual hallucination, and multimodal reasoning abilities. Code: https://github.com/Supltz/BlinkOPD

Figures & tables

Explore similar work

CardsList
  1. Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation

    May 18, 2026Qianhao Yuan, Jie Lou, Xing Yu +4Multimodal Large Language ModelsOn-Policy Distillation

  2. Is Better Teacher Supervision Enough? Unlocking Student-side Learning in Multimodal On-Policy Distillation

    Sep 30, 2026Siyuan Liu, Kanghui Tian, Yue Duan +4Vision-Language ModelsVisual Reasoning

  3. BAS-OPD: Budget-Aware Selective On-Policy Self-Distillation for Fine-Grained Multimodal Perception

    Sep 22, 2026Zihan Chen, Hengguang Zhou, Yuan Kang +5VLM DistillationMultimodal Large Language Models