cs.CVNov 7, 2026

Tiny Evidence Region Localization with Selective Contrastive On-Policy Self-Distillation

Authors: Yujuan Pang, Wenhao Xu, Yinglei Teng

Organizations: Beijing University of Posts and Telecommunications, Beijing, China

Abstract

Multimodal large language models have made progress in general visual tasks. However, limitations remain in fine-grained visual tasks requiring evidence from minute image regions. Existing methods primarily train models using evidence-related regions while overlooking the influence of extensive background regions, resulting in a limited ability to resist interference from irrelevant regions. We propose selective positive-negative on-policy self-distillation (SPN-OPSD), which considers both evidence and background regions. Supervision from both views improves the model's ability to resist interference from irrelevant regions. Furthermore, selective token-level distillation provides more informative supervision, improving the model's ability to attend to evidence regions in Fine-Grained Visual Tasks. On Qwen3.5-2B, SPN-OPSD improves accuracy on V* Bench, ZoomBench, and MME-RealWorld-Lite from 76.44%, 45.92%, and 57.43% to 84.82%, 54.56%, and 59.56%, respectively. Similar gains are also observed on Qwen3.5-0.8B.

Figures & tables

Explore similar work

CardsList
  1. Look Where It Counts: A Free, Label-Free Visual Evidence Signal for Fine-Grained Vision-Language Reasoning

    Sep 21, 2026Santi Ram Tiwari, Nihal Naik, Devbrat Pandey +1Vision-Language GroundingVLM Reasoning

  2. Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation

    May 18, 2026Qianhao Yuan, Jie Lou, Xing Yu +4Multimodal Large Language ModelsOn-Policy Distillation

  3. Evidence-Aligned Multimodal On-Policy Self-Distillation for Fine-Grained Visual Understanding

    Sep 28, 2026Nanxing Hu, Qiwei Yan, Jinchao Zhang +1On-Policy Self-DistillationVLM Distillation