VLM Distillation

VLM: Vision-Language Model

Vision-language model distillation studies how to transfer image-and-text understanding from a teacher model to a smaller or more efficient student, often for tasks such as visual question answering, retrieval, video reasoning, and robotics. Methods align teacher and student features, logits, attention, or relationships between visual and text representations; others train on teacher-generated rationales or student-generated responses, sometimes using multiple teachers or privileged visual cues. Work also targets practical constraints such as fewer visual tokens, quantization, and deployment on devices with limited memory or compute. A central challenge is deciding which teacher signals are reliable and grounded in evidence the student can access, since copying a teacher too closely can preserve its errors or encourage shortcuts rather than visual reasoning.

Latest papers

126 papers29 papers in the last four weeks, up 142% on the four weeks before. 0.2% of all new papers.

ViCuR: Visual Cues as Recoverable Privilege for Multimodal On-Policy Distillation
Jun 4, 2026Kanghui Tian, Siyuan Liu, Ziang Yan +3VLM Distillation Multimodal Reasoning
1Shanghai AI Laboratory · 3Nanjing University · 2Fudan University
Geometry-Aware Distillation for Prompt Tuning Biomedical Vision-Language Models
Jun 3, 2026Tran Dinh Tien, Zhiqiang ShenVLM Adaptation VLM Distillation
Department of Machine Learning Mohamed bin Zayed University of Artificial Intelligence
Decomposed On-Policy Distillation for Vision-Language Reasoning: Steering Gradients for Visual Grounding
May 30, 2026Hee Suk Yoon, Eunseop Yoon, Jaehyun Jang +6Cross-Modal Knowledge Distillation VLM Distillation
Korea Advanced Institute of Science and Technology (KAIST) · University of Illinois Urbana-Champaign (UIUC) · Microsoft Research Asia (MSRA)
Multimodal Distribution Matching for Vision-Language Dataset Distillation
May 22, 2026Jongoh Jeong, Hoyong Kwon, Minseok Kim +1Cross-Modal Learning Cross-Modal Alignment
Visual Intelligence Lab., KAIST
Visual-Advantage On-Policy Distillation for Vision-Language Models
May 21, 2026Ruiqi Liu, Xiaolei Lv, Gengsheng Li +8Vision-Language Models VLM Distillation
Institute of Automation, CAS · School of Advanced Interdisciplinary Sciences, UCAS · 3Hello +1
Distill to Think, Foresee to Act: Cognitive-Physical Reinforcement Learning for Autonomous Driving
May 20, 2026Yang Wu, Qiang Meng, Zhaojiang Liu +3End-to-End Autonomous Driving NavSim
SJTU · FDU · NJU
Preferences Order, Ratings Anchor: From Fused Expert Aesthetic Ground Truth to Self-Distillation
May 19, 2026Yuanpei Zhao, Jie Lin, Chao Zhang +5Pairwise Preference Learning Personalized Image Aesthetics Assessment
1Sichuan University · 2NetEase Fuxi AI Lab · †Work done during an internship at NetEase Fuxi AI Lab. +1
HEED: Density-Weighted Residual Alignment for Hybrid Vision-Language Model Distillation
May 16, 2026Yihao Liang, Niraj K. JhaCross-Modal Knowledge Distillation Efficient VLM Inference
Princeton University
GeoWorld-VLM: Geometry from World Models for Vision-Language Models
May 15, 2026Renjie Gu, Kaichen Zhou, Yan Luo +1Visual Spatial Reasoning VLM Adaptation
Harvard AI and Robotics Lab · Kempner Institute for the Study of Natural and Artificial Intelligence · Harvard University
Offline Semantic Guidance for Efficient Vision-Language-Action Policy Distillation
May 15, 2026Jin Shi, Brady Zhang, Yishun LuVLM Distillation Efficient VLA Model Inference
Department of Mechanical Engineering University College London London, United Kingdom · Department of Engineering Science University of Oxford Oxford, United Kingdom
DeltaPrompts: Escaping the Zero-Delta Trap in Multimodal Distillation
May 15, 2026Jaehun Jung, Hyunwoo Kim, Brandon Cui +4VLM Distillation VLM Reasoning
NVIDIA Research
Hide to See: Reasoning-prefix Masking for Visual-anchored Thinking in VLM Distillation
May 12, 2026Seonghoon Yu, Dongjun Nam, Byung-Kwan Lee +1Cross-Modal Knowledge Distillation CoT Distillation
1KAIST · 3POSTECH · 2NVIDIA
OmniThoughtVis: A Scalable Distillation Pipeline for Deployable Multimodal Reasoning Models
May 12, 2026Yuanhao Yue, Chengyu Wang, Yuanjie Lyu +2Multimodal CoT Reasoning Multimodal Reasoning
Alibaba Group, Hangzhou, China
LLaVA-CKD: Bottom-Up Cascaded Knowledge Distillation for Vision-Language Models
May 11, 2026Nikolaos Gkalelis, Vasileios MezarisVision-Language Models Efficient VLM Inference
CERTH-ITI, Thessaloniki, Greece, 57001
Distilling 3D Spatial Reasoning into a Lightweight Vision-Language Model with CoT
May 10, 2026Alaa Asfour, Christopher Indris, Leihan Chen +2Cross-Modal Knowledge Distillation 3D Spatial Reasoning
Department of Computer Science, Toronto Metropolitan University, Toronto, ON, Canada
Switch-KD: Visual-Switch Knowledge Distillation for Vision-Language Models
Apr 16, 2026Haoyi Sun, Xiaoxiao Wang, Ning Mao +5Cross-Modal Knowledge Distillation Vision-Language Models
Li Auto Inc.
BARD: Bridging AutoRegressive and Diffusion Vision-Language Models Via Highly Efficient Progressive Block Merging and Stage-Wise Distillation
Apr 15, 2026Baoyou Chen, Hanchen Xia, Peng Tu +5Vision-Language Models Diffusion VLMs
Shanghai Academy of AI for Science · Fudan University · Shanghai Innovation Institute +1
TIPSv2: Advancing Vision-Language Pretraining with Enhanced Patch-Text Alignment
Apr 13, 2026Bingyi Cao, Koert Chen, Kevis-Kokitsi Maninis +16Vision-Language Pretraining Vision-Language Alignment
Google DeepMind · xAI · Epsilon Health +1
Orion-Lite: Distilling LLM Reasoning into Efficient Vision-Only Driving Models
Apr 9, 2026Jing Gu, Niccolò Cavagnero, Gijs DubbelmanEnd-to-End Autonomous Driving VLM Distillation
Eindhoven University of Technology
LinguDistill: Recovering Linguistic Ability in Vision-Language Models via Selective Cross-Modal Distillation
Apr 1, 2026Patrick Amadeus Irawan, Erland Hilman Fuadi, Shanu Kumar +2VLM Adaptation VLM Distillation
Mohamed bin Zayed University of Artificial Intelligence
MEDiC: Multi-objective Exploration of Distillation from CLIP
Mar 30, 2026Konstantinos Georgiou, Maofeng Tang, Hairong QiMasked Image Modeling VLM Distillation
Min H. Kao Department of Electrical Engineering and Computer Science The University of Tennessee Knoxville, TN 37996
CLIP-RD: Relational Distillation for Efficient CLIP Knowledge Distillation
Mar 26, 2026Jeannie Chung, Hanna Jang, Ingyeong Yang +2Cross-Modal Learning Cross-Modal Knowledge Distillation
Ewha Womans University, Seoul 03760, South Korea
GoldiCLIP: The Goldilocks Approach for Balancing Explicit Supervision for Language-Image Pretraining
Mar 25, 2026Deen Dayal Mohan, Hossein Souri, Vitali Petsiuk +4Vision-Language Pretraining VLM Distillation
AI Center – Mountain View, Samsung Electronics
Adaptive Dual-Teacher Distillation with Subnetwork Rectification for Bridging Semantic Gaps in Black-Box Domain Adaptation
Mar 24, 2026Zhe Zhang, Jing Li, Wanli Xue +4Source-Free Domain Adaptation Knowledge Distillation
School of Computer Science and Engineering, Tianjin University of Technology, Tianjin 300384, China. · School of Artificial Intelligence, Tianjin University, Tianjin 300350, China
Uncertainty-Aware Knowledge Distillation for Multimodal Large Language Models
Mar 22, 2026Jingchen Sun, Shaobo Han, Deep Patel +3Uncertainty-Aware Knowledge Distillation Knowledge Distillation
NEC Laboratories America, Inc., USA · University at Buffalo, SUNY · Rutgers University
VSD-MOT: End-to-End Multi-Object Tracking in Low-Quality Video Scenes Guided by Visual Semantic Distillation
Mar 21, 2026Jun DuMulti-Object Tracking Visual Object Tracking
DAIT: Distillation from Vision-Language Models to Lightweight Classifiers with Adaptive Intermediate Teacher Transfer
Mar 16, 2026Zhengxu He, Jun Li, Zhijian WuVLM Distillation Fine-Grained Image Classification
School of Computer and Electronic Information, Nanjing Normal University, Nanjing, China · Medical Artificial Intelligence Lab, Westlake University
NanoVDR: Distilling a 2B Vision-Language Retriever into a 70M Text-Only Encoder for Visual Document Retrieval
Mar 13, 2026Zhuchenyang Liu, Yao Zhang, Yu XiaoVisual Document Retrieval Vision-Language Retrieval
Aalto University, Espoo, Finland
Beyond Single-Sample: Reliable Multi-Sample Distillation for Video Understanding
Mar 12, 2026Songlin Li, Xin Zhu, Zechao Guan +2VLM Distillation Knowledge Distillation
University of Electronic Science and Technology of China · Robotics Center, XPeng Motors
From Images to Words: Efficient Cross-Modal Knowledge Distillation to Language Models from Black-box Teachers
Mar 11, 2026Ayan Sengupta, Shantanu Dixit, Md Shad Akhtar +1Cross-Modal Knowledge Distillation Knowledge Distillation
Indian Institute of Technology Delhi, India · Indraprastha Institute of Information Technology Delhi, India

Jun 4, 2026J/K move · Enter open · S save

ViCuR: Visual Cues as Recoverable Privilege for Multimodal On-Policy Distillation

Kanghui Tian, Siyuan Liu, Ziang Yan, Sheng Xia+2

1Shanghai AI Laboratory · 3Nanjing University · 2Fudan University

VLM Distillation Multimodal Reasoning On-Policy Distillation Learning Using Privileged Information Teacher-Student Learning Knowledge Distillation

Open paper AskPDF arXiv

On-policy distillation (OPD) improves reasoning by training a student on trajectories sampled from its own policy under supervision from a teacher. In multimodal reasoning, a common extension is to use a privileged teacher that observes training-time-only signals such as reference answers or rationales. However, such answer-side privilege creates a train-test mismatch: the teacher's supervision may depend on signals unavailable to the student, encouraging shortcut imitation rather than visually grounded reasoning. We propose ViCuR, a visually grounded privileged-teacher distillation framework that replaces answer-side privilege with visual cues (query-related evidence in the input). Because these cues are derived from the same visual input available at inference, their evidence is recoverable by the student. To support this, ViCuR introduces a lightweight cue recovery module that uses dedicated sink-token cross-attention during prefill to aggregate task-relevant visual evidence into an internal representation, without changing the inference interface or requiring auxiliary cue-generation losses. Across seven benchmarks with Qwen3-VL-2B and 8B students, ViCuR consistently improves over answer-based on-policy self-distillation by +1.19 and +1.24 on overall average performance. It also extends naturally to stronger-teacher OPD, surpassing OPD baselines by +0.64 and +1.08, with consistent out-of-domain gains at the 8B scale. These results show that, in multimodal on-policy distillation, the design of teacher privilege is as important as teacher strength.